Logging | HoldMyBeer

Category Archives: Logging

Leave a comment

Incident Response, IR Tales, Logging, System Administration, Tools

IR Tales: The Quest for the Holy SIEM: Splunk + Sysmon + Osquery + Zeek

This blog post is the season finale in a series to demonstrate how to install and setup common SIEM platforms. The ultimate goal of each blog post is to empower the reader to choose their own adventure by selecting the best SIEM based on their goals or requirements. Each blog post in the series will provide Docker-compose v2, Docker-compose for Swarm, Ansible, and manual instructions to allow the reader to setup each platform with the deployment method of their choosing. In addition to setting up Splunk, I will cover fundamental Splunk concepts such as the Common Information Model (CIM). Lastly, I will provide step-by-step instructions to install Sysmon + Splunk Universal Forwarder on Windows, Osquery + Splunk Universal Forwarder on Ubuntu, and Zeek + Filebeat to ship logs to Splunk.

Continue reading →

Leave a comment

Logging, Network Security, System Administration, Tools

Implementing Logstash and Filebeat with mutual TLS (mTLS)

Do you know if your Filebeat client is connecting to a rogue Logstash server? Do you know if your Logstash server is accepting random logs from random devices? If you have answered “I don’t know” to either of these questions then this blog post is for you. The purpose of this blog post is to provide instructions on how to setup Logstash and Filebeat with mutual TLS (mTLS). The step-by-step instructions in this post, will demonstrate how to create the certificate chain of trust using Vault. Lastly, I will cover the Python script I created to automate constructing this logging certificate chain of trust.

Continue reading →

DevOps, DevOps Tales, Logging, Tools

DevOps Tales: Install/Setup Gitlab + Gitlab runners on Docker, Windows, Linux and macOS

Are you tired of manually pushing code to production? Are you always searching through your BASH history to find the commands you used to test your code? Do you wish the process to merge code into production had a defined process? Well I have the solution for you! Introducing Gitlab CI/CD pipelines! With Gitlab you can setup Gitlab runners to create a CI/CD pipeline. A CI/CD pipeline will revolutionize your workflow to push code to production.

The purpose of this blog post is to provide instructions on how to setup the necessary components (Gitlab and Gitlab runners) to create a CI/CD pipeline. One of the deliverables from this blog post is Docker composes for Swarm and non-swarm deployments of Gitlab. Additionally, there are manual instructions on how to setup Gitlab runners on Ubuntu 20.04, Ubuntu 20.04 with Docker, Windows 10, Windows 10 with Docker, and macOS Big Sur. In addition, a Docker Registry is setup and integrated into the CI/CD pipeline for custom Docker images. The instructions and the infra-as-code provided in this post will create the foundation for future blogs that will contain a CI/CD component.

Continue reading →

Leave a comment

IR Tales, Logging, System Administration, Tools

IR Tales: The Quest for the Holy SIEM: Graylog + AuditD + Osquery

This blog post is the second in a series to demonstrate how to install and setup common SIEM platforms. The ultimate goal of each blog post is to empower the reader to choose their own adventure by selecting the best SIEM based on their goals or requirements. Each blog post in the series will provide Docker-compose v2, Docker-compose for Swarm, Ansible, and manual instructions to allow the reader to setup each platform with the deployment method of their choosing. This blog post will also cover how to setup the Graylog with Elasticsearch and Mongo. In addition to setting up the Graylog I will provide instructions to install Osquery + Filebeat on Windows and AuditD + Auditbeat on Ubuntu to ship logs to Elastic.

Continue reading →

Leave a comment

IR Tales, Logging, System Administration, Tools

IR Tales: The Quest for the Holy SIEM: Elastic stack + Sysmon + Osquery

This blog post is the first in a series to demonstrate how to install and setup common SIEM platforms. The ultimate goal of each blog post is to empower the reader to choose their own adventure by selecting the best SIEM based on their goals or requirements. Each blog post in the series will provide Docker-compose v2, Docker-compose for Swarm, Ansible, and manual instructions to allow the reader to setup each platform with the deployment method of their choosing. This blog post will cover how to setup the Elastic stack formerly known as ELK. In addition to setting up the Elastic stack I will provide instructions to install Sysmon + Winlogbeat on Windows and Osquery + Filebeat on Ubuntu to ship logs to Elastic.

Continue reading →

Leave a comment

Incident Response, Logging, System Administration, Tools

Getting started with FleetDM v3.6.0

The purpose of this blog post is to provide multiple methods on how to install/setup FleetDM, how to deploy Osquery, and demonstrate how to use features of FleetDM + FleetCTL. This blog post generated an Ansible playbook, Docker-composes for Swarm and non-swarm, and manual instructions for installing FleetDM on Ubuntu 20.04. Additionally, there are Ansible playbooks for deploying the Osquery agent on Windows and Ubuntu with manual instructions as well. Lastly, I will end by demonstrating how to use the FleetDM WebGUI and FleetCTL tool to manage FleetDM and interact with your Osquery agents.

Continue reading →

Leave a comment

Incident Response, Logging, PoC, Threat Detection Engineering, Tools

Create a custom Splunk search commands with Python3

This blog post will demonstrate how to create a custom Python search command for Splunk and will demystify common roadblocks such as: how to create a custom search command with Python, how to store secrets for a custom search command, and how to install external Python libraries. With each roadblock discussed we will also cover the solution as code examples and hands-on exercises. To do this, we must first start with an introduction to the architecture of a custom Python search command.

Continue reading →

Leave a comment

Incident Response, Logging, Threat Detection Engineering, Tools

Demystifying the Kolide Fleet API with CURL, Python, Fleetctl, and Ansible

A common question in the #Kolide channel in the Osquery Slack is how to use the Kolide Fleet API. Kolide Fleet is written in GoLang and utilizes the GoKit framework to build the application. Therefore, almost every action that can be performed via the WebGUI is an API call. This blog post is going to demonstrate how to use the Kolide Fleet API to perform actions such as creating a live query and obtaining the results using Python websockets, obtaining the Osquery enroll secret, and creating a saved query. In addition to the API, this blog post will demonstrate how to use the Fleetctl command-line tool to perform the same actions. Lastly, this blog post includes an Ansible playbook to automate deploying Oquery agents and registering them with Kolide.

Continue reading →

Leave a comment

Incident Response, Logging, PoC, System Administration, Tools

Setting up Kolide and Osquery with client certificates for mutual TLS (mTLS)

Do you know if your Osquery client is connecting to the right server? Do you know if your Kolide server is accepting requests from rogue devices? If you have answered “I don’t know” to either of these questions then this blog post is for you. This blog post will be demonstrating how to setup Kolide + Osquery with mutual TLS (mTLS). Mutual TLS is a mechanism that can be implemented to verify the identity of the server and clients. Continue reading →

Incident Response, Logging, Network Security, PoC, Tools

PoC: Using KSQL to enrich Zeek logs with Osquery and Sysmon data

In incident response, time is precious and something you can never get back. Typically, when I receive a security alert about an endpoint, it requires manual lookups on multiple data sources for critical pieces of information. These manual lookups can be time-consuming, create fatigue, and don’t use the power of technology to your advantage. This blog post will demonstrate a proof-of-concept (POC) by using the power of a network community ID hash by Corelight to fuse endpoint and network-based data sources.

KSQL by Confluent provides the ability to enrich independent data sources by correlating common attributes. In this POC, we are going to use Sysmon or Osquery to monitor the endpoint and Zeek to monitor the network. Not only will this blog post serve as a POC but it will discuss the architecture, design decisions, working infrastructure-as-code, and the knowledge I accumulated from this project. The hope is that this POC will serve as a framework for the infosec community to use to perform log enrichment. Lastly, I will demonstrate the power of this POC by detecting a Powershell Empire agent that has been injected into explorer.exe.

Continue reading →

Leave a comment

Incident Response, Logging, Tools

Generating CommunityIDs with Sysmon and Winlogbeat

While working on another logging project, I discovered a mechanism to generate CommunityIDs with Sysmon and Winlogbeat. Winlogbeat provides a feature called processors which can enrich log events before they are sent to the SIEM/logging server. This blog post will demonstrate a proof-of-concept (PoC) to enrich Sysmon network logs with a Community ID Network Flow Hash.

Continue reading →

Leave a comment

DevOps, Incident Response, Logging, Network Security, PoC, System Administration, Threat Detection Engineering, Tools

Reducing your alert fatigue with AskJeevesSecBot

In incident response, there is a disconnect between a security alert being generated and a user’s confirmation of the security alert. For example, generating an alert every time a user runs “curl” on a production system would generate a bunch of false positives that can lead to what is called “alert fatigue”. But if we extend our incident response capabilities to include the user as part of the triage process we could reduce the number of alerts. This blog post is going to demonstrate AskJeevesSecBot which is an open-source proof of concept (POC) of how to integrate Slack and user responses into your security pipeline, specifically during the triage phase of the incident response process. In addition to a PoC, this blog post will also provide a deep dive into the architecture of this project, design decisions, and lessons learned as an evolving threat detection engineer.

Continue reading →

Incident Response, Logging, PoC, System Administration, Tools

My logging pipeline: Splunk, Logstash, and Kafka

Over the years I have built several logging pipelines within my homelab and each used different technologies and methodologies but now I have finally built a pipeline that suites my needs. When I tell people about my pipeline they usually ask if I have a blog post on it because they want to know more or replicate it. This blog post is my attempt to share my logging pipeline as a framework for newcomers. The hope is that the explanation of the architecture, design decisions, working infrastructure-as-code, and the knowledge I accumulated over the years will be beneficial to the community. Continue reading →

Incident Response, Logging, Network Security, PoC, Threat Detection Engineering, Tools

Creating my first Osquery extension to generate CommunityIDs with Osquery-python on Windows

Osquery is my favorite open-source security tool and Python is my favorite programming language so fusing them together allows us to engineer tools to detect threats. This blog post will build an Osquery-python extension to calculate the CommunityID of a network connection utilizing the Osquery-polylogyx extension pack to monitor network connections. In blog posts to follow, we will correlate network-based events generated by Zeek and host-based events generated by Osquery using the CommunityID. So follow me as your adventure guide on this development journey to make an Osquery extension with osquery-python.

Continue reading →

Leave a comment

Incident Response, Logging, Network Security, PoC, Tools

PoC: Monitoring user browser activity with Osquery

This proof-of-concept (PoC) will demonstrate how to use Osquery to monitor the browser activity of users. Not only will this PoC collect browser activity, but it will also use VirusTotal to rank each URL to detect malicious activity. In addition to VirusTotal, this PoC will utilize Rsyslog, Osquery, Kafka, Splunk, Virustotal, Python3, and Docker as a logging pipeline. Once this pipeline has been implemented, your security team will have the ability to protect your user’s from today’s most serious threats on the web.

Continue reading →

Incident Response, Logging, Tools

Back in the saddle: Install/Setup Elastic stack 7.0 on Ubuntu 18.04

Wow, the last time I really used the Elastic Stack it was called the ELK stack, and it was version 2.0. A lot of things have changed since then, so I am going to do an updated post on installing and setting up the Elastic stack.

Continue reading →

Leave a comment

Incident Response, Logging, PoC, Tools

Detecting malicious downloads with Osquery, Rsyslog, Kafka, Python3, and VirusTotal

This blog post will explore how to set up a simple logging pipeline to detect maliciously downloaded files. This setup will utilize technologies such as Osquery, Rsyslog, Kafka, Docker, Python3, and VirusTotal for a logging pipeline. If this pipeline detects a malicious file, a Slack alert will be triggered.

Continue reading →

Logging, Network Security, PoC, Tools

Detecting SSH brute forcing with Zeek

In this blog post, we will explore how Zeek detects SSH brute forcing. We will explore the SSH handshake to understand how it works. Next, I will demonstrate several test cases of Zeek detecting SSH brute forcing. Finally, this post will lay down the foundation to implement active defense controls with Zeek in future posts.

Continue reading →

Logging, Network Security, Tools

Part 1: Install/Setup Zeek + pf_ring on Ubuntu 18.04 on Proxmox 5.3 + openVswitch

Monitoring your home network can be challenging without enterprise-grade equipment. Although monitoring your home network can prove to be difficult, Proxmox and Zeek provide the perfect solution to monitor your home network. This blog post will cover how to setup Zeek+PF_Ring to monitor network traffic on Proxmox.

Continue reading →

Leave a comment

Logging, Tools

Logging OSquery with Rsyslog v8 – Love at first sight

This blog post is going to cover how to ingest OSquery logs with Rsyslog v8. Most setups I have come across have Rsyslog ingesting the logs from disk, but this setup will ingest logs via the system journal. OSquery supports writing logs to disk and to the system journal. This post also contains a setup via Ansible and a manual walkthrough. Lastly, explanations of Rsyslog and OSquery configs.

Continue reading →