Incident Response | HoldMyBeer

Category Archives: Incident Response

Leave a comment

Incident Response, Logging, Network Security, Tools, WebSec

Making Damn Vulnerable Web Application (DVWA) almost unhackable with Cilium and Tetragon

In this universe of ever-changing landscapes and unlimited hackers, defenders are searching for a hero to help defend the Kubernetes universe. To paraphrase Nick Fury, we need to assemble a group of remarkable tools who can work together to fight battles the vulnerable apps can’t. In this blog post, we are going to assemble our own Avengers team using Cilium and Tetragon to defend the Damn Vulnerable Web Application (DVWA) against the unearthly invaders, rendering it almost unhackable. Tetragon + Cilium will provide process, file, HTTP, and network-based defenses to thwart the known evil OWASP’s top 10. DVWA is a web app that was intentionally designed to be vulnerable to OWASP’s top 10 as a training resource. Lastly, I will end with a director’s commentary on my opinions to integrate Cilium + Tetragon in an enterprise and some ideas to close the gap between developers and security.

Continue reading →

Leave a comment

ForFunAndWumbos, Incident Response, Logging, Threat Hunting, Tools

Part 3: Intro to threat hunting – Hunting the imposter among us with the Elastic stack and Sysmon

This blog post series is for anyone who has ever had an interest in threat hunting but did not have the knowledge of how or where to start, what tools they need, or what to hunt for. In this blog post, I will introduce an informal threat hunting process by hunting the APT-style attack performed during the red team exercise in the previous blog post. The theme of this blog post is to demonstrate how to hunt and detect malicious activity at each stage of the Mandiant Attack Lifecycle to create a fundamental framework for hunting adversaries. This blog post is a written adaptation of my DefCon 2020 Blue Team village workshop. It will utilize the same ideas and techniques used for that workshop reiterating specifics and points for the greater InfoSec community to use.

In this blog series, we have a fictitious advanced persistent threat (APT) code-named Goofball. They have been known to steal intellectual property and the Hackinglab corporation just released a press statement about a new widget that will revolutionize the world. This blog post is going to embark on a quest to hunt for the existence of Goofball in the Hackinglab corporation network. Additionally, this quest will introduce you to an informal threat hunting process to demonstrate the tools and techniques using Sysmon and the Elastic stack. The hope is that this informal process demonstrates how to apply a threat hunting mindset to search for malicious activity in your environment but also understand your findings to investigate further.

Continue reading →

Tagged AttackLifecycle, PSEmpire, ThreatHunting

DevOps, Incident Response, Memory Forensics, PoC, Tools

Getting started with Autopsy multi-user cluster

The purpose of this blog post is to provide multiple methods on how to install/setup an Autopsy multi-client cluster. This blog post generated an infrastructure-as-code in the form of an Ansible playbook, Docker-compose, and manual instructions for setting up a cluster. In addition, this blog post will demonstrate how to setup the Autopsy client to connect to the Autopsy cluster and how to ingest disk images.

Continue reading →

Leave a comment

Incident Response, IR Tales, Logging, System Administration, Tools

IR Tales: The Quest for the Holy SIEM: Splunk + Sysmon + Osquery + Zeek

This blog post is the season finale in a series to demonstrate how to install and setup common SIEM platforms. The ultimate goal of each blog post is to empower the reader to choose their own adventure by selecting the best SIEM based on their goals or requirements. Each blog post in the series will provide Docker-compose v2, Docker-compose for Swarm, Ansible, Vagrant, and manual instructions to allow the reader to setup each platform with the deployment method of their choosing. In addition to setting up Splunk, I will cover fundamental Splunk concepts such as the Common Information Model (CIM). Lastly, I will provide step-by-step instructions to install Sysmon + Splunk Universal Forwarder on Windows, Osquery + Splunk Universal Forwarder on Ubuntu, and Zeek + Filebeat to ship logs to Splunk.

Continue reading →

Incident Response, Logging, System Administration, Tools

Getting started with FleetDM

The purpose of this blog post is to provide multiple methods on how to install/setup FleetDM, how to deploy Osquery, and demonstrate how to use features of FleetDM + FleetCTL. This blog post generated an Ansible playbook, Docker-composes for Swarm and non-swarm, Vagrant to create a VM, and manual instructions for installing FleetDM on Ubuntu 20.04. Additionally, there are Ansible playbooks for deploying the Osquery agent on Windows and Ubuntu with manual instructions as well. Lastly, I will end by demonstrating how to use the FleetDM WebGUI and FleetCTL tool to manage FleetDM and interact with your Osquery agents.

Continue reading →

Leave a comment

Incident Response, Logging, PoC, Threat Detection Engineering, Tools

Create a custom Splunk search commands with Python3

This blog post will demonstrate how to create a custom Python search command for Splunk and will demystify common roadblocks such as: how to create a custom search command with Python, how to store secrets for a custom search command, and how to install external Python libraries. With each roadblock discussed we will also cover the solution as code examples and hands-on exercises. To do this, we must first start with an introduction to the architecture of a custom Python search command.

Continue reading →

Leave a comment

DevOps, Incident Response, PoC, Threat Detection Engineering, Tools

Integrating Vault secrets into Jupyter Notebooks for Incident Response and Threat Hunting

The industry has gravitated towards using Jupyter notebooks for automating incident response and threat hunting. However, one of the biggest barriers for any application/automation is the ability to store secrets (username+passwords, API keys, etc) to access other services. This blog post will demonstrate how to use Vault to store secrets and integrate the ability to retrieve secrets from Vault with Jupyter Notebooks to assist in automating your security operations.

Continue reading →

Leave a comment

Incident Response, Logging, Threat Detection Engineering, Tools

Demystifying the Kolide Fleet API with CURL, Python, Fleetctl, and Ansible

A common question in the #Kolide channel in the Osquery Slack is how to use the Kolide Fleet API. Kolide Fleet is written in GoLang and utilizes the GoKit framework to build the application. Therefore, almost every action that can be performed via the WebGUI is an API call. This blog post is going to demonstrate how to use the Kolide Fleet API to perform actions such as creating a live query and obtaining the results using Python websockets, obtaining the Osquery enroll secret, and creating a saved query. In addition to the API, this blog post will demonstrate how to use the Fleetctl command-line tool to perform the same actions. Lastly, this blog post includes an Ansible playbook to automate deploying Oquery agents and registering them with Kolide.

Continue reading →

Leave a comment

Incident Response, PoC, Threat Detection Engineering, Tools

Setup my GoLang Osquery-file-carving server with Kolide

Facebook released an awesome open-source tool named Osquery that is being maintained by a thriving community supported by the Linux Foundation and several product leaders such as Kolide, TrailOfBits, and Uptycs. However, Facebook did not release the server component of Osquery and that has led to the creation of many projects: Kolide, Uptycs, Doorman, OSCRTL, and SGT just to name a few. Furthermore, not all projects have the ability to support the Osquery file carve functionality, more specifically the open-source version of Kolide Fleet. This project set out on a mission to provide an open-source Osquery file carving server for file uploads and downloads that could be used with Kolide.

This blog will provide a deep dive into the architecture of this project, design decisions, and lessons learned as an evolving incident response engineer. This project has been a 6-month long effort that resulted in the creation of 4 blog posts, 3 Udemy certificates/courses, and 3 separate Github repos. The collection of these experiences and research has led to the creation of this project. My hope is that this project benefits the community and provides an additional capability to Osquery that may not be supported by all fleet managers.

Continue reading →

Incident Response, Network Security, Tools

Compile Suricata v5.0.3 with PF_RING v7.6.0 on Ubuntu 20.04

While working on my TOR relay project I was trying to compile Suricata with pf_ring but couldn’t find any documentation for the latest releases. This blog post will provide instructions to compile the latest stable version of Suricata and pf_ring. In addition, this blog post contains instructions for the infrastructure-as-code written in Ansible to automate the installation and step-by-step instructions for a manual installation.

Continue reading →

Leave a comment

Incident Response, Logging, PoC, System Administration, Tools

Setting up Kolide and Osquery with client certificates for mutual TLS (mTLS)

Do you know if your Osquery client is connecting to the right server? Do you know if your Kolide server is accepting requests from rogue devices? If you have answered “I don’t know” to either of these questions then this blog post is for you. This blog post will be demonstrating how to setup Kolide + Osquery with mutual TLS (mTLS). Mutual TLS is a mechanism that can be implemented to verify the identity of the server and clients. Continue reading →

Incident Response, Logging, Network Security, PoC, Tools

PoC: Using KSQL to enrich Zeek logs with Osquery and Sysmon data

In incident response, time is precious and something you can never get back. Typically, when I receive a security alert about an endpoint, it requires manual lookups on multiple data sources for critical pieces of information. These manual lookups can be time-consuming, create fatigue, and don’t use the power of technology to your advantage. This blog post will demonstrate a proof-of-concept (POC) by using the power of a network community ID hash by Corelight to fuse endpoint and network-based data sources.

KSQL by Confluent provides the ability to enrich independent data sources by correlating common attributes. In this POC, we are going to use Sysmon or Osquery to monitor the endpoint and Zeek to monitor the network. Not only will this blog post serve as a POC but it will discuss the architecture, design decisions, working infrastructure-as-code, and the knowledge I accumulated from this project. The hope is that this POC will serve as a framework for the infosec community to use to perform log enrichment. Lastly, I will demonstrate the power of this POC by detecting a Powershell Empire agent that has been injected into explorer.exe.

Continue reading →

Leave a comment

Incident Response, Logging, Tools

Generating CommunityIDs with Sysmon and Winlogbeat

While working on another logging project, I discovered a mechanism to generate CommunityIDs with Sysmon and Winlogbeat. Winlogbeat provides a feature called processors which can enrich log events before they are sent to the SIEM/logging server. This blog post will demonstrate a proof-of-concept (PoC) to enrich Sysmon network logs with a Community ID Network Flow Hash.

Continue reading →

Leave a comment

DevOps, Incident Response, Logging, Network Security, PoC, System Administration, Threat Detection Engineering, Tools

Reducing your alert fatigue with AskJeevesSecBot

In incident response, there is a disconnect between a security alert being generated and a user’s confirmation of the security alert. For example, generating an alert every time a user runs “curl” on a production system would generate a bunch of false positives that can lead to what is called “alert fatigue”. But if we extend our incident response capabilities to include the user as part of the triage process we could reduce the number of alerts. This blog post is going to demonstrate AskJeevesSecBot which is an open-source proof of concept (POC) of how to integrate Slack and user responses into your security pipeline, specifically during the triage phase of the incident response process. In addition to a PoC, this blog post will also provide a deep dive into the architecture of this project, design decisions, and lessons learned as an evolving threat detection engineer.

Continue reading →

Incident Response, Logging, PoC, System Administration, Tools

My logging pipeline: Splunk, Logstash, and Kafka

Over the years I have built several logging pipelines within my homelab and each used different technologies and methodologies but now I have finally built a pipeline that suites my needs. When I tell people about my pipeline they usually ask if I have a blog post on it because they want to know more or replicate it. This blog post is my attempt to share my logging pipeline as a framework for newcomers. The hope is that the explanation of the architecture, design decisions, working infrastructure-as-code, and the knowledge I accumulated over the years will be beneficial to the community. Continue reading →

Incident Response, Memory Forensics, PoC, Threat Detection Engineering, Tools

Adventures of the Sherlock Holmes Memory Gopher: Dumping and analyzing memory with Osquery and Kolide

For several years I have always wanted to write an Osquery extension to perform memory dumps and analysis. I never got the time to do a deep dive into my idea but since I have been creating some Osquery-go extensions lately, I decided to take a crack at my idea. This blog post will provide a high overview of the architecture of these Osquery extensions for this project, how to generate memory dumps, and how to remotely analyze these memory dumps with Osquery. Follow me with another threat detection engineering experience with Osquery-go.

Continue reading →

Leave a comment

Incident Response, PoC, Threat Detection Engineering, Tools Image

Operation cleanup: Eradicating malware with Osquery and Kolide

This blog post is going to cover an Osquery extension that I engineered with osquery-go to eradicate malware. This extension has the ability to delete files, kill processes, delete directories, and can be used with the builtin YARA table. This blog post will act as documentation for the setup and operation of this Osquery extension.

Continue reading →

Leave a comment

Incident Response, macOS, Network Security, PoC, Threat Detection Engineering, Tools

Creating my second Osquery extension with osquery-go

Here we go again! This blog post is tangential to my previous blog post on creating an Osquery extension with Python but this time we are using golang. Osquery is my favorite open-source security tool and golang is becoming a popular programming language so fusing them together allows us to engineer tools to detect threats. This blog post will build an Osquery-go extension to calculate the CommunityID of a network connection utilizing the Osquery-polylogyx extension pack to monitor network connections. In blog posts to follow, we will correlate network-based events monitored by Zeek and host-based events generated by Osquery using the CommunityID. So follow me again as your adventure guide on this development journey to make an Osquery extension with osquery-go.

Continue reading →

Incident Response, Logging, Network Security, PoC, Threat Detection Engineering, Tools

Creating my first Osquery extension to generate CommunityIDs with Osquery-python on Windows

Osquery is my favorite open-source security tool and Python is my favorite programming language so fusing them together allows us to engineer tools to detect threats. This blog post will build an Osquery-python extension to calculate the CommunityID of a network connection utilizing the Osquery-polylogyx extension pack to monitor network connections. In blog posts to follow, we will correlate network-based events generated by Zeek and host-based events generated by Osquery using the CommunityID. So follow me as your adventure guide on this development journey to make an Osquery extension with osquery-python.

Continue reading →

Incident Response, Threat Intelligence, Tools

Install/Setup MISP on Ubuntu 18.04 with an intro to PyMISP

In this blog post, we are going to cover how to install MISP on Ubuntu 18.04. Once MISP is installed, we will do an introduction to the PyMISP API to store indicators of compromise (IOCs) in MISP and query IOCs from MISP. This blog post will serve as the foundation for future blog posts moving forward.

Continue reading →