Jun 02 2021
Leave a comment
By spartan2194 ForFunAndWumbos, Incident Response, Logging, Threat Hunting, Tools

Part 3: Intro to threat hunting – Hunting the imposter among us with the Elastic stack and Sysmon

This blog post series is for anyone who has ever had an interest in threat hunting but did not have the knowledge of how or where to start, what tools they need, or what to hunt for. In this blog post, I will introduce an informal threat hunting process by hunting the APT-style attack performed during the red team exercise in the previous blog post. The theme of this blog post is to demonstrate how to hunt and detect malicious activity at each stage of the Mandiant Attack Lifecycle to create a fundamental framework for hunting adversaries. This blog post is a written adaptation of my DefCon 2020 Blue Team village workshop. It will utilize the same ideas and techniques used for that workshop reiterating specifics and points for the greater InfoSec community to use.

In this blog series, we have a fictitious advanced persistent threat (APT) code-named Goofball. They have been known to steal intellectual property and the Hackinglab corporation just released a press statement about a new widget that will revolutionize the world. This blog post is going to embark on a quest to hunt for the existence of Goofball in the Hackinglab corporation network. Additionally, this quest will introduce you to an informal threat hunting process to demonstrate the tools and techniques using Sysmon and the Elastic stack. The hope is that this informal process demonstrates how to apply a threat hunting mindset to search for malicious activity in your environment but also understand your findings to investigate further. 

DISCLAIMERS

The information contained in this blog post is for educational purposes ONLY! HoldMyBeerSecurity.com/HoldMyBeer.xyz and its authors DO NOT hold any responsibility for any misuse or damage of the information provided in blog posts, discussions, activities, or exercises. 

The red team exercise performed in the previous blog posts occurred in a home lab network. Therefore, the malicious activity performed was on machines that I own and have permission to perform these actions.

DISCLAIMERS

Blog post series

Goals

  • Understand the mindset of a threat hunter
  • Understand the phases of the Mandiant Attack Lifecycle from a threat hunters perspective
  • Hunt the artifacts left behind by the red team exercise and recognize their importance
  • Connecting evidence discovered at one phase to possible activities in past or future phases.
  • Develop an informal threat hunting process and methodology to hunt for adversaries in your own environment
    • Learn hunting techniques to detect the artifacts left behind by the red team exercise
    • Use the Elastic stack and Sysmon v12 to detect the artifacts left behind by the red team exercise

Background

What is Sysmon?

System Monitor (Sysmon) is a Windows system service and device driver that, once installed on a system, remains resident across system reboots to monitor and log system activity to the Windows event log. It provides detailed information about process creations, network connections, and changes to file creation time. By collecting the events it generates using Windows Event Collection or SIEM agents and subsequently analyzing them, you can identify malicious or anomalous activity and understand how intruders and malware operate on your network.

Important Sysmon v12 event codes for this blog post

Fictitious scenario

The fictitious scenario is that you’re a threat hunter who has just received an intel report on APT with a code name of “GoofBall”. This adversary has been known to steal intellectual property to produce products at a cheaper rate. In addition, this group is known to be highly skilled in Windows environments and their custom tooling evades security controls. Recently your organization HackingLab has announced a fancy new widget that the Wall Street Journal reports is worth billions. To ensure that you and everyone else at your organization have a job within the next 6 months you decide to hunt your environment for any sign of Goofball.

This blog post will simulate what it’s like to hunt an advanced persistent threat (APT) style attack within a Windows environment by hunting the red team exercise from the previous post. In this fictitious scenario, we will pretend that Powershell Empire is a “next-gen APT malware” that is undetectable. In the sections to follow, we will leverage the telemetry collected by Sysmon v12 agents to hunt for the malicious activity performed by our fictitious attacker.

My approach

The previous blog post performed an APT style attack against a Windows environment following the phases of the Mandiant Attack Lifecycle. The reason I am using the Attack Lifecycle is to demonstrate how to hunt for malicious activity at each stage of an attack’s lifecycle and NOT focus on hunting specific TTPs. This model will help you create mental maps of attacker activity.

For example, if you detect Mimikatz it usually indicates the following three things: One the attacker has gained a foothold within the network. Two the attacker was able to escalate privileges to run Mimikatz (which means they have reached the privilege escalation phase of the attack lifecycle). Three most likely they are using Mimikatz to obtain credentials to attempt to move laterally in the environment. At this point, the detection of Mimikatz can give you (the defender) an idea of where the attacker is in the lifecycle and what they might do next. Again, this isn’t always true but I am trying to paint a picture.

In this post you will walk through each stage of the attack lifestyle using the following format:

  • Phase Name
    • Abstract concept of each phase
    • Understand common techniques used in each phase
    • A simple explanation of each technique
    • Queries to detect techniques discussed
    • Explanation of data discovered
    • Using data from previous phases to hunt for activity in the current phase
    • Additional/alternative hunt queries/things to investigate further – won’t be covered here

The hope is that by walking through each phase from a defenders perspective that you will take away the following:

  • Understanding the phases of the Attack Lifecycle
    • High-level understanding of the purpose of each phase
    • Understand the tools and techniques used by adversaries at each phase
  • Understanding what an adversary has accomplished and what they may do next
    • How to leverage data from previous phases to hunt for malicious activity in later phases
  • How to craft Elastic queries to query Sysmon v12 data for malicious activity
    • Understanding the artifacts generated by attacker activity
    • Understand the artifacts/logs generated from malicious activity

Lastly, as the exercise progresses it may seem that there is an incident response undertone. However, if you analyze each query by itself or in tandem, each query is demonstrating not only how to hunt for malicious activity left behind by GoofBall but also how to identify malicious activity and how to use your findings for additional hunts.

Phase 1: Initial recon

As discussed in the “Fictitious scenario” section above it is NOT at random that a nation-state would target the HackingLab corporation. This brings up an important concept that is not typically discussed when discussing the Attack Lifecycle, which is the “pre-recon” phase. This phase is the actions of an adversary or red team to observe a set of targets. In this case, the nation-state is monitoring multiple targets for the announcement of a new product. If a target announces a new product that the nation-state would like to produce, it will start the planning stages of a targeted attack. For this threat hunting exercise, there are no logs for us to hunt or analyze pertaining to this phase. In the next Attack Lifecycle phase, we will speculate on the actions performed by the attackers based on the initial compromise vector.

Phase 2: Initial compromise

Initial compromise is when the attacker attempts to execute malicious code on one or more systems. One way that initial compromise can be accomplished is through a social engineering technique called phishing. Phishing techniques include: sending a malicious document to a target user(s) or luring a target user(s) to a malicious website. Alternative methods of initial compromise include: exploiting a vulnerability or utilizing default credentials on publicly facing services.

Weaponization

As I stated in the previous blog post in this series, weaponization is not an actual phase of the Attack Lifecycle but it is an important concept that should be recognized by defenders. The Attack Lifecycle is designed from the perspective of the defender and the actions performed by an adversary during this “phase” can NOT be detected. It’s not until the attacker attempts to perform the initial compromise that the defender has something to detect. Before we start hunting the initial compromise vector, I am going to review the generation and execution of the malicious document. This review will provide us the necessary details to set the stage for this threat hunting exercise and it will allow us to start hunting.

In the red team exercise in the previous blog post, a malicious document was generated, sent to [email protected], and opened by our fictitious user. Once the security warning was accepted by the user, the malicious macro within the document was executed, which makes a Windows call to WMI (WmiPrvSE.exe) to spawn a Powershell instance (Powershell.exe). This Powershell instance executes the Powershell Empire stager packaged in the malicious macro. The Powershell Empire stager executes and calls back to the Powershell Empire server to download the Powershell Empire agent.

Now that we have an understanding of the initial compromise vector, we can now speculate on the actions taken by the adversary to perform initial recon. We know the attacker created an Office 365 Word document so they were able to determine that the HackingLab organization uses Microsoft Office 365. This could have been accomplished by Google Dorking using the following query: hackinglab.beer filetype: [pdf, docx, doc, pptp, xlsx]. We know the document was not blasted to all employees but to a single employee who was [email protected]. We can assume the attacker was able to use open-source intelligence (OSINT) to find the user’s e-mail on a platform like LinkedIn.

Our first hunting query

This blog post assumes you have the Elastic stack or a SIEM of your choice already setup, if you do not, take a look at the first post in this series: Part1: Intro to Threat Hunting: Setting up the Environment. Additionally, the logs needed for this blog post are linked at the bottom as bg-sysmon and bg-wineventlogs. This blog post is going to assume you are using the Elastic stack as your SIEM of choice. If you are using a different SIEM the queries in this blog post can be converted to your SIEM of choice query language.

  1. Open up a web browser
  2. Browser to https://<Docker/ELK IP addr>:8443
  3. Select “Discover” on the left
  4. Select the bg-sysmon index
  5. Set the time windows to “1 year”
  6. Enter the following query into the search bar: process.parent.name: wmiprvse.exe AND process.name: powershell.exe

Breaking down your findings

Preceding each query in the Attack Lifecycle phases to follow there will be an explanation of the finding and its discrete components. Each explanation will be color-coded to correlate the explanation text and the text within the screenshot. Lastly, a “breaking down your findings” section may not always be included in each phase but if it is, its purpose is to summarize the importance of all your findings.

Based on the findings from the query above, I have hand-picked several key-value pairs that are important to set the stage for our threat hunting exercise. I am going to cover each key-value pairing in the screenshot above and their importance and keep in mind the colored text below will match the screenshot. First, is the timestamp key-value pair, which is very important because in this instance it defines the start of the timeline for our investigation/hunt. Next, it’s always important to understand the software that generated this event log which in our case is Sysmon. The logging source is an important attribute for later when transitioning your findings from a successful hunt into a detection. 

Sysmon records all new processes created on a machine and assigns these events Event code:1 , which is an important piece of information to use when writing detections. The hostname for this machine is important because it can be used for further hunting on this infected machine. The username for the account that was compromised in order to have a discussion with the user and possibly see if their account has been used anywhere else in the environment. For example, is the user accessing their Office 365 account from a new IP address, is the user account accessing internal resources, and etc?  As a threat hunter, you should recognize that the process ID can be used to track all activity performed by this malicious process, which we will do in the next Attack Lifecycle phase. 

Next, understanding parent and child process relationships in Windows is SUPER important. It’s not inherently malicious if WMI (WmiPrvSE.exe) spawns a Powershell instance but it is a technique used by adversaries. Lastly, the process command line confirms that this Powershell instance spawned by WMI is malicious. The command line string (powershell -noP -sta -w 1 -enc *) follows a common format used by red team frameworks like Powershell Empire.

Additional hunt queries based on findings

The purpose of this section is to help you understand how to use your findings and develop new hunts to search alternative data sources or new search parameters to use as indicators of compromise (IOCs). These IOCs can be used to hunt for the adversary in the environment and can be used as early warning detections.

Lastly, if this blog post covered all the possible avenues for hunting pertaining to the red team exercise, I would be able to write a small book. However, that is not effective because the goal of this post is not to provide all the queries to hunt an adversary but to help you build an understanding of how to hunt. A more common way to phrase it is “Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime”.

  • Hunt query 1: powershell -noP -sta -w 1 -enc * – Detect other instances of Powershell Empire running on other machines in the environment
  • Hunt query 2: event.code: 1 AND parent.process: wmiprvse.exe AND process.name: powershell.exe – This query will not only find when Microsoft Office Word uses WMI to create Powershell instances but when applications like Adobe PDF reader use this same technique
  • Hunt query 3: event.code: 11 AND file.name: "Invoice - July 2020.docm" – Sysmon event code 11 tracks file creation. We know the document name so we can search our environment for any other users that have this document. The hunt doesn’t stop there you could pivot to the e-mail server to perform a search via the filename for all users who received this document. Furthermore, we could go ahead and block the sender and again pivot on the sender to see all e-mails received.
  • Hunt query 4: event.code: 11 AND "docm" – Sysmon event code 11 tracks file creation. We know documents with the file extension of “.docm” are macro-enabled. You could hunt for all macro-enabled documents in your environment.

Phase 3: Establish foothold

To ensure the attacker maintains access to the network, they will implant persistent mechanisms on the infected machine(s). Some of the most common persistent mechanisms may take the form of Scheduled Tasks, adding a local user to the system, or adding an AutoRun to the user’s Registry, more techniques can be found here. It is important to understand that the Powershell Empire agent is running in memory. If the infected machine is rebooted, the attacker loses that access because the contents of the memory are wiped on reboot. Persistent mechanisms allow attackers to maintain access through a reboot, user logging out, loss of network connectivity, process dying unexpectedly, or interactions by the security team.

Following the bread crumb trail

In the previous Attack Lifecycle phase, we discovered a Powershell Empire agent running on a machine with a Process ID (PID) of 3,965. Sysmon records all the activity that occurs on a machine such as all the actions performed by a specific process. The query below is searching for any new processes that were created by PID 3965 or the process itself performing any activity.  The screenshot below shows all the malicious actions performed by the malicious Powershell Empire process (the chronological order of these events is from the bottom-up). First, the Powershell Empire stager packed in the malicious Word document is executed, the Powershell Empire stager makes a DNS request for malwarelove.xyz, creates a Windows Registry value, and creates a Windows Scheduled Task. In the section below, we will break down and analyze each of these actions. 

  • Query: (process.pid: 3956 OR process.parent.pid:3956) AND NOT event.code:3

Scheduled task

As we discussed above, the Powershell Empire agent utilized the schtasks.exe binary to create a Scheduled Task. Unfortunately, Sysmon does not have an event code for newly created Scheduled Tasks. However, it does track process execution like when the Scheduled Task binary was used via the command line or when new files are created. The first query below is searching for when Powershell (process.parent.name: powershell.exe) calls the Scheduled Task binary via the command line and attempts to create (/Create) a Scheduled Task.

The screenshot below for the first query has the command string that will be executed when this Scheduled task runs. More specifically, the Scheduled Task will create a Powershell instance, read the contents from the following Registry key HKCU:\Software\Microsoft\Windows\CurrentVersion\debug, base64 decode the contents from the Registry key, and execute it. The second query below is searching for the creation of new files at the following path which is used to store Scheduled Tasks: C:\\Windows\\System32\\Tasks. As you can see both queries result in the same finding but you can use two different methods to get the same result.

  • Query 1: event.code: 1 AND process.parent.name: powershell.exe AND process.name: schtasks.exe AND process.args: "/Create"
  • Query 2: event.code:11 AND "C:\\Windows\\System32\\Tasks"

Windows logon item

Windows Logon is another common technique used by adversaries to maintain persistence. A Windows Logon item will execute every time the user logs in. At the beginning of this Attack Lifecycle phase, we observed Powershell creating a Registry key at the following path HKCU:\Software\Microsoft\Windows\CurrentVersion\Run\*, which is the Registry path for Windows Logon items. The Registry Key Updater (highlighted in yellow) contains malicious Powershell code. The malicious Powershell code is executing the contents of the same Registry Key used in the Scheduled Task above.

  • Query: event.code:13 AND "Software\\Microsoft\\Windows\\CurrentVersion\\Run"

New user account

Creating a new user account is a common technique used by adversaries to establish persistence. Unfortunately, Sysmon doesn’t have an event code to track the creation of new user accounts. However, Sysmon tracks modifications to the Registry and when new accounts are created they are added to the following path in the Registry: "HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\ProfileList".

If you read the previous blog post you know the red team exercise did not create a user account. So you might be wondering why are we hunting for new user accounts?!?!?! The purpose of this section is to demonstrate how to disprove a hunting hypothesis with data. Therefore, our hypothesis is the attackers created a local user account on the initially compromised machine as a method of persistence. Now keep in mind, you must be careful when you say you have disproved something. More accurately you want to say based on the data available to me at the current moment in the Sysmon index and the specified time range there is no evidence of any new accounts being created.

It’s important to use this CYA (Cover Your Ass) language because what if the attacker disabled logging or a firewall rule is preventing a client from sending logs? So yes, from the perspective of the SIEM no account was created but in the realm of reality, it may not be true. It’s also important to understand that threat hunting is based on data therefore if you have bad or missing data you will likely have incomplete, impartial, or misinformed results, which are used to inform decisions.

  • Query: event.code: 12 AND registry.path: "HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\ProfileList"

Breaking down your findings

  • Scheduled Task
    • Name of the scheduled task: Updater
    • Interval/condition for the task to execute: Every day at 9 am
    • The task will run with admin privileges
    • Task runs in the background (-w hidden) or (-w 1)
    • Registry path to payload: HKCU:\Software\Microsoft\Windows\CurrentVersion\debug
    • In file created in the following directory: C:\\Windows\\System32\\Tasks
  • Registry Winlogon
    • Registry path to payload: HKCU:\Software\Microsoft\Windows\CurrentVersion\debug
    • Logon item name: Updater
  • Powershell Empire process
    • The process ID of Powershell instance: 3956
    • The process is set to run in the background (-w 1)

ALL OF THESE FINDINGS CAN BE USED FOR NEW HUNTS

Additional hunt queries based on findings

  • Hunt query 1: event.code: 1 AND process.parent.name: powershell.exe AND process.name: schtasks.exe AND ( process.command_line: "-W hidden" OR process.command_line: "-W 1") – Find all Scheduled Tasks that execute Powershell in the background
  • Hunt query 2: event.code:11 AND file.path: "C:\\Windows\\System32\\Tasks\\Updater" – Look for other hosts that have this Scheduled Task by name.
  • Hunt query 3: event.code: 12 AND registry.key: "Software\\Microsoft\\Windows\\CurrentVersion\\Debug" – The existence of the Registry Key used by the Scheduled Task and Winlogon event

Phase 4: Privilege escalation

At this stage, the attacker attempts to escalate their privileges from a normal user context to an administrator context. This is typically accomplished with a local operating system vulnerability to escalate privileges on the local machine. Unfortunately, our logging did not log anything that could be used to detect privilege escalation at this stage. However, later in this blog post, we will find evidence of privilege escalation. I would like to take this opportunity and remind you that even though we don’t have any actionable findings in our SIEM this is still a great finding to document. This finding can be used to improve your security controls or to document a known vulnerability in your environment.

Phase 5: Internal recon

At this stage, the attacker explores the network environment to gain an understanding of how users interact with the network. Ideally, attackers want to discover network services such as storage shares with documents or an internal server used to store documentation about the network (internal wiki). Gaining access to the internal knowledge base tools (Sharepoint, Confluence, wiki, etc) can be a GOLD MINE of information.

For example, let’s say an attacker was trying to dump a database that stores credit cards. The internal wiki would include documentation on these databases such as the network subnet (location), how to access them (VPN, jump box, VLAN, configuration management, etc), type of database (Postgres, MySQL, Mongo, etc), administrators (future targets) who have access to the servers, or backups of the database that may not be as protected.

An attacker may also sniff the network traffic to observe DNS queries, to locate where the DNS server is to request a zone transfer, learn how machines are remotely managed (PsExec, Chef, Ansible), and what internal services users interact with. The fundamental goals are to understand the network layout, locate information of interest, and see how users typically interact with the network and services.

The red team exercise utilized a broad range of Powershell Empire situational_awareness modules to perform reconnaissance on the hackinglab.beer network. Each module returns results about hosts discovered, services being utilized, information about the environment, services that can be used for lateral movement, SMB shares, and the location of users. The first situational_awareness module that was used during the red team exercise was an ARP scan on the local network to discover all hosts on the same subnet. Second, information about the domain was collected such as the Windows server version, the forest domain, the IP address of the domain controller, the FQDN of the domain controller, and the LDAP schema for this domain. Third, recursive DNS queries were performed on the domain controller to get a list of machine hostnames within the domain. In fact, each hostname returned has a correlating IP address so the attacker knows the network location of each machine.

Once a list of machines to target has been compiled from host discovery, a more targeted style of reconnaissance can occur. A targeted reconnaissance will use techniques such as port scanning, SMB share enumeration, and the query command to locate Sheldon Jobs’s workstation. First, the attacker may start by running a port scan to discover open ports on the network followed by a service enumeration scan. These port scans can provide a lot of information about the environment. The service enumeration will allow us to identify web services, file shares, and potential services the attacker Goofball could use for lateral movement. Lastly, the query command allows the attacker to identify the workstation that is used by the attacker’s target so they can move laterally to it.

ARP scan

Sysmon only monitors traffic at layer 3 of the OSI model and ARP requests are layer 2. Additionally, I did a search for any commands using the ARP command and found none.

Get-Domain controller

Was not unable to find any definitive logs for this activity

Port scanning

Sysmon tracks network connections (event ID: 3) based on predefined conditions. Since we know internal recon will target our local network of 10.150.100.0/24 we can search the Sysmon index for our infected machine (10.150.100.135) scanning the local network. The screenshot below shows network connections from the infected machine (10.150.100.135) which is attempting to port scan the destination machine (10.150.100.165) on the following common Windows ports: 135 (RPC), 139 (NETBIOS), 445 (SMB), 3389 (RDP), and 5985 (WinRM). Some of these ports are commonly used for remote code execution or remote access, which can lead to lateral movement. 

  • Query: event.code:3 AND source.ip: 10.150.100.135 AND destination.ip : 10.150.100.*

SMB scan

Sysmon tracks network connections (event ID: 3) based on predefined conditions. In the query above, we observed the infected host (10.150.100.135) scanning port 445 and it was able to discover that another host (10.150.100.165) had port 445 open. The query below is searching the Sysmon index for network connections from the infected host (10.150.100.135) to perform SMB actions with this host. 

SMB shares can be enumerated by an attacker giving them even more information about your networks, such as shares accessible to the current user, share rights on that share, and scan for administrative shares such as the admin$ share or the IPC$ share. These SMB administrative shares can be used for remote code execution and lateral movement. We will cover this later in the blog post but for right now remember the process name that initiated these connections which is powershell.exe.

For the purposes of this blog post, I am pointing out a specific detail that will come into play later. As a threat hunter, you will need to maintain detailed documentation and notes.  Sometimes the data collected may not seem relevant but as the hunt progresses it could be. As result, developing good note taking and documentation habits will be valuable for your career.

  • Query: event.code:3 AND destination.port:445 AND source.ip: 10.150.100.135

Reverse DNS

Sysmon tracks DNS requests (EventID: 22) made by the host machine it’s monitoring. A common internal recon technique is collecting all DNS records for the local domain and/or resolving all IP addresses to DNS hostnames. Additionally, DNS provides context as to what an IP address may be hosting. For example, mail.hackinglab.beer is most likely the e-mail server and WinDC.hackinglab.beer maybe the domain controller. Resolving IP addresses to DNS hostnames may help the attacker determine their next target or source of information. The query below is searching the Sysmon index for DNS requests (EventID: 22), DNS requests targeting the local domain (hackinglab.beer), and focus on all DNS requests generated by our infected machine (10.150.100.135).

  • Query: event.code:22 AND dns.question.registered_domain : "hackinglab.beer" AND host.ip:10.150.100.135

User hunting

Sysmon records all commands executed (EventID: 1) by Powershell and the Command Prompt (CMD) on Windows hosts. There are several procedures adversaries can use to look up the machines that a user typically resides at. One of these procedures is to use the query command with the /server:<computername> flag to query what users are logged into which machines.

Since we know the actions performed by the red team exercise, we are going to take a shortcut in this section. The use of the query command is one of the hundreds, therefore, you wouldn’t necessarily hunt all of these. It would be more efficient to write a detection rule to alert you if any of these procedures are used, which would lead to further investigation. For the sake of this blog post and demonstrating how to hunt for this activity, we are going to hunt for the following hypothesis: the attacker used the query command to determine which machine sjobs resides at. The screenshot below displays the query command being used to query information on several hosts.

  • Query: event.code:1 AND process.parent.command_line: "query.exe\" user"

Breaking down the findings

One of the biggest takeaways from this section is that the attacker scanned the local network and was able to detect open ports that could be used to perform lateral movement. We discovered the attacker attempting to scan SMB shares which indicates the attacker may move laterally with SMB. The attacker also performed a DNS zone transfer so they have a list of potential targets. Lastly, we know the attackers used the list of machines from the DNS zone to query for logged-in users on each host. The discovery of this technique could indicate several things, for our scenario it means the attacker was able to determine the location of Sheldon Job’s (sjobs) machine.

As a threat hunter, it’s important to understand the inner workings of your networking because the use of RDP (Remote Desktop Protocol) or WinRM (Windows Remote Management) by the IT department could be leveraged by an attacker for lateral movement. It’s also good to note that threat hunters don’t always focus on finding evil. A successful hunt could be as simple as learning something new about your network such as all the Windows machines in the Finance department have RDP enabled. Maybe you discover an internal web application that developers can use to rotate their credentials or add a trusted SSH key to production machines. These types of findings by themselves are not indicators of malicious activity but there is potential for an attacker to abuse them. Lastly, a good threat hunter is always working to improve the overall security posture of the organization, and if any findings are discovered such as the ones mentioned you should discuss the next steps with your security team for remediation or creating detections to alert on malicious activity.

Additional hunt queries based on findings

  • Hunt query 1: Using a Kibana visualization the Y-axis would be the count of DNS requests made by each host and the X-axis would be each host. This type of graph would allow you to visually spot anomalies such as hosts with lots of queries. For better results exclude your local DNS servers and focus on all queries for your local domain.
  • Hunt query 2: Hosts initiating AND establishing connections to known lateral movement services. This requires good documentation on which hosts should be talking to who and how.
  • Hunt query 3: Collect Windows logs for hosts attempting to connect to admin shares
  • Use Zeek formerly known as BRO to monitor SMB activity from a network perspective and detect SMB lateral movement.

Phase 6: Lateral movement

At this stage, an attacker will use the intelligence discovered during the internal recon phase to move from system to system. Common protocols and services used by computers on the network can be used by attackers. This allows the attacker to move around the network evading detection and blending into normal day-to-day activities. Obtaining access to multiple systems may allow an attacker to: obtain access to privileged domain accounts, sensitive documents, or place additional persistent mechanisms.

There are several known techniques that can be used for lateral movement in a Windows environment. The previous Attack Lifecycle phase found logs for port scanning activity for the following ports 445 (SMB), 3389 (RDP), and 5985 (WinRM), which were scanned by the infected host. SMB can be used by system administrators to remotely administer machines on the domain with PsExec. WinRM can also be used by system administrators to remotely manage machines on the domain with Powershell. Remote Desktop can be used by the IT department to assist with triaging issues remotely but can also be abused by attackers to remotely control a system. All three of these services are viable options to move laterally in a Windows environment.

If you would like to follow the threat hunting journey with no spoilers, please skip to the next section. If you would like more background on the lateral movement techniques used in the red team exercise keep reading. The red team exercise in the previous blog post demonstrated how to move laterally with PSRemoting and SMBexec, but first, credentials are required. To obtain credentials there are two methods within Powershell Empire which are: Mimikatz and a SAM database dump. The first credential dumping method would allow the Powershell Empire agent running with SYSTEM level privileges to inject a Powershell Empire agent into the LSASS.exe process.

LSASS (Local Security Authority Subsystem Service) is a Windows process that may contain password hashes of users which can be obtained with Mimikatz. Mimikatz is a common tool used among APT actors, based on the 2019 FireEye M-Trends report. Next, Powershell Empire can be instructed to use PSRemoting which utilizes WinRM to launch a Powershell Empire agent on a remote machine. The second credential dumping method will dump the NTLM hashes from the SAM database of users on the machines. Since the local hashed password of the local administrator on all Windows machines in a domain is usually the same, an adversary could use it to move laterally with SMBexec.

It should be noted that this environment has been configured to allow these techniques to work. The user accounts created for this simulated environment were added to the Windows Remote Management group which provides them the necessary permissions to use WinRM on the network without being domain administrators. To share the $admin share for PsExec, I shared the entire C drive on each machine. These configurations are not set by default but I set them to demonstrate these techniques. Lastly, as we progress through this Attack Lifecycle phase keep in mind that this phase is where everything comes together. However, the path that leads to this point is not an easy one to follow. I will do my best to keep it simple. If you ever get lost there is a diagram at the end of this Attack Lifecycle phase that puts everything into perspective. 

Remote Desktop (RDP) lateral movement

Sysmon does not track Remote Desktop Protocol (RDP) activity but as we know already, Sysmon tracks (EventID: 12) modifications to the Windows Registry. The following Registry path: HKEY_USERS\{SID}\Software\Microsoft\Terminal Server Client\Default\ tracks all RDP connections TO remote machines. However, as you can see from the screenshot below there are zero results. What this means, again, since we don’t have a network sensor monitoring traffic, is that we have disqualified the most likely source of evidence for RDP lateral movement. Another method that could be used to hunt for RDP activity is Windows event logs, specifically event IDs: 1149, 4624, and 4625, for more information, see this blog post here. 

  • Query: event.code:12 AND registry.path: "Software\\Microsoft\\Terminal Server Client\\Default"

PSRemoting/WinRM lateral movement

Windows Remote Management (WinRM) commonly referred to as Powershell Remoting (PSRemtoting) is a common Windows service in a Windows domain used for lateral movement. PSRemoting utilizes WinRM to execute PowerShell commands on remote machines. The query below is searching the Sysmon index for any network events (EventID: 3), where the source IP address is our infected machine, with the destination IP address of our local network, and the destination port set as the default HTTP port for WinRM, which 5985 – port 5986 is the HTTPS port.

Depending on whether you have been taking good notes or not the IP addresses in the screenshot below came from our SMB scan section during the internal recon phase.

  • Query: event.code:3 AND source.ip: 10.150.100.135 AND destination.ip: 10.150.100.* AND destination.port: 5985

Even though our Sysmon index contains several WinRM connections, we don’t have any evidence to support if the attempts were successful. Unfortunately, our Windows environment was not logging Microsoft-Windows-WinRM/Operational events with Windows Event Log. Typically, this logging source would verify on the infected host which connection attempts were successful. As a side note, if you are collecting this logging source it will generate Windows Events with Event ID 169. The trigger for the generation of Event ID 169 is the beginning of remoting activity on the target system. This Windows Event also includes the type of auth mechanism and user name used to access WinRM.

Another useful Windows Event is Event ID 142, which logs whenever a client attempts to initiate a WinRM connection but it is disabled on the remote server. Unfortunately, our infected machine was not logging these Windows events, so we can not look them up to confirm/deny the WinRM activity. However, we still have alternative log sources we can analyze for this activity. Again as mentioned before, these type of findings are great for reviewing to make adjusts to the logging infrastructure to gain better visibility.

To determine if the WinRM connection was successful on the host we can perform the query below. This query searches the Sysmon index for the process creation (EventID: 1), process name of Powershell.exe, and a parent process name of WsmProvhost.exe. The existence of these logs would indicate that the WinRM attempt was established successfully and a Powershell instance was created to execute code. However, the screenshot below is displaying that no logs exist for this query, which means the attempts were unsuccessful. 

  • Query: event.code:1 AND process.parent.name: wsmprovhost.exe AND process.name: powershell.exe

To provide some background to this exercise that ended with a dead end. When I was executing the red team exercise for some reason WinRM was not working in my Windows homelab and I was not able to move laterally with WinRM. I did not spend timing looking for a root cause. These failed attempts can be strong indicators of malicious activity even if the attacker never has a successful outcome.

Server Message Block (SMB) lateral movement

Remember in the Internal recon phase when hunting for Server Message Block (SMB) scanning activity we saw Powershell.exe initiating connections to remote hosts on port 445 and I told you it would be important to note? Well, let’s circle back to that activity and go one step deeper into our journey. SMB is a common Windows service in a Windows domain commonly used for lateral movement. SMB is commonly used for interacting with Windows file shares but it can be used to execute commands on a remote machine.

The queries below are demonstrating different methods of detecting PsExec or in our case SMBexec. The first query below is searching the Sysmon index for any network events (EventID: 3), where the source IP address is our infected machine, with the destination IP address of our local network, and the destination port set to the default port for SMB which is 445. The screenshot associated with the first query is displaying network activity from the infected host connecting to other hosts via SMB. Note that this query is attempting to demonstrate a method that can be used to determine which hosts to focus on (potentially compromised hosts), the results show us that we need to focus on 10.150.100.165 (Jupiter).

  • Query 1: event.code:3 AND source.ip: 10.150.100.135 AND destination.ip: 10.150.100.* AND destination.port: 445

At this point, we need to pivot our hunt to look for signs of PsExec/SMBexec on the Jupiter host. If you are not familiar with how PsExec works, I would strongly recommend reading this blog post for more information. The TL;DR version: SMBexec is very similar to PsExec, except PsExec sends a binary to the remote host to be executed while SMBexec lives off the land by starting an interactive shell. Like PsExec, SMBexec uses the admin shares which means all processes created will run as SYSTEM, which allows a new Windows service to be created. This new Windows service will execute all the commands on the host with SYSTEM level privileges.

The second query below is searching the Windows Event Logs index for any event ID of 7045 which is generated when a new service is created and the ImagePath of the new service must contain the common Powershell Empire string. The screenshot below is displaying two new services that were created on the Jupiter host with randomly generated service names.

In contrast to the Windows Event Logs, Sysmon does not track new services being created but it does track modifications to the Registry (EventID: 13). The third query below searches the Sysmon index for all modifications to the Registry made to the following path: HKLM\System\CurrentControlSet\Services\ (which is a list of all the services on the machine), and that the ImagePath contains that famous PowerShell Empire string to initiate an agent.

The screenshot for the third query shows the same service names that were found by the second query. The purpose of the second and third queries is to demonstrate two different methods to hunt for this activity that resulted in the same outcome. It’s important to note that the ImagePaths for each service contains %%COMSPEC%% a common environment variable used in Windows, which normally points to the command-line interpreter. This means when the service is started it creates a Windows Command Prompt (cmd.exe) running as SYSTEM that initiates a Powershell Empire agent.

  • Query 2: winlog.event_id: 7045 AND winlog.event_data.ImagePath: "*-noP -sta -w 1 -enc*"
  • Query 3: event.code: 13 AND registry.path: "HKLM\System\CurrentControlSet\Services\*" AND registry.value: "ImagePath" AND winlog.event_data.Details: "*-noP -sta -w 1 -enc*"

 

The fourth and final query for this phase is searching the Sysmon index for the creation of these Powershell Empire agents which were created by the services discovered above. The fourth query below is searching for new processes (EventID: 1), the parent process is cmd.exe, the command used to start the parent process is the string seen in the previous section (cmd.exe /C start /b), a child process with the name powershell.exe, the child process is running as SYSTEM, and the command to start the child process is the common Powershell Empire string. It’s important to understand that the results from this query not only confirm the attacker technique used, but that lateral movement was successful. So you may be wondering “doesn’t SMB lateral movement require credentials?!?!?!?”. If you are now asking, “did I miss a section on how the credentials were obtained?!?!?” So let’s zero in on those questions in the following section to answer them.

  • Query 4: event.code:1 AND process.parent.command_line: "cmd.exe /C start /b" AND process.name: powershell.exe AND user.name: SYSTEM

Credential dumping

If you are not familiar with how PsExec works, I would strongly recommend reading this blog post for more information. The TL;DR summary, if you want to move laterally with SMB you need the following requirements: admin credentials, port 445 exposed on a remote host, and the remote host needs to be sharing the appropriate admin shares. We know from the internal recon phase that port 445 is open and we know the admin shares are exposed. We also know from the previous section that the attacker was able to successfully move laterally using SMBexec.

However, we don’t know how the attacker gained access to credentials to move laterally. There are several techniques that can be used to dump credentials from a Windows machine. The red team exercise used common credential dumping techniques on Windows which are Mimikatz being the most common and dumping the SAM database. Below we are going to explore different hunting techniques to detect this malicious activity.

Mimikatz

As described above Mimikatz is the most common tool used for credential dumping. Mimikatz requires access to the LSASS.exe process memory space. If you are not familiar with LSASS, it handles passwords and token creation which means it caches these secrets in memory. To obtain these secrets there are two methods which are: inject Powershell Empire agent into LSASS and run Mimikatz or dump the LSASS process memory space and use memory forensics, but this latter method will not be covered in this blog post.

When Mimikatz is loaded in a process like LSASS it loads a very specific set of DLLs, which is a strong indicator. However, unless specified in the config, Sysmon is not tracking DLLs being loaded into LSASS. Please keep in mind there are other ways to hunt for Mimikatz, but will be using the process injection technique.

Process injection

Process injection is a Windows capability to add/load code into an existing process by loading malicious DLLs, .exes, etc. Process injection allows code to run with elevated privileges and evade defenses by hiding in legitimate processes to run malicious code. Process injection is a series of steps and one of those steps is a Windows API call for “CreateRemoteThread”. Sysmon does track the usage of “CreateRemoteThread”. The query below is searching the Sysmon index for CreateRemoteThread (EventID: 8) events, specifically on the ground-zero compromised machine, the process performing the action should be Powershell.exe running as SYSTEM, and the process having an action performed against it is LSASS.exe.

  • Query: event.code: 8 AND host.hostname: saturn AND process.name: powershell.exe AND winlog.event_data.TargetImage: "C:\\Windows\\System32\\lsass.exe"

Prove it!

Okay, great you found some random Powershell process performing ONE step of the process injection steps. In the screenshot above, we saw a Powershell.exe process with a PID of 5100 performing the “CreateRemoteThread” action. As we have seen several times throughout this blog post, when the Powershell Empire agent starts it does so by using the following string: powershell -noP -sta -w 1 -enc *. The query below is querying the Sysmon index for the process creation of powershell.exe with a PID of 5100. You can see from the screenshot below the command executed to start this process is the common Powershell Empire string. The results from this query and the query in the previous section are not indisputable evidence but it does provide a reasonable amount. In threat hunting, you won’t always have the data you need to determine something but having an understanding of attacker tools and techniques allows you to make educated guesses. 

  • Query: event.code: 1 AND process.name: powershell.exe AND process.pid: 5100

Privilege escalation

BUT WAIT doesn’t the Powershell process have to be running as SYSTEM to inject into LSASS.exe?!?!? Now we circle back to that pesky phase 4: privilege escalation we didn’t have any logs for. Unfortunately, I was unable to find any logs/evidence of the privilege escalation. Based on the findings in the previous section with Powershell Empire being successfully injected into LSASS.exe we can assume that Powershell instance was running as SYSTEM.  We can also reasonably assume (no conclusive evidence) the attacker was able to use some technique like an exploit to escalate privileges.

For those of you who like a challenge, make an attempt to detect the privilege escalation technique with the logs provided. Good info to know:

Dumping the SAM database

Dumping the Windows SAM database is a common technique to obtain credentials. For those who are not familiar with the SAM database, it contains the local user’s hashed password. Typically, in a domain, the local admin password is the same on all machines which means if you can obtain it on one machine you have it for all machines. The SAM database is located at the following file path C:\Windows\system32\config\SAM on Windows. The SAM database is also loaded into the Windows Registry at the following location: HKLM\:\\SAM\\SAM.

The query below is searching the Windows Event logs index for all activity pertaining to Event ID 4104. Event ID 4104 is Powershell Script Block Logging, which logs all the blocks of PowerShell code as they are executing. The full contents of the code, including the entire script, and all commands are captured. Script block logging also captures all de-obfuscated code due to the object-oriented nature of its implementation. Our compromised host did have Powershell Script Block logging enabled and captured the Powershell Empire payload attempting to capture the SAM database.

The screenshot below is displaying that our ground-zero compromised host had a process ID that correlates to a Powershell instance we discovered earlier, and we can see the process attempting to access the SAM database. 

  • Query: winlog.event_id:4104 AND "HKLM\:\\SAM\\SAM"

Breaking down the findings

Okay WOW, up until this point we have COVERED ALOT of content and have definitely backtracked from the lateral movement phase to the privilege escalation phase of the Attack Lifecycle. For those who may be a tad bit fuzzy about the timeline take a look at the diagram constructed below. At this point, we were able to detect and confirm the attacker was able to move laterally from Saturn to Jupiter, which is the machine used by Sheldon Jobs (sjobs). We were also able to detect and confirm the method used for lateral movement and credential dumping.

Additional hunt queries based on findings

  • Hunt query 1: event.code:3 AND destination.port: 5986 – Hunt for WinRM activity using the HTTPS port which is 5986.
  • Hunt query 2: winlog.event_id: 7045 AND message: "Service Name: PSEXESVC.exe" – Hunt for services created by PsExec
  • Hunt query 3: winlog.event_id: 7045 AND message: /[A-Z]{20}/ – Hunt for services created by SMBexec
  • Hunt query 4: C:\Windows\Prefetch[Executable File Name of Tool]-[RANDOM].pf – Hunt for the existence of the PsExec binary in the Windows Prefetch
  • Hunt query 5: HKEY_USERS[User SID]\SOFTWARE\Sysinternals\PsExec\EulaAccepted – Hunt for the existence of this Registry to determine if PsExec was executed and accepted the EULA agreement.

Phase 7: Maintain persistence

At this stage, the attacker wants to maintain their presence with their newly conquered machines. Depending on the attacker’s objective this phase may be similar to the “Establish Foothold” phase. Typically an attacker will continue to loop through the internal recon, lateral movement, and maintain presence phases. The hope is to gain access to a machine that can perform an action or contains the objective. For example, the attacker may move laterally to obtain the credentials of a user account to access the credit card database.

In the case of an APT actor whose objective is to maintain access to a network to exfiltrate data over a long period of time, the persistent mechanisms may take a different form. Some examples include creating a domain user with VPN access, trusted third-party access, and a self spun-up virtual machine (VM).

An account with VPN access would provide the APT actor consistent access to the network and would not require any additional type of persistence on any machine in the environment. A VM in the target environment controlled (disabling logging and security controls) by the attacker can act as a backdoor. A trusted third party may have access to the target environment and the attackers could use that as a pivot into the network. However, the red team exercise in the previous blog post implanted the same type of persistence mechanisms used above in the Establish foothold section. Nothing to hunt for in this phase.

Phase 8: Complete mission

In the final phase, the adversary has obtained access to the machine with intellectual property, financial data, or personally identifiable information (PII). However, the attacker still needs to exfiltrate the data out of the network without detection. In order to stay undetected, attackers may send the data in small chunks, encrypt the payload, or compress the payload to reduce the possibility of being detected.

Document search

We know from the fictitious scenario at the beginning of this blog post that Goofball is interested in stealing intellectual property (IP). Typically, this is by exfiltrating documents/schematics that contains intellectual property from the network. The query below is searching the Sysmon index for any Powershell.exe process that executed a command that searched the following string .docx. The screenshot below is displaying a Powershell command that recursively searched sjob’s user directory (C:\Users\sjobs) for documents with the Microsoft Office Word document extension. Additionally, this process was created by another Powershell process and the PID correlates to a Powershell Empire instance created bt SMBexec seen in the lateral movement phase (Server Message Block (SMB) lateral movement – query 4).

  • Query: process.name: powershell.exe AND process.command_line: "*.docx"

Exfiltration

Again, we pivot back to the Windows Event Logs specifically the Event ID 4104 logs which record Powershell scripting activity. The query below is searching the Windows Event Log index for all Event ID 4104 log events that contain the string .docx. The result of the query is in the screenshot below showing that a process was attempting to read the contents of C:\Users\sjobs\Documents\super_secret_widget_design.doc. Unfortunately, since this is a Windows Event Log we don’t know the parent process or process that performed this action. However, we can reasonably assume based on the findings in the previous section this is the attacker. Lastly, we have found evidence that the attacker was able to obtain their objective and steal the intellectual property/design for the new widget.

  • Query: winlog.event_id: 4104 AND *.docx

Breaking down the findings

At this point, we have detected the imposter in our network and we have discovered evidence that suggests the adversary accomplished their missions. The next steps are to contact all interested stakeholders to discuss possible remediation, public relations with the breach, the legal team to pursue legal actions, or potentially a third-party incident response team for the clean-up.

Additional hunt queries based on findings

  • Hunt query 1: event.code: 1 AND process.command_line: "Compress-Archive" – Powershell command to create a ZIP
  • Hunt query 2: Powershell commands containing an array of file extensions
  • Hunt query 3: event.code: 1 AND process.command_line: "Invoke-RestMethod *-InFile" – Powershell command for HTTP POST with ZIP

Retrospective

Threat hunt perspective

  • Hunting the red team exercise was mostly successful
  • Some visibility gaps were identified.
    • As result, it also provides a prioritization road map
  • The findings from our hunts contain enough data to develop detections
  • Suspicious activity was detected and should be reported to IR
  • Demonstrated that threat hunting can be an effective practice to uncover attackers that have circumvented pre-existing security controls and/or preventing the attacker from accomplishing their object.

Incident response perspective

  • Threat hunting uncovered an incident for the IR team to investigate
    • Ensure that clearly defined processes exist for escalation of findings
  • Unclear who the actor(s) are
  • As the IR investigation/hunt progresses, it will uncover findings to emerge into a bigger picture hopefully indicating the significance of the activity

Take away

We may never get a guarantee of true security, but we can sleep better at night knowing that we are bringing the fight to the attackers by being proactive. Threat hunting allows us to utilize the knowledge of how our network operates and the tactics of our adversaries to our advantage. When we combine this knowledge with host and network tools that provide the proper telemetry we can develop detections to spot adversaries, ideally before they accomplish their mission.

While this blog post is hunting for an attacker following the phases of the attack Lifecycle you do not need to hunt in this manner. You can start hunting at ANY stage depending on visibility and tooling. This blog post is demonstrating how one might hunt for malicious behavior at each phase of the Attack Lifecycle. My approach to hunting the red team exercise from the previous blog post was to help the reader understand and develop the following:

  • Understanding the mindset of a threat hunter
  • Creating an informal threat hunting methodology and process
  • Understand the phases of the Mandiant Attack Lifecycle from a threat hunters perspective
  • How to hunt for artifacts left behind by the red team exercise and recognize their importance
  • Connecting evidence discovered at one phase to possible findings in past or future phases
  • How to craft Elastic queries to query for malicious activity
  • Focus on TTPs (Tactics, Techniques, and Procedures) over IOCs
  • How to convert successful hunts to detections

Threat hunters mindset

A threat hunter’s goal should not be solely focused on finding “evil” or malicious behavior. In fact, this type of mindset will only have negative impacts on developing capabilities, detections, and proper architectural visibility on the network you’re trying to protect. I HIGHLY HIGHLY HIGHLY recommend reading the following blog post: How WW2 aircraft can teach us about data driven insights today. The TL;DR summary is that in WW2 many bombers were getting shot down on runs over Germany. “The naval researchers knew they needed hard data to solve this problem and went to work. After each mission, the bullet holes and damage from each bomber was painstakingly reviewed and recorded.”. However, this approach meant they were trying to solve a solution for planes that returned home and not analyzing planes that didn’t. Threat hunting is no different, if you only focus on the data in front of you, you will fall into the same trap told by this wise tale.

As a threat hunter take time to understand your environment. In the process, you will understand what is normal or what is not normal, why that is the case, and use that knowledge to baseline and create not only valuable detections but architectural changes to proactively make it much more difficult for an adversary to take advantage of any of the phases in the attack life cycle. For example, if you only focus on using EDR data (therefore you only have data from devices with an EDR solution) you might miss that a gaming console in your employee lounge area has been compromised to exfiltrate (exfil) documents to Goofball. If you only focus on the assets in your asset management directory what about your assets in the cloud, assuming they aren’t in the directory?!?!?

Being a threat hunter is about learning the environment, finding deficiencies in your security controls – like a Windows domain controller has logging disabled, hunting for malicious behavior using multiple sources of telemetry. Additionally, as a threat hunter, you want to always be re-evaluating your assumptions. For example, what if your EDR vendor says they record file creation.  Are you sure that is the case?  Some EDR vendors will only create file events if the file being created matches a set of pre-defined and un-configurable parameters. What if the problem is your EDR is not properly configured? What if a network ACL (network access list) is blocking a section of your network from communicating with your EDR server thus making you blind to attacks?  Threat hunting helps you revisit these assumptions to confirm you have optimal visibility and your detection dwell time is as low as possible.

Active Countermeasures (commonly known as Black Hills Infosec) states that threat hunting is the glue between protection and response. Protection being security controls like firewalls that prevent the adversaries from getting into the network. The response being the incident response portion that when we detect malicious activity we need someone to remediate it. Threat hunting fits into the spot of “what if my security controls don’t prevent X malicious activity nor detect it?!?!?”. Threat hunting is a proactive capability that allows us to identify areas of improvement that can improve security controls while augmenting detection and response capabilities.

It should be noted that threat hunting programs at each organization will have varying levels of maturity but mature programs will clearly define what are the roles and expectations of their program. In addition, mature programs will have defined processes for escalations, remediations, and lessons learned. In short, when threat hunting is done correctly it can be an integral process to improving the overall security posture for an organization.

DFIR knowledge is crucial

Before I started this blog post series, I had been helping a professor do a re-design of the Windows DFIR class and I was reading the “Incident Response & Computer Forensics” book. This book is a GOLD MINE of DFIR knowledge for Windows. When I combined my DFIR knowledge with the understanding of how attacks are conducted a HUGE light bulb went off for me on how to hunt. Having a solid foundation in Windows DFIR artifacts is essential to hunting in Windows environments. I understand that reading about the AmCache is boring by itself but when you can apply that knowledge as we did to the red team exercise it makes all the differences. This general recommendation applies to any technology or platform that you are trying to hunt for malicious activity. It is paramount to understand it and where forensic artifacts/logs are located.

Threat hunting methodology and process

As I stated several times throughout this blog post, I was demonstrating an informal process to threat hunting. My next blog post will demonstrate how to hunt the red team exercise using the Endgame threat hunting process and methodology and Osquery. Here is a short preview of the Endgame process and methodology and the outline of what is to come:

  1. Select a framework – spoiler alert it will be MITRE ATT&CK
  2. Generate a hypothesis
  3. Identify evidence to prove the hypothesis
  4. Develop analytics
  5. Automate
  6. Document
  7. Communicate and report
  8. My own personal and unofficial step – Creating detections

Implementing a threat hunting methodology and process is fundamental in demonstrating to the business the value threat hunting brings to the table. Beyond business value, it will help your team properly scope, quantify and document hunts. We will explore how the scientific process and threat hunting align as well as the basis for a successful program using the Endgame threat hunting process and methodology in my next blog post. Lastly, as I alluded to above, one of the final stages of any threat hunting process should be converting your findings into a detection. Each phase of the attack lifecycle above included queries and these queries can be converted to alerts to detect malicious activity.

Lessons learned

I am currently reading a book called “Cracking the Coding Interview” and it is a great book. One interesting part of the book is their matrix to describe projects you worked on and the matrix contains the following sections which are: challenges, mistakes/failures, enjoyed, leadership, conflicts, and what would you’d do differently. I am going to try and use this model at the end of my blog posts to summarize and reflect on the things I learn. I don’t blog to post things that I know, I blog to learn new things and to share the knowledge of my security research.

New skills/knowledge

  • Learned the different phases of an APT style attack
  • Learned the Mandiant Attack Lifecycle
  • Learned the tools and techniques used by attackers and how to detect them
  • Learned how to connect findings from one aspect of the attack lifecycle to future/past actions
  • How to setup and configure Sysmon v12.0.1
  • Used the latest version of ELK v7.10.0
  • Learned Terraform v0.13+
  • Learned how to deploy infrastructure on AWS for a DEFCON scale workshop
  • Learned how to create a workshop at DEFCON scale
  • Got a refresher on how PsExec and SMBexec work

Challenges

  • Keep this blog post exciting and in-depth but simple for beginners

Enjoyed

This blog post series has been three years in the making. It was exciting to watch me write this blog post over the past couple of months and see how much I have grown. Not to mention this blog post series caught the eye of the DEFCON Blue Team Village and in the summer of 2020, I had the pleasure of leading a workshop based on my work pertaining to this blog post series.

Projects inspired by this work

DISCLAIMERS

The information contained in this blog post is for educational purposes ONLY! HoldMyBeerSecurity.com/HoldMyBeer.xyz and its authors DO NOT hold any responsibility for any misuse or damage of the information provided in blog posts, discussions, activities, or exercises. 

The red team exercise performed in the previous blog posts occurred in a home lab network. Therefore, the malicious activity performed was on machines that I own and have permission to perform these actions.

DISCLAIMERS

Datasets

References

Tagged , ,

Leave a Reply

Your email address will not be published. Required fields are marked *