One of the biggest trends in infosec, besides the word cyber, is threat hunting. First, I want to start by defining threat hunting as the action of “investigation without cause” and this concept is nothing new. It’s been around for years but we didn’t have a catchy marketing term associated with it. In this post, I will breakdown the Sqrrl threat hunting model, Powershell Empire for adversary activity, and instructions on setting up Graylog for log aggregation and a search platform to perform threat hunting. Finally, I would like to point out all Ansible playbooks used in this post are publicly accessible on my Github page in a repo called “AgileFalcon“.

Definitions

• TTP – Tactics, Techniques and Procedures also sometimes referred to as Tools, Techniques, Procedures.
• Data – Data is facts or statistics that can be stored/processed
  • Anti-virus flags a PDF as malicious but this isn’t enough because anti-virus can produce false positives.
• Context – Context helps you interpreter the data in a certain way
  • Context 1: The PDF was downloaded from an e-mail that is known to be connected with a phishing campaign. (positive)
  • Context 2: The PDF makes a javascript call which is not malicious but can be used for malicious activity. (false positive)
• Information – A combination of data and context, data is something we can act upon.
  • Data + context 1: We can add this e-mail to our spam filter, search machines for file hash, and update PDF reader(if updates are available).
  • Data + context 2: Ignore the false positive and update documentation/ticket to reflect the reason why it was flagged.
• Event – An observable occurrence in an information system that happened at some point in time.

Sqrrl threat hunting model

Create hypothesis

A hunt starts by creating a hypothesis about malicious activity that might be occurring within your network. You may be asking the question of “how do I make this hypothesis”. Fortunately, the MITRE ATT&CK framework is a great tool for beginners. Each square on the matrix is a potential hunt for your team to do on your environment.

For this blog post we want to hunt for the use of Powershell Empire within our environment.  If we look at the MITRE ATT&CK framework we see that “Powershell” is categorized under the “Execution” column. Powershell Empire is one tool used by adversaries to run Powershell commands for malicious activity. For the purpose of this post, our hypothesis is how to detect Powershell Empire being used within our environment.

Investigate: tools and techniques

Our hypothesis will be investigated using tools and techniques that our environment supports. Tools such as ELK, Graylog(this post), Bro, and OSQuery can be used, but in this post we will focus on Graylog + Winbeatlog + Windows event logs. Techniques such as visualizations of the datasets, statistical analysis, and machine learning may be applied to the data to create information which we can act on. With the combination of tools and techniques we will create TTPs to detect malicious activity within our environment currently and in the future.

Uncover: New patterns and TTPs

This stage is uncovering tools and techniques used by the adversary. Once the tools and techniques are identified we want to implement this new TTP into our security platform. For our hunt this may include looking for Powershell commands that are Base64 encoded. Furthermore, during this blog post we will take an in-depth look at how Powershell Empire works and by understanding our adversaries tool we will make alerts.

Inform and Enrich: Analytics

This stage is about automating your hunt to catch adversaries in the future. During this stage, you will update your logging system to collect certain events, update your data processing engine, and/or modify input data used for machine learning. In this blog post we will not have a pipeline like this, but we use a smaller example. However, we will use Graylog to our advantage to setup alerts, and dashboards.

Install/Setup Graylog on CentOS 7

Install/Setup Graylog

1. git clone https://github.com/Benster900/AgileFalcon.git
2. cd AgileFalcon
3. vim hosts and set manager_node
4. cp group_vars/all.example group_vars/all
5. vim group_vars/all and set base_hostname, base_domain, slack_token, and slack_channel
6. cp group_vars/manager.example group_vars/manager
7. vim group_vars/manager set graylog_admin_password
8. ansible-playbook -i hosts deploy_management.yml -u root

Setup/Configure Graylog Input

1. Browse to https://<graylog fqdn> and login
   1. Username: admin
   2. Password: <group_vars/manager: graylog_admin_password>
   3. Select “System” then “Inputs”
   4. Select “beats” for input then select “Launch new input”
      1. Select a node for Node
         1. Should only be one node
      2. Enter “Winlogbeat” for title
      3. Select “Save”

Windows agents setup

Install/Setup Winlogbeat with Powershell 

1. vim hosts and add ip addresses of all windows agents
2. cp group_vars/windows.example group_vars/windows
3. vim group_vars/windows set ansible_user, ansible_pass
4. vim deploy_agent.yml
   1. Comment out - import_tasks: roles/windows/deploy_powershell_logging.yml
   2. We enable Powershell logging later on after we do some threat hunting.
5. ansible-playbook -i hosts deploy_agent.yml

Powershell Empire

Install/Setup Powershell Empire on Kali Linux rolling

1. Bootup a Kali Linux instance
2. cd /opt
3. git clone https://github.com/EmpireProject/Empire.git
4. cd Empire
5. ./setup/install.sh
   1. Hit enter to set a random server password
6. ./empire

Setup/Configure HTTP listener

1. listeners
2. userlistener http
3. info
   1. set Name http
   2. set Host http://<IP addr of Kali Linux>:80
   3. execute

Create Powershell stager

1. usestager multi/launcher
   1. set Listener http
   2. execute
2. Copy Powershell output string

Detonate Powershell stager

1. Go to the Windows 10 machine
2. Open a Powershell prompt as Administrator
3. Copy Powershell output string and hit enter
4. Enter interact <agent ID> into Powershell Empire
5. Enter sysinfo
6. Enter ps

Let the hunt begin

Investigate: tools and techniques

Understating the Empire

As the name implies, Powershell Empire uses Windows Powershell, recognizing this helps us understand how to scope our hunt. However, our hunt is not focused enough for us to start looking for Powershell Empire in Graylog. Let’s take a look at at the actions we did above with Powershell Empire in an effort to narrow our search.

Powershell

As we observed above Powershell Empire will generate a stager of powershell -noP -sta -w 1 -enc <base64 string>.  The -enc flag is telling Powershell that the string that follows will be base64 encoded and Powershell will need to decode the string to execute the code. The -noP flag tells Powershell to not use a profile. A profile may contain environment variables and aliases for the current shell; it’s like the .bashrc equivalent on Linux.  The “-sta” starts the Powershell command as a single thread on the machine.  The -w 1 flag tells Powershell to run the command in the background and to hide the Powershell prompt.

HTTP

As we observed above, Powershell Empire will generate a listener that is HTTP based for all communication between infected hosts and Powershell Empire. The listener has the agent call back to /admin/get.php or /news.php to obtain the next batch of commands to run. The HTTP listener sets the server header to Microsoft-IIS/7.5 and the user-agent is set to Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0;rv:11.0) like Gecko. The HTTP listener also sets a session key for all HTTP traffic. Unfortunately, this post isn’t collecting proxy logs to use this data effectively, but it’s important to understand these small things about Powershell Empire that can be used to detect it. However, keep in mind all these settings can be modified.

To the logs

Windows Event logs

1. Enter eventvwr into a Windows run prompt
2. Expand Microsoft > Windows Powershell
3. Look for event ID 400

Graylog searching

In the section above we used the Windows Event Log to confirm Powershell Empire detonated on the machine. However, for most hunts your going to use your logging service to search all the logs of all the machines your currently collecting from.

1. Log into Graylog
2. Select “search” at the top
3. Enter "powershell.exe -noP -sta -w 1 -enc" into the search
   1. YES YOU NEED THE DOUBLE QUOTES which are encapsulated in single quotes
4. This Graylog messages provides the following information:
   1. The Powershell Empire command that was run in Powershell
   2. The source aka the machine the log was generated from
   3. A timestamp of when the event occurred
   4. Provided a Windows Event ID(EID) which is 400

Uncover: New patterns and TTPs

The Graylog message above indicates that we may have Powershell Empire running our environment. However, we don’t have enough information to make this jump yet, so let’s try decoding the base64 string. Go to “https://gchq.github.io/CyberChef/“, enter base64 string from the Graylog message above into the input box, and add “FROM base64” into the recipes box. Once the string is decoded we are provided with following data that we can use for further hunting:

• $u is the user-agent used by Powershell Empire and it’s set to Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0, like above
• $ser is the Powershell Empire callback domain/IP addr and it’s set to http://172.16.77.168:80
• $t is the URLs Powershell Empire will callback to when requesting new tasks to run on agents. This variable is set to /admin/get.php, like above.
• $session is the cookie session key being used by Powershell Empire which is set to J9HyOewbMOL6y/9fcITaMx7DRP8=.

Let’s reach out

Above we have three pieces of information that we can use to our advantage to “act” like Powershell Empire. I am going to make a curl request by using the known host Powershell Empire is calling back to, the session cookie, and a known path. Open a terminal and enter curl http://<IP addr of $ser>:80/admin/get.php then enter curl http://<IP addr of $ser>:80/admin/get.php --cookie "session=<$session>" --output tmp.

 

As you can see above, when we make a curl request without a cookie we get a generic response from the server. Conversely, when we make a curl request with the session cookie we get a binary output. Unfortunately, the output is in binary format and we have no indicators this is malicious. I even uploaded the data file to Virustotal and it was green across the board, so onto the next thing.

URL dumpster diving

Above we can see that Powershell Empire is calling out to a path of “/admin/get.php”, so let’s search for that in Graylog. This search actually returned a result that allows us to make the verdict that we are in fact dealing with Powershell Empire. Unfortunately, we are unable to tell the actions done by Powershell Empire from the logs. In the next section, we take this into consideration by enabling verbose logging of Powershell. But moving forward, this Graylog message contains the following information:

• • $profile is the profile set in the Powershell Empire stager and it’s set to /admin/get.php,/news.php,/login/process.php|Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
  • $AgentDelay is the beacon interval set in Powershell Empire listener and it’s set to 5 (seconds).
  • function Get-Sysinfo – This a command used by Powershell Empire
  • function Invoke-ShellCommand

• • ipconfig
  • ps
  • whoami

Inform and enrich: Analytics

The most important part of any successful hunt is identifying malicious activity, determining if we have the ability to stop it, and  alerting all malicious activity detected. In this section we will setup Graylog streams to trigger alerts based on TTPs we found during our hunt. Additionally, during this stage, we may wish to enable logging of things that are not currently being logged, link datasets together, or start ingesting logs that you had been discarding till this hunt.

Setup/Configure Graylog Stream

Powershell command line stream

1. Select “Streams” at the top
2. Select “Create stream”
   1. Enter “Windows Powershell logging” for title
   2. Select “Default index set” for Index set
   3. Select “Save”
3. Select “Manage Rules” for “Windows Powershell logging” stream
   1. Select “Winlogbeat” for input
   2. Select “Add stream rule”
   3. Enter “winlogbeat_event_id” for field
   4. Select “match exactly” for type
   5. Enter “403” for value
   6. Select “save”
   7. Select “I’m done”

Setup/Configure Graylog alerts

Slack alerts

1. Select “Alerts” at the top
2. Select “Manage notifications”
3. Select “Add new notification”
   1. Select “Windows Powershell logging” for stream
   2. Select “Slack alarm callback” for notification type
   3. Select “Add alert notification”
   4. Enter “Windows Powershell logging” for name
   5. Enter “<slack url>” for webhook url
   6. Enter “#<channel>” for channel
   7. Enter “Graylog” for reporting user
   8. Select “save”

Create Graylog Slack alert

1. Select “Alerts” at the top
2. Select “Manage Conditions”
3. Select “Windows Powershell Logging” stream
4. Select “Field content alert condition” for condition type
5. Select “Add alert condition”
   1. Enter “Powershell Empire alert” for title
   2. Enter “winlogbeat_event_data_ScriptBlockText” for field
   3. Enter ” -noP -sta -w 1 -enc” for Value
   4. Select “Save”

Testing Slack alert

1. Open Powershell prompt on Windows 10 machine.
2. Paste Powershell Empire stager and hit enter

Moving forward

Enable more verbose Powershell logging

1. vim hosts and add ip addresses of all windows agents
2. cp group_vars/windows.example group_vars/windows
3. vim group_vars/windows set ansible_user, ansible_pass
4. vim deploy_agent.yml
   1. UNcomment out - import_tasks: roles/windows/deploy_powershell_logging.yml
5. ansible-playbook -i hosts deploy_agent.yml

The cycle continues

Enabling this Powershell logging will log a more verbose output of Powershell activity. Run through this blog post again but use the verbose logging to detect the type of commands.

Future work

In future blog posts I hope to go more in depth with the verbose Powershell logging and implementing malleable C2 profiles using SSL based communications. If these posts interest you can follow me on Twitter(@spartan2194) to get a live feed when I add new posts.

Resources/Sources

• Graylog Documentation
• Github repo: Agile Falcon
• Powershell Empire: Quick start
• Greater Visibility Through PowerShell Logging
• A Framework for Cyber Threat Hunting
• ATT&CK Matrix