Jun 04 2020
Leave a comment
By spartan2194 Incident Response, Logging, Tools

Generating CommunityIDs with Sysmon and Winlogbeat

While working on another logging project, I discovered a mechanism to generate CommunityIDs with Sysmon and Winlogbeat. Winlogbeat provides a feature called processors which can enrich log events before they are sent to the SIEM/logging server. This blog post will demonstrate a proof-of-concept (PoC) to enrich Sysmon network logs with a Community ID Network Flow Hash.

Background

What is Sysmon

System Monitor (Sysmon) is a Windows system service and device driver that, once installed on a system, remains resident across system reboots to monitor and log system activity to the Windows event log. It provides detailed information about process creations, network connections, and changes to file creation time. By collecting the events it generates using Windows Event Collection or SIEM agents and subsequently analyzing them, you can identify malicious or anomalous activity and understand how intruders and malware operate on your network.

What is Winlogbeat?

Winlogbeat ships Windows event logs to Elasticsearch or Logstash. You can install it as a Windows service. Winlogbeat reads from one or more event logs using Windows APIs, filters the events based on user-configured criteria, then sends the event data to the configured outputs (Elasticsearch or Logstash). Winlogbeat watches the event logs so that new event data is sent in a timely manner. The read position for each event log is persisted to disk to allow Winlogbeat to resume after restarts.

What are Beat processors?

You can define processors in your configuration to process events before they are sent to the configured output. The libbeat library provides processors for: reducing the number of exported fields, enhancing events with additional metadata, and performing additional processing and decoding. Each processor receives an event, applies a defined action to the event, and returns the event.

What is a CommunityID?

CommunityID is a new feature being implemented by networking security applications such as Zeek and Suricata. A CommunityID is a hash of the tuple (destination IP address, source IP address, destination port, source port, protocol)  and this tuple defines a unique connection. For example, let’s say Suricata detects malicious activity and when you examine the details of the alert it will include a unique hash as the value of communityID.

Spin up Elastic stack with Docker

Skip this section if you have a SIEM/logging server

  1. git clone https://github.com/CptOfEvilMinions/BlogProjects
  2. cd ElasticStackv7
  3. docker-compose build
  4. docker-compose up -d
  5. We will come back to finish this setup later in the post

Install/Setup Sysmon and Winlogbeat on Windows  10 1909

Automated method

I wrote a Powershell script to automate setting up Sysmon and Winlogbeat. ONLY run this Powershell script on a test machine because it will overwrite configs. Skip to the next section for a step-by-step setup.

  1. Invoke-Webrequest https://raw.githubusercontent.com/CptOfEvilMinions/BlogProjects/master/sysmon-winlogbeat-communityid/setup-sysmon-winlogbeat.ps1 -Uri  -OutFile setup-sysmon-winlogbeat.ps1
  2. powershell -Exec bypass -File .\setup-sysmon-winlogbeat.ps1
    1. Enter IP address of Logstash
    2. Enter Port for Logstash

Install/Setup Sysmon

  1. Spin up a Windows 10 machine
  2. Open Powershell as Administrator
  3. cd $ENV:TMP
  4. Invoke-WebRequest -Uri https://download.sysinternals.com/files/Sysmon.zip -OutFile Sysmon.zip
    1. Download latest Sysmon release
  5. Expand-Archive .\Sysmon.zip -DestinationPath .
    1. Unzip Sysmon
  6. Invoke-WebRequest -Uri https://raw.githubusercontent.com/SwiftOnSecurity/sysmon-config/master/sysmonconfig-export.xml -OutFile sysmonconfig-export.xml
    1. SwiftOnSecurity Sysmon config
  7. .\Sysmon.exe -accepteula -i .\sysmonconfig-export.xml
  8. Enter eventvwr into Powershell
  9. Expand Application and Services Logs > Microsoft > Windows > Sysmon
  10. Find a Sysmon event with an Event ID of 3

Install/Setup Winlogbeat

  1. cd $ENV:TEMP
  2. Invoke-WebRequest -Uri https://artifacts.elastic.co/downloads/beats/winlogbeat/winlogbeat-7.7.0-windows-x86_64.zip -OutFile winlogbeat-7.7.0-windows-x86_64.zip
    1. Go here to get a newer version
  3. Expand-Archive .\winlogbeat-7.7.0-windows-x86_64.zip -DestinationPath .
  4. mv .\winlogbeat-7.7.0-windows-x86_64 'C:\Program Files\winlogbeat'
  5. cd 'C:\Program Files\winlogbeat\'
  6. Invoke-WebRequest -Uri https://raw.githubusercontent.com/CptOfEvilMinions/BlogProjects/master/sysmon-winlogbeat-communityid/conf/winlogbeat/winlogbeat.yml -OutFile winlogbeat.yml
  7. Using your favorite text editor open 'C:\Program Files\winlogbeat\winlogbeat.yml'
    1. Open the document from the command line with Visual Studio Code: code .\winlogbeat.yml
    2. Open the document from the command line with Notepad: notepad.exe.\winlogbeat.yml
  8. Scroll down to the output.logstash:
    1. Replace logstash_ip_addr with the IP address of FQDN of Logstash
    2. Replace logstash_port with the port Logstash uses to ingest Beats (default 5044)
  9. Scroll down to the bottom and find the processors: section
  10. Add the following code:
  11. Here is the code that makes all the magic happen
  12. powershell -Exec bypass -File .\install-service-winlogbeat.ps1
  13. Set-Service -Name "winlogbeat" -StartupType automatic
  14. Start-Service -Name "winlogbeat"
  15. Get-Service -Name "winlogbeat"

Setup Elastic index for Sysmon

  1. Open browser to https://<Docker IP addr>:8443
  2. Select “Settings” in the bottom left
  3. Select “Index management” in the top left
    1. Ensure Sysmon data is in Elasticsearch
  4. Select “Index patterns” on the left under “Kibana”
  5. Select “Create index pattern” in top right
    1. Step 1: Define index pattern
      1. Enter sysmon-* into index pattern
      2. Select “Next step”
    2. Step 2: Configure settings
      1. Select “@timestamp” for Time filter field name
      2. Select “Create index pattern”
  6. Select “Discover” on the left

CommunityID verification

Simple test

  1. Enter event.code: 3 into the search
  2. Expand one of the events
    1. Obtain the source IP address (src_ip), destination IP address (dest_ip), source port (src_port), destination port (dest_port), and protocol (preferably a TCP connection)
    2. My connection for testing:
      1. src_ip: 172.16.125.135
      2. dest_ip: 172.217.12.163
      3. src_port: 52910
      4. dest_port: 80
      5. community_id: 1:D/J5+bP0lm3oa5tt0feE5yzySO8=
  3. pip3 install -U communityid
  4. ipython
  5. import communityid

cid = communityid.CommunityID()
tpl = communityid.FlowTuple.make_tcp('<src_ip>', '<dest_ip>', <src_port>, <dest_port>)

print(cid.calc(tpl))

Comprehensive test

First, to create a more comprehensive test, I added chrome.exe and python.exe to the Symon config to monitor network connections. Second, I let Google Chrome play videos on Youtube for 30 minutes and I used the Python package Scrapy to scrape websites, which generated ~500 network events. Third, I searched for all Sysmon events with an  event.code: 3 and downloaded those events as a CSV. For each CSV log entry, I generated the community ID and compared it to the one in the Sysmon log. After running this test on my sample of data it appears that this method is working correctly.

Discussion

Don’t let perfection be the enemy of the good

This implementation is restricted to TCP and UDP protocol excluding common protocols like ICMP. I tried to look for a Sysmon config that would record ICMP traffic or an example log of ICMP traffic for the key names but I was not able to. Since a majority of protocols are built on TCP and UDP, I don’t think this will be a huge loss in visibility. Furthermore, if your environment has a network monitoring platform like Zeek, you will have visibility but it will not contain information about the process that generated the event.

Lessons learned

I am currently reading a book called “Cracking the Coding Interview” and it is a great book. One interesting part of the book is their matrix to describe projects you worked on and the matrix contains the following sections which are: challenges, mistakes/failures, enjoyed, leadership, conflicts, and what would you do differently. I am going to try and use this model at the end of my blog posts to summarize and reflect on the things I learn. I don’t blog to post things that I know, I blog to learn new things and to share the knowledge of my security research.

New skills/knowledge

  • Learned how to install, setup and configure Sysmon
  • How to configure Winlogbeat and Logstash to use TLS without mutual TLS for testing
  • Dockerized Elastic Stack v7.7

What You’d Do Differently

  • Add support for ICMP
  • If I had an enterprise environment I would do a more thorough test to verify the CommunityID generated by Sysmon
  • I wish Sysmon provided a mechanism to download specific versions.

References

Leave a Reply

Your email address will not be published. Required fields are marked *