Feb 20 2019
Leave a comment
By spartan2194 Incident Response, Network Security, Threat Hunting, Tools

Part 1: Threat hunting with BRO/Zeek and EQL

One of the biggest challenges for blue teams is using logs to hunt for malicious activity. Tools like BRO provide fantastic logging of the events that transpired on a network but don’t provide a mechanism to ask those logs a question. Threat hunting is the process of generating a series of hypotheses about malicious activity that might be occurring on your network. EQL provides a tool that can ingest logs and provide the threat hunter a mechanism to ask questions to prove or disprove their hypotheses. Furthermore, I have extended the EQL platform to support Zeek/BRO logs for network-based threat hunting.

Tools

Event Query Langauge(EQL)

EQL is a language that can match events, generate sequences, stack data, build aggregations, and perform analysis. EQL is schemaless and supports multiple database backends. It supports field lookups, boolean logic, comparisons, wildcard matching, and function calls. EQL also has a preprocessor that can perform parse and translation time evaluation, allowing for easily sharable components between queries.

BRO/Zeek

Zeek is a powerful system that on top of the functionality it provides out of the box, also offers the flexibility to customize analysis pretty much arbitrarily. For the rest of this post I will refer to Zeek as BRO because it more commonly known.

EQL terms

  • Domain – A data source such as Sysmon, BRO, Osquery, etc
  • Source – Event types that are recorded by the domain
  • Event type – A specific event such as a network connection, file creation, process creation, etc

Install/Setup EQLLIB for BRO logs

Install/Setup EQLLIB

  1. cd /tmp && git clone https://github.com/endgameinc/eqllib
  2. cd eqllib
  3. python3 setup.py install
  4. cd /tmp && git clone https://github.com/CptOfEvilMinions/ThreatHuntingEQLandBro.git
  5. cd ThreatHuntingEQLandBro
  6. python3
    1. import eqllib
    2. print(eqllib.__file__)
  7. cp <Python3.7 base_dir>/site-packages/eql-*.egg/eql/etc/schema.json <Python3.7 base_dir>/site-packages/eql-*.egg/eql/etc/schema.json.bak
    1. Create a backup of schema.json
  8. cp bro-schema.json <Python3.7 base_dir>/site-packages/eql-*.egg/eql/etc/schema.json
    1. MacOS Python 3.7 base_dir: /usr/local/lib/python3.7
    2. Schema.json contains a list of event_types
  9. cp bro-domain.toml <Python3.7 base_dir>/site-packages/eqllib-*.egg/eqllib/domains/bro-domain.toml
    1. A domain is a record of the schema for each event in a log
  10. cp bro-source.json <Python3.7 base_dir>/site-packages/eqllib-*.egg/eqllib/sources/bro.toml
    1. Source bonds the key names in a log to the schema names

Verify BRO + EQL

  1. cd example_logs
  2. Check if new schema, BRO domain, and BRO source are working
    1. eqllib query -s "Bro events" -f conn.jsonl "bro_conn where true"
  3. Count the connections in the conn.log
    1. eqllib query -s "Bro events" -f conn.jsonl "bro_conn where true | count"
  4. Connections with a destination IP addr
    1. eqllib query -s "Bro events" -f conn.jsonl "bro_conn where destination_address == '217.20.147.1'
  5.  Count the connections with a destination IP addr
    1. eqllib query -s "Bro events" -f conn.jsonl "bro_conn where destination_address == '217.20.147.1' | count"
  6. Unique DNS queries
    1. eqllib query -s "Bro events" -f dns.jsonl "bro_dns where true | unique"

Converting BRO logs on MacOS

At the time of this writing, EQLLIB(version 0.6.2), does not handle BRO keys that contain a “.” like “id.resp_h”. I have documented below how I use SED to convert keys from “id.resp_h” to “src_addr”. Additionally, in the repo, I have a RSYLOG config for a client to ship the logs correctly :).

  1. sed -i '' 's:id\.orig_h:dest_addr:g' *.jsonl
  2. sed -i '' 's:id\.orig_p:dest_port:g' *.jsonl
  3. sed -i '' 's:id\.resp_h:src_addr:g' *.jsonl
  4. sed -i '' 's:id\.resp_p:src_port:g' *.jsonl
  5. sed -i '' 's/\(:[0-9][0-9]\)\.[0-9]\{6\}/\1/g' *.jsonl

Threat hunting for Trickbot

Generating a hypothesis

The first step to threat hunting is generating a hypothesis. A bad hypothesis would be to hunt for all the “bad” things on a network. A better hypothesis is to hunt for the existence of “Trickbot” on the network. From threat intelligence feeds, YARA rules, Twitter, and etc, we know network artifacts unique to Trickbot. Our hypothesis(H1) will be: BRO logs and EQL can be used to detect the existence of Trickbot on the network or prove no existence.

Known intelligence about Trickbot

  • MalwareBytes report
    • Targets Windows systems
    • Uses SMB for lateral movement
    • Distributed via malspam or malicious documents
      • Disguised as a Microsoft word document
    • Banking trojan
    • Malware hashes
  • PasteBin post for Trickbot
    • List of servers to call back too
  • F5 networks
    • Communicates via HTTP

Subhypotheses

  • H1.1: Detect the existence of Trickbot with a list of known C2 IP addresses.
  • H1.2: Detect the existence of Trickbot with a list of known file hashes.
  • H1.3: Detect the existence of Trickbot by detecting anomalies in HTTP
    • H1.3.1: User-agents
    • H1.3.2: URIs
    • H1.3.3: POST requests for resources
  • H1.4: Detect the existence of Trickbot by sharing a malicious binary via SMB
    • H1.4.1: A new file is pushed via SMB to a remote host with a random filename

Converting PCAP with BRO on MacOS

The repo for this blog post contains BRO logs that have been converted from a PCAP for this exercise. However, if you’re interested in steps I performed, follow the instructions in this section below .

  1. brew install bro bro-aux
  2. mkdir -p /usr/local/Cellar/bro/2.5.5/share/bro/site/scripts
sudo tee /usr/local/Cellar/bro/2.5.5/share/bro/site/scripts/json-logs.bro << EOF
@load tuning/json-logs

redef LogAscii::json_timestamps = JSON::TS_ISO8601;
redef LogAscii::use_json = T;
EOF
  3. wget https://www.malware-traffic-analysis.net/2018/05/24/2018-05-24-Trickbot-infection-traffic-AD-environment.pcap.zip
  4. unzip 2018-05-24-Trickbot-infection-traffic-AD-environment.pcap.zip
  5. bro -C -r 2018-05-24-Trickbot-infection-traffic-AD-environment.pcap policy/tuning/json-logs.bro

EQL + BRO logs

Extracting destination addresses

PasteBin provided a list of known IP addresses that Trickbot calls back too. Let’s see if conn.log contains an IP address known to be associated with Trickbot.

  1. eqllib query -s "Bro events" -f example_logs/trickbot-conn.jsonl "bro_conn where destination_address in ('187.188.162.150', '185.28.63.109','83.0.245.234','213.241.29.89','62.109.31.123','92.63.107.14')"
    1. Result was none

Extracting file hashes

Malwarebytes provided a list of known hashes for Trickbot. Let’s see if files.conn contains a hash known to be associated with Trickbot.

  1. eqllib query -s "Bro events" -f example_logs/trickbot-files.jsonl "bro_files where files_md5 in ('9aac1e00d62e0b4049781cc5eff99bc7', '9b3659936354dceb1063a42f15d0f12a', '60bd4480035e82393636b0fb60d351ba','ba36cf1afb6b6eed38b0a8d54152335b','74933912ad87ec0b3a1b570a0ea0832b','b6f9ba3fd8af478147c59b2f3b3043c7','ac32c723c94e2c311db78fb798f2dd63','f8e58af3ffefd4037fef246e93a55dc8','25570c3d943c0d83d69b12bc8df29b9d','5ac93850e24e7f0be3831f1a7c463e9c','69086a1e935446067ecb1d20bfa99266','b34d36c1c76b08e7b8f28d74fbf808d8')"
    1. Result was none

Analyzing HTTP traffic

Detecting anomalous HTTP user-agents

HTTP is probably the most common protocol used by malware. The user-agent in the HTTP protocol is a field controlled by the client and not the server. Malware can use a custom user-agent like Trickbot.

  1. eqllib query -s "Bro events" -f example_logs/trickbot-http.jsonl "bro_http where true | unique_count http_user_agent"

The query above displays the unique user-agents and how many times it occurred in this dataset. The interesting thing is the first entry occurred 4 times and has NO user-agent but, has a URI of “/traur.bin”. Having a blank user-agent isn’t a known indicator of Trickbot but, it’s not normal.

Detecting anomalous HTTP URIs

eqllib query -s "Bro events" -f example_logs/trickbot-http.jsonl "bro_http where true | unique_count http_uri"

The query above will display all the unique URIs within each HTTP headers and how many times it occurred. The first entry has a URI for “/traur.bin” and an http_mime_type of “application/x-dosexec” which is confirming this is a binary .

The second entry above is an HTTP request to “http://ip.anysrc.net/plain/clientip” to request the public IP address of this network. This may be a mechanism by the malware to detect if it’s in a sandbox.

The third entry above has a count of two, a URI of “/table.png”, and an http_mime_type of “application/x-dosexec”. The request being requested(“/table.png”) does not match with the mime type.

Detecting HTTP POSTs for resources

  1. eqllib query -s "Bro events" -f example_logs/trickbot-http.jsonl "bro_http where http_method == 'POST'"

We have ONE entry for an HTTP packet that is a POST and it has several irregularities: the source IP address is from our domain controller, the POST body is being sent to an odd port of 8082, and the user-agent is “WinHTTP sender/1.0” LIKE the user-agent above, the URI seems to contain the hostname of the DC and a UID.

Detecting SMB lateral movement

Trickbot will use SMB to worm the malware throughout the network. BRO has the ability to record SMB mapping events and SMB file transfers. SMB mapping events are when a host connects to a remote network drive. SMB file transfers, as the name implies, is when SMB is used to transfer files across the network. To detect Trickbot, we will use the sequence function in EQL. This function will allow us to look at SMB events interacting with the “IPC$” share followed by a SMB file event with an action of “SMB::FILE.OPEN”.

  1. cat example_logs/trickbot-smb_mapping.jsonl > example_logs/trickbot-smb.jsonl
  2. cat example_logs/trickbot-smb_files.jsonl >> example_logs/trickbot-smb.jsonl
  3. eqllib query -s "Bro events" -f example_logs/trickbot-smb.jsonl "sequence with maxspan=30s [bro_smb_mapping where smb_mapping_share_type == 'PIPE'] [bro_smb_files where smb_files_action =='SMB::FILE.OPEN' and not smb_files_name in ('samr','lsarpc')]"
    1. eqllib query -s "Bro events" -f example_logs/trickbot-smb.jsonl "bro_smb_mapping where smb_mapping_share_type == 'PIPE'"
    2. eqllib query -s "Bro events" -f example_logs/trickbot-smb.jsonl "bro_smb_files where smb_files_action =='SMB::FILE.OPEN'"

In the screenshot above, we can see host “10.0.100.223” pushing a binary named “44783m8uh77g8l8.nkubyhu5vfxxbh878xo6hlttkppzf28tsdu5kwppk.11c1jl.exe” to the remote host “10.0.100.5”(our domain controller). From our intelligence we know Trickbot uses SMB to worm itself throughout the network and we known the filenames are arbitrary.

Conclusion

Not all hunts will end with a finding of malicious activity. As stated above, threat hunting is the process of creating a hypothesis and applying the scientific method to prove or disprove it. This hunt didn’t result in a definite answer but it did discover some irregularities which should be followed up with a more thorough investigation. A thorough investigation may include: analyzing the hosts themselves, collecting more intelligence on Trickbot to generate additional hypotheses, or this hunt lead to the discovery of malicious activity occurring on your network that you were unaware of.

Resources/Sources

Leave a Reply

Your email address will not be published. Required fields are marked *