Intro to Threat Intelligence with Bro and ELK

One of the biggest trends in cyber security is threat intelligence. A lot of security professionals and enterprises are asking what is threat intelligence, do I need it, and can it improve my security?  First let’s start by defining threat intelligence and the rest of this guide will provide a practical use case for threat intelligence. Threat intelligence is utilizing information to detect security threats that traditional methods and technologies may not and providing decision driven incident response based off data.

Components

  • Cobalt strike is software for Adversary Simulations and Red Team Operations.
    • Teamserver is the controller for the Beacon payload and the host for Cobalt Strike’s social engineering features. The team server also stores data collected by Cobalt Strike and it manages logging.
  • Bro is a passive, open-source network traffic analyzer. It is primarily a security monitor that inspects all traffic on a link in depth for signs of suspicious activity. More generally, however, Bro supports a wide range of traffic analysis tasks even outside of the security domain, including performance measurements and helping with trouble-shooting.
    • Packet capture – This is the actual acquisition of packets off the network for analysis
    • Event engine –  The engine decodes protocols to create events from the packets collected.
    • Policy script interpreter –  Events from the event engine are sent to the interpreter to create notifications and logs based on events.
  • Criticalstack is intel marketplace that utilizes the bro intelligence framework. Users subscribe to intel feeds provided by Criticalstack and then Bro scripts are generated from the feeds. The Criticalstack utility is used to add these Bro scripts to Bro to start collecting data.
    • Sensors – Our mapping of an API key to a Bro sensor
    • Collections – A container of feeds that you are subscribed too
    • Feeds – An available feed of threat intelligence data. Feeds can include malicious domains, phishing websites, Tor exit node IP addresses, and scam domains.
  • ELK stack
    • Elasticsearch – As stated by the creators “Elasticsearch is the heart of the ELK stack”. Elasticsearch is a search engine based on Lucene. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents. Elasticsearch is a NoSQL type database and contains all your data for your ELK stack.
      • This is the most important component of the entire stack! Once you understand how data is ingested and retrieved from Elasticsearch then all the components of the ELK stack make more sense.
    • Logstash – A service used for log collection, processing, and ingesting data into Elasticsearch.
    • Kibana – Provides visualization capabilities on top of the content indexed on an Elasticsearch cluster.
    • Filebeat – A lightweight way to forward and centralize logs and files to Logstash.
      • Filebeat can send logs to various service but this guide will use Logstash.
    • Elastalert – A simple framework for alerting on anomalies, spikes, or other patterns of interest from data in Elasticsearch.
  • Domains
    • Security.beer
      • This is our local domain for our network.
    • Selak.info, icloudhistory.com
      • Actual C2 domains we use to make queries and trigger the Bro intel framework.

Domain Name System Explanation

DNS provides a mechanism to translate hostnames to IP addresses. Hostnames provide a more friendly way to name hosts instead of remembering IP addresses. For example, when we want to use a search engine we browse to “google.com” and not “172.217.7.238”. Humans historically have been better at remembering names they can associate things with and not numbers like IP address.

Now let’s take this DNS resolve one step further and see how we actually end up at “google.com”. First the browser will check if the domain is in it’s cache. If not found, the browser will ask the OS to look in it’s host file for “google.com”. If not found, the OS will make a query to the local DNS server for “google.com”. If not found, the local DNS server can be setup to be a recursive or an iterative server for DNS requests. A recursive server will resolve the hostname on behalf of the client and return an answer. An iterative server will tell the client the next DNS server to ask to obtain the answer, such as a local ISP DNS server.

Test network

First, let’s start by creating a small test network to understand what a DNS beacon is and how it works. For our test network I am going to setup a Windows 7 client, CentOS 7 as a DNS server, and Kali Linux with Cobalt Strike as our attacker.

For our small test network I will utilize the domain “elite.hackers.tech” as our malicious domain. Our Kali Linux box will have an IP of 172.16.0.154, our CentOS 7 DNS server will have an IP of 172.16.0.134, and our Windows 7 client will have an IP of 172.16.0.228.

Network diagram

Install/Setup DNSmasq on Centos 7

  1. yum install dnsmasq -y
  2. echo “server/elite.hackers.tech/172.16.0.154” >> /etc/dnsmasq/dnsmasq.conf
  3. systemctl start dnsmasq
  4. systemctl stop firewalld

Setup Windows 7 client DNS

  1. Open run and enter “ncpa.cpl”
  2. Right-click “Local Area Connection” and select “Properties”
  3. Double-click  “Internet Protocol Version 4(TCP/IP)”
  4. Enter “172.16.0.134” for Preferred DNS serverScreen Shot 2017-05-05 at 1.22.37 AM.png
  5. Select “Ok”

Setup Cobalt Strike on Kali Linux

  1. Download Cobalt Strike
  2. tar -xvzf cobaltstrike.tar.gz
  3. cd cobaltstrike
  4. ./teamserver <IP of Kali Linux VM> <password>Screen Shot 2017-05-07 at 2.50.44 PM.png
  5. Open a new terminal tab
  6. ./cobaltstrike
    1. Enter “127.0.0.1” for host
    2. Leave port as default
    3. Enter a username into user
    4. Enter teamserver password into passwordScreen Shot 2017-05-05 at 1.28.56 AM.png
    5. Select connect
  7. Select “Cobalt Strike” in the top left then select “Listeners”
  8. Select “Add” at the bottomScreen Shot 2017-05-05 at 1.30.03 AM.png
    1. Enter “dns_beacon_listener” for name
    2. Select “windows/beacon_dns/reverse_dns_txt” for payload
    3. Leave host as default
    4. Enter “8080” for portScreen Shot 2017-05-05 at 1.31.50 AM.png
    5. Select “save”
  9. Enter “elite.hackers.tech” for DNS callback domain
  10. Select “Attacks” then “Web Drive-by” then “Scripted Web Delivery”
    1. Enter “/dns” for URI path
    2. Leave localhost as default
    3. Leave port as default
    4. Select “dns_beacon_listener” for listener
    5. Select “powershell” for typeScreen Shot 2017-05-05 at 1.36.59 AM.png
    6. Select “connect”
  11. A box will pop-up with a powershell command inside. Copy and paste this text and select “Ok”Screen Shot 2017-05-05 at 1.38.01 AM.png

Pwning our Windows 7 Client

  1. Open Wireshark and start a capture
  2. Open a Powershell prompt
  3. Paste the text from above and hit enterScreen Shot 2017-05-05 at 1.39.34 AM.png

Interact with our DNS beacon

  1. Go back to Cobalt Strike to see our beacon has called back successful
  2. Right-click our beacon select “explore” then “process list”
    1. Now remember DNS beacons by default work on a 60 second interval
    2. Here we can see our beacon has returned results of the process listScreen Shot 2017-05-05 at 1.46.28 AM.png

Analyzing DNS beacon PCAP

Analyzing the PCAP allows us to get an understanding of how Cobalt Strike DNS beacons work. After entering the command into Powershell we can see a DNS TXT query of “aaa.stage.10554586.elite.hackers.tech” to our local DNS server. Our local DNS server has been instructed to treat our Kali Linux VM as the authoritative nameserver for “elite.hackers.tech”. Additionally, recursion is enabled on the DNS server so it relays(like a proxy) the query to our Kali Linux VM.

The attacker receives this query and responds to our DNS server with an encoded stager. The DNS server receives the response from our attacker and relays it to the Windows 7 client. The Windows 7 client receives the encoded stager from our attacker, decodes the stager, and runs the code. This process continues until an effective C2 channel is created between our Windows 7 client and our Kali Linux VM. Once the C2 channel is established the DNS beacon sets a default callback for every 60 seconds. This means the infected host will reach out to our Kali Linux VM every 60 seconds for the next set of instructions or to return data from a request.    

Infected host querying malicious domain

The photo below shows the Windows 7 client making a query for “aaa.stage.10554586.elite.hackers.tech” to our local DNS server.

Screen Shot 2017-05-07 at 3.30.53 PM.png

Local DNS server relaying request to attacker

The photo below shows the local DNS server receiving the query from the client. The local DNS server then relays the query to “elite.hackers.tech” authoritative name server which is our attacker VM.Screen Shot 2017-05-07 at 3.33.26 PM.png

Attacker responding to DNS server

The photo below shows the attacker responding with an answer to the local DNS server. The new addition to this process is the answer section with encoded data. The encoded data is the second part of the payload process to be executed on our Windows 7 client. Screen Shot 2017-05-07 at 3.38.16 PM.png

Local DNS server relaying stager to infected host

The photo below shows the local DNS server relaying the response from the attacker to our Windows 7 client. The answer section will be decoded by our Windows 7 client and executed. Screen Shot 2017-05-07 at 3.43.44 PM.png

Building the threat intelligence platform

Network diagram

pfSense Router

System Resources and settings

  • CPU: 2 cores
  • Ram: 2048 MB
  • HDD: 40 GB
  • Eth0: 00:0C:29:80:F4:6E – VMware ESXi VM Network
  • Eth1: 0:0C:29:80:F4:78 – NetSecAuditNetwork
  • External DNS1: 8.8.8.8
  • External DNS2: 8.8.4.4
  • Domain: security.beer
  • Hostname: pfsense

Install/Setup pfSense

  1. Start VM
  2. When prompted press “I” to enter installer
  3. Select “Accept these settings” for Configure consoleScreen Shot 2017-03-25 at 1.21.51 PM.png
  4. Select “Quick/Install” for Select Task
  5. Select “Standard Kernel”
  6. Select “Reboot”
  7. Select “2” for “Set Interface(s) IP address”
    1. Select “1” for WAN interface
    2. Enter “y” to configure IPv4 address WAN interface via DHCP
    3. Enter “n” to configure IPv6 address WAN interface via DHCP
    4. Leave new WAN IPv6 address blank
      1. We are not supporting IPv6 for this
    5. Enter “n” to revert to HTTP for webconfigurator
  8. Select “2” for “Set Interface(s) IP address”
    1. Select “2” for LAN interface
    2. Enter “172.25.1.254” for LAN IPv4 Address
    3. Enter “24” for subnet mask of 255.255.255.0
    4. Leave upstream gateway address blank
    5. Leave new LAN IPv6 address blank
    6. Enter “y” to enable DHCP Server on LAN
    7. Enter “172.25.1.100” for start address of IPv4 client range
    8. Enter “172.25.1.150” for end address of IPv4 client range
    9. Enter “n” to revert to HTTP for webconfigurator

Temporary Client and pfsense Webconfig

  1. Configure a temporary client on the pfSense internal LAN
  2. Open web browser and browse to “172.25.1.254”
    1. Login with “admin” as user and “pfsense” as password
  3. General info
    1. Enter “pfsense” for hostname
    2. Enter “security.beer” for domain
    3. Enter “8.8.8.8” for Primary DNS server
    4. Enter “8.8.4.4” for Secondary DNS serverScreen Shot 2017-05-07 at 3.51.39 PM.png
  1. Time server info
    1. Leave settings as default
  2.  Configure WAN interface
    1. Select “DHCP” for Selected Type
    2. Uncheck “Block private networks from entering via WAN”
    3. Uncheck “Block non-Internet routed networks from entering via WAN”Screen Shot 2017-05-07 at 3.53.59 PM.png
  1. Configure LAN interface
    1. Enter “172.25.1.254” for LAN IP address
    2. Select “24” for subnet maskScreen Shot 2017-05-07 at 3.55.06 PM.png
  1. WebGUI password
    1. Enter a password

Setup LAN Firewall rules

  1. Block all rule
    1. Select “Block” for action
    2. Select “LAN” for interface
    3. Select “IPv4 + IPv6” for address family
    4. Select “any” for protocol
    5. Select “any” for source
    6. Select “any” for destinationScreen Shot 2017-05-07 at 4.03.02 PM.png
    7. Select “save”
  2.  Add DNS outbound rule
    1. Select “Pass” for action
    2. Select “LAN” for interface
    3. Select “IPv4” for address family
    4. Select “UDP” for protocol
    5. Select “single host or alias” for source
      1. Enter “172.25.1.253” for host
    6.  Select “single host or alias” for destination
      1. Enter “8.8.8.8” for host
      2. Enter “DNS” for destination port rangeScreen Shot 2017-05-07 at 4.09.41 PM.png
    7. Select “save”
  3. Add rule to allow all HTTP/S traffic from LAN to world
  4. Add rule to allow all NTP traffic from LAN to world
  5. Add rule to allow all git traffic from LAN to world
  6. Add rule to allow ICMP from anywhere to anywhereScreen Shot 2017-05-07 at 4.11.31 PM.png

Disable DNS Server on router

  1. Select “Services” then “DNS Resolver”
  2. Un-check “Enable DNS resolver”Screen Shot 2017-03-25 at 4.13.25 PM.png
  3. Select “Save”

Setup DHCP

  1. Select “Services” then “DHCP”
  2. Enter “172.25.1.253” for DNS ServerScreen Shot 2017-05-07 at 4.16.44 PM.png
  3. Select “Save”

CentOS 7 Bind9 DNS Server, Bro, and Criticalstack

System resources

  • CPU: 2 cores
  • Ram: 1024MB
  • HDD: 40GB
  • Eth0: 00-0C-29-5D-94-A0

Install/setup BIND DNS

  1. yum update -y && yum upgrade -y
  2. yum install net-tools vim epel-release -y
  3. yum install bind bind-utils -y
  4. vim /etc/named.conf
    1. Use configuration file in the appendix
      1. Added query logging
      2. Enabled recursion
      3. Added forwarders
      4. Set listening to any interface(0.0.0.0)
    2. save, exit
  5. sed -i ‘s/SELINUX=enforcing/SELINUX=disabled/g’ /etc/sysconfig/selinux
    1. To lazy to deal with SELinux today 🙂
  6. setenforce 0

Install/setup Bro

  1. yum install cmake make gcc gcc-c++ flex bison libpcap-devel openssl-devel python-devel swig zlib-devel git -y
  2. cd /opt
  3. git clone –recursive git://git.bro.org/bro
  4. cd bro
  5. ./configure
  6. make
  7. make install
  8. export PATH=/usr/local/bro/bin:$PATH
  9. echo “PATH=/usr/local/bro/bin:$PATH” >> /etc/profile
  10. broInterface=$(ip a | grep ‘2:’ | grep ‘en’ | awk ‘{print $2}’ |rev | cut -c 2- | rev)
  11. sed -i “s#interface=eth0#interface=$broInterface#g” /usr/local/bro/etc/node.cfg
  12. sed -i ‘s#const use_json = F#const use_json = T#g’ /usr/local/bro/share/bro/base/frameworks/logging/writers/ascii.bro
    1. Bro will log in JSON
  13. /usr/local/bro/bin/broctl install
  14. /usr/local/bro/bin/broctl start
  15. /usr/local/bro/bin/broctl status

Install/Setup Filebeat logging

  1. rpm –import https://packages.elastic.co/GPG-KEY-elasticsearch
  2. cat > /etc/yum.repos.d/elastic.repo << EOF
    [elastic-5.x]
    name=Elastic repository for 5.x packages
    baseurl=https://artifacts.elastic.co/packages/5.x/yum
    gpgcheck=1
    gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
    enabled=1
    autorefresh=1
    type=rpm-md
    EOF
  1. yum install filebeat -y
  2. mkdir /etc/filebeat/conf.d/
  3. cp /etc/filebeat/filebeat.yml /etc/filebeat/filebeat.yml.bak
  4. cat > /etc/filebeat/filebeat.yml << EOF
    filebeat:
    registry_file: /var/lib/filebeat/registry
    config_dir: /etc/filebeat/conf.d
    output.logstash:
    hosts: ["172.25.1.252:5044"]
    EOF
  5. cat > /etc/filebeat/conf.d/bro_dns.yml << EOF
    filebeat.prospectors:
    - paths:
    - /usr/local/bro/logs/current/dns.log
    document_type: bro_dns
    EOF

  6. cat > /etc/filebeat/conf.d/bro_intel.yml << EOF
    filebeat.prospectors:
    - paths:
    - /usr/local/bro/logs/current/intel.log
    document_type: bro_intel
    EOF

  7. systemctl enable filebeat
  8. systemctl start filebeat

CriticalStack threat intelligence feeds setup

CriticalStack create collection

  1. Browse to “https://intel.criticalstack.com
  2. Create an account and login
  3. Select “Collections” at the top
  4. Select “Create New Collection” on the left
    1. Enter “Malicious Domains Collection” for name
    2. Enter a descriptionScreen Shot 2017-05-06 at 11.32.51 PM.png
    3. Select “Create collection”

CriticalStack subscribe to feeds

  1. Select your newly created collection called “Create New Collection”
  2. Select “Add more feeds” on the left
  3. Hover over a feed with the cursor and select “Subscribe”Screen Shot 2017-05-06 at 11.36.11 PM.png
  4. I going to add the “malicious domains” feed by Sophos and “DShields Domain List” by SANs
  5. Select “My feeds” on the left to see a list of feeds for a collectionScreen Shot 2017-05-06 at 11.39.02 PM.png

CriticalStack add Bro sensor

  1. Select “Sensors” at the top
  2. Select “Create a new sensor”
    1. Select “Malicious Domains Collection” for collection
    2. Enter “Bro Bind9 Sensor”Screen Shot 2017-05-06 at 11.40.36 PM.png
    3. Select “Create sensor”
  3. Copy the key from the newly created sensor

Install/Setup Criticalstack client on Bro sensor

  1. Select “Client” tab for a list of directions or continue with directions below
  2. curl https://packagecloud.io/install/repositories/criticalstack/critical-stack-intel/script.rpm.sh | sudo bash
  3. yum install critical-stack-intel -y
  4. critical-stack-intel config –set=bro.path=/usr/local/bro
    1. The default Bro location has moved from /opt to /usr/local
  5.  critical-stack-intel config –set bro.restart=true
    1. Criticalstack will periodically pull down updates from the feeds and restart Bro with the new feed data.
  6.  critical-stack-intel api <critical stack sensor key from above>Screen Shot 2017-05-06 at 11.55.46 PM.png
  1. critical-stack-intel list
    1. Ignore the warning with DShield because an update fixed it 🙂
      Screen Shot 2017-05-06 at 11.56.40 PM.png

View malicious domains list

  1. cat /opt/critical-stack/frameworks/intel/master-public.bro.datScreen Shot 2017-05-07 at 12.08.44 AM.png

Testing Criticalstack and Bro

  1. From the malicious domain list above I am going to query a domain
    1. I am choosing to query “selak.info”
  2. I will query this domain from a client in our network
    1. Remember that our local DNS server is the only machine allowed to make outbound DNS connections.
  3.  Open a command prompt on Windows 7 client
  4. Enter “nslookup selak.info”
    1. We can see that this malicious domain resolvedScreen Shot 2017-05-07 at 1.20.47 AM.png
  1. Now we need to move to our Bro sensor box
  2. cat /usr/local/bro/logs/current/dns.logScreen Shot 2017-05-07 at 1.23.13 AM.png
  3. cat /usr/local/bro/logs/current/intel.lo
    1. As we can see Bro used the Criticalstack feed and created an indicator for the malicious domainScreen Shot 2017-05-07 at 1.22.21 AM.png

Configure FirewallD

  1. yum install firewalld -y
  2. systemctl enable firewalld
  3. systemctl start firewalld
  4. firewall-cmd –set-default-zone=internal
  5. firewall-cmd –permanent –add-service=ssh
  6. firewall-cmd –permanent –add-service=dns
  7. firewall-cmd –reload

Install/Setup Threat Intel Stack with ELK

Install/Setup ELK

Setup NTP sync

  1. yum install ntp ntpdate ntp-doc -y
  2. systemctl enable ntpd
  3. systemctl start ntpd
  4. ntpdate pool.ntp.org

Install/Setup Elasticsearch

  1. yum install java-devel -y
  2. rpm –import https://packages.elastic.co/GPG-KEY-elasticsearch
  3. cat > /etc/yum.repos.d/elastic.repo << EOF
    [elastic-5.x]
    name=Elastic repository for 5.x packages
    baseurl=https://artifacts.elastic.co/packages/5.x/yum
    gpgcheck=1
    gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
    enabled=1
    autorefresh=1
    type=rpm-md
    EOF
    ssl_certificate /etc/nginx/ssl/nginx.crt;
  4. yum -y install elasticsearch
  5. sed -i ‘s/#network.host: 192.168.0.1/network.host: localhost/g’ /etc/elasticsearch/elasticsearch.yml
  6. systemctl enable elasticsearch
  7. systemctl start elasticsearch

Install/Setup Kibana

  1. yum -y install kibana
  2. /bin/systemctl daemon-reload
  3. /bin/systemctl enable kibana.service
  4. systemctl start kibana.service

Install/Setup Nginx, Basic Auth, and OpenSSL

  1. yum -y install epel-release
  2. yum -y install nginx httpd-tools
  3. htpasswd -c /etc/nginx/htpasswd.users <username>
  4. cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak
  5. sed -i -e ‘38,87d’ /etc/nginx/nginx.conf
    1. This will delete the default server block in nginx.conf
  6.  mkdir /etc/nginx/ssl
  7. openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/ssl/nginx.key -out /etc/nginx/ssl/nginx.crt
  8. openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

  9. cat > /etc/nginx/conf.d/kibana.conf << \EOF
    server {
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name _;
    return 301 https://$server_name/$uri;
    }
    server {
    listen 443 ssl;
    server_name _;root /usr/share/nginx/html;
    index index.html index.htm;ssl_certificate /etc/nginx/ssl/nginx.crt;
    ssl_certificate_key /etc/nginx/ssl/nginx.key;
    ssl_dhparam /etc/ssl/certs/dhparam.pem;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

    ssl_prefer_server_ciphers on;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_stapling on;
    ssl_stapling_verify on;
    add_header Strict-Transport-Security max-age=15768000;auth_basic "Restricted";auth_basic_user_file

    /etc/nginx/htpasswd.users;
    location / {
    proxy_pass http://localhost:5601;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
    }
    }
    EOF

    1. systemctl start nginx
    2. systemctl enable nginx
    3. setsebool -P httpd_can_network_connect 1

    Install/Setup Logstash

    1.  yum -y install logstash
    2. cd /usr/share/logstash
    3. ./bin/logstash-plugin install logstash-input-beats
    4. cd /etc/logstash/conf.d/


    5. cat > /etc/logstash/conf.d/02-beats-input.conf << EOF
      input {
      beats {
      port => 5044
      ssl => false
      }
      }
      EOF
    6. cat > /etc/logstash/conf.d/13-bro-filter.conf << EOF
      filter {
      if [message] =~ /^#/ {
      drop {  }
      }
      else {
      if [type] == "bro_dns" {
      json { source => "message" }
      }
      if [type] == "bro_intel" {
      json { source => "message" }
      }
      }
      }
      EOF



    7. cat /etc/logstash/conf.d/30-elasticsearch.conf << EOF
      output {
      elasticsearch {
      hosts = ["http://localhost:9200"]
      index = "%{type}-%{[@metadata][beat]}-%{+YYYY.MM.dd}"
      document_type = "%{[@metadata][type]}"
      }
      }
      EOF
    8. systemctl restart logstash
    9. systemctl enable logstash

    Install/Setup FirewallD

    1. yum install firewalld -y
    2. systemctl start firewalld
    3. systemctl enable firewalld
    4. firewall-cmd –zone=public –permanent –add-service=http
    5. firewall-cmd –zone=public –permanent –add-service=https
    6. firewall-cmd –zone=public –permanent –add-service=ssh
    7. firewall-cmd –zone=public –permanent –add-port=5044/tcp
    8. firewall-cmd –reload

    Kibana Dashboards

    Setup Index Pattern

    • Friendly reminder: Indexes are created by a daily index scheme of <year>-<month>-<day>. Indexes will only be created when there is data to store in the index. So until data is following from Bro into ELK there won’t be any indexes to setup in Kibana.
    1. Browse to “https://172.25.1.252
    2. Select “Management” on the left then “Index patterns”
    3. Select “Add new +”
      1. Enter “bro_dns-*” for index name
      2. Select “@timestampt” for tile-field nameScreen Shot 2017-05-07 at 6.10.42 PM.png
      3. Select “Create”
    4.  Select “Discover” on the left
    5. Select the drop-down menu and select “bro_dns-*” for indexScreen Shot 2017-05-07 at 6.11.41 PM.pngScreen Shot 2017-05-07 at 6.12.00 PM.png

    Creating a DNS dashboard

    Dashboards are made up by a collection of visualizations that we create and define. We will start by creating a visualization for DNS queries but it will display the least queried domains. The reason for this is DNS beacons are meant to be stealthy and are unlikely to make the top 10 queried domains. Additionally, we want to see the least queried domains because that is the most likely area for our C2 domains to show up. Furthermore, we saw above that Cobalt Strike can set the beacon to call back on a specified interval.

    Second, we will create a visualization for known malicious domains. This will help stop huge malware campaigns in there tracks and see if there is a phishing campaign against our organization.

    Finally, the data collected and analyzed by these visualizations can create alerts and notifications. These alerts and notifications can be used to help our incident responders act swiftly within a timely manner. Finally over time trends can emerge from these visualizations and can be used to detect anomalies.

    Create visualization for potentially unknown malicious domains

    1. Select “Visualize” on the left
    2. Select “Vertical bar Chart” and select “dns_query-*” for index
      1. Select “Count” for Y-Axis
      2. Select “Terms” for Aggregation
      3. Select “query.keyword” for Field
      4. Select “Ascending” and enter “30” for size in the order sectionScreen Shot 2017-05-07 at 6.17.40 PM.png
      5. Select “Save” in the top right
      6. Enter “DNS Query Histogram” for Name
      7. Select “Save”
    3. Select “Dashboard” on the left
    4. Select “Add” in the top-right
      1. Enter “DNS Query Histogram” into the search
      2. Select “DNS Query Histogram” under “Visualization”  
      3. Select “Save” in the top right
      4. Enter “DNS Dashboard” into the name fieldScreen Shot 2017-05-07 at 1.58.12 AM.png
      5. Select “Save”

    Create visualization for known malicious domains

    1. Select “Visualize” on the left
    2. Select “Data table” and select “bro_intel-*” for index
      1. Select “Split rows”
      2. Select “Terms” for aggregation
      3. Select “seen.indicator_keyword” for field
      4. Select “descending” and enter “100” for order fieldsScreen Shot 2017-05-07 at 1.51.02 AM.png
      5. Select “Save” on top right
      6. Enter “Known Malicious Domains” for nameScreen Shot 2017-05-07 at 1.52.02 AM.png
      7. Select “save”
    3.  Select “Dashboard” on the left
      1. Select “Add” in the top right
      2. Enter “Known Malicious Domains” into search
      3. Select “Known Malicious Domains”under “Visualization”  
      4. Select “Save” in the top right
      5. Enter “DNS Dashboard” into the name fieldScreen Shot 2017-05-07 at 1.57.34 AM.png
      6. Select “Save”

    Install/setup Elastalert and config rules

    Install/setup Elastalert on CentOS 7 64-bit

    1. yum install gcc python-devel openssl-devel libffi-devel python-pip -y
    2. pip install –upgrade pip
    3. pip install virtualenv
    4. cd /opt
    5. git clone https://github.com/Yelp/elastalert.git
    6. cd elastalert
    7. virtualenv virtualenvelastalert
    8. source virtualenvelastalert/bin/activate
    9. python setup.py install
    10. pip install -r requirements.txt
    11. elastalert-create-index
      1. Enter “127.0.0.1” for elasticsearch host
      2. Enter “9200” for elasticsearch port
      3. Enter “f” for SSL
      4. Leave username blank
      5. Leave password blank
      6. Leave URL prefix blank
      7. Leave index name as default
    12.  mkdir rules
    13. cp config.yaml.example config.yaml
    14. sed -i ‘s/rules_folder: example_rules/rules_folder: rules/g’ config.yaml
    15. sed -i ‘s/es_host: elasticsearch.example.com/es_host: 127.0.0.1/g’ config.yaml

    Configure Elastalert rules

    Elastalert rule for known malicious domains via Slack

    1. cat > rules/alert_malicious_domain.yaml << EOF
      <Elastalert appendix: Slack notification for known malicious domain>
      EOF

    Screen Shot 2017-05-07 at 8.07.17 PM.png

    Configure Elastalert SystemD

    1. useradd elastalert
    2. cat > /etc/systemd/system/elastalert.target << EOF
      <Appendix Elastalert: Elastalert systemD>
      EOF
    3. systemctl start elastalert

    Cobalt Strike unknown malicious domain test

    Create authoritative zone for “hackers.tech” on bind server

    1. cat > /var/named/db.hackers.tech << EOF
      $TTL    604800
      @       IN      SOA     hackers.tech. root.hackers.tech. (
      2         ; Serial
      604800         ; Refresh
      86400         ; Retry
      2419200         ; Expire
      604800 )       ; Negative Cache TTL
      ; Name servers
      hackers.tech.        IN        NS        hackers.tech.
      ; A records for name servers
      @                IN        A        10.140.100.105
      EOF

    2. echo “
      zone "hackers.tech" {
      type master;
      file "/var/named/db.hackers.tech";
      /etc/named.conf
    1. systemctl restart named

    Setup Cobalt Strike

    1. ./teamserver <IP of Kali Linux VM> <password>Screen Shot 2017-05-07 at 10.56.58 PM.png
    1. Open a new terminal tab
    2. ./cobaltstrike
      1. Enter “127.0.0.1” for host
      2. Leave port as default
      3. Enter a username into user
      4. Enter teamserver password into passwordScreen Shot 2017-05-05 at 1.28.56 AM.png
      5. Select connect
    3.  Select “Cobalt Strike” in the top left then select “Listeners”Screen Shot 2017-05-05 at 1.30.03 AM.png
    4. Select “Add” at the bottom
      1. Enter “dns_beacon” for name
      2. Select “windows/beacon_dns/reverse_dns_txt” for payload
      3. Leave host as default
      4. Enter “8080” for portScreen Shot 2017-05-07 at 10.58.28 PM.png
      5. Select “save”
    5. Enter “hackers.tech” for DNS callback domain
    6. Select “Attacks” then “Web Drive-by” then “Scripted Web Delivery”
      1. Enter “/dns” for URI path
      2. Leave localhost as default
      3. Leave port as default
      4. Select “dns_beacon_listener” for listener
      5. Select “powershell” for typeScreen Shot 2017-05-07 at 10.59.25 PM.png
      6. Select “connect”
    7. A box will pop-up with a powershell command inside. Copy and paste this text and select “Ok”

    Pawning our Windows 7 Client

    1. Open a Powershell prompt
    2. Paste the text from above and hit enter
      1. Yes, I know it’s the lazy method but this post is not about red teaming.

    Interact with our DNS beacon

    1. Go back to Cobalt Strike to confirm our beacon has called back successfully
    2. Right-click our beacon select “explore” then “process list”
      1. Now remember DNS beacons by default work on a 60 second interval
      2. Here we can see our beacon has returned results of the process listScreen Shot 2017-05-05 at 1.46.28 AM.png

    ELK results of DNS beacon

    The photo below shows the domains that have the least amount of queries. We can see that “hackers.tech: showed up in this section reinforcing the idea malicious DNS beacons will be among the least queried domains.

    Screen Shot 2017-05-07 at 11.09.43 PM.png

    Conclusion

    As was previously stated, my hypothesis is whether we can use open source intelligence feeds and tools like Bro to detect malicious DNS beacons. I built a network that utilizes Bro to monitor DNS traffic being transmitted and received. I used Criticalstack’s feeds with Bro to detect maliciously known domains such as “icloudhistory.org”. Once the data was analyzed by Bro it was shipped off to ELK for further analysis and alerting. We saw that Elastalert was used to create notifications when Bro detected a malicious domain being used. Furthermore, the ELK stack provides incident responders with actionable intelligene to respond too. The setup outlined above can be implemented in an enterprise environment looking for an open-source solution to threat intelligence.

    Resources/Sources

    Appendix

    Bind server

    Named.conf



    //
    // named.conf
    //
    // Provided by Red Hat bind package to configure the ISC BIND named(8) DNS
    // server as a caching only nameserver (as a localhost DNS resolver only).
    //
    // See /usr/share/doc/bind*/sample/ for example named configuration files.
    //
    // See the BIND Administrator's Reference Manual (ARM) for details about the
    // configuration located in /usr/share/doc/bind-{version}/Bv9ARM.html

    options {
    listen-on port 53 { any; };
    listen-on-v6 port 53 { ::1; };
    directory "/var/named";
    dump-file "/var/named/data/cache_dump.db";
    statistics-file "/var/named/data/named_stats.txt";
    memstatistics-file "/var/named/data/named_mem_stats.txt";
    allow-query     { any; };

    forwarders {
    8.8.8.8;
    8.8.4.4;
    };

    /*
    - If you are building an AUTHORITATIVE DNS server, do NOT enable recursion.
    - If you are building a RECURSIVE (caching) DNS server, you need to enable recursion.
    - If your recursive DNS server has a public IP address, you MUST enable access control to limit queries to your legitimate users. Failing to do so will cause your server to become part of large scale DNS amplification attacks. Implementing BCP38 within your network would greatlyreduce such attack surface
    */

    recursion yes;

    dnssec-enable yes;
    dnssec-validation yes;

    /* Path to ISC DLV key */
    bindkeys-file "/etc/named.iscdlv.key";
    managed-keys-directory "/var/named/dynamic";
    pid-file "/run/named/named.pid";
    session-keyfile "/run/named/session.key";
    };

    logging {
    channel default_debug {
    file "data/named.run";
    severity dynamic;
    };

    channel query.log {
    file "/var/named/query.log";
    severity debug 3;
    };

    category queries { query.log; };
    };

    zone "hackers.tech" {
    type master;
    file "/var/named/db.hackers.tech";
    };

    zone "." IN {
    type hint;
    file "named.ca";
    };

    include "/etc/named.rfc1912.zones";
    include "/etc/named.root.key";

    db.hackers.tech



    $TTL    604800
    @       IN      SOA     hackers.tech. root.hackers.tech. (

    2         ; Serial

    604800         ; Refresh

    86400         ; Retry

    2419200         ; Expire

    604800 )       ; Negative Cache TTL

    ; Name servers
    hackers.tech. IN NS hackers.tech.
    ; A records for name servers
    @ IN A 10.140.100.105

    Nginx

    Kibana.conf



    server {
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name _;
    return 301 https://;
    }

    server {

    listen 443 ssl;
    server_name _;

    root /usr/share/nginx/html;
    index index.html index.htm;

    ssl_certificate /etc/nginx/ssl/nginx.crt;
    ssl_certificate_key /etc/nginx/ssl/nginx.key;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_dhparam /etc/ssl/certs/dhparam.pem;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_session_timeout 1d;
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_stapling on;
    ssl_stapling_verify on;
    add_header Strict-Transport-Security max-age=15768000;

    auth_basic "Restricted";
    auth_basic_user_file /etc/nginx/htpasswd.users;

    location / {
    proxy_pass http://localhost:5601;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
    }
    }

    Logstash

    02-beats-input.conf



    input {
    beats {
    port = 5044
    ssl = false
    }
    }

    13-bro-filter.conf



    ############################ Bro Network Security Monitor filter ############################
    filter {
    if [message] =~ /^#/ {
    drop {  }
    }
    else {
    if [type] == "bro_dns" {
    json { source => "message" }
    }
    if [type] == "bro_intel" {
    json { source => "message" }
    }
    }
    }

    30-elasticsearch-output.conf



    output {
    elasticsearch {
    hosts = ["http://localhost:9200"]
    index = "%{type}-%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type = "%{[@metadata][type]}"
    }
    }

    Filebeat



    filebeat:
    registry_file: /var/lib/filebeat/registry
    config_dir: /etc/filebeat/conf.d

    output.logstash:
    hosts: ["172.25.1.252:5044"]

    Filbeat bro_dns.conf



    filebeat.prospectors:
    - paths:
    - /usr/local/bro/logs/current/dns.log
    document_type: bro_dns

    Filebeat bro_intel.conf



    filebeat.prospectors:
    - paths:
    - /usr/local/bro/logs/current/intel.log
    document_type: bro_intel

    Elastalert configs

    Elastalert systemD



    [Unit]
    Description=Elastalert
    After=elasticsearch.service

    [Service]
    Type=simple
    User=elastalert
    Group=elastalert
    Restart=on-failure
    ExecStart=/opt/elastalert/elastalert

    [Install]
    WantedBy=multi-user.target

    Slack notification for known malicious domain



    # (Required)
    # Rule name, must be unique
    name: Example frequency rule

    # (Required)
    # Type of alert.
    # the frequency rule type alerts when num_events events occur with timeframe time
    type: frequency

    # (Required)
    # Index to search, wildcard supported
    index: bro_intel-*

    # (Required, frequency specific)
    # Alert when this many documents matching the query occur within a timeframe
    num_events: 1

    # (Required, frequency specific)
    # num_events must occur within this amount of time to trigger an alert
    timeframe:
    minutes: 5

    # (Required)
    # A list of Elasticsearch filters used for find events
    # These filters are joined with AND and nested in a filtered query
    # For more info: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html
    filter:
    - term:
    _type: "bro_intel"

    # (Required)
    # The alert is use when a match is found
    alert:
    #- "email"
    - "slack"
    slack_webhook_url: ""

    # (required, email specific)
    # a list of email addresses to send alerts to
    #email:
    #- "[email protected]"

2 thoughts on “Intro to Threat Intelligence with Bro and ELK

  1. Mike says:

    Hi,

    I just wanted to take the time to say thank you for a great post. I got experience with most parts of this but the ELK bit really helps me to get a first idea of how to get that up and running.

    Having said that – all the other parts are equally as good.

    Many thanks for taking the time to get this together and sharing with the rest of the community.

    Cheers, Mike

  2. Janith Malinga says:

    Hi,

    This is a great blog post. This post helped me to understand most areas related to Bro-ELK.

    Thanks for sharing.

Leave a Reply

Your email address will not be published. Required fields are marked *