HoldMyBeer

If you have been following my blog you know that I am trying to increase my Incident Response(IR) skillz and experience. For a class project we had to create/improve a piece of software in the forensic community for Windows(Windows forensic class). From my short time of searching the internet I never found a guide to setting up a logging system for Windows from start to finsh. An effective logging system has an agent/collector, a log aggregator, a data visualizer, and a good alerting mechnism.

 

The following sytem I have setup has Wazuh(OSSEC fork) for log collection, Wazuh Management for a log aggregator, the ELK stack for data retention and vizualiztion, and elastalert for e-mail alerting. In this guide I will walk you through on how to setup an effective logging system for all operating systems but mainly Windows for free. Additionally, we will be discussing the type of things that should be logged depending on your enviornment. As final note I have included my github repo at the bottom if you want to automated scripts for all of this.

About/why Wazuh

Linux is without a doubt the easiest operating system for system administrators to administrate.

 

Wazuh components

• Wazuh HIDS: Performs log analysis, file integrity checking, policy monitoring, rootkits/malware detection and real-time alerting. The alerts are written in an extended JSON format, and stored locally on the box running as the OSSEC manager.
• Logstash: Is a data pipeline used for processing logs and other event data from a variety of systems. Logstash will read and process OSSEC JSON files, adding IP Geolocation information and modeling data before sending it to the Elasticsearch Cluster.
• Elasticsearch: Is the search engine used to index and store our OSSEC alerts. It can be deployed as a cluster, with multiple nodes, for better performance and data replication.
• Kibana: Kibana is a WEB framework used to explore all elasticsearch indexes. We will use it to analyze OSSEC alerts and to create custom dashboards for different use cases, including compliance regulations like PCI DSS or benchmarks like CIS.

 

Install/Setup Wazuh Manager

1. yum update -y && yum upgrade -y
2. yum install epel-release -y
3. yum install vim wget net-tools -y
4. yum install make gcc git
5. yum install openssl-devel
6. cd ~
7. mkdir ossec_tmp && cd ossec_tmp
8. git clone -b stable https://github.com/wazuh/wazuh.git ossec-wazuh
9. cd ossec-wazuh
10. sudo ./install.sh
    1. Enter “en” for english
       
       
    2. Enter “server” installation type
    3. Accept default location for ossec install
    4. Enter “n” for e-mail notification
    5. Enter “y” to run integrity check daemon
    6. Enter “y” to run rootkit detection
    7. Enter “y” to run active response
    8. Enter “n” to disable the firewall-drop response
    9. Enter “y” to add critical ip adrresses to servers and services
       1. The install should list your DNS servers. Be sure to add any additional server but I don’t have any in this network.
    10. Accept default port for remote syslog port
    11. Press “Enter” to build Wazuh manager from source
11. sudo /var/ossec/bin/ossec-control start
12. ps aux | grep ossec
13. /var/ossec/bin/manage_agent
    1. Enter “A” to add agent
    2. Enter a name for the new node
    3. Accept default id
    4. Enter “y” to confirm the new agent
    5. Enter “E” to extract a key for an agent
    6. Enter an agent id
    7. Copy the agent key information

 

Install/Setup Wazuh agent

Windows

1. Browse to “http://ossec.wazuh.com/windows/”
2. Download “ossec-win32-agent-*.exe”
3. Run installer to install the agent
4. Agent Manager
   1. Enter “<Wazuh management IP addr>” for ossec server ip
   2. Enter key for agent for authentication key
   3. Select “Save”
5. Select Manage > Restart
6. Select Manage > Exit

 

Ubuntu 14.04

1. sudo apt-key adv –fetch-keys http://ossec.wazuh.com/repos/apt/conf/ossec-key.gpg.key
2. sudo sh -c ‘echo -e “deb http://ossec.wazuh.com/repos/apt/ubuntu trusty main” >> /etc/apt/sources.list.d/ossec.list’
3. sudo apt-get update
4. sudo apt-get install ossec-hids-agent
   1. Enter Management node IP addr
5. sudo /var/ossec/bin/manage_agents
   1. Enter “I” to import key
   2. Enter the key from the management node
   3. Enter “y” to confirm adding the key
6. sudo /var/ossec/bin/ossec-control restart

 

CentOS

1. sudo echo ‘[wazuh] name = WAZUH OSSEC Repository – www.wazuh.com baseurl = http://ossec.wazuh.com/el/$releasever/$basearch gpgcheck = 1 gpgkey = http://ossec.wazuh.com/key/RPM-GPG-KEY-OSSEC enabled = 1
   ‘ | tee /etc/yum.repos.d/wazuh.repo
2. sudo yum install ossec-hids
3.  sudo /var/ossec/bin/manage_agents
   1. Enter “I” to import key
   2. Enter the key from the management node
   3. Enter “y” to confirm adding the key
4. sudo /var/ossec/bin/ossec-control restart

 

Adding new Wazuh agent

1. Go on to the management node
2. /var/ossec/bin/manage_agents

 

Install/Setup ELK stack

Install/Setup java

1. cd ~
2. wget –no-cookies –no-check-certificate –header “Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie” “http://download.oracle.com/otn-pub/java/jdk/8u60-b27/jdk-8u60-linux-x64.rpm“
3. sudo yum localinstall jdk-8u60-linux-x64.rpm
4. rm ~/jdk-8u60-linux-x64.rpm
5. export JAVA_HOME=/usr/java/jdk1.8.0_60/jre
6. Echo “export JAVA_HOME=/usr/java/jdk1.8.0_60/jre” >> /etc/profile

 

Install/Setup Logstash

1. sudo rpm –import https://packages.elasticsearch.org/GPG-KEY-elasticsearch
2. echo ‘[logstash-2.1]
   name=Logstash repository for 2.1.x packages
   baseurl=https://packages.elastic.co/logstash/2.1/centos
   gpgcheck=1
   gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
   enabled=1
   ‘ | sudo tee /etc/yum.repos.d/logstash.repo
3. sudo yum install logstash
4. cd ~
5. git clone https://github.com/wazuh/wazuh
6. sudo cp ~/ossec_tmp/ossec-wazuh/extensions/logstash/01-ossec-singlehost.conf /etc/logstash/conf.d/
7. sudo cp ~/ossec_tmp/ossec-wazuh/extensions/logstash/01-ossec-singlehost.conf /etc/logstash/conf.d/
8. sudo cp ~/ossec_tmp/ossec-wazuh/extensions/elasticsearch/elastic-ossec-template.json /etc/logstash/
9. sudo curl -O “http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz”
10. sudo gzip -d GeoLiteCity.dat.gz && sudo mv GeoLiteCity.dat /etc/logstash/
11. sudo usermod -a -G ossec logstash

 

Install/Setup Elasticsearch

1. sudo rpm –import http://packages.elastic.co/GPG-KEY-elasticsearch
2. echo ‘[elasticsearch-2.x]
   name=Elasticsearch repository for 2.x packages
   baseurl=http://packages.elastic.co/elasticsearch/2.x/centos
   gpgcheck=1
   gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
   enabled=1
   ‘ | sudo tee /etc/yum.repos.d/elasticsearch.repo
3. yum -y install elasticsearch
4. sed -i ‘s/# network.host: 192.168.0.1/network.host: localhost/g’ /etc/elasticsearch/elasticsearch.yml
5. sed -i ‘s/# cluster.name: my-application/cluster.name: ossec/g’ /etc/elasticsearch/elasticsearch.yml
6. sed -i ‘s/# node.name: node-1/node.name: ossec_node1/g’ /etc/elasticsearch/elasticsearch.yml
7. echo “index.number_of_shards: 1
   index.number_of_replicas: 0
   ” >> /etc/elasticsearch/elasticsearch.yml
8. sudo systemctl start elasticsearch
9. sudo systemctl enable elasticsearch
10. curl -XGET localhost:9200
11. curl -XGET ‘http://localhost:9200/_cluster/health?pretty=true’
12. cd ossec_tmp/ossec-wazuh/extensions/elasticsearch/ && curl -XPUT “http://localhost:9200/_template/ossec/” -d “@elastic-ossec-template.json”
13. systemctl start logstash

 

Install/Setup Kibana

1. sudo rpm –import https://packages.elastic.co/GPG-KEY-elasticsearch
2. echo ‘[kibana-4.4]
   name=Kibana repository for 4.4.x packages
   baseurl=http://packages.elastic.co/kibana/4.4/centos
   gpgcheck=1
   gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
   enabled=1
   ‘ | tee /etc/yum.repos.d/kibana.repo
3. yum -y install kibana
4. sed -i ‘s/# server.host: “0.0.0.0”/server.host: “localhost”/g’ /opt/kibana/config/kibana.yml
5. systemctl enable kibana
6. systemctl start kibana

 

Install/Setup Nginx and Let’s Encrypt

1. yum -y install epel-release
2. yum -y install nginx httpd-tools
3. yum install certbot -y
4. htpasswd -c /etc/nginx/htpasswd.users kibanaadmin
5. cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak
6. echo ‘# For more information on configuration, see:
   # * Official English Documentation: http://nginx.org/en/docs/
   # * Official Russian Documentation: http://nginx.org/ru/docs/user nginx;
   worker_processes auto;
   error_log /var/log/nginx/error.log;
   pid /run/nginx.pid;# Load dynamic modules. See /usr/share/nginx/README.dynamic.
   include /usr/share/nginx/modules/*.conf;events {
   worker_connections 1024;
   }http {
   log_format main ‘\$remote_addr – \$remote_user [\$time_local] “\$request” ‘
   ‘\$status \$body_bytes_sent “\$http_referer” ‘
   ‘”\$http_user_agent” “\$http_x_forwarded_for”‘;access_log /var/log/nginx/access.log main;sendfile on;
   tcp_nopush on;
   tcp_nodelay on;
   keepalive_timeout 65;
   types_hash_max_size 2048;include /etc/nginx/mime.types;
   default_type application/octet-stream;# Load modular configuration files from the /etc/nginx/conf.d directory.
   # See http://nginx.org/en/docs/ngx_core_module.html#include
   # for more information.
   include /etc/nginx/conf.d/*.conf;}
   ‘ | tee /etc/nginx/nginx.conf
7. echo ‘server {
   listen 80;
   location ~ /.well-known {
   allow all;
   }
   }
   ‘ | tee /etc/nginx/conf.d/letsencrypt.conf
8. systemctl start nginx
9. mkdir -p .well-known/acme-challenge
10. domain=”<domain>”
11. certbot certonly -a webroot –webroot-path=/usr/share/nginx/html -d $domain
12. openssl dhparam -out /etc/nginx/ssl/dhparam.pem 2048
13. rm -rf /etc/nginx/conf.d/letsencrypt.conf
14. echo “server {listen 443 ssl;server_name “$domain”;ssl_certificate /etc/letsencrypt/live/”$domain”/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/”$domain”/privkey.pem;
    ssl_dhparam /etc/nginx/ssl/dhparam.pem;ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_ciphers ‘EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH’;
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_stapling on;
    ssl_stapling_verify on;
    add_header Strict-Transport-Security max-age=15768000;location ~ /.well-known {
    allow all;
    }auth_basic ‘Restricted Access’;
    auth_basic_user_file /etc/nginx/htpasswd.users;location / {
    proxy_pass http://localhost:5601;
    proxy_http_version 1.1;
    proxy_set_header Upgrade \$http_upgrade;
    proxy_set_header Connection ‘upgrade’;
    proxy_set_header Host \$host;
    proxy_cache_bypass \$http_upgrade;
    }
    }
    ” | tee /etc/nginx/conf.d/kibana.conf
15. systemctl enable nginx
16. systemctl restart nginx
17. setsebool -P httpd_can_network_connect 1
18. Browser to “https://<kibana domain>”
    1. Select “Index contains time-based events”
    2. Enter “ossec-*” for Index name or pattern
    3. Enter “@timestamp” for time-field-name
    4. Select “Create”

Setup FirewallD

1. yum install firewalld -y
2. systemctl enable firewalld
3. systemctl start firewalld
4. firewall-cmd –zone=public –permanent –add-service=https
5. firewall-cmd –zone=public –permanent –add-service=ssh
6. firewall-cmd –permanent –zone=public –add-port=1514/udp
7. firewall-cmd –reload

 

Kibana Discover

1. Browser to “https://<kibana domain>”
2. Since I know the Wazuh Agent name I entered it into Kibana
   1. As you can see below within the past 12 hours I have had 12 events from this agent
   2. We can expand an event for more information

 

 

Setup Kibana Dashboards

We are going to create some simple dashboards to get your feet wet with the visualization power of Kibana. On my WindowsTestNode I have entered the incorrect password to create some events. If you search for “rule.groups:’windows, authentication_success'” in the discover tab we get two hits. But I want a counter of how many incorrect login.

1. Select “visualize”
2. Select “Metric” for “new visualization”.
3. Select “From a new search” for search source
4. Enter “rule.groups:’windows, authentication_success'” into search
5. Select the save icon in the top right
6. Enter a name for the new visualization and hit save
7. Select “Dashboard” at the top
8. Select “+” to add
9. Select the new visualization you just made
10. Select save in the top right and give the dashboard a name

 

Setup e-mail alerting with elastalert

Install/Setup Elastalert

1. cd /opt
2. yum install python-devel -y python-pip
3. git clone https://github.com/Yelp/elastalert.git
4. cd elastalert/
5. easy_install -U setuptools
6. python setup.py install
7. elastalert-create-index
   1. Enter “127.0.0.1” for Elasticsearch host
   2. Enter “9200” for Elasticsearch port
   3. Enter “f” for SSL
   4. Leave user name blank
   5. Leave password blank
   6. Leave prefix blank
   7. Leave index name as default
   8. Leave existing index blank

Setup ElastAlert and SystemD

1. vim /lib/systemd/system/elastalert.service
   1. Add
[Unit]
Description=elastalert
After=multi-user.target[Service]
      Type=simple
      WorkingDirectory=/opt/elastalert
      ExecStart=/usr/bin/elastalert

      [Install]
      WantedBy=multi-user.target

   2. Save, exit
2. systemctl enable elastalert
3. systemctl start elastalert
4. systemctl status elastalert
   1. You may get errors about multiple rules having the same name if you use the preexisting ruleset.

 

Setup e-mail notifications

1. yum remove sendmail -y
2. yum install postfix -y
3. postconf -e “mydomain = wazuh.student.rit.edu”
4. systemctl enable postfix
5. systemctl start postfix

 

Setup Slack notifications

1. Login into your slack account online
2. Then go to “https://<slack team>.slack.com/services/new/incoming-webhook “
3. Since it’s jsut me for right now selected the channel for my user
   1. Feel free to add your own channel and use that
4. Select “Add incming webhooks integration”
5. The next page will provide you with a webhook link and bunch of features for your webhook.

 

Automatic Wazuh ruleset updating

1. Log onto the OSSEC management node
2. sudo mkdir -p /var/ossec/update/ruleset && cd /var/ossec/update/ruleset
3. sudo wget https://raw.githubusercontent.com/wazuh/ossec-rules/stable/ossec_ruleset.py
4. sudo chmod +x /var/ossec/update/ruleset/ossec_ruleset.py
5. sudo /var/ossec/update/ruleset/ossec_ruleset.py –help
   1. Only run this command if you want to see all the options for the updater
6.  ./var/ossec/update/ruleset/ossec_ruleset.py
   1. Update decoders/rules/rootchecks
7. ./var/ossec/update/ruleset/ossec_ruleset.py -a
   1. Update and prompt menu to activate new Rules & Rootchecks:
8.  ./var/ossec/update/ruleset/ossec_ruleset.py –backups list
   1. restore a backup
9. ./var/ossec/update/ruleset/ossec_ruleset.py -a
   1. Actually install all rule sets
10. sudo crontab -e
    1. Add “@weekly root cd /var/ossec/update/ruleset && ./ossec_ruleset.py -s”
    2. save,exit

 

 

Resources/Sources

• https://github.com/Benster900/ossecKibanaElkonWindows-475-2161_bornholm
• http://documentation.wazuh.com/en/latest/about.html
• http://documentation.wazuh.com/en/latest/ossec_reference.html
• http://wazuh-documentation.readthedocs.io/en/latest/ossec_ruleset.html
• http://elastalert.readthedocs.io/en/latest/running_elastalert.html