Part 1: Install/Setup Wazuh with ELK Stack

If you have been following my blog you know that I am trying to increase my Incident Response(IR) skillz and experience. For a class project we had to create/improve a piece of software in the forensic community for Windows(Windows forensic class). From my short time of searching the internet I never found a guide to setting up a logging system for Windows from start to finsh. An effective logging system has an agent/collector, a log aggregator, a data visualizer, and a good alerting mechnism.

 

The following sytem I have setup has Wazuh(OSSEC fork) for log collection, Wazuh Management for a log aggregator, the ELK stack for data retention and vizualiztion, and elastalert for e-mail alerting. In this guide I will walk you through on how to setup an effective logging system for all operating systems but mainly Windows for free. Additionally, we will be discussing the type of things that should be logged depending on your enviornment. As final note I have included my github repo at the bottom if you want to automated scripts for all of this.

About/why Wazuh

Linux is without a doubt the easiest operating system for system administrators to administrate.

 

Wazuh components

  • Wazuh HIDS: Performs log analysis, file integrity checking, policy monitoring, rootkits/malware detection and real-time alerting. The alerts are written in an extended JSON format, and stored locally on the box running as the OSSEC manager.
  • Logstash: Is a data pipeline used for processing logs and other event data from a variety of systems. Logstash will read and process OSSEC JSON files, adding IP Geolocation information and modeling data before sending it to the Elasticsearch Cluster.
  • Elasticsearch: Is the search engine used to index and store our OSSEC alerts. It can be deployed as a cluster, with multiple nodes, for better performance and data replication.
  • Kibana: Kibana is a WEB framework used to explore all elasticsearch indexes. We will use it to analyze OSSEC alerts and to create custom dashboards for different use cases, including compliance regulations like PCI DSS or benchmarks like CIS.

 

Install/Setup Wazuh Manager

  1. yum update -y && yum upgrade -y
  2. yum install epel-release -y
  3. yum install vim wget net-tools -y
  4. yum install make gcc git
  5. yum install openssl-devel
  6. cd ~
  7. mkdir ossec_tmp && cd ossec_tmp
  8. git clone -b stable https://github.com/wazuh/wazuh.git ossec-wazuh
  9. cd ossec-wazuh
  10. sudo ./install.sh
    1. Enter “en” for english
      screen-shot-2016-11-26-at-3-14-08-pm
    2. Enter “server” installation typescreen-shot-2016-11-26-at-3-14-22-pm
    3. Accept default location for ossec install
    4. Enter “n” for e-mail notification
    5. Enter “y” to run integrity check daemon
    6. Enter “y” to run rootkit detection
    7. Enter “y” to run active response
    8. Enter “n” to disable the firewall-drop responsescreen-shot-2016-11-26-at-3-14-46-pm
    9. Enter “y” to add critical ip adrresses to servers and services
      1. The install should list your DNS servers. Be sure to add any additional server but I don’t have any in this network.
    10. Accept default port for remote syslog portscreen-shot-2016-11-26-at-3-23-15-pm
    11. Press “Enter” to build Wazuh manager from source
  11. sudo /var/ossec/bin/ossec-control start
  12. ps aux | grep ossecscreen-shot-2016-11-26-at-3-26-04-pm
  13. /var/ossec/bin/manage_agent
    1. Enter “A” to add agent
    2. Enter a name for the new node
    3. Accept default id
    4. Enter “y” to confirm the new agentscreen-shot-2016-11-26-at-3-42-05-pm
    5. Enter “E” to extract a key for an agent
    6. Enter an agent id
    7. Copy the agent key informationscreen-shot-2016-11-26-at-3-42-21-pm

 

Install/Setup Wazuh agent

Windows

  1. Browse to “http://ossec.wazuh.com/windows/”
  2. Download “ossec-win32-agent-*.exe”
  3. Run installer to install the agent
  4. Agent Manager
    1. Enter “<Wazuh management IP addr>” for ossec server ip
    2. Enter key for agent for authentication keyscreen-shot-2016-11-26-at-4-32-57-pm
    3. Select “Save”
  5. Select Manage > Restart
  6. Select Manage > Exit

 

Ubuntu 14.04

  1. sudo apt-key adv –fetch-keys http://ossec.wazuh.com/repos/apt/conf/ossec-key.gpg.key
  2. sudo sh -c ‘echo -e “deb http://ossec.wazuh.com/repos/apt/ubuntu trusty main” >> /etc/apt/sources.list.d/ossec.list’
  3. sudo apt-get update
  4. sudo apt-get install ossec-hids-agent
    1. Enter Management node IP addr
  5. sudo /var/ossec/bin/manage_agents
    1. Enter “I” to import key
    2. Enter the key from the management node
    3. Enter “y” to confirm adding the keyscreen-shot-2016-11-26-at-3-45-34-pm
  6. sudo /var/ossec/bin/ossec-control restart

 

CentOS

  1. sudo echo ‘[wazuh] name = WAZUH OSSEC Repository – www.wazuh.com baseurl = http://ossec.wazuh.com/el/$releasever/$basearch gpgcheck = 1 gpgkey = http://ossec.wazuh.com/key/RPM-GPG-KEY-OSSEC enabled = 1
    ‘ | tee /etc/yum.repos.d/wazuh.repo
  2. sudo yum install ossec-hids
  3.  sudo /var/ossec/bin/manage_agents
    1. Enter “I” to import key
    2. Enter the key from the management node
    3. Enter “y” to confirm adding the keyscreen-shot-2016-11-26-at-4-23-12-pm
  4. sudo /var/ossec/bin/ossec-control restart

 

Adding new Wazuh agent

  1. Go on to the management node
  2. /var/ossec/bin/manage_agents

 

Install/Setup ELK stack

Install/Setup java

  1. cd ~
  2. wget –no-cookies –no-check-certificate –header “Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie” “http://download.oracle.com/otn-pub/java/jdk/8u60-b27/jdk-8u60-linux-x64.rpm
  3. sudo yum localinstall jdk-8u60-linux-x64.rpm
  4. rm ~/jdk-8u60-linux-x64.rpm
  5. export JAVA_HOME=/usr/java/jdk1.8.0_60/jre
  6. Echo “export JAVA_HOME=/usr/java/jdk1.8.0_60/jre” >> /etc/profile

 

Install/Setup Logstash

  1. sudo rpm –import https://packages.elasticsearch.org/GPG-KEY-elasticsearch
  2. echo ‘[logstash-2.1]
    name=Logstash repository for 2.1.x packages
    baseurl=https://packages.elastic.co/logstash/2.1/centos
    gpgcheck=1
    gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
    enabled=1
    ‘ | sudo tee /etc/yum.repos.d/logstash.repo
  3. sudo yum install logstash
  4. cd ~
  5. git clone https://github.com/wazuh/wazuh
  6. sudo cp ~/ossec_tmp/ossec-wazuh/extensions/logstash/01-ossec-singlehost.conf /etc/logstash/conf.d/
  7. sudo cp ~/ossec_tmp/ossec-wazuh/extensions/logstash/01-ossec-singlehost.conf /etc/logstash/conf.d/
  8. sudo cp ~/ossec_tmp/ossec-wazuh/extensions/elasticsearch/elastic-ossec-template.json /etc/logstash/
  9. sudo curl -O “http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz”
  10. sudo gzip -d GeoLiteCity.dat.gz && sudo mv GeoLiteCity.dat /etc/logstash/
  11. sudo usermod -a -G ossec logstash

 

Install/Setup Elasticsearch

  1. sudo rpm –import http://packages.elastic.co/GPG-KEY-elasticsearch
  2. echo ‘[elasticsearch-2.x]
    name=Elasticsearch repository for 2.x packages
    baseurl=http://packages.elastic.co/elasticsearch/2.x/centos
    gpgcheck=1
    gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
    enabled=1
    ‘ | sudo tee /etc/yum.repos.d/elasticsearch.repo
  3. yum -y install elasticsearch
  4. sed -i ‘s/# network.host: 192.168.0.1/network.host: localhost/g’ /etc/elasticsearch/elasticsearch.yml
  5. sed -i ‘s/# cluster.name: my-application/cluster.name: ossec/g’ /etc/elasticsearch/elasticsearch.yml
  6. sed -i ‘s/# node.name: node-1/node.name: ossec_node1/g’ /etc/elasticsearch/elasticsearch.yml
  7. echo “index.number_of_shards: 1
    index.number_of_replicas: 0
    ” >> /etc/elasticsearch/elasticsearch.yml
  8. sudo systemctl start elasticsearch
  9. sudo systemctl enable elasticsearch
  10. curl -XGET localhost:9200
  11. curl -XGET ‘http://localhost:9200/_cluster/health?pretty=true’
  12. cd ossec_tmp/ossec-wazuh/extensions/elasticsearch/ && curl -XPUT “http://localhost:9200/_template/ossec/” -d “@elastic-ossec-template.json”
  13. systemctl start logstash

 

Install/Setup Kibana

  1. sudo rpm –import https://packages.elastic.co/GPG-KEY-elasticsearch
  2. echo ‘[kibana-4.4]
    name=Kibana repository for 4.4.x packages
    baseurl=http://packages.elastic.co/kibana/4.4/centos
    gpgcheck=1
    gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
    enabled=1
    ‘ | tee /etc/yum.repos.d/kibana.repo
  3. yum -y install kibana
  4. sed -i ‘s/# server.host: “0.0.0.0”/server.host: “localhost”/g’ /opt/kibana/config/kibana.yml
  5. systemctl enable kibana
  6. systemctl start kibana

 

Install/Setup Nginx and Let’s Encrypt

  1. yum -y install epel-release
  2. yum -y install nginx httpd-tools
  3. yum install certbot -y
  4. htpasswd -c /etc/nginx/htpasswd.users kibanaadmin
  5. cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak
  6. echo ‘# For more information on configuration, see:
    # * Official English Documentation: http://nginx.org/en/docs/
    # * Official Russian Documentation: http://nginx.org/ru/docs/user nginx;
    worker_processes auto;
    error_log /var/log/nginx/error.log;
    pid /run/nginx.pid;# Load dynamic modules. See /usr/share/nginx/README.dynamic.
    include /usr/share/nginx/modules/*.conf;events {
    worker_connections 1024;
    }http {
    log_format main ‘\$remote_addr – \$remote_user [\$time_local] “\$request” ‘
    ‘\$status \$body_bytes_sent “\$http_referer” ‘
    ‘”\$http_user_agent” “\$http_x_forwarded_for”‘;access_log /var/log/nginx/access.log main;sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;include /etc/nginx/mime.types;
    default_type application/octet-stream;# Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;}
    ‘ | tee /etc/nginx/nginx.conf
  7. echo ‘server {
    listen 80;
    location ~ /.well-known {
    allow all;
    }
    }
    ‘ | tee /etc/nginx/conf.d/letsencrypt.conf
  8. systemctl start nginx
  9. mkdir -p .well-known/acme-challenge
  10. domain=”<domain>”
  11. certbot certonly -a webroot –webroot-path=/usr/share/nginx/html -d $domain
  12. openssl dhparam -out /etc/nginx/ssl/dhparam.pem 2048
  13. rm -rf /etc/nginx/conf.d/letsencrypt.conf
  14. echo “server {listen 443 ssl;server_name “$domain”;ssl_certificate /etc/letsencrypt/live/”$domain”/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/”$domain”/privkey.pem;
    ssl_dhparam /etc/nginx/ssl/dhparam.pem;ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_ciphers ‘EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH’;
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_stapling on;
    ssl_stapling_verify on;
    add_header Strict-Transport-Security max-age=15768000;location ~ /.well-known {
    allow all;
    }auth_basic ‘Restricted Access’;
    auth_basic_user_file /etc/nginx/htpasswd.users;location / {
    proxy_pass http://localhost:5601;
    proxy_http_version 1.1;
    proxy_set_header Upgrade \$http_upgrade;
    proxy_set_header Connection ‘upgrade’;
    proxy_set_header Host \$host;
    proxy_cache_bypass \$http_upgrade;
    }
    }
    ” | tee /etc/nginx/conf.d/kibana.conf
  15. systemctl enable nginx
  16. systemctl restart nginx
  17. setsebool -P httpd_can_network_connect 1
  18. Browser to “https://<kibana domain>”
    1. Select “Index contains time-based events”
    2. Enter “ossec-*” for Index name or pattern
    3. Enter “@timestamp” for time-field-namescreen-shot-2016-11-26-at-4-14-31-pm
    4. Select “Create”

Setup FirewallD

  1. yum install firewalld -y
  2. systemctl enable firewalld
  3. systemctl start firewalld
  4. firewall-cmd –zone=public –permanent –add-service=https
  5. firewall-cmd –zone=public –permanent –add-service=ssh
  6. firewall-cmd –permanent –zone=public –add-port=1514/udp
  7. firewall-cmd –reload

 

Kibana Discover

  1. Browser to “https://<kibana domain>”
  2. Since I know the Wazuh Agent name I entered it into Kibana
    1. As you can see below within the past 12 hours I have had 12 events from this agentscreen-shot-2016-11-26-at-5-34-50-pm
    2. We can expand an event for more informationscreen-shot-2016-11-26-at-5-36-40-pm

 

 

Setup Kibana Dashboards

We are going to create some simple dashboards to get your feet wet with the visualization power of Kibana. On my WindowsTestNode I have entered the incorrect password to create some events. If you search for “rule.groups:’windows, authentication_success'” in the discover tab we get two hits. But I want a counter of how many incorrect login.

  1. Select “visualize”
  2. Select “Metric” for “new visualization”.
  3. Select “From a new search” for search source
  4. Enter “rule.groups:’windows, authentication_success'” into searchscreen-shot-2016-11-26-at-5-49-45-pm
  5. Select the save icon in the top right
  6. Enter a name for the new visualization and hit savescreen-shot-2016-11-26-at-5-50-52-pm
  7. Select “Dashboard” at the top
  8. Select “+” to add
  9. Select the new visualization you just madescreen-shot-2016-11-26-at-5-58-45-pm
  10. Select save in the top right and give the dashboard a namescreen-shot-2016-11-26-at-6-00-52-pm

 

Setup e-mail alerting with elastalert

Install/Setup Elastalert

  1. cd /opt
  2. yum install python-devel -y python-pip
  3. git clone https://github.com/Yelp/elastalert.git
  4. cd elastalert/
  5. easy_install -U setuptools
  6. python setup.py install
  7. elastalert-create-index
    1. Enter “127.0.0.1” for Elasticsearch host
    2. Enter “9200” for Elasticsearch port
    3. Enter “f” for SSL
    4. Leave user name blank
    5. Leave password blank
    6. Leave prefix blank
    7. Leave index name as default
    8. Leave existing index blankscreen-shot-2016-11-27-at-12-26-39-am

Setup ElastAlert and SystemD

  1. vim /lib/systemd/system/elastalert.service
    1. Add
      [Unit]
      Description=elastalert
      After=multi-user.target
      [Service]
      Type=simple
      WorkingDirectory=/opt/elastalert
      ExecStart=/usr/bin/elastalert

      [Install]
      WantedBy=multi-user.target

    2. Save, exit
  2. systemctl enable elastalert
  3. systemctl start elastalert
  4. systemctl status elastalert
    1. You may get errors about multiple rules having the same name if you use the preexisting ruleset.

 

Setup e-mail notifications

  1. yum remove sendmail -y
  2. yum install postfix -y
  3. postconf -e “mydomain = wazuh.student.rit.edu”
  4. systemctl enable postfix
  5. systemctl start postfix

 

Setup Slack notifications

  1. Login into your slack account online
  2. Then go to “https://<slack team>.slack.com/services/new/incoming-webhook “screen-shot-2016-11-27-at-12-43-38-am
  3. Since it’s jsut me for right now selected the channel for my user
    1. Feel free to add your own channel and use that
  4. Select “Add incming webhooks integration”
  5. The next page will provide you with a webhook link and bunch of features for your webhook.

 

Automatic Wazuh ruleset updating

  1. Log onto the OSSEC management node
  2. sudo mkdir -p /var/ossec/update/ruleset && cd /var/ossec/update/ruleset
  3. sudo wget https://raw.githubusercontent.com/wazuh/ossec-rules/stable/ossec_ruleset.py
  4. sudo chmod +x /var/ossec/update/ruleset/ossec_ruleset.py
  5. sudo /var/ossec/update/ruleset/ossec_ruleset.py –help
    1. Only run this command if you want to see all the options for the updater
  6.  ./var/ossec/update/ruleset/ossec_ruleset.py
    1. Update decoders/rules/rootchecks
  7. ./var/ossec/update/ruleset/ossec_ruleset.py -a
    1. Update and prompt menu to activate new Rules & Rootchecks:
  8.  ./var/ossec/update/ruleset/ossec_ruleset.py –backups list
    1. restore a backup
  9. ./var/ossec/update/ruleset/ossec_ruleset.py -a
    1. Actually install all rule sets
  10. sudo crontab -e
    1. Add “@weekly root cd /var/ossec/update/ruleset && ./ossec_ruleset.py -s”
    2. save,exit

 

 

Resources/Sources

  • https://github.com/Benster900/ossecKibanaElkonWindows-475-2161_bornholm
  • http://documentation.wazuh.com/en/latest/about.html
  • http://documentation.wazuh.com/en/latest/ossec_reference.html
  • http://wazuh-documentation.readthedocs.io/en/latest/ossec_ruleset.html
  • http://elastalert.readthedocs.io/en/latest/running_elastalert.html

3 thoughts on “Part 1: Install/Setup Wazuh with ELK Stack

  1. infolookup says:

    The code block above for sed as well as downloading java via the terminal came back with an error. I did it a GUI XFCE on the server and installed it that way.

  2. I’m excited to uncover this page. I want to to thank you for ones time for this wonderful read!!
    I definitely really liked every little bit of it and i also have you saved to fav to
    look at new information on your website.

  3. Lee says:

    Stumbled upon this article after running into issues with trying to get Wazuh on CentOS as a unRAID VM. Love the name of your blog/site! Thank you for sharing. #NEIPA #IPA #HAZYASF$*#

Leave a Reply

Your email address will not be published.