In this post I am going to explore the tool OSquery. OSquery allows you to easily ask questions about your Linux, Windows, and macOS infrastructure. Whether your goal is intrusion detection, infrastructure reliability, or compliance, OSquery gives you the ability to empower and inform a broad set of organizations within your company. It is a tool that is used by system administrators, incident responders, and ole mighty threat hunters. However, in this post I will not be posting how to use OSquery for threat hunting. I hope to utilize the tool in my environment and write a later post :).

Doorman/OSquery components and terms

Terms

• Node– A single machine
• Fleet – All the machines controlled and owned by an enterprise
• Queries – A query runs a set of tasks on fleet of machines on a specified interval
• Distributed – An on the fly query
• Packs – OSquery query packs are groups of queries to be added to the OSquery schedule

Install/Setup Doorman Remote API server

Install/Setup Doorman on CentOS 7 64-bit with Docker

Install/Setup NTPd on Centos

1. yum install ntp ntpdate ntp-doc -y
2. systemctl enable ntpd
3. systemctl start ntpd
4. ntpdate pool.ntp.org || true

Install/Setup Postgres and Redis

Install/Setup Postgres database

1. yum update -y && yum upgrade -y
2. rpm -Uvh https://yum.postgresql.org/9.6/redhat/rhel-7-x86_64/pgdg-centos96-9.6-3.noarch.rpm
3. yum install postgresql96-server postgresql96 -y
   1. MUST install Postgresql 9.4 or later
4. /usr/pgsql-9.6/bin/postgresql96-setup initdb
5. sed -i 's#host all all 127.0.0.1/32 ident#host all all 127.0.0.1/32 md5#'g /var/lib/pgsql/9.6/data/pg_hba.conf
6. systemctl enable postgresql-9.6.service
7. systemctl start postgresql-9.6.service
8. su – postgres
9.  psql
   1. CREATE ROLE doorman WITH LOGIN PASSWORD ‘<password>’;
      1. password can NOT contain “@” or “#”
   2. CREATE DATABASE doorman;
   3. ALTER DATABASE doorman OWNER TO doorman;
   4. GRANT ALL PRIVILEGES ON DATABASE doorman TO doorman;
   5. \q
10. exit
11. psql -U doorman -h 127.0.0.1 -d doorman -W
    1. Test to make sure you can connect as doorman user on postgres
12. useradd doorman

Install/Setup Redis

1. yum install redis -y
2. systemctl enable redis
3. systemctl start redis

Install/Setup Doorman

1. yum install python-pip python-devel libffi-devel gcc postgresql-devel npm -y
   1. For Centos 7.3: rpm -ivh https://kojipkgs.fedoraproject.org//packages/http-parser/2.7.1/3.el7/x86_64/http-parser-2.7.1-3.el7.x86_64.rpm && yum -y install nodejs
2. pip install –upgrade pip
3. cd /opt
4. git clone https://github.com/mwielgoszewski/doorman.git
5. cd doorman
6. pip install virtualenv
7. virtualenv env
8. source env/bin/activate
9. pip install -r requirements.txt
10. chown doorman:doorman -R /opt/doorman
11. vim doorman/settings.py
    1. scroll to “class ProdConfig(Config):”
SQLALCHEMY_DATABASE_URI = 'postgresql://doorman:<doorman password>@127.0.0.1:5432/doorman'
DOORMAN_ENROLL_SECRET = ['<randomly generated secret key>']
BROKER_URL = 'redis://localhost:6379/0'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
12. mkdir /var/log/doorman
13. chown doorman:doorman -R /var/log/doorman
14. export DOORMAN_ENV=prod
    1. Set this variable in /etc/profile to be permanent
15. su – doorman -c “cd /opt/doorman; source env/bin/activate; python manage.py db upgrade”
16. npm install bower -g
17. bower install
18. npm install -g less

Install/Setup Nginx + WSGI/Flask + OpenSSL

Install/Setup Nginx and OpennSSL

1. yum install nginx -y
2. mkdir /etc/nginx/ssl
3. openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/ssl/private.key -out /etc/nginx/ssl/certificate.crt
4. sed -i -e ‘38,87d’ /etc/nginx/nginx.conf
5. cat > /etc/nginx/conf.d/osquery.conf <<\EOF
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
return 301 https://$host$request_uri;
}server {
listen 443 ssl;
server_name _;ssl_certificate /etc/nginx/ssl/certificate.crt;
ssl_certificate_key /etc/nginx/ssl/private.key;ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!MD5;location / {
include uwsgi_params;
uwsgi_pass unix:/opt/doorman/doorman.sock;
}
}
EOF


Install/Setup WSGI and Flask

1. cd /opt/doorman
2. pip install uwsgi flask
3. cat > doorman.ini << EOF
[uwsgi]
master = true
processes = 5
home = env
wsgi-file = manage.py
callable = app
socket = doorman.sock
chmod-socket = 660
vacuum = true
die-on-term = true
smart-attach-daemon = /opt/doorman/celery.pid celery worker -A doorman.worker:celery --pidfile=/opt/doorman/celery.pid
env = DOORMAN_ENV=prod
EOF

4. cat > /etc/systemd/system/doorman.service << EOF
[Unit]
Description=uWSGI instance to serve Doorman
After=network.target[Service]
User=doorman
Group=nginx
WorkingDirectory=/opt/doorman
Environment="PATH=/opt/doorman/env/bin:/usr/bin"
ExecStart=/opt/doorman/env/bin/uwsgi --ini doorman.ini[Install]
WantedBy=multi-user.target
EOF

5. systemctl enable doorman
6. systemctl start doorman
7. systemctl enable nginx
8. systemctl start nginx
9. setsebool httpd_can_network_connect 1 -P

Install/Setup FirewallD

1. yum install firewalld -y
2. systemctl start firewalld
3. systemctl enable firewalld
4. firewall-cmd –zone=public –permanent –add-service=http
5. firewall-cmd –zone=public –permanent –add-service=https
6. firewall-cmd –zone=public –permanent –add-service=ssh
7. firewall-cmd –reload

Accessing Doorman

1. Browse to “https://<IP addr of doorman>/manage

Install/Setup OSQuery on Linux, Mac, and Windows

Install/Setup OSQuery on CentOS 7 Server 64-bit

1. yum update -y && yum upgrade -y
2. yum install yum-utils -y
3. curl https://s3.amazonaws.com/osquery-packages/rpm/RPM-GPG-KEY-osquery | sudo tee /etc/pki/rpm-gpg/RPM-GPG-KEY-osquery
4. yum install install yum-utils -y
5. yum-config-manager –add-repo https://s3.amazonaws.com/osquery-packages/rpm/osquery-s3-rpm.repo
6. yum-config-manager –enable osquery-s3-rpm
7. yum install osquery -y
8. openssl s_client -showcerts -connect <doorman IP addr>:443 </dev/null 2>/dev/null|openssl x509 -outform PEM >/etc/osquery/certificate.crt
9. cat > /etc/osquery/osquery.flags << 'EOF'
--host_identifier=uuid
--config_plugin=tls
--config_tls_endpoint=/config
--config_tls_refresh=10
--config_tls_max_attempts=3
--enroll_tls_endpoint=/enroll
--enroll_secret_path=/etc/osquery/osquery.key
--disable_distributed=false
--distributed_plugin=tls
--distributed_interval=10
--distributed_tls_max_attempts=3
--distributed_tls_read_endpoint=/distributed/read
--distributed_tls_write_endpoint=/distributed/write
--logger_plugin=tls
--logger_tls_endpoint=/log
--logger_tls_period=5
--tls_hostname=<doorman IP addr>:443
--tls_server_certs=/etc/osquery/certificate.crt
--log_result_events=false
--pack_delimiter=/
--utc
--verbose
EOF

10. cat > /etc/osquery/osquery.key << 'EOF'
<randomly generated secret key for Doorman>
EOF

11. systemctl enable osqueryd
12. systemctl start osqueryd
13. Browse to “https://<doorman IP addr>:443/manage/nodes” to confirm node was added

Install/Setup OSQuery on Ubuntu 16.04 64-bit

1. sudo apt-key adv –keyserver keyserver.ubuntu.com –recv-keys 1484120AC4E9F8A1A577AEEE97A80C63C9D8B80B
2. sudo add-apt-repository “deb [arch=amd64] https://osquery-packages.s3.amazonaws.com/deb deb main”
3. sudo apt-get update -y
4. sudo apt-get install osquery -y
5. openssl s_client -showcerts -connect <doorman IP addr>:443 </dev/null 2>/dev/null|openssl x509 -outform PEM | sudo tee /etc/osquery/certificate.crt
6. cat << EOF | sudo tee/etc/osquery/osquery.flags
--host_identifier=uuid
--config_plugin=tls
--config_tls_endpoint=/config
--config_tls_refresh=10
--config_tls_max_attempts=3
--enroll_tls_endpoint=/enroll
--enroll_secret_path=/etc/osquery/osquery.key
--disable_distributed=false
--distributed_plugin=tls
--distributed_interval=10
--distributed_tls_max_attempts=3
--distributed_tls_read_endpoint=/distributed/read
--distributed_tls_write_endpoint=/distributed/write
--logger_plugin=tls
--logger_tls_endpoint=/log
--logger_tls_period=5
--tls_hostname=<doorman IP addr>:443
--tls_server_certs=/etc/osquery/certificate.crt
--log_result_events=false
--pack_delimiter=/
--utc
--verbose
EOF

7. cat << EOF | sudo tee /etc/osquery/osquery.key
<randomly generated secret key for Doorman>
EOF

8. sudo systemctl enable osqueryd
9. sudo systemctl start osqueryd

Install/Setup OSQuery on Mac OSX

1. brew update
2. brew install osquery
3. openssl s_client -showcerts -connect <doorman IP addr>:443 </dev/null 2>/dev/null|openssl x509 -outform PEM | sudo tee /var/osquery/certificate.crt
4. rm -rf /var/osquery/osquery.example.conf
5. cat << EOF | sudo tee /var/osquery/osquery.flags
--host_identifier=uuid
--config_plugin=tls
--config_tls_endpoint=/config
--config_tls_refresh=10
--config_tls_max_attempts=3
--enroll_tls_endpoint=/enroll
--enroll_secret_path=/var/osquery/osquery.key
--disable_distributed=false
--distributed_plugin=tls
--distributed_interval=10
--distributed_tls_max_attempts=3
--distributed_tls_read_endpoint=/distributed/read
--distributed_tls_write_endpoint=/distributed/write
--logger_plugin=tls
--logger_tls_endpoint=/log
--logger_tls_period=5
--tls_hostname=<doorman IP addr>:443
--tls_server_certs=/var/osquery/certificate.crt
--log_result_events=false
--pack_delimiter=/
--utc
--verbose
EOF

6. cat << EOF | sudo tee /var/osquery/osquery.key
<randomly generated secret key for Doorman>
EOF

7. sudo cp /var/osquery/com.facebook.osqueryd.plist /Library/LaunchDaemons/
8. sudo launchctl load /Library/LaunchDaemons/com.facebook.osqueryd.plist
9. sudo launchctl start /Library/LaunchDaemons/com.facebook.osqueryd.plist

Install/Setup OSQuery on Windows

Install/Setup Choclately the package manager for Windows

1. Open Powershell as an Administrator
2. Set-ExecutionPolicy RemoteSigned
3. Copy certificate.pem from Doorman server to Windows
4. Copy and Paste: iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
5. Close Powershell and re-open as Administrator
6. choco install osquery --params='/InstallService'
   1. This will install OSquery as a Windows service
7. $config = "
--host_identifier=uuid`r`n
--config_plugin=tls`r`n
--config_tls_endpoint=/config`r`n
--config_tls_refresh=10`r`n
--config_tls_max_attempts=3`r`n
--enroll_tls_endpoint=/enroll`r`n
--enroll_secret_path=C:\ProgramData\osquery\osquery.key`r`n
--disable_distributed=false`r`n
--distributed_plugin=tls`r`n
--distributed_interval=10`r`n
--distributed_tls_max_attempts=3`r`n
--distributed_tls_read_endpoint=/distributed/read`r`n
--distributed_tls_write_endpoint=/distributed/write`r`n
--logger_plugin=tls`r`n
--logger_tls_endpoint=/log`r`n
--logger_tls_period=5`r`n
--tls_hostname=<doorman IP addr>:443`r`n
--tls_server_certs=C:\ProgramData\osquery\certificate.crt`r`n
--log_result_events=false`r`n
--pack_delimiter=/`r`n
--utc`r`n
--verbose`r`n"

8. $config | Out-File -FilePath C:\Program Data\osquery\osquery.flags
9. Start-service osqueryd
10. Get-Service | Where-Object {$_.name -eq “osqueryd”}

Setup/Configure/Secure Doorman

Setup LDAP/local user authentication

Local authentication

1. cd /opt/doorman
2. vim doorman/settings.py
   1. Add DOORMAN_AUTH_METHOD = 'doorman' to “Class ProcConfig():” section
   2. save, exit
3. systemctl restart doorman
4. python manage adduser –email [email protected] test
   1. Enter password for user
5. Browse to “http://<DOORMAN IP addr>/manage”
6. Enter login credentials from above and select “Login”

LDAP authentication

1. cd /opt/doorman
2. vim doorman/settings.py
   1. Add DOORMAN_AUTH_METHOD = 'ldap' to “Class ProcConfig():” section
   2. Then scroll up to the LDAP section and set your settings. The settings below are a basic setup for Freeipa
      
LDAP_HOST = '<hostname of FreeIPA>'
LDAP_PORT = 636
LDAP_USE_SSL = True
LDAP_BASE_DN = 'cn=users,cn=accounts,dc=<domain, example>,dc=<tld, com>'

   3. save, exit
3. systemctl restart doorman
4. Browse to “http://<DOORMAN IP addr>/manage”
5. Enter LDAP credentials and select “Login”

Add OSQuery packs

Setup new pack

1. Browse to “https://github.com/facebook/osquery/tree/master/packs”
2. For our example we will install the “hardware-monitoring.conf” pack
3. Download the hardware-monitoring.conf
4. Login into Doorman and select “Packs” at the top
5. Select “Choose file” and select the pack on disk
6. Select “Update Query Pack”

 

Doorman scanning

Distributed scans

1. Login into Doorman and select “Add” then “Distributed”
2. Enter “SELECT uid, name FROM listening_ports l, processes p WHERE l.pid=p.pid; into Query
   1. For more information about queries look here
3. Select specific nodes in the node section
4. Select specific tags to scan a set of nodes with a particular tag
   1. If you select nothing from above it will scan everything
5. Select “Add distributed query”

Interval scans

1. Select “Add” then “Query”
2. Enter “Get all listening ports” for name
3. Enter “select * from listening_ports”
4. Enter “3600” for interval
   1. The interval is in seconds
5. Select “All” for platforms
6. Select a hardware pack to run but for this scan we will not
7. Select specific tags to scan a set of nodes with a particular tag
8. Select “Add query”

 

Resources/Sources

• OSQuery homepage
• OSQuery Documentation
• Doorman Github
• Getting Started with PostgreSQL