In this blog post series, I am documenting my novice pursuit for knowledge to become a threat hunter. This series has a strong focus on utilizing OSQuery to hunt for different techniques used by adversaries. The first post of this series, will consist of instructions to set up a lab environment. The second post of this series will be walking through red team scenarios with Powershell Empire, and MITRE’s cyber adversary emulation system called Caldera.
The third post in this series, will utilize OSQuery to hunt red team activity by discovering artifacts and modifications made to the system. The fourth post in this series, will integrate our successful hunts into Graylog for logging and alerting. Lastly, this series will be generating threat-hunting playbooks and scoring our detection capabilities using Cyb3rWard0g’s system.
Requirements
- Hypervisor with the following hardware:
- CPU: Quad-core
- Ram: 16GBs
- HDD: 160GBs
- Some Ansible experience
- Some Linux experience
- Some Windows Group policy and server system administration experience
- ISOs
- Ubuntu Server 16.04 64-bit
- Windows 10 64-bit 1511
Network diagram
Setup environment
VM creation
- Windows Server 2016
- Services: AD + DNS, WinRM, RDP
- Description: This server will be used as a domain controller for central auth and Group Policy
- Hardware
- CPU: 2 cores
- RAM: 4GBs
- HDD: 40GBs
- Windows 10 client x2
- Version: 1511
- Some of the red teaming tactics require an older version that is vulnerable 🙂
- Some of the red teaming tactics require an older version that is vulnerable 🙂
- Services: WinRM
- Description: Client computers that will be attacked by Powershell Empire and Caldera
- Client 1 – Machine that will be initially compromised
- Client 2 – Machine that is pivoted to from client 1
- Hardware
- CPU: 2 cores
- RAM: 4GBs
- HDD: 40GBs
- Version: 1511
- Ubuntu Server 16.04 Graylog
- Services: Graylog, SSH
- Description: Used for OSQuery log aggregation and alerting via Slack
- Hardware
- CPU: 4 cores
- RAM: 4GBs
- HDD: 40GBs
- Ubuntu Server 16.04 Kolide Fleet manager
- Services: Kolide + Nginx, SSH
- Description: Used to manage/query OSQuery agents in the environment
- Hardware
- CPU: 2 cores
- RAM: 4GBs
- HDD: 40GBs
Initial setup
pip install pywinrm
pip install ansible
git clone https://github.com/CptOfEvilMinions/ThreatWaffle.git
cd ThreatWaffle
Windows Server 2016
Setup domain controller
- Set hostname of the
- server to “WinDC”
- Open Powershell as Administrator
- Run this command:
powershell -NoProfile -ExecutionPolicy Bypass -Command "iex ((new-object net.webclient).DownloadString('https://github.com/ansible/ansible/raw/devel/examples/scripts/ConfigureRemotingForAnsible.ps1'))"
- Run this command:
mv group_vars/all.example group_vars/all
vim group_vars/all
 and set:- base_domain
- timezone
- slack(Optional)
- cert info
mv group_vars/win_dc.example group_vars/win_dc
vim group_vars/win_dc
and set:- ad_domain_name (by default set to base_domain)
- ad_safe_mode_password
vim group_vars/windows
and set :- asnsible_user
- ansible_password
vim hosts
set IP address for[win_dc]
ansible-playbook -i hosts deploy_windows_dc.yml
Setup DNS
Create DNS reverse zone
- Open Server Manager
- Select “Tools” then “DNS”
- WinDC > Reverse Lookup Zone
- Right-Click “Reverse Lookup Zone” and select “New zone”
- Select “Primary zone” for the type of zone
- Select “To all DNS servers running on domain controllers in domain” for data replication
- Select “IPv4 reverse lookup zone” for the type of reverse zone
- Enter “10.100.3” for the Network ID
- Select “Allow only secure dynamic updates” for dynamic updates
- Select “Finish”
DNS entries
- WinDC > Forward Lookup Zone > hackinglab.beer
- Select “Action” then “New Host(A)”
- Enter
graylog
for Name - Enter “<IP addr of Graylog>” for IP address
- Check “Create associated pointer (PTR) record” if UNchecked
- Select “Add host”
- Enter
- Â Repeat for Kolide
Group Policy settings
Enable RDP through the firewall
- Select “Tools” then “Group policy manager”
- Expand:Â Group policy manager > Forest: hackinglab.beer > Domains >Â hackinglab.beer
- Right-click “Default Domain Policy”
- Computer Configuration > Policies > Administrative Templates > Network > Network Connections > Windows Firewall > Domain Profile
- Enable “Allow inbound Remote Desktop exceptions” and set IP addresses to “*”
- Enable “Allow inbound Remote Desktop exceptions” and set IP addresses to “*”
Powershell script block logging
- Computer Configuration > Policies > Administrative Templates -> Windows Components -> Windows PowerShell
- Enable “Turn on PowerShell Script Block Logging”
Process creation logging
- Computer Configuration > Windows Settings > Security Settings > Advanced Audit Policy Configuration > Audit Policies > Detailed Tracking
- Enable for successful “Audit Process Creation”
- Enable for successful “Audit Process Creation”
SMB access for PSexec
- Computer Configuration > Policies > Windows Settings > Security Settings > Windows Firewall with Advanced Security > Windows Firewall with Advanced Security LDAP
- Right-click “Inbound Rules” and select “New Rule”
- Select “Port” for rule type
- Select “TCP” for protocol and enter “445” for port
- Select “Allow the connection”
- Select all profiles
- Enter “Allow PSexec” for name
Create domain users(non-admin)
- Open server manager
- Select “Tools” then “Active Directory Users and Computers”
- Expand: hackinglab.beer > Users
- Select “Users” on the left
- Select “Action” then “New User”
- User info
- Enter “Bill” for Firstname
- Enter “Gates” for the Last Name
- Enter “Bgates” for logon
- Password
- Enter a password for user
- UNcheck “User must change password at next logon”
- User info
- Select “Users” on the left
- Select “Action” then “New User”
- User info
- Enter “Steve” for Firstname
- Enter “Jobs” for the Last Name
- Enter “Sjobs” for logon
- Password
- Enter a password for user
- UNcheck “User must change password at next logon”
- User info
- Shutdown and create snapshot
Ubuntu Server 16.04 Graylog
Install/Setup Graylog
- Set DNS server to WinDC
vim group_vars/all
and set “graylog_hostname”mv group_vars/graylog.example group_vars/graylog
vim group_vars/graylog
set:- graylog_admin_password
- graylog_admin_password can not contain the following special characters: )(;
- graylog_admin_password
vim hosts
and set:- IP address for
[graylog]
- IP address for
ansible-playbook -i hosts deploy_graylog.yml -u <username>
Setup/Configure Graylog input
- Browse to https://<graylog FQDN – MUSTÂ BE accessed via FQDN> and login
- User: admin
- Pass: {{Â graylog_admin_password }}
- Select “System” then “Inputs”
- Select “Beats” for input than “Launch new input”
- Select “<node>” for Node
- Enter “Beats” for title
- Leave the default for bind-address
- Leave the default for port
- Select “save”
- Shutdown and create snapshot
Ubuntu Server 16.04 Kolide Fleet
Kolide terms
- Node– A single machine
- Fleet – All the machines controlled and owned by an enterprise
- Queries – A query runs a set of tasks on a fleet of machines on a specified interval
- Distributed queries – An on the fly query
- Packs – Â Group queries into packs to perform ongoing monitoring.
Install/Setup Kolide Fleet
- Set DNS server to WinDC
openssl rand -base64 32
- Copy the output from above
vim group_vars/all
and set “fleet_hostname”mv group_vars/kolide.example group_vars/kolide
vim group_vars/kolid
e and set:- kolide_jwt_key to output from above
- Set username and password for Kolide, MySQL
vim hosts
- Set management IP address under [kolide]
ansible-playbook -i hosts deploy_kolide.yml -u <ubuntu local user>
Setup/Configure webGUI
- https://<Hostname/IP addr of Kolide>
- Setup user
- Enter “admin” for username
- Enter a password
- Enter an e-mail for the admin user
- Select “Submit”
- Setup organization
- Enter an organization name
- Enter the organizations URL
- DO NOT enter the same URL to access Kolide
- DO NOT enter the same URL to access Kolide
- Select “Submit”
- Set Kolide URL
- Enter the Kolide FQDN into “Kolide web address”Â
- Select “Submit”
- Enter the Kolide FQDN into “Kolide web address”Â
- Setup user
- Shutdown and create snapshot
Deploy OSQuery agents
Initial setup
- Browse to https://<Hostname/IP addr of Kolide>
- Select “Add new host” in top right
- Select “Reveal secret” and copy the string
mv group_vars/agents.example group_vars/agents
vim group_vars/agents
- set osquery_enroll_secret with string from Kolide
Client 1
- Change hostname to client1
- Install trial of Office 365
- Open Powershell as Administrator
- Run this command
powershell -NoProfile -ExecutionPolicy Bypass -Command "iex ((new-object net.webclient).DownloadString('https://github.com/ansible/ansible/raw/devel/examples/scripts/ConfigureRemotingForAnsible.ps1'))"
- Run this command
Client 2
- Change hostname to client2
- Install trial of Office 365
- Place some “intellectual property”(IP) like a Microsoft Word document in the user’s home directory
- Open Powershell as AdministratorRun this command
powershell -NoProfile -ExecutionPolicy Bypass -Command "iex ((new-object net.webclient).DownloadString('https://github.com/ansible/ansible/raw/devel/examples/scripts/ConfigureRemotingForAnsible.ps1'))"
Install/Setup OSQuery
- Copy contents of
/etc/nginx/ssl/kolide.crt
for Kolide server mv conf/agents/certificate.example conf/agents/certificate.crt
vim conf/agents/certificate.crt
and paste the contents from abovemv group_vars/win_agents.example group_vars/win_agents
vim group_vars/win_agents
and set:- ansible_user
- Ansible_password
vim hosts
- Add hosts to win_agents
- Browse to “https://<Hostname/IP addr of Kolide>” and login
- Select “Add new host in the top right”
- Copy the secret key
- Copy the secret key
mv group_vars/agents.example group_vars/agents
vim group_vars/agents
and setosquery_enroll_secret
to the key from aboveansible-playbook -i hosts deploy_windows_osquery_agents.yml
- Shutdown each host and snapshot
Future road map
In this blog post, we created a lab environment to perform red teaming activities for us to hunt. Before I start talking about the future, let’s understand our environment. This environment is predominantly Windows machines and they will be our targets, more specifically Windows client 1 and 2. The Windows domain controller provides central authentication and Group Policy management for our Windows clients. Next, we have our Ubuntu servers running Kolide and Graylog.
Kolide is our OSQuery fleet manager for our OSQuery agents, and also provides the ability to run specified configs on a particular interval, or the ability to ask our environment a question on the fly(distributed query). We will use this tool and techniques from the MITRE ATT&CK matrix to hunt for our red teaming activity. Once we have identified malicious activity from our hunts, we will create an OSQuery pack(multiple queries) that will run on a specified interval. OSQuery agents will keep track of each query’s output and will only send results when the query state changes. Next, we will ingest the changes sent by OSQuery agents into Graylog.
Graylog will be configured to ingest these logs and alert based on certain conditions. This part is essential because we want to automate our hunts and move from a threat hunting role to an incident response role. By moving our hunts to an incident response role, we will have more time to search for more advanced threats that haven’t been discovered within our environment. Furthermore, we will use Cyb3rWard0g’s threat-hunting playbook template to create our own threat-hunting playbooks. It’s important to document all hunts and our criteria for determining malicious activity. Finally, we will evaluate our hunts using Cyb3rWard0g scoring system.
Resources/Sources
- Github: ThreatWaffle
- Managing Windows machines with Ansible
- Greater Visibility Through PowerShell Logging
- Kolide fleet documentation
- Enable Remote Desktop using Group Policy
- Powershell script block logging via Group Policy
- Process creation logging via Group Policy
- Allow PSexec on domain via Group Policy
Hi.
I am trying to follow your steps to deploy the osquery on Windows 10 and to enroll them to Kolide, however I do not understand what you mean in Deploy OSQuery agents -> Initial Setup -> Step 4 -> vim group_vars/agents -> set osquery_enroll_secret with string from Kolide.
Where is this group_vars? It is really confusing this Deployment steps, could you please clarify?
Thanks.
The directions have been updated to include:
4. mv group_vars/agents.example group_vars/agents
5. vim group_vars/agents
I have difficulty understanding the lingo here.
mv group_vars/all.example group_vars/all
mv group_vars/win_agents.example group_vars/win_agents
vim group_vars/win_agents
mv ==> move
group_vars/all.example ==> [folder/file.example] ==> group_vars/all [folder] ?
Do i make a file name group_vars with a file name all.example?
What is vim an editor. Do i create a file in folder groups_vars name
win_agents? My understanding of linux goes off the rails here. Can you please
provide some explanation. I would like to try your method in my personal lab.
Hey ralph23,
The move(mv) command within Linux can be used to move files OR rename files, like done above. The files within the current group_vars directory end with a “.example” extension. But Ansible is looking for a file called “group_vars/all”, which does not contain the extension. Continue reading if you want to know why I do this :).
Files within the “group_vars” directory may contain sensitive information. To avoid pushing code with potentially sensitive information, I make a copy of each group_var file and call it .example with fake/blank values. Next, I add “group_vars” files WITHOUT the “.example” extension to my “.gitignore” file. This ensures I don’t accidentally push code to Github with sensitive values such as a Slack token.
Happy Hunting!