Part 1: Google GRR Incident Response Tool

 

GRR Rapid Response is an incident response framework focused on remote live forensics. It based on client server architecture, so there’s an agent which is installed on target systems and a Python server infrastructure that can manage and communicate with the agents.

This is my introduction into setting up Google Grr on my home network. Google Grr is a great open-source tool released for incident response. However as you may have noticed the documentation on Google Grr is not that helpful to beginners. Here I hope to clear up any confusion and help beginners get started on their own networks.

 

Google Grr Terminology

  • Flow –
  • Hunt –

 

Google Grr Server Install/Setup

  1. First grab a copy on Ubuntu Server 16.04 and setup a VM.
    1. Update on code ONLY works on 16.04.
  2. sudo apt-get update && sudo apt-get upgrade -y
  3. wget https://raw.githubusercontent.com/google/grr/master/scripts/install_script_ubuntu.sh
  4. sudo bash install_script_ubuntu.sh
    1. Enter “1” for SQLite database
    2. Accept default location for Database location
    3. Accept default for hostname unless you need to change it.
    4. Accept default  front-end URL
    5. Accept default AdminUI URL
    6. Accept default for e-mails
      1. I don’t have a mail server :/
    7. Enter password for admin user
    8. Enter “y” to install template packeage and repacking clients with new configs

 

UFW Setup

  1. sed -i ‘s/IPV6=yes/IPV6=no/g’ /etc/default/ufw
  2. sudo ufw enable
  3. sudo ufw allow ssh
  4. sudo ufw allow https
  5. sudo ufw allow 8080/tcp
  6. sudo ufw allow 44449/tcp
  7. sudo ufw default deny incoming
  8. sudo ufw reload

 

Nginx and UFW Install/Setup

  1. sudo apt-get install nginx -y
  2. sudo mkdir /etc/nginx/ssl
  3. sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/ssl/nginx.key -out /etc/nginx/ssl/nginx.crt
  4. sudo vim /etc/nginx/sites-enabled/default
  5. Add
    server  {
        listen 443 ssl default_server;listen [::]:443 ssl default_server;server_name googlegrr googlegrr.hackinglab.com;

    ssl_certificate /etc/nginx/ssl/nginx.crt;

    ssl_certificate_key /etc/nginx/ssl/nginx.key;#

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

    ssl_ciphers ‘EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH’;

    ssl_prefer_server_ciphers on;

    ssl_session_cache shared:SSL:10m;

    location / {

    proxy_pass http://127.0.0.1:8000;

    }

    }

    1. save,exit
  6. sudo service nginx restart
  7. Browse to “https://<IP of Google Grr Server>”
    1. Ignore warning about untrusted certificate.
    2. Prompted to loginScreen Shot 2016-06-25 at 3.17.19 PM

 

Google GRR UI Locations

  • Google Grr Clients: Manage Binaries > Executables > <OS> > Installers.
  • Google Grr Hunts: Hunt Manager.
  • Google Grr Flows: Start Global Flow.
  • Google Grr Clients: To show all clients enter “.” in search. To show a specific client enter an ip or hostname.

 

Samba Share for GoogleGRR Binaries(optional)

  1. sudo mkdir /srv/GoogleGrr
  2. sudo apt-get install samba
  3. sudo vim /etc/samba/smb.conf
    1. Add to bottom”

      [GoogleGrr]

              comment = Google Grr Share

              path = /srv/GoogleGrr

              guest ok = yes

              read only = yes

    2. save,exit
  4. Copy GRR installers to the share folder.
    1. sudo cp grr_*_amd64.deb /srv/GoogleGrr/

    2. sudo cp grr_*_amd64.rpm /srv/GoogleGrr/

    3. sudo cp GRR_*_amd64.exe /srv/GoogleGrr/

  5.  sudo service smbd restart
  6. tespam
  7. sudo ufw allow from <Network IP> to any app Samba

 

Install/Setup Google Client

  1. smbclient //<google grr server>/<google grr share>
  2. get “grr_*_amd64.[.deb|.rpm]”
    1. My example will be centos.
  3. exit
  4. sudo yum install grr_*_amd64.rpmScreen Shot 2016-06-25 at 4.09.47 PM
  5. As you can see below the Google Grr client will automatically call back.Screen Shot 2016-06-25 at 4.09.41 PM

Screen Shot 2016-06-26 at 12.12.11 PM

Setup Google Grr Flow

  1. Get  a process list
    1. Start new flows > Processes > List Processes
    2. Select under general if you wish to wetch binaries.
    3. Under the Advance tab select the “Today” button.
    4. Set the priority for this flow.
    5. Select “Launch”.Screen Shot 2016-06-26 at 12.10.17 PM
    6. The notification box in the top right will let you know when a flow is completed.Screen Shot 2016-06-26 at 12.28.24 PM
    7. Select the notification box to see all the completed flows.Screen Shot 2016-06-26 at 12.29.02 PM
    8. Select “List Processes” to see the details.Screen Shot 2016-06-26 at 12.33.10 PM
    9.  As you can see above these are the details collected for the bash process.
    10. Select the hyperlink above to go to the downloaded process.Screen Shot 2016-06-26 at 12.35.02 PM
    11. As you can see from above we can see the path of the binary, hash of the binary, and more.
    12. If you scroll up you have the following options which are to download the binary, the text view of the binary, and the hex view of the binary.

Download

Screen Shot 2016-06-26 at 12.37.46 PM

Text view

Screen Shot 2016-06-26 at 12.39.07 PM

Hex view

Screen Shot 2016-06-26 at 12.39.14 PM

  1. Get network connections
    1. Start new flows > Network > Netstat.
    2. Under advance select the priority for this flow.
    3. Select the “Today” button to start this flow.
    4. Select “Launch”.
      Screen Shot 2016-06-26 at 12.14.38 PM
    5. Manage launched flows to see a flow’s progress.Screen Shot 2016-06-26 at 12.26.08 PM
    6. When the flow completes this is the output of the results.Screen Shot 2016-06-26 at 12.41.30 PM

 

Setup Google Grr Hunt

 

 

Setup Google Grr Hunt Cron Jobs

  1. Grab processes and netstat connections from all machines every 24hrs.
  2. Select “Cron Job Viewer”
  3. Select “+” to add a job
  4. Enter “Process Collector” into name.
    1. Set periodicity to 24hrs.
    2. Set Lifetime to 1w.Screen Shot 2016-06-26 at 1.47.55 PM
  5. Processes > ListProcesses.
    1. I want to capture all binaries for each processes.
    2. Each binary will be held for a week.Screen Shot 2016-06-26 at 1.48.47 PM
  6. Accept default for output plugin.
  7. Select all machines for the hunt.
  8. Select “Schedule Hunt”.Screen Shot 2016-06-26 at 1.50.00 PM

 

Setup Google Grr Alerting System

 

 

Setup Google Grr Data Retention

 

 

 

 

Resources

 

Sources

Tagged

15 thoughts on “Part 1: Google GRR Incident Response Tool

  1. Frances says:

    Hi there,

    I have a question about Ubuntu Server 16.04. Can I use the Ubuntu Desktop 16.04 instead of the server?

    The server is CLI only, so I thought maybe the desktop system is a better choice. Do you agree? What is your recommendation?

    Also, when you Browse to “https://”, do you do this on a separate machine as the server is CLI only?

    Looking forward to your response

    Frances

    • spartan2194 says:

      Hi Frances,

      The biggest difference between Ubuntu Desktop and Ubuntu Server is the GUI. Ubuntu Server as you mentioned is a headless system so you have no GUI. You can use an Ubuntu Desktop to setup Google Grr. However a desktop environment will take up unnecessary resources so it is recommended to install Google Grr on an Ubuntu server and access it via browser from another pc on the network. So from another machine on the network do the following “https://” witout quotes. Have FUN!

  2. Frances says:

    Hi again,

    Just following up my previous comment/question:

    1. when you Browse to “http://”, is it done on a separate machine?

    2. For testing purpose, can I install both the GRR server and the GRR client on the same VM?

    Thanks and looking forward to you response.

    Frances

    • Arganas says:

      Hi Frances
      Yes it is possible to install the server and client on the same machine. This is not normally recommended due to limitation of resources, especially disc IO. I personally recommend testing this in two separate VM’s to ensure connectivity outside of the server.

      Arganas

  3. sangeeta says:

    HI ,

    can you help me in understanding why we are using GRR, what is the main purpose ?

    What i have understood is it is for memory foreign forensics .

    Kindly share your knowledge .

    Thank you .

    • spartan2194 says:

      Hey Sangeeta,

      Google Grr is an incident response framework that can be utilized for remote IR. Google Grr allows IR teams to investigate an incident on a machine(s) remotely from the Grr web console. For example, we can remotely request the process list, network connection list, a memory dump, or do memory analysis on a machine. The awesome thing about Google grr is rekall is installed on all agents so you can run rekall plugins on a remote box. The benefit of this is if a machine has a rootkit on it. Rootkits are known to hide themselves from tools such as netstat and ps. However, this security concern is mitigated with the use of tools like rekall that analyze the memory of a machine.

      Additionally, you can give Google Grr indicators of compromise to look for on a fleet(one or more) machines. You can schedule certain types of tasks to be run on an interval to return actionable data about a set of machines. Google Grr is an open-source product being used currently by Google. This ensures that Google is going to keep providing updates and features to the framework. Furthermore, the creators of the framework aren’t trying to re-invent the analysis wheel. Meaning they aren’t trying to create or replace existing forensic tools. Pre-existing tools being used by the IR community are built into Google Grr or the data received can be used by these tools.

  4. TD says:

    Great write-up, thanks!!!

  5. prats says:

    Could GRR be used to push out custom script to endpoints and execute them. Eg. a bash script to gather certain data which is not part of the Artefacts ?

  6. Archer says:

    So idk if it’s because of an update or what has happened, but your guide no longer works.
    You have to go get the DEB release now. So I go and do that, it’s sets up fine until I get to nginx.
    I’ve tried it twice now. It simply won’t work. I’ll go try it again and post my errors. Once I install nginx and edit the /etc/nginx/sites-enabled/default with your config, nginx no longer starts.
    I’ve googled the hell out of this and gotten nowhere, if you off the top of your head please let me know what to do.
    I’ll be back in a few with the errors.
    Thanks for your help!

    • spartan2194 says:

      Hey Archer,

      Without a screenshot/message block of the error I cannot provide any potential solutions. However, Google is probably still your best option to resolve the issue :).

  7. 123 says:

    Why we don’t just use RDP or some kind of remote desktop instead of this limited tasks software?

    • spartan2194 says:

      Hey 123,

      This is a great question! RDP specifically requires the sys admin/security personnel to initiate a connection to the machine with RDP. If the machine is behind a NAT/firewall device device, you probably won’t be able to access the machine without proper port forwarding. Since, Google GRR is an agent that runs on the machine behind the NAT/firewall device, it can initiate a connection outbound to your server for new tasks. TLDR: Ingress vs. egress.

      Furthermore, RDP is an intensive network application, so your user may be in a remote location(overseas) with a limited connection. Google GRR uses HTTP which is a light protocol so it can perform better on limited network connections.

      Finally, I would recommend watching this Youtube on Google GRR for a more comprehensive answer.
      Youtube video on Google Grr: https://www.youtube.com/watch?v=ren6QSvwFvg

Leave a Reply to spartan2194 Cancel reply

Your email address will not be published. Required fields are marked *