In this blog post, we are going to cover how to install MISP on Ubuntu 18.04. Once MISP is installed, we will do an introduction to the PyMISP API to store indicators of compromise (IOCs) in MISP and query IOCs from MISP. This blog post will serve as the foundation for future blog posts moving forward.

Goals

• Install/Setup MISP on Ubuntu 18.04
• Learn how to use PyMISP to add IOCs to MISP
• Learn how to use PyMISP to query IOCs from MISP

Background

What is an IOC?

Indicators of Compromise (IOC) are pieces of forensic data that can be used to identify potentially malicious activity on a system or network. The following pieces of data can be used as an IOC:

• File paths
• File hashes
• File names
• IP addresses
• Domains
• URLs
• e-mail addresses
• Hex string
• x509 cert
• etc

What is Malware Information Sharing Platform and Threat Sharing (MISP)?

MISP is an open-source software solution for collecting, storing, distributing and sharing cybersecurity indicators and threats about cybersecurity incidents analysis and malware analysis. MISP is designed by and for incident analysts, security and ICT professionals or malware reversers to support their day-to-day operations to share structured information efficiently.

The objective of MISP is to foster the sharing of structured information within the security community and abroad. MISP provides functionalities to support the exchange of information but also the consumption of said information by Network Intrusion Detection Systems (NIDS), LIDS but also log analysis tools, SIEMs.

MISP event

MISP events are encapsulations for contextually linked information. Linked information will include things such as IP addresses, domains, malicious binaries, file hashes, and etc. For example, let’s say your NIDS detects web scanner activity. The event would container for the attributes associated with this event such as source IP address, URIs scanned, HTTP methods, and etc.

MISP object

MISP objects are in addition to MISP attributes to allow advanced combinations of attributes. The creation of these objects and their associated attributes are based on real cyber security use-cases and existing practices in information sharing. The objects are just shared like any other attributes in MISP even if the other MISP instances don’t have the template of the object.

MISP attributes

Attributes in MISP can be network indicators such as (e.g. IP address), system indicators (e.g. a string in memory) or even bank account details. A type (e.g. MD5, url) is how an attribute is described. An attribute is always in a category (e.g. Payload delivery) which puts it in a context. A category is what describes an attribute. An IDS flag on an attribute allows to determine if an attribute can be automated (such as being exported as an IDS ruleset or used for detection). If the IDS flag is not present, the attribute can be useful for contextualisation only.

MISP feed

MISP includes a set of public OSINT feeds in its default configuration. The feeds can be used as a source of correlations for all of your events and attributes without the need to import them directly into your system. The MISP feed system allows for fast correlation but also for a quick comparison of the feeds against one another.

Install/Setup MISP on Ubuntu 18.04.3

Install MISP with install.sh

1. sudo apt-get update -y && sudo apt-get upgrade -y
2. sudo apt-get install mysql-client  -y
3. wget https://raw.githubusercontent.com/MISP/MISP/2.4/INSTALL/INSTALL.sh
4. chmod +x INSTALL.sh
5. ./INSTALL.sh -A
   1. Enter “<FQDN of MISP>” into MISP_BASEURL
   2. Enter “Y” to create misp user
6. sudo ufw allow 80/tcp
7. sudo ufw allow 443/tcp

Change admin password

1. Browse to https://<FQDN of MISP>/users/login
   1. Username:  [email protected]
   2. Password: admin
2. Enter new password

Create an organization

1. Select Administration > Add Organisations
   1. Enter “<ORG name> into Organisation Identifier
   2. Select “Generate UUID”
   3. Select “Submit” at the bottom

Create admin for new org

1. Administration> Add user
   1. Enter “admin@<fqdn>” for email
   2. Check “Set password”
   4. Select “<new org name>” for Organisation
   5. Select “Org” for Role
   6. Select submit

Create API user for new org

1. Administration> Add user
   1. Enter “api_user@<fqdn>” for email
   2. Select “<new org name>” for Organisation
   3. Select “Publisher” for Role
   4. Select submit

Enable threat intel feeds

To enable feeds you will need to login to MISP with the “superadmin” account which is the “[email protected]” account.

1. Sync Actions > List feeds
2. Find a feed such as “Feodo IP Blocklist”
3. Select the “Edit” icon
   1. Check “Enabled”
   2. Check “Caching Enabled”
   3. Select “Edit” at the bottom

IPython + PyMISP

1. In the MISP console select Administrator then List Users
2. Look for “api_user@<fqdn>” and copy “auth key”
3. Open a terminal
4. brew install python3 ipython
5. pip3 install -U pymisp
6. ipython

Connect a MISP instance with PyMISP

from pymisp import ExpandedPyMISP

misp_url = 'https://<FQDN of MISP>' 
misp_key = "<Enter MISP API key>" 
misp_verifycert = False 

# Init misp connector 
misp = ExpandedPyMISP(misp_url, misp_key, misp_verifycert)

Create a MISP event

from pymisp import ExpandedPyMISP, PyMISP, MISPEvent 


# Create event
event_obj = MISPEvent()
event_obj.distribution = 1
event_obj.threat_level_id = 1
event_obj.analysis = 1
event_obj.info = "Event from notebook 2"

# Add event to MISP
event = misp.add_event(event_obj)
event_id, event_uuid = event['Event']['id'], event['Event']['uuid']
print (event_id, event_uuid)

Add an object to MISP event

from pymisp import MISPAttribute


# Define attributes
attr_type = "ip-src"
value = "8.8.8.8"
category = "Network activity"
to_ids = False

# Create attribute object
attribute = MISPAttribute()
attribute.type = attr_type
attribute.value = value
attribute.category = category
attribute.to_ids = to_ids

# Add attributes to event
attribute_to_change = misp.add_attribute(event_id, attribute)

# Print event
print(attribute_to_change['Attribute']['id'], attribute_to_change)

Search MISP for IOC

# Search for an IOC in MISP 
misp.search(controller='attributes', type_attribute="ip-src", value="8.8.8.8")

Future topics

• Ingesting IOCs from Malware Traffic Analysis
• Ingesting IOCs from @MalwareHunterTeam on Twitter
• Ingesting Suricata alerts into IOCs with a model to decay old events
• Generate Suricata rules from MISP intel
• Generate OSquery rules from MISP intel
• Correlating MISP intel with MITRE ATT&CK
• Ingest honeypot data into MISP
• Ingest WAF data into MISP
• Make your own MISP intel publicly available

Resources/Sources

• MISP Install Documentation
• Github – MISP
• MISP Quick start
• MISP Quick start – Add user
• Github – PyMISP examples
• MISP Jupiter Notebook
• How To Install MySQL on Ubuntu 18.04
• MISP Glossary