Quick Start Overview of MSTICPy
Installing
Install from PyPI:
pip install msticpy
Note
Ensure that the correct version of pip
is being run.
The safest approach is to avoid running pip
directly from
the command line. Instead, first ensure that you have activated
the Python environment that you want to use then type:
python -m pip install msticpy
For more details see Installing
Importing MSTICPy
You can install MSTICPy as it is or rename it to something
easier to type such as mp
.
import msticpy as mp
Simple help is available:
help(mp)
This product includes GeoLite2 data created by MaxMind, available from https://www.maxmind.com.
Help on package msticpy:
NAME
msticpy - msticpy - Jupyter and Python Tools for InfoSec.
....
Many commonly-used classes are available as attributes of
the msticpy
top-level package:
QueryProvider
MpConfigEdit - the configuration editor
init_notebook - see Initializing MSTICPy
Searching for a MSTICPy module
Use the search
function to find the module to import
mp.search("geo")
Initializing MSTICPy
The initialization function is intended to prepare things when you are using MSTICPy interactive (usually in a notebook). It does a few things by default:
Imports some common (non-MSTICPy) packages such as pandas, numpy, ipywidgets
Imports a number of MSTICPy components such as Entities
Checks for a valid msticpyconfig file
Initializes MSTICPy notebook magics and pandas accessors
Hooks notebook exception handling to display friendly MSTICPy exceptions (other exceptions are unaffected).
mp.init_notebook()
If you are having issues with this you can increase the verbosity
of the output with verbosity=1
or verbosity=1
parameter.
Use help(mp.init_notebook)
for more detailed usage.
Setup msticpyconfig.yaml
For many of the most interesting MSTICPy features to work you will need configuration for query providers and things like Threat Intelligence and other data providers.
See the following documents for more details:
Running a data query
To run a query you must load and instantiate a Query Provider for the data source that you want to query from. Several providers are supported by MSTICPy including: Microsoft Sentinel, Splunk, Microsoft Defender, Azure Monitor, Cybereason and Sumologic.
qry_sent = mp.QueryProvider("MSSentinel")
qry_splunk = mp.QueryProvider("Splunk")
You will need to connect and authenticate to the data service. The exact parameters depend on the data provider and what details you’ve added to your msticpconfig.yaml
qry_sent.connect(mp.WorkspaceConfig(workspace="MySentinelWorkspace"))
qry_splunk.connect(host=splunk_uri, user="splunkuser", password=MYPWD)
Use query_prov.list_queries()
to see the built-in queries for the
provider type.
Built-in queries are functions that usually require additional parameters. Run the query passing “?” for more help (if you run a query that is missing required parameters, the missing parameters will be listed in the exception thrown)
results_df = qry_sent.WindowsSecurity.list_host_logons(host_name="myhost")
If successful, the query returns results as a pandas DataFrame.
For more information see: Querying and Importing Data
Visualizing Data
MSTICPy has several visualizations that are simple to use. Most
are based on the Bokeh package. Visualizations can be run directly
from DataFrames using the MSTICPy custom accessor mp_plot
.
results_df.mp_plot.timeline(group_by="LogonType")
results_df.mp_plot.folium_map(ip_column="IpAddress")
For more information see: Displaying/Visualizing Data
Enriching data with Context and Pivot Functions
When investigating an incident, it is normal for analysts to need more contextual information about the entities that they see. In MSTICPy you can run many context queries directly from Entities. Entities are Python classes such as IpAddress, Host, Url, that encapsulate attributes and methods relevant to the corresponding objects in the real (virtual?) world.
For example:
IpAddress.ip_type("172.217.14.206")
ip |
result |
---|---|
172.217.14.206 |
Public |
IpAddress.whois("172.217.14.206")
asn |
asn_cidr |
asn_country_code |
asn_date |
asn_description |
asn_registry |
nets |
---|---|---|---|---|---|---|
15169 |
172.217.14.0/24 |
US |
2012-04-16 |
GOOGLE, US |
arin |
[{‘cidr’: ‘172.217.0.0/16’, ‘name’: ‘GOOGLE’ |
If you have any Threat Intelligence provider accounts configured in your msticpconfig.yaml you can use the same mechanism to lookup Threat Intelligence reports on one or more observables.
iocs = ['162.244.80.235', '185.141.63.120', '82.118.21.1', '85.93.88.165']
IpAddress.ti.lookup_ip(iocs)
Ioc |
IocType |
SanitizedValue |
Provider |
Severity |
Details |
Reference |
---|---|---|---|---|---|---|
162.244.80.235 |
ipv4 |
162.244.80.235 |
RiskIQ |
high |
{‘summary’: {‘resolutions’: 12, ‘certificates’: 12, ‘malware_hashes’: 2, ‘projects’… |
|
185.141.63.120 |
ipv4 |
185.141.63.120 |
RiskIQ |
high |
{‘summary’: {‘resolutions’: 2, ‘certificates’: 6, ‘malware_hashes’: 1, ‘projects’: … |
|
82.118.21.1 |
ipv4 |
82.118.21.1 |
RiskIQ |
high |
{‘summary’: {‘resolutions’: 13, ‘certificates’: 20, ‘malware_hashes’: 0, ‘projects’… |
|
85.93.88.165 |
ipv4 |
85.93.88.165 |
RiskIQ |
high |
{‘summary’: {‘resolutions’: 24, ‘certificates’: 25, ‘malware_hashes’: 2, ‘projects’… |
|
162.244.80.235 |
ipv4 |
162.244.80.235 |
VirusTotal |
high |
{‘verbose_msg’: ‘IP address in dataset’, ‘response_code’: 1, ‘positives’: 34, ‘dete… |
|
185.141.63.120 |
ipv4 |
185.141.63.120 |
VirusTotal |
high |
{‘verbose_msg’: ‘IP address in dataset’, ‘response_code’: 1, ‘positives’: 19, ‘dete… |
|
82.118.21.1 |
ipv4 |
82.118.21.1 |
VirusTotal |
high |
{‘verbose_msg’: ‘IP address in dataset’, ‘response_code’: 1, ‘positives’: 41, ‘dete… |
|
85.93.88.165 |
ipv4 |
85.93.88.165 |
VirusTotal |
high |
{‘verbose_msg’: ‘IP address in dataset’, ‘response_code’: 1, ‘positives’: 7, ‘detec… |
You can also use the individual modules directly from the TILookup module:
ioc_lookup = mp.TILookup()
ioc_lookup.lookup_iocs(iocs)
You can also use additional queries as context lookups. For example:
Host.qry_sent_myworkspace.LinuxSyslog.list_logons_for_source_ip(iocs)
MSTICPy also includes more specialized context functions for Azure and Azure Sentinel resources.
For more details about context functions see:
Analysis and Data Transformation
MSTICPy has several generic analysis and transformation functions. Some examples of transforms are:
Process Tree building
Decoding encoded and compressed data
IoC/observable extraction from data
Syslog parsing
Some examples of analysis functions are:
Anomaly patterns in Office activity
Time series decomposition
Event clustering
For more details see: