MSTIC Jupyter and Python Security Tools
msticpy is a library for InfoSec investigation and hunting in Jupyter Notebooks. It includes functionality to:
query log data from multiple sources
enrich the data with Threat Intelligence, geolocations and Azure resource data
extract Indicators of Activity (IoA) from logs and unpack encoded data
perform sophisticated analysis such as Anomalous session detection and time series decomposition
visualize data using interactive timelines, process trees and multi-dimensional Morph Charts
It also includes some time-saving notebook tools such as widgets to set query time boundaries, select and display items from lists, and configure the notebook environment.
Background - Why we created msticpy
The package was originally built for authoring notebooks in Azure Sentinel. While Azure Sentinel is still a big focus of our work, we are have extended the data query/acquisition components to pull log data from other sources (e.g. Splunk, Microsoft 365 Defender and Microsoft Graph, SumoLogic, and are actively working on support for data from other SIEM platforms). Most of the components can also be used with data from any source. Pandas dataframes are used as the ubiquitous input and output format of almost all components.
Most of the content of the package began life as inline code in a notebook. However, having a lot of code in your notebooks creates a few problems:
notebooks dominated by big code blocks make it difficult to see the results and text of your notebook - which is really what you want your audience to see
it makes the notebook especially intimidating for non-programmers
it makes code re-use very hard.
The philosophy behind this package is to create a repository for re-usable pieces of functionality that make it both quicker to author notebooks, and make resultant notebooks easier to read.
What is/who are MSTIC?
MSTIC == Microsoft Threat Intelligence Center. We are mostly security analysts and engineers working on:
authoring security detections for several Microsoft platforms
threat identification and investigation
- Getting Started
- Querying and Importing Data
- Data Provider Library
- Data Queries Reference
- Setting up Process Auditing for Linux in Azure Sentinel
- Data Uploaders
- Data Masking Functions
- Splunk - Data Connector
- Resource Graph Explorer Data Provider
- Mordor data provider and browser
- Reading from and writing to Azure Blob AzureBlobStorage
- SQL TO KQL Conversion (Experimental)
- Enriching Data
- Analyzing Data
- Displaying/Visualizing Data
- msticpy API
- Notebook Examples
- Articles, Labs and other Resources