Jupyter, msticpy and Microsoft Sentinel
You can run notebooks from Microsoft Sentinel in the Azure Machine Learning studio hosted environment.
If you have a local installation of Python 3.8 or later, you can also download the notebooks and run these locally. Our recommendation is to use the Anaconda distribution since it contains the Jupyter packages and many others needed for the Microsoft Sentinel notebooks.
Running notebooks in Azure Machine Learning (AML)
Please follow the guidance given in this document to get started Tutorial: Get started with Jupyter notebooks and MSTICPy in Microsoft Sentinel.
Running notebooks locally
The easiest way of getting to the Microsoft Sentinel template and sample notebooks is to use Git to clone the Azure-Sentinel-Notebooks repository
cd <root-path-to-create-repo-folder> git clone https://github.com/Azure/Azure-Sentinel-Notebooks.git
You can also download the repo as a ZIP file and extract the contents into a folder of your choice.
View the notebooks on GitHub or in NBViewer
You can also view the notebooks without executing them natively in GitHub just by clicking on them. You can also use nbviewer. Copy the URL from a notebook in the repo, then go to nbviewer.org and paste the notebook URL into the text box.
When it comes to running one of the notebooks in against real data, you will need some preparatory steps.
Permissions - in order to read any data, you will need to have at least LogAnalytics Reader role for your account.
Configuring your Python Environment for the First Time
You will need to carry out this procedure every time you start working in a fresh Python environment.
Ensure that you have a version of Python 3.8 or later.
For more details see Installing
msticpyconfig.yaml configuration file
To use Microsoft Sentinel you need at least to configure the Sentinel Workspace details in this file.
See the section Authenticating to MS Sentinel below. This is covered in more detail in:
Import and initialize MSTICPy
import msticpy as mp mp.init_notebook()
Creating a Query Provider
qry_prov = mp.QueryProvider("MSSentinel")
A range of built-in queries come with the Sentinel query provider.
You can view these with the
list_queries function or interactively
browse the queries and help with the query browser.
qry_prov.list_queries() # or qry_prov.browse()
Authenticating to MS Sentinel
This assumes that you have configured at least one Sentinel Workspace
msticpyconfig.yaml. The contents should look something
AzureSentinel: Workspaces: Default: WorkspaceId: "52b1ab41-869e-4138-9e40-2a4457f09bf3" TenantId: "72f988bf-86f1-41af-91ab-2d7cd011db49" SubscriptionId: "cd928da3-dcde-42a3-aad7-d2a1268c2f48" ResourceGroup: MyResourceGroup WorkspaceName: Workspace1
At minimum you must have WorkspaceId and TenantId configured.
You can authenticate to Sentinel using the query provider
No parameters are needed for WorkspaceConfig if you have a Workspace entry in your msticpyconfig name “Default”. If you do not, use the workspace parameter to pick a named workspace.
AzureSentinel: Workspaces: MyHuntingWorkspace: WorkspaceId: "52b1ab41-869e-4138-9e40-2a4457f09bf3" TenantId: "72f988bf-86f1-41af-91ab-2d7cd011db49" ...
From version 2.0 you can also use a shortcut parameter to
connect to specify the workspace directly.
to use the Default workspace, use “Default” as the workspace name.
Most queries require additional parameters (you can check which parameters are needed by using the help in the query browser or calling the query with a “?” parameter).
The queries use a built-in time range as their default time boundary.
You can change this by opening and modifying the
Run a query
results_df = qry_prov.WindowsSecurity.list_host_logons(host_name="MyHost") results_df.head()
This will run the query with start and end times defined by the settings in
Run an ad hoc query
exec_query function to run arbitrary KQL queries.
query_time settings have no impact on ad hoc queries.
You must supply required
where clauses to restrict the time
range for your query.
results_df = qry_prov.exec_query("SecurityAlert | where TimeGenerated > ago(1day)") results_df.head()
Microsoft Sentinel Jupyter notebooks can be found here on GitHub.
Other sample notebooks with saved data are in the Sample-Notebooks folder.