Microsoft Sentinel Provider - Legacy Version
Warning
This provider has been replaced by one based on the Azure Monitor SDK. Unless you have a specific need to use this version, we recommend that you use the new version. See Microsoft Sentinel Provider for more information.
Note
This provider is still supported but will not be updated
with new features.
In order to use it you to have install KqlMagic installed. This is
no longer installed by default with MSTICPy.
You can install this with pip install Kqlmagic
or
pip install msticpy[kql]
.
Sentinel Configuration
You can store configuration for your workspace (or workspaces) in either
your msticpyconfig.yaml
or a config.json
file. The latter
file is auto-created in your Azure Machine Learning (AML) workspace when
you launch a notebook from the Sentinel portal. It can however, only
store details for a single workspace.
Sentinel Configuration in msticpyconfig.yaml
This is the simplest place to store your workspace details.
You likely need to use a msticpyconfig.yaml anyway. If you are using other msticpy features such as Threat Intelligence Providers, GeoIP Lookup, Azure Data, etc., these all have their own configuration settings, so using a single configuration file makes managing your settings easier. If you are running notebooks in an AML workspace and you do not have a msticpyconfig.yaml MSTICPy will create one and import settings from a config.json, if it can find one.
For more information on using and configuring msticpyconfig.yaml see msticpy Package Configuration and MSTICPy Settings Editor
The MS Sentinel connection settings are stored in the
AzureSentinel\\Workspaces
section of the file. Here is an example.
AzureSentinel:
Workspaces:
# Workspace used if you don't explicitly name a workspace when creating WorkspaceConfig
# Specifying values here overrides config.json settings unless you explicitly load
# WorkspaceConfig with config_file parameter (WorkspaceConfig(config_file="../config.json")
Default:
WorkspaceId: 271f17d3-5457-4237-9131-ae98a6f55c37
TenantId: 335b56ab-67a2-4118-ac14-6eb454f350af
ResourceGroup: soc
SubscriptionId: a5b24e23-a96a-4472-b729-9e5310c83e20
WorkspaceName: Workspace1
# To use these launch with an explicit name - WorkspaceConfig(workspace_name="Workspace2")
Workspace1:
WorkspaceId: "c88dd3c2-d657-4eb3-b913-58d58d811a41"
TenantId: "335b56ab-67a2-4118-ac14-6eb454f350af"
ResourceGroup: soc
SubscriptionId: a5b24e23-a96a-4472-b729-9e5310c83e20
WorkspaceName: Workspace1
TestWorkspace:
WorkspaceId: "17e64332-19c9-472e-afd7-3629f299300c"
TenantId: "4ea41beb-4546-4fba-890b-55553ce6003a"
ResourceGroup: soc
SubscriptionId: a5b24e23-a96a-4472-b729-9e5310c83e20
WorkspaceName: Workspace2
If you only use a single workspace, you only need to create a Default
entry and
add the values for your WorkspaceID and TenantID. You can add other entries here,
for example, SubscriptionID, ResourceGroup. These are recommended but not required
for the QueryProvider (they may be used by other MSTICPy components however).
Note
The property names are spelled differently to the values in the config.json so be sure to enter these as shown in the example. These names are case-sensitive.
Note
The section names (Default, Workspace1 and TestWorkspace) do not have to be the same as the workspace name - you can choose friendlier aliases, if you wish.
If you use multiple workspaces, you can add further entries here. Each workspace entry is normally the name of the Azure Sentinel workspace but you can use any name you prefer.
Sentinel Configuration in config.json
When you load a notebook from the MS Sentinel UI a configuration file config.json is provisioned for you with the details of the source workspace populated in the file. An example is shown here.
{
"tenant_id": "335b56ab-67a2-4118-ac14-6eb454f350af",
"subscription_id": "b8f250f8-1ba5-4b2c-8e74-f7ea4a1df8a6",
"resource_group": "ExampleWorkspaceRG",
"workspace_id": "271f17d3-5457-4237-9131-ae98a6f55c37",
"workspace_name": "ExampleWorkspace"
}
If no msticpyconfig.yaml is found MSTICPy will automatically look for a config.json file in the current directory. If not found here, it will search the parent directory and in all its subdirectories. It will use the first config.json file found.
Loading a QueryProvider for Microsoft Sentinel
qry_prov = QueryProvider(
data_environment="MSSentinel_Legacy",
)
Connecting to a MS Sentinel Workspace
Once we have instantiated the QueryProvider we need to authenticate to Sentinel Workspace. This is done by calling the connect() function of the Query Provider.
connect() requires a connection string as its parameter. For MS Sentinel
we can use the WorkspaceConfig
class.
WorkspaceConfig
Note
From v2.0.0 of MSTICPy the MS Sentinel QueryProvider
will automatically create a WorkspaceConfig from your settings.
Simply call connect
with a workspace="YourWorkspace"
parameter
WorkspaceConfig
handles loading your workspace configuration and generating a
connection string from your configuration.
See WorkspaceConfig API documentation
WorkspaceConfig
works with workspace configuration stored in msticpyconfig.yaml
or config.json (although the former takes precedence).
To use WorkspaceConfig
, simple create an instance of it. It will automatically build
your connection string for use with the query provider library.
>>> ws_config = WorkspaceConfig()
>>> ws_config.code_connect_str
"loganalytics://code().tenant('335b56ab-67a2-4118-ac14-6eb454f350af').workspace('271f17d3-5457-4237-9131-ae98a6f55c37')"
You can use this connection string in the call to QueryProvider.connect()
When called without parameters, WorkspaceConfig loads the “Default”
entry in your msticpyconfig.yaml (or falls back to loading the settings
in config.json). To specify a different workspace pass the workspace
parameter
with the name of your workspace entry. This value is the name of
the section in the msticpyconfig Workspaces
section, which may
not necessarily be the same as your workspace name.
>>> ws_config = WorkspaceConfig(workspace="TestWorkspace")
To see which workspaces are configured in your msticpyconfig.yaml use
the list_workspaces()
function.
Tip
list_workspaces
is a class function, so you do not need to
instantiate a WorkspaceConfig to call this function.
>>> WorkspaceConfig.list_workspaces()
{'Default': {'WorkspaceId': '271f17d3-5457-4237-9131-ae98a6f55c37',
'TenantId': '335b56ab-67a2-4118-ac14-6eb454f350af'},
'Workspace1': {'WorkspaceId': 'c88dd3c2-d657-4eb3-b913-58d58d811a41',
'TenantId': '335b56ab-67a2-4118-ac14-6eb454f350af'},
'TestWorkspace': {'WorkspaceId': '17e64332-19c9-472e-afd7-3629f299300c',
'TenantId': '4ea41beb-4546-4fba-890b-55553ce6003a'}}
Entries in msticpyconfig always take precedence over settings in your
config.json. If you want to force use of the config.json, specify the path
to the config.json file in the config_file
parameter to WorkspaceConfig
.
If you need use a specific instance of a config.json you can specify a full
path to the file you want to use when you create your WorkspaceConfig
instance.
Connecting to the workspace
When connecting you can just pass the name of your workspace or
an instance of WorkspaceConfig to the query provider’s connect
method.
qry_prov.connect(workspace="Default")
qry_prov.connect(workspace="MyOtherWorkspace")
# or, passing WorkspaceConfig
qry_prov.connect(WorkspaceConfig())
# or
qry_prov.connect(WorkspaceConfig(workspace="MyOtherWorkspace"))
MS Sentinel Authentication options
By default, the data provider tries to use chained authentication, attempting to use existing Azure credentials, if they are available.
If you are running in an AML workspace, it will attempt to use integrated MSI authentication, using the identity that you used to authenticate to AML.
If you have logged in to Azure CLI, the Sentinel provider will try to use your AzureCLI credentials
If you have your credentials stored as environment variables, it will try to use those
Finally, it will fall back on using interactive browser-based device authentication.
If you are using a Sovereign cloud rather than the Azure global cloud, you should select the appropriate cloud in the Azure section of the msticpyconfig.
Warning
Although msticpy allows you to configure multiple entries for workspaces in different tenants, you cannot currently authenticate to workspaces that span multiple tenants in the same notebook. If you need to do this, you should investigate Azure Lighthouse. This allows delegated access to workspaces in multiple tenants from a single tenant.
For more details on Azure authentication see Azure Authentication in MSTICPy.
Other MS Sentinel Documentation
For examples of using the MS Defender provider, see the sample M365 Defender Notebook<https://github.com/microsoft/msticpy/blob/master/docs/notebooks/Data_Queries.ipynb>
Built-in Queries for Microsoft Sentinel.