msticpy.context.azure.sentinel_core module

Uses the Microsoft Sentinel APIs to interact with Microsoft Sentinel Workspaces.

msticpy.context.azure.sentinel_core.AzureSentinel: alias of MicrosoftSentinel

Bases: SentinelAnalyticsMixin, SentinelHuntingMixin, SentinelBookmarksMixin, SentinelDynamicSummaryMixin, SentinelIncidentsMixin, SentinelUtilsMixin, SentinelWatchlistsMixin, SentinelSearchlistsMixin, SentinelWorkspacesMixin, SentinelTIMixin, AzureData

Class for returning key Microsoft Sentinel elements.

Initialize connector for Azure APIs.

Parameters:

res_id (str, optional) – Set the Sentinel workspace resource ID you want to use, if not specified defaults will be looked for or details can be passed separately with functions.
connect (bool, optional) – Set true if you want to connect to API on initialization, by default False
cloud (str, optional) – Specify cloud to use, overriding any configuration value. Default is to use configuration setting or public cloud if no configuration setting is available.
sub_id (str, optional) – If not specifying a resource ID the Subscription ID of the Sentinel Workspace by default None
res_grp (str, optional) – If not specifying a resource ID the Resource Group name of the Sentinel Workspace, by default None
ws_name (str, optional) – If not specifying a resource ID, the Workspace name of the Sentinel Workspace, by default None
workspace (str, optional) – Alias of ws_name

add_bookmark_to_incident(incident: str, bookmark: str)

Add a bookmark to an incident.

Parameters:

incident (str) – Either an incident name or an incident GUID
bookmark (str) – Either a bookmark name or bookmark GUID

Raises:

CloudError – If API returns error

add_tag(indicator_id: str, tag: str)

Add a tag to an existing indicator.

Parameters:

indicator_id (str) – The GUID of the indicator to add a tag to.
tag (str) – The tag to add.

add_watchlist_item(watchlist_name: str, item: Dict | Series | DataFrame, overwrite: bool = False)

Add or update an item in a Watchlist.

Parameters:

watchlist_name (str) – The name of the watchlist to add items to
item (Union[Dict, pd.Series, pd.DataFrame]) – The item to add, this can be a dictionary of valies, a Pandas Series, or DataFrame
overwrite (bool, optional) – Wether you want to overwrite an item if it already exists in the watchlist, by default False

Raises:

MsticpyUserError – If the specified Watchlist does not exist.
MsticpyUserError – If the item already exists in the Watchlist and overwrite is set to False
CloudError – If the API returns an error.

bulk_create_indicators(data: DataFrame, indicator_column: str = 'Observable', indicator_type_column: str = 'IoCType', **kwargs)

Bulk create indicators from a DataFrame.

Parameters:

data (pd.DataFrame) – A dataframe containing indicators and indicator types
indicator_column (str, optional) – The column containing indicator values to create, by default “Observable”
indicator_type_column (str, optional) – The column containing indicator type values, by default “IoCType”
confidence_column (str, optional) – The column containing indicator confidence values, by default 0 value used.

check_connected(): Check that Sentinel workspace is connected.

check_search_status(search_name: str) → bool

Check the status of a search job.

Parameters:: search_name (str) – The name of the search job to check.
Returns:: Returns True if search is ready.
Return type:: bool
Raises:: CloudError – If error in checking the search job status.

connect(auth_methods: List | None = None, tenant_id: str | None = None, silent: bool = False, **kwargs)

Authenticate with the SDK & API.

Parameters:

auth_methods (List, optional) – list of preferred authentication methods to use, by default None
tenant_id (str, optional) – Specify cloud tenant to use
silent (bool, optional) – Set true to prevent output during auth process, by default False
cloud (str, optional) – What Azure cloud to connect to. By default it will attempt to use the cloud setting from config file. If this is not set it will default to Azure Public Cloud
credential (AzureCredential, optional) – Credentials to use for authentication. This will use the credential directly and bypass the MSTICPy Azure credential selection process.

See also

msticpy.auth.azure_auth.az_connect: function to authenticate to Azure SDK

create_analytic_rule(template: str | None = None, name: str | None = None, enabled: bool = True, query: str | None = None, query_frequency: str = 'PT5H', query_period: str = 'PT5H', severity: str = 'Medium', suppression_duration: str = 'PT1H', suppression_enabled: bool = False, trigger_operator: str = 'GreaterThan', trigger_threshold: int = 0, description: str | None = None, tactics: list | None = None) → str | None

Create a Sentinel Analytics Rule.

Parameters:

template (str, optional) – The GUID or name of a templated to create the analytic from, by default None
name (str, optional) – The name to give the analytic, by default None
enabled (bool, optional) – Whether you want the analytic to be enabled once deployed, by default True
query (str, optional) – The KQL query string to use in the anlaytic, by default None
query_frequency (str, optional) – How often the query should run in ISO8601 format, by default “PT5H”
query_period (str, optional) – How far back the query should look in ISO8601 format, by default “PT5H”
severity (str, optional) – The severity to raise incidents as, by default “Medium” Options are; Informational, Low, Medium, or High
suppression_duration (str, optional) – How long to suppress duplicate alerts in ISO8601 format, by default “PT1H”
suppression_enabled (bool, optional) – Whether you want to suppress duplicates, by default False
trigger_operator (str, optional) – The operator for the trigger, by default “GreaterThan”
trigger_threshold (int, optional) – The threshold of events required to create the incident, by default 0
description (str, optional) – A description of the analytic, by default None
tactics (list, optional) – A list of MITRE ATT&CK tactics related to the analytic, by default None

Returns:

The name/ID of the analytic rule.

Return type:

Optional[str]

Raises:

MsticpyUserError – If template provided isn’t found.
CloudError – If the API returns an error.

create_bookmark(name: str, query: str, results: str | None = None, notes: str | None = None, labels: List[str] | None = None) → str | None

Create a bookmark in the Sentinel Workspace.

Parameters:

name (str) – The name of the bookmark to use
query (str) – The KQL query for the bookmark
results (str, optional) – The results of the query to include with the bookmark, by default None
notes (str, optional) – Any notes you want associated with the bookmark, by default None
labels (List[str], optional) – Any labels you want associated with the bookmark, by default None

Returns:

The name/ID of the bookmark.

Return type:

Optional[str]

Raises:

CloudError – If API returns an error.

Create a Dynamic Summary in the Sentinel Workspace.

Parameters:

summary (DynamicSummary) – DynamicSummary instance.
name (str) – The name of the dynamic summary to create
description (str) – Dynamic Summary description
data (pd.DataFrame) – The summary data

Returns:

The name/ID of the dynamic summary.

Return type:

Optional[str]

Raises:

MsticpyAzureConnectionError – If API returns an error.

Create a Sentinel Incident.

Parameters:

title (str) – The title of the incident to create
severity (str) –

The severity to assign the incident, options are:
Informational, Low, Medium, High
status (str, optional) – The status to assign the incident, by default “New” Options are: New, Active, Closed
description (str, optional) – A description of the incident, by default None
first_activity_time (datetime, optional) – The start time of the incident activity, by default None
last_activity_time (datetime, optional) – The end time of the incident activity, by default None
labels (List, optional) – Any labels to apply to the incident, by default None
bookmarks (List, optional) – A list of bookmark GUIDS you want to associate with the incident

Returns:

The name/ID of the incident.

Return type:

Optional[str]

Raises:

CloudError – If the API returns an error

create_indicator(indicator: str, ioc_type: str, name: str = 'TI Indicator', confidence: int = 0, silent: bool = False, **kwargs) → str

Create a new indicator within the Microsoft Sentinel workspace.

Parameters:

indicator (str) – The indicator to create - i.e. IP address, domain name etc.
ioc_type (str) – The type of indicator to create - can be: “dns”, “url”, “ipv4”, “ipv6”, “md5_hash”, “sha1_hash”, “sha256_hash”
name (str, optional) – A common name to give to the indicator default is ‘TI Indicator’
confidence (int, optional) – A score between 0-100 of the confidence in the indicator, defualt is 0
silent (bool, optional) – If True no output is displayed, defaults to False
description (str, optional) – An description of the indicator
labels (list, optional) – A list of string object labels to associate with the indicator
kill_chain_phases (list, optional) – A list of string objects relating to the kill chain phases an indicator is assocaited with
threat_types (list, optional) – A list of threat types associated with the indicator (list of string objects)
external_references (list, optional) – A list of URLs that provide an external reference for the indicator
valid_from (datetime, optional) – A datetime from which the indicator is valid from, defaults to now
valid_to (datetime, optional) – A datetime to which the indicator is valid until

Return type:

The ID of the created indicator

Raises:

MsticpyUserError – If invalid ioc_type or confidence value provided
CloudError – If API call fails

create_search(query: str, start: datetime | None = None, end: datetime | None = None, search_name: str | None = None, **kwargs)

Create a Search job.

Parameters:

query (str) – The KQL query to run as a job.
start (datetime, optional) – The start time for the query, by default 90 days ago.
end (datetime, optional) – The end time for the query, by default now.
search_name (str, optional) – A name to apply to the search, by default a random GUID is generated.

Raises:

CloudError – If there is an error creating the search job.

create_watchlist(watchlist_name: str, description: str, search_key: str, provider: str = 'MSTICPy', source: str = 'Notebook', data: DataFrame | None = None) → str | None

Create a new watchlist.

Parameters:

watchlist_name (str) – The name of the watchlist you want to create, this can’t be the name of an existing watchlist.
description (str) – A description of the watchlist to be created.
search_key (str) – The search key is used to optimize query performance when using watchlists for joins with other data. This should be the key column that will be used in the watchlist when joining to other data tables.
provider (str, optional) – This is the label attached to the watchlist showing who created it, by default “MSTICPy”
source (str, optional) – The source of the data to be put in the watchlist, by default “Notebook”
data (pd.DataFrame, optional) – The data you want to upload to the watchlist

Returns:

The name/ID of the watchlist.

Return type:

Optional[str]

Raises:

MsticpyUserError – Raised if the watchlist name already exists.
CloudError – If there is an issue creating the watchlist.

property default_workspace_name: Return the default workspace name.

property default_workspace_settings: Dict[str, Any] | None: Return current default workspace settings.

delete_analytic_rule(analytic_rule: str)

Delete a deployed Analytic rule from a Sentinel workspace.

Parameters:: analytic_rule (str) – The GUID or name of the analytic.
Raises:: CloudError – If the API returns an error.

delete_bookmark(bookmark: str)

Delete the selected bookmark.

Parameters:: bookmark (str, optional) – The name or GIUD of the bookmark to delete.
Raises:: CloudError – If the API returns an error.

delete_dynamic_summary(summary_id: str)

Delete the Dynamic Summary for summary_id.

Parameters:: summary_id (str, optional) – The UUID of the summary to delete.
Raises:: MsticpyAzureConnectionError – If the API returns an error.

delete_indicator(indicator_id: str)

Delete a specific TI indicator.

Parameters:: indicator_id (str) – The GUID of the indicator to delete
Raises:: CloudError – If API call fails

delete_search(search_name: str)

Delete a search result.

Parameters:: search_name (str) – The name of the search to delete.
Raises:: CloudError – If an error occurs when attempting to delete the search

delete_watchlist(watchlist_name: str)

Delete a selected Watchlist.

Parameters:

watchlist_name (str) – The name of the Watchlist to deleted

Raises:

MsticpyUserError – If Watchlist does not exist.
CloudError – If the API returns an error.

delete_watchlist_item(watchlist_name: str, watchlist_item_id: str)

Delete a Watchlist item.

Parameters:

watchlist_name (str) – The name of the watchlist with the item to be deleted
watchlist_item_id (str) – The watchlist item ID to delete

Raises:

MsticpyUserError – If the specified Watchlist does not exist.
CloudError – If the API returns an error.

df_to_dynamic_summaries() → List[DynamicSummary]

Return a list of DynamicSummary objects from a DataFrame of summaries.

Parameters:: data (pd.DataFrame) – DataFrame containing dynamic summaries
Returns:: List of Dynamic Summary objects.
Return type:: List[DynamicSummary]

Examples

Use the following steps to obtain a list of dynamic summaries from MS Sentinel and convert to DynamicSummary objects.

query = \"\"\"
    DynamicSummary
    | where <some filter criteria>
    | where SummaryStatus == "Active" or SummaryDataType == "SummaryItem"
\"\"\"
data = qry_prov.exec_query(query)
dyn_summaries = df_to_dynamic_summaries(data)

df_to_dynamic_summary() → DynamicSummary

Return a single DynamicSummary object from a DataFrame.

Parameters:: data (pd.DataFrame) – DataFrame containing a single dynamic summary plus summary items.
Returns:: The DynamicSummary object.
Return type:: DynamicSummary

Examples

Use the following steps to query a single dynamic summary from MS Sentinel and convert to a DynamicSummary object.

query = \"\"\"
    DynamicSummary
    | where SummaryId == "26b95b5e-2645-4d33-91a7-ea3c1b8b4b8b"
    | where SummaryStatus == "Active" or SummaryDataType == "SummaryItem"
\"\"\"
data = qry_prov.exec_query(query)
dyn_summaries = df_to_dynamic_summary(data)

get_alert_rules() → DataFrame

Return all Microsoft Sentinel alert rules for a workspace.

Returns:: A table of the workspace’s alert rules.
Return type:: pd.DataFrame

get_all_indicators(limit: int | None = None, orderby: str | None = None) → DataFrame

Return all TI indicators in a Microsoft Sentinel workspace.

Parameters:

limit (int, optional) – If set returns top n results
orderby (Optional[str], optional) – Order results by a specific column

Returns:

A table of the custom hunting queries.

Return type:

pd.DataFrame

get_analytic_rules() → DataFrame

Return all Microsoft Sentinel alert rules for a workspace.

Returns:: A table of the workspace’s alert rules.
Return type:: pd.DataFrame

get_bookmarks() → DataFrame

Return a list of Bookmarks from a Sentinel workspace.

Returns:: A set of bookmarks.
Return type:: pd.DataFrame

get_dynamic_summary(summary_id: str, summary_items=False) → DynamicSummary

Return DynamicSummary for ID.

Parameters:

summary_id (str) – The ID of the Dynamic summary object.
summary_items (bool, optional) – Use a data query to retrieve the dynamic summary along with summary items (data records), by default, false.

Returns:

DynamicSummary object.

Return type:

DynamicSummary

Raises:

MsticpyAzureConnectionError – If API returns an error.

get_entities(incident: str) → list

Get the entities from an incident.

Parameters:: incident (str) – Incident GUID or Name .
Returns:: A list of entities.
Return type:: list

get_hunting_queries() → DataFrame

Return all custom hunting queries in a Microsoft Sentinel workspace.

Returns:: A table of the custom hunting queries.
Return type:: pd.DataFrame

get_incident(incident: str, entities: bool = False, alerts: bool = False, comments: bool = False, bookmarks: bool = False) → DataFrame

Get details on a specific incident.

Parameters:

incident (str) – Incident ID GUID.
entities (bool, optional) – If True include all entities in the response. Default is False.
alerts (bool, optional) – If True include all alerts in the response. Default is False.
comments (bool, optional) – If True include all comments in the response. Default is False.
bookmarks (bool, optional) – If True include all bookmarks in the response. Default is False.

Returns:

Table containing incident details.

Return type:

pd.DataFrame

Raises:

CloudError – If incident could not be retrieved.

get_incident_alerts(incident: str) → list

Get the alerts from an incident.

Parameters:: incident (str) – Incident GUID or Name.
Returns:: A list of alerts.
Return type:: list

get_incident_bookmarks(incident: str) → list

Get the comments from an incident.

Parameters:: incident (str) – Incident GUID or name.
Returns:: A list of bookmarks.
Return type:: list

get_incident_comments(incident: str) → list

Get the comments from an incident.

Parameters:: incident (str) – Incident GUID or Name.
Returns:: A list of comments.
Return type:: list

get_incidents(params: dict | None = None) → DataFrame

Get a list of incident for a Sentinel workspace.

Parameters:: params (Optional[dict], optional) – Additional parameters to pass to the API call, by default None
Returns:: A table of incidents.
Return type:: pd.DataFrame
Raises:: CloudError – If incidents could not be retrieved.

get_indicator(indicator_id: str) → dict

Get a specific indicator by its ID.

Parameters:: indicator_id (str) – The GUID of the indicator to get
Returns:: Indicator details
Return type:: dict
Raises:: CloudError – If API call fails.

get_metrics(metrics: str, resource_id: str, sub_id: str, sample_time: str = 'hour', start_time: int = 30) → Dict[str, DataFrame]

Return specified metrics on Azure Resource.

Parameters:

metrics (str) – A string list of metrics you wish to collect (https://docs.microsoft.com/en-us/azure/azure-monitor/platform/metrics-supported)
resource_id (str) – The resource ID of the resource to collet the metrics from
sub_id (str) – The subscription ID that the resource is part of
sample_time (str (Optional)) – You can select to collect the metrics every hour of minute - default is hour Accepted inputs = ‘hour’ or ‘minute’
start_time (int (Optional)) – The number of days prior to today to collect metrics for, default is 30

Returns:

results – A Dictionary of DataFrames containing the metrics details

Return type:

dict

get_network_details(network_id: str, sub_id: str) → Tuple[DataFrame, DataFrame]

Return details related to an Azure network interface and associated NSG.

Parameters:

network_id (str) – The ID of the network interface to return details on
sub_id (str) – The subscription ID that the network interface is part of

Returns:

details – A dictionary of items related to the network interface

Return type:

dict

get_resource_details(sub_id: str, resource_id: str | None = None, resource_details: dict | None = None) → dict

Return the details of a specific Azure resource.

Parameters:

resource_id (str, optional) – The ID of the resource to get details on
resource_details (dict, optional) –

If ID is unknown provide the following details:
-resource_group_name -resource_provider_namespace -resource_type -resource_name -parent_resource_path
sub_id (str) – The ID of the subscription to get resources from

Returns:

resource_details – The details of the requested resource

Return type:

dict

classmethod get_resource_id_from_url(portal_url: str) → str: Return resource ID components from Sentinel portal URL.

get_resources(sub_id: str, rgroup: str | None = None, get_props: bool = False) → DataFrame

Return details on all resources in a subscription or Resource Group.

Parameters:

sub_id (str) – The subscription ID to get resources for
rgroup (str (Optional)) – The name of a Resource Group to get resources for
get_props (bool (Optional)) – Set to True if you want to get the full properties of every resource Warning this may be a slow process depending on the number of resources

Returns:

A dataframe of resource details

Return type:

pd.DataFrame

get_sentinel_workspaces(sub_id: str) → Dict[str, str]

Return a list of Microsoft Sentinel workspaces in a Subscription.

Parameters:: sub_id (str) – The subscription ID to get a list of workspaces from. If not provided it will attempt to get sub_id from config files.
Returns:: A dictionary of workspace names and ids
Return type:: Dict

get_subscription_info(sub_id: str) → dict

Get information on a specific subscription.

Parameters:: sub_id (str) – The ID of the subscription to return details on.
Returns:: Details on the selected subscription.
Return type:: dict
Raises:: MsticpyNotConnectedError – If .connect() has not been called.

get_subscriptions() → DataFrame

Get details of all subscriptions within the tenant.

Returns:: Details of the subscriptions present in the users tenant.
Return type:: pd.DataFrame
Raises:: MsticpyNotConnectedError – If .connect() has not been called

get_ti_metrics() → DataFrame

Return metrics about TI indicators in a Microsoft Sentinel workspace.

Returns:: A table of the custom hunting queries.
Return type:: pd.DataFrame

classmethod get_workspace_details_from_url(portal_url: str) → Dict[str, Dict[str, str]]

Return workspace settings from portal URL.

Parameters:: portal_url (str) – URL from Sentinel Azure portal
Return type:: Dict[str, Dict[str, str]]

classmethod get_workspace_id(workspace_name: str, subscription_id: str = '', resource_group: str = '') → str | None

Return the workspace ID given workspace name.

Parameters:

workspace_name (str) – Workspace name (case insensitive)
subscription_id (str, optional) – Azure subscription UUID, by default “”
resource_group (str, optional) – Azure resource group name, by default “”

Returns:

The ID of the workspace if found, else None

Return type:

Optional[str]

classmethod get_workspace_name(workspace_id: str | None = None, resource_id: str | None = None) → str | None

Return resolved name from workspace ID or resource ID.

Parameters:

workspace_id (Optional[str], optional) – The UUID of the Sentinel workspace, by default None
resource_id (Optional[str], optional) – The Resource ID string of the workspace, by default None

Returns:

The workspace name, if found, else None

Return type:

Optional[str]

Raises:

ValueError – If neither workspace_id or resource_id parameters are supplied.

classmethod get_workspace_settings(workspace_id: str | None = None, resource_id: str | None = None)

Return resolved workspace settings from workspace ID or resource ID.

Parameters:

workspace_id (Optional[str], optional) – The UUID of the Sentinel workspace, by default None
resource_id (Optional[str], optional) – The Resource ID string of the workspace, by default None

Returns:

The workspace name, if found, else None

Return type:

Dict[str, str]

Raises:

ValueError – If neither workspace_id or resource_id parameters are supplied.

classmethod get_workspace_settings_by_name(workspace_name: str, subscription_id: str = '', resource_group: str = '')

Return the workspace ID given workspace name.

Parameters:

workspace_name (str) – Workspace name (case insensitive)
subscription_id (str, optional) – Azure subscription UUID, by default “”
resource_group (str, optional) – Azure resource group name, by default “”

Returns:

The ID of the workspace if found, else None

Return type:

Optional[str]

list_alert_rules() → DataFrame

Return all Microsoft Sentinel alert rules for a workspace.

Returns:: A table of the workspace’s alert rules.
Return type:: pd.DataFrame

list_analytic_rules() → DataFrame

Return all Microsoft Sentinel alert rules for a workspace.

Returns:: A table of the workspace’s alert rules.
Return type:: pd.DataFrame

list_analytic_templates() → DataFrame

List Analytic Templates.

Returns:: A DataFrame containing the analytics templates
Return type:: pd.DataFrame
Raises:: CloudError – If a valid result is not returned.

list_bookmarks() → DataFrame

Return a list of Bookmarks from a Sentinel workspace.

Returns:: A set of bookmarks.
Return type:: pd.DataFrame

list_data_connectors() → DataFrame

List deployed data connectors.

Returns:: A DataFrame containing the deployed data connectors
Return type:: pd.DataFrame
Raises:: CloudError – If a valid result is not returned.

list_dynamic_summaries() → DataFrame

Return current list of Dynamic Summaries from a Sentinel workspace.

Returns:: The current Dynamic Summary objects.
Return type:: pd.DataFrame

list_hunting_queries() → DataFrame

Return all custom hunting queries in a Microsoft Sentinel workspace.

Returns:: A table of the custom hunting queries.
Return type:: pd.DataFrame

list_incidents(params: dict | None = None) → DataFrame

Get a list of incident for a Sentinel workspace.

Parameters:: params (Optional[dict], optional) – Additional parameters to pass to the API call, by default None
Returns:: A table of incidents.
Return type:: pd.DataFrame
Raises:: CloudError – If incidents could not be retrieved.

list_saved_queries() → DataFrame

Return all saved queries in a Microsoft Sentinel workspace.

Returns:: A table of the custom hunting queries.
Return type:: pd.DataFrame

list_sentinel_workspaces(sub_id: str) → Dict[str, str]

Return a list of Microsoft Sentinel workspaces in a Subscription.

Parameters:: sub_id (str) – The subscription ID to get a list of workspaces from. If not provided it will attempt to get sub_id from config files.
Returns:: A dictionary of workspace names and ids
Return type:: Dict

list_watchlist_items(watchlist_name: str) → DataFrame

List items in a watchlist.

Parameters:: watchlist_name (str) – The name of the watchlist to get items from
Returns:: A DataFrame containing the watchlists
Return type:: pd.DataFrame
Raises:: CloudError – If a valid result is not returned.

list_watchlists() → DataFrame

List Deployed Watchlists.

Returns:: A DataFrame containing the watchlists
Return type:: pd.DataFrame
Raises:: CloudError – If a valid result is not returned.

classmethod new_dynamic_summary(**kwargs)

Return a new DynamicSummary object.

Notes

See the DynamicSummary class documentation for details of expected parameters.