msticpy.context.lookup module

Generic Module for Lookup classes.

Input can be a single item or a pandas DataFrame containing multiple items. Processing may require a an API key and processing performance may be limited to a specific number of requests per minute for the account type that you have.

class msticpy.context.lookup.Lookup(providers: list[str] | None = None, *, primary_providers: list[Provider] | None = None, secondary_providers: list[Provider] | None = None)

Bases: object

Item lookup from providers.

Initialize TILookup instance.

Parameters:

primary_providers (Optional[list[Provider]], optional) – Primary Providers, by default None
secondary_providers (Optional[list[Provider]], optional) – Secondary Providers, by default None
providers (Optional[list[str]], optional) – List of provider names to load, by default all available providers are loaded. To see the list of available providers call TILookup.list_available_providers(). Note: if primary_provides or secondary_providers is specified This will override the providers list.

CUSTOM_PROVIDERS: ClassVar[dict[str, type[Provider]]]

PACKAGE: ClassVar[str] = ''

PROVIDERS: ClassVar[dict[str, tuple[str, str]]] = {}

add_provider(provider: Provider, name: str | None = None, *, primary: bool = True) → None

Add a provider to the current collection.

Parameters:

provider (TIProvider) – Provider instance
name (str, optional) – The name to use for the provider (overrides the class name of provider)
primary (bool, optional) – “primary” or “secondary” if False, by default “primary”

property available_providers: list[str]

Return a list of builtin and plugin providers.

Returns:: List of TI Provider classes.
Return type:: list[str]

classmethod browse(data: pd.DataFrame, severities: list[str] | None = None, *, height: str = '300px') → SelectItem | None

Return TI Results list browser.

Parameters:

data (pd.DataFrame) – TI Results data from TIProviders
severities (Optional[list[str]], optional) – A list of the severity classes to show. By default these are [‘warning’, ‘high’]. Pass [‘information’, ‘warning’, ‘high’] to see all results.
height (str, Optional) – Height of the widget

Returns:

SelectItem browser for TI Data.

Return type:

SelectItem

classmethod browse_results(data: pd.DataFrame, severities: list[str] | None = None, *, height: str = '300px') → SelectItem | None

Return TI Results list browser.

Parameters:

data (pd.DataFrame) – TI Results data from TIProviders
severities (Optional[list[str]], optional) – A list of the severity classes to show. By default these are [‘warning’, ‘high’]. Pass [‘information’, ‘warning’, ‘high’] to see all results.
height (str, Optional) – Height of the widget

Returns:

SelectItem browser for TI Data.

Return type:

SelectItem

property configured_providers: list[str]

Return a list of available providers that have configuration details present.

Returns:: List of TI Provider classes.
Return type:: list[str]

disable_provider(providers: str | Iterable[str]) → None

Set the provider as secondary (not used by default).

Parameters:: providers (Union[str, Iterable[str]) – Provider name or list of names. Use list_available_providers() to see the list of loaded providers.
Raises:: ValueError – If the provider name is not recognized.

enable_provider(providers: str | Iterable[str]) → None

Set the provider(s) as primary (used by default).

Parameters:: providers (Union[str, Iterable[str]) – Provider name or list of names. Use list_available_providers() to see the list of loaded providers.
Raises:: ValueError – If the provider name is not recognized.

classmethod import_provider(provider: str) → type[Provider]: Import provider class.

classmethod list_available_providers(*, show_query_types: bool = False, as_list: bool = False) → list[str] | None

Print a list of builtin providers with optional usage.

Parameters:

show_query_types (bool, optional) – Show query types supported by providers, by default False
as_list (bool, optional) – Return list of providers instead of printing to stdout. Note: if you specify show_query_types this will be printed irrespective of this parameter setting.

Returns:

A list of provider names (if return_list=True)

Return type:

Optional[list[str]]

property loaded_providers: dict[str, Provider]

Return dictionary of loaded providers.

Returns:: [description]
Return type:: dict[str, TIProvider]

lookup_item(item: str, item_type: str | None = None, query_type: str | None = None, providers: list[str] | None = None, default_providers: list[str] | None = None, prov_scope: str = 'primary', *, show_not_supported: bool = False, start: dt.datetime | None = None, end: dt.datetime | None = None) → pd.DataFrame

Lookup single item in active providers.

Parameters:

item (str) – item to lookup
item_type (str, optional) – One of ItemExtract.ItemType, by default None If none, the Item type will be inferred
query_type (str, optional) – The query type (e.g. rep, info, malware)
providers (list[str]) – Explicit list of providers to use
default_providers (Optional[list[str]], optional) – Used by pivot functions as a fallback to providers. If providers is specified, it will override this parameter.
prov_scope (str, optional) – Use “primary”, “secondary” or “all” providers, by default “primary”
show_not_supported (bool) – If True, display unsupported items. Defaults to False
start (dt.datetime) – If supported by the provider, start time for the item’s validity
end (dt.datetime) – If supported by the provider, end time for the item’s validity

Returns:

DataFrame of results

Return type:

pd.DataFrame

Lookup a collection of items.

Parameters:

data (Union[pd.DataFrame, Mapping[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in item_col parameter) 2. Mapping (e.g. a dict) of [item, ItemType] 3. Iterable of items - ItemTypes will be inferred
item_col (str, optional) – DataFrame column to use for items, by default None (“col” and “column” are also aliases for this parameter)
item_type_col (str, optional) – DataFrame column to use for ItemTypes, by default None
query_type (str, optional) – The item query type (e.g. rep, info, malware)
providers (list[str]) – Explicit list of providers to use
default_providers (Optional[list[str]], optional) – Used by pivot functions as a fallback to providers. If providers is specified, it will override this parameter.
prov_scope (str, optional) – Use “primary”, “secondary” or “all” providers, by default “primary”
show_not_supported (bool) – If True, display unsupported items. Defaults to False
start (dt.datetime) – If supported by the provider, start time for the item’s validity
end (dt.datetime) – If supported by the provider, end time for the item’s validity
progress (bool) – Use progress bar to track completion, by default True

Returns:

DataFrame of results

Return type:

pd.DataFrame

Lookup a collection of items.

Parameters:

data (Union[pd.DataFrame, Mapping[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in item_col parameter) 2. Mapping (e.g. a dict) of [item, ItemType] 3. Iterable of items - ItemTypes will be inferred
item_col (str, optional) – DataFrame column to use for items, by default None (“col” and “column” are also aliases for this parameter)
item_type_col (str, optional) – DataFrame column to use for ItemTypes, by default None
query_type (str, optional) – The item query type (e.g. rep, info, malware)
providers (list[str]) – Explicit list of providers to use
default_providers (Optional[list[str]], optional) – Used by pivot functions as a fallback to providers. If providers is specified, it will override this parameter.
prov_scope (str, optional) – Use “primary”, “secondary” or “all” providers, by default “primary”
col (str, Optional) – Name of the column holding the data
column (str, Optional) – Name of the column holding the data
show_not_supported (bool, Optional) – Set to True to include unsupported items in the result DF. Defaults to False
show_bad_item (bool, Optional) – Set to True to include invalid items in the result DF. Defaults to False

Returns:

DataFrame of results

Return type:

pd.DataFrame

property provider_status: Iterable[str]

Return loaded provider status.

Returns:: List of providers and descriptions.
Return type:: Iterable[str]

provider_usage() → None: Print usage of loaded providers.

classmethod reload_provider_settings() → None: Reload provider settings from config.

reload_providers() → None: Reload settings and provider classes.

static result_to_df(item_lookup: DataFrame) → DataFrame

Return DataFrame representation of Lookup response.

Parameters:: item_lookup (pd.DataFrame) – Output from lookup_item
Returns:: The response as a DataFrame with a row for each provider response.
Return type:: pd.DataFrame

set_provider_state(prov_dict: dict[str, bool]) → None

Set a dict of providers to primary/secondary.

Parameters:: prov_dict (dict[str, bool]) – Dictionary of provider name and bool - True if enabled/primary, False if disabled/secondary.

class msticpy.context.lookup.ProgressCounter(total: int)

Bases: object

Progress counter for async tasks.

Initialize the class.

async decrement(increment: int = 1) → None: Decrement the counter.

async get_remaining() → int: Get the current remaining count.