msticpy.context.contextlookup module
Module for ContextLookup classes.
Input can be a single Observable observable or a pandas DataFrame containing multiple observables. Processing may require a an API key and processing performance may be limited to a specific number of requests per minute for the account type that you have.
- class msticpy.context.contextlookup.ContextLookup(providers: List[str] | None = None, **kwargs)
Bases:
Lookup
Observable lookup from providers.
Initialize TILookup instance.
- Parameters:
primary_providers (Optional[List[Provider]], optional) – Primary Providers, by default None
secondary_providers (Optional[List[Provider]], optional) – Secondary Providers, by default None
providers (Optional[List[str]], optional) – List of provider names to load, by default all available providers are loaded. To see the list of available providers call TILookup.list_available_providers(). Note: if primary_provides or secondary_providers is specified This will override the providers list.
- PACKAGE: str = 'contextproviders'
- add_provider(provider: Provider, name: str | None = None, primary: bool = True)
Add a provider to the current collection.
- Parameters:
provider (TIProvider) – Provider instance
name (str, optional) – The name to use for the provider (overrides the class name of provider)
primary (bool, optional) – “primary” or “secondary” if False, by default “primary”
- property available_providers: List[str]
Return a list of builtin and plugin providers.
- Returns:
List of TI Provider classes.
- Return type:
List[str]
- classmethod browse(data: DataFrame, severities: List[str] | None = None, **kwargs)
Return TI Results list browser.
- Parameters:
data (pd.DataFrame) – TI Results data from TIProviders
severities (Optional[List[str]], optional) – A list of the severity classes to show. By default these are [‘warning’, ‘high’]. Pass [‘information’, ‘warning’, ‘high’] to see all results.
kwargs – passed to SelectItem constructor.
- Returns:
SelectItem browser for TI Data.
- Return type:
- classmethod browse_results(data: DataFrame, severities: List[str] | None = None, **kwargs)
Return TI Results list browser.
- Parameters:
data (pd.DataFrame) – TI Results data from TIProviders
severities (Optional[List[str]], optional) – A list of the severity classes to show. By default these are [‘warning’, ‘high’]. Pass [‘information’, ‘warning’, ‘high’] to see all results.
kwargs – passed to SelectItem constructor.
- Returns:
SelectItem browser for TI Data.
- Return type:
- property configured_providers: List[str]
Return a list of available providers that have configuration details present.
- Returns:
List of TI Provider classes.
- Return type:
List[str]
- disable_provider(providers: str | Iterable[str])
Set the provider as secondary (not used by default).
- Parameters:
providers (Union[str, Iterable[str]) – Provider name or list of names. Use list_available_providers() to see the list of loaded providers.
- Raises:
ValueError – If the provider name is not recognized.
- enable_provider(providers: str | Iterable[str])
Set the provider(s) as primary (used by default).
- Parameters:
providers (Union[str, Iterable[str]) – Provider name or list of names. Use list_available_providers() to see the list of loaded providers.
- Raises:
ValueError – If the provider name is not recognized.
- classmethod list_available_providers(show_query_types=False, as_list: bool = False) List[str] | None
Print a list of builtin providers with optional usage.
- Parameters:
show_query_types (bool, optional) – Show query types supported by providers, by default False
as_list (bool, optional) – Return list of providers instead of printing to stdout. Note: if you specify show_query_types this will be printed irrespective of this parameter setting.
- Returns:
A list of provider names (if return_list=True)
- Return type:
Optional[List[str]]
- property loaded_providers: Dict[str, Provider]
Return dictionary of loaded providers.
- Returns:
[description]
- Return type:
Dict[str, TIProvider]
- lookup_item(item: str, item_type: str | None = None, query_type: str | None = None, providers: List[str] | None = None, default_providers: List[str] | None = None, prov_scope: str = 'primary', **kwargs) DataFrame
Lookup single item in active providers.
- Parameters:
item (str) – item to lookup
item_type (str, optional) – One of ItemExtract.ItemType, by default None If none, the Item type will be inferred
query_type (str, optional) – The query type (e.g. rep, info, malware)
providers (List[str]) – Explicit list of providers to use
default_providers (Optional[List[str]], optional) – Used by pivot functions as a fallback to providers. If providers is specified, it will override this parameter.
prov_scope (str, optional) – Use “primary”, “secondary” or “all” providers, by default “primary”
kwargs – Additional arguments passed to the underlying provider(s)
- Returns:
DataFrame of results
- Return type:
pd.DataFrame
- lookup_items(data: DataFrame | Mapping[str, str] | Iterable[str], item_col: str | None = None, item_type_col: str | None = None, query_type: str | None = None, providers: List[str] | None = None, default_providers: List[str] | None = None, prov_scope: str = 'primary', **kwargs) DataFrame
Lookup a collection of items.
- Parameters:
data (Union[pd.DataFrame, Mapping[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in item_col parameter) 2. Mapping (e.g. a dict) of [item, ItemType] 3. Iterable of items - ItemTypes will be inferred
item_col (str, optional) – DataFrame column to use for items, by default None (“col” and “column” are also aliases for this parameter)
item_type_col (str, optional) – DataFrame column to use for ItemTypes, by default None
query_type (str, optional) – The item query type (e.g. rep, info, malware)
providers (List[str]) – Explicit list of providers to use
default_providers (Optional[List[str]], optional) – Used by pivot functions as a fallback to providers. If providers is specified, it will override this parameter.
prov_scope (str, optional) – Use “primary”, “secondary” or “all” providers, by default “primary”
progress (bool) – Use progress bar to track completion, by default True
kwargs – Additional arguments passed to the underlying provider(s)
- Returns:
DataFrame of results
- Return type:
pd.DataFrame
- lookup_items_sync(data: DataFrame | Mapping[str, str] | Iterable[str], item_col: str | None = None, item_type_col: str | None = None, query_type: str | None = None, providers: List[str] | None = None, default_providers: List[str] | None = None, prov_scope: str = 'primary', **kwargs) DataFrame
Lookup a collection of items.
- Parameters:
data (Union[pd.DataFrame, Mapping[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in item_col parameter) 2. Mapping (e.g. a dict) of [item, ItemType] 3. Iterable of items - ItemTypes will be inferred
item_col (str, optional) – DataFrame column to use for items, by default None (“col” and “column” are also aliases for this parameter)
item_type_col (str, optional) – DataFrame column to use for ItemTypes, by default None
query_type (str, optional) – The item query type (e.g. rep, info, malware)
providers (List[str]) – Explicit list of providers to use
default_providers (Optional[List[str]], optional) – Used by pivot functions as a fallback to providers. If providers is specified, it will override this parameter.
prov_scope (str, optional) – Use “primary”, “secondary” or “all” providers, by default “primary”
kwargs – Additional arguments passed to the underlying provider(s)
- Returns:
DataFrame of results
- Return type:
pd.DataFrame
- lookup_observable(observable: str, observable_type: str | None = None, query_type: str | None = None, providers: List[str] | None = None, default_providers: List[str] | None = None, prov_scope: str = 'primary', **kwargs) DataFrame
Lookup single observable in active providers.
- Parameters:
observable (str) – observable
observable_type (str, optional) – One of IoCExtract.IOCType, by default None If none, the Observable type will be inferred
query_type (str, optional) – The observable query type (e.g. rep, info, malware)
providers (List[str]) – Explicit list of providers to use
default_providers (Optional[List[str]] = None,) – Used by pivot functions as a fallback to providers. If providers is specified, it will override this parameter.
prov_scope (str, optional) – Use “primary”, “secondary” or “all” providers, by default “primary”
kwargs – Additional arguments passed to the underlying provider(s)
- Returns:
The result returned as a DataFrame: bool indicates whether a TI record was found in any provider list has an entry for each provider result
- Return type:
pd.DataFrame
- lookup_observables(data: DataFrame | Mapping[str, str] | Iterable[str], obs_col: str | None = None, obs_type_col: str | None = None, query_type: str | None = None, providers: List[str] | None = None, default_providers: List[str] | None = None, prov_scope: str = 'primary', **kwargs) DataFrame
Lookup a collection of Observables.
- Parameters:
data (Union[pd.DataFrame, Mapping[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in obs_col parameter) 2. Mapping (e.g. a dict) of [observable, ObservableType] 3. Iterable of observables - ObservableTypes will be inferred
obs_col (str, optional) – DataFrame column to use for observables, by default None (“col” and “column” are also aliases for this parameter)
obs_type_col (str, optional) – DataFrame column to use for ObservableTypes, by default None
query_type (str, optional) – The observable query type (e.g. rep, info, malware)
providers (List[str]) – Explicit list of providers to use
default_providers (Optional[List[str]], optional) – Used by pivot functions as a fallback to providers. If providers is specified, it will override this parameter.
prov_scope (str, optional) – Use “primary”, “secondary” or “all” providers, by default “primary”
kwargs – Additional arguments passed to the underlying provider(s)
- Returns:
DataFrame of results
- Return type:
pd.DataFrame
- lookup_observables_sync(data: DataFrame | Mapping[str, str] | Iterable[str], obs_col: str | None = None, obs_type_col: str | None = None, query_type: str | None = None, providers: List[str] | None = None, default_providers: List[str] | None = None, prov_scope: str = 'primary', **kwargs) DataFrame
Lookup a collection of IoCs.
- Parameters:
data (Union[pd.DataFrame, Mapping[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in obs_col parameter) 2. Mapping (e.g. a dict) of [observable, IoCType] 3. Iterable of observables - IoCTypes will be inferred
obs_col (str, optional) – DataFrame column to use for observables, by default None (“col” and “column” are also aliases for this parameter)
obs_type_col (str, optional) – DataFrame column to use for IoCTypes, by default None
query_type (str, optional) – The ioc query type (e.g. rep, info, malware)
providers (List[str]) – Explicit list of providers to use
default_providers (Optional[List[str]], optional) – Used by pivot functions as a fallback to providers. If providers is specified, it will override this parameter.
prov_scope (str, optional) – Use “primary”, “secondary” or “all” providers, by default “primary”
kwargs – Additional arguments passed to the underlying provider(s)
- Returns:
DataFrame of results
- Return type:
pd.DataFrame
- property provider_status: Iterable[str]
Return loaded provider status.
- Returns:
List of providers and descriptions.
- Return type:
Iterable[str]
- provider_usage()
Print usage of loaded providers.
- classmethod reload_provider_settings()
Reload provider settings from config.
- reload_providers()
Reload settings and provider classes.