msticpy.context.http_provider module

HTTP Lookup base class.

Input can be a single item or a pandas DataFrame containing multiple items. Processing may require a an API key and processing performance may be limited to a specific number of requests per minute for the account type that you have.

class msticpy.context.http_provider.APILookupParams(path='', verb='GET', full_url=False, headers=<factory>, params=<factory>, data=<factory>, auth_type='', auth_str=<factory>, sub_type='')

Bases: object

HTTP Lookup Params definition.

Parameters:

  • path (str)

  • verb (str)

  • full_url (bool)

  • headers (dict[str, str])

  • params (dict[str, str | float])

  • data (dict[str, str])

  • auth_type (str)

  • auth_str (list[str])

  • sub_type (str)

auth_str: list[str]
auth_type: str = ''
data: dict[str, str]
full_url: bool = False
headers: dict[str, str]
params: dict[str, str | float]
path: str = ''
sub_type: str = ''
verb: str = 'GET'
class msticpy.context.http_provider.HttpProvider(*, timeout=None, ApiID=None, AuthKey=None, Instance=None)

Bases: Provider

HTTP Generic lookup provider base class.

For subclasses:

Define Base URL of the service

_BASE_URL = "https://my.api.org/"

Define query parameters for different item types (keys)

..code:: python

_QUERIES: dict[str, APILookupParams] = {}

For example:

_QUERIES = {
# Community API
"ipv4": APILookupParams(
    path="/v3/community/{observable}",
    headers={"key": "{AuthKey}"},
),
# Enterprise API Quick Lookup
"ipv4-quick": APILookupParams(
    ...

Define list of required __init__ params

_REQUIRED_PARAMS: list[str] = []

For example:

_REQUIRED_PARAMS = ["AuthKey"]

In __init__

Be sure to call

super().__init__(**kwargs)``

Supply any additional checkers/pre-processors with

See also

PreProcessor, HttpTIProvider

Initialize the class.

Parameters:

  • timeout (int | None)

  • ApiID (str | None)

  • AuthKey (str | None)

  • Instance (str | None)

classmethod is_known_type(item_type)

Return True if this a known IoC Type.

Parameters:

item_type (str) – IoCType string to test

Returns:

True if known type.

Return type:

bool

is_supported_type(item_type)

Return True if the passed type is supported.

Parameters:

  • item_type (Union[str, IoCType]) – type name or instance

  • self (Self)

Returns:

True if supported.

Return type:

bool

property item_query_defs: dict[str, Any]

Return current dictionary of IoC query/request definitions.

Returns:

IoC query/request definitions keyed by IoCType

Return type:

dict[str, Any]

abstractmethod lookup_item(item, item_type=None, query_type=None)

Lookup from an item value.

Parameters:

  • item (str) – item to lookup

  • item_type (str, optional) – The Type of the item to lookup, by default None (type will be inferred)

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the item_value will be returned.

  • self (Self)

Returns:

The lookup result: result - Positive/Negative, details - Lookup Details (or status if failure), raw_result - Raw Response reference - URL of the item

Return type:

pd.DataFrame

Raises:

NotImplementedError – If attempting to use an HTTP method or authentication protocol that is not supported.

Notes

Note: this method uses memoization (lru_cache) to cache results for a particular item to try avoid repeated network calls for the same item.

lookup_items(data, item_col=None, item_type_col=None, query_type=None)

Lookup collection of items.

Parameters:

  • data (Union[pd.DataFrame, dict[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in item_col parameter) 2. Dict of items 3. Iterable of items

  • item_col (str, optional) – DataFrame column to use for items, by default None

  • item_type_col (str, optional) – DataFrame column to use for types, by default None

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the type will be returned.

  • self (Self)

Returns:

DataFrame of results.

Return type:

pd.DataFrame

async lookup_items_async(data, item_col=None, item_type_col=None, query_type=None, *, prog_counter=None, item_type=None)

Lookup collection of items.

Parameters:

  • data (Union[pd.DataFrame, dict[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in item_col parameter) 2. Dict of items, Type 3. Iterable of items - Types will be inferred

  • item_col (str, optional) – DataFrame column to use for items, by default None

  • item_type_col (str, optional) – DataFrame column to use for Types, by default None

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the item will be returned.

  • prog_counter (ProgressCounter, Optional) – Progress Counter to display progess of IOC searches.

  • item_type (str, Optional) – Type of item

  • self (Self)

Returns:

DataFrame of results.

Return type:

pd.DataFrame

property name: str

Return the name of the provider.

static resolve_item_type(item)

Return IoCType determined by ItemExtract.

Parameters:

item (str) – Item string

Returns:

IoCType (or unknown if type could not be determined)

Return type:

str

property supported_types: list[str]

Return list of supported types for this provider.

Returns:

List of supported type names

Return type:

list[str]

classmethod usage()

Print usage of provider.

Return type:

None