msticpy.sectools.tiproviders package

Submodules

msticpy.sectools.tiproviders.alienvault_otx module

AlienVault OTX Provider.

Input can be a single IoC observable or a pandas DataFrame containing multiple observables. Processing may require a an API key and processing performance may be limited to a specific number of requests per minute for the account type that you have.

class msticpy.sectools.tiproviders.alienvault_otx.OTX(**kwargs)

Bases: msticpy.sectools.tiproviders.http_base.HttpProvider

AlientVault OTX Lookup.

Set OTX specific settings.

property ioc_query_defs: Dict[str, Any]

Return current dictionary of IoC query/request definitions.

Returns

IoC query/requist definitions keyed by IoCType

Return type

Dict[str, Any]

classmethod is_known_type(ioc_type: str) → bool

Return True if this a known IoC Type.

Parameters

ioc_type (str) – IoCType string to test

Returns

True if known type.

Return type

bool

is_supported_type(ioc_type: Union[str, msticpy.sectools.iocextract.IoCType]) → bool

Return True if the passed type is supported.

Parameters

ioc_type (Union[str, IoCType]) – IoC type name or instance

Returns

True if supported.

Return type

bool

lookup_ioc(ioc: str, ioc_type: str = None, query_type: str = None, **kwargs) → msticpy.sectools.tiproviders.ti_provider_base.LookupResult

Lookup a single IoC observable.

Parameters

  • ioc (str) – IoC observable

  • ioc_type (str, optional) – IocType, by default None (type will be inferred)

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

The lookup result: result - Positive/Negative, details - Lookup Details (or status if failure), raw_result - Raw Response reference - URL of IoC

Return type

LookupResult

Raises

NotImplementedError – If attempting to use an HTTP method or authentication protocol that is not supported.

Notes

Note: this method uses memoization (lru_cache) to cache results for a particular observable to try avoid repeated network calls for the same item.

lookup_iocs(data: Union[pandas.core.frame.DataFrame, Dict[str, str], Iterable[str]], obs_col: Optional[str] = None, ioc_type_col: Optional[str] = None, query_type: Optional[str] = None, **kwargs) → pandas.core.frame.DataFrame

Lookup collection of IoC observables.

Parameters

  • data (Union[pd.DataFrame, Dict[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in obs_col parameter) 2. Dict of observable, IoCType 3. Iterable of observables - IoCTypes will be inferred

  • obs_col (str, optional) – DataFrame column to use for observables, by default None

  • ioc_type_col (str, optional) – DataFrame column to use for IoCTypes, by default None

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

DataFrame of results.

Return type

pd.DataFrame

parse_results(response: msticpy.sectools.tiproviders.ti_provider_base.LookupResult) → Tuple[bool, msticpy.sectools.tiproviders.ti_provider_base.TISeverity, Any]

Return the details of the response.

Parameters

response (LookupResult) – The returned data response

Returns

bool = positive or negative hit TISeverity = enumeration of severity Object with match details

Return type

Tuple[bool, TISeverity, Any]

static resolve_ioc_type(observable: str) → str

Return IoCType determined by IoCExtract.

Parameters

observable (str) – IoC observable string

Returns

IoC Type (or unknown if type could not be determined)

Return type

str

property supported_types: List[str]

Return list of supported IoC types for this provider.

Returns

List of supported type names

Return type

List[str]

classmethod usage()

Print usage of provider.

msticpy.sectools.tiproviders.azure_sent_byoti module

Microsoft Sentinel TI provider class.

Input can be a single IoC observable or a pandas DataFrame containing multiple observables. Processing may require a an API key and processing performance may be limited to a specific number of requests per minute for the account type that you have.

class msticpy.sectools.tiproviders.azure_sent_byoti.AzSTI(**kwargs)

Bases: msticpy.sectools.tiproviders.kql_base.KqlTIProvider

Microsoft Sentinel TI provider class.

Initialize a new instance of the class.

property ioc_query_defs: Dict[str, Any]

Return current dictionary of IoC query/request definitions.

Returns

IoC query/requist definitions keyed by IoCType

Return type

Dict[str, Any]

classmethod is_known_type(ioc_type: str) → bool

Return True if this a known IoC Type.

Parameters

ioc_type (str) – IoCType string to test

Returns

True if known type.

Return type

bool

is_supported_type(ioc_type: Union[str, msticpy.sectools.iocextract.IoCType]) → bool

Return True if the passed type is supported.

Parameters

ioc_type (Union[str, IoCType]) – IoC type name or instance

Returns

True if supported.

Return type

bool

lookup_ioc(ioc: str, ioc_type: str = None, query_type: str = None, **kwargs) → msticpy.sectools.tiproviders.ti_provider_base.LookupResult

Lookup a single IoC observable.

Parameters

  • ioc (str) – IoC observable

  • ioc_type (str, optional) – IocType, by default None (type will be inferred)

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

The lookup result: result - Positive/Negative, details - Lookup Details (or status if failure), raw_result - Raw Response reference - URL of IoC

Return type

LookupResult

Raises

LookupError – If a query could not be found for the ioc_type.

Notes

Note: this method uses memoization (lru_cache) to cache results for a particular observable to try avoid repeated network calls for the same item.

lookup_iocs(data: Union[pandas.core.frame.DataFrame, Dict[str, str], Iterable[str]], obs_col: Optional[str] = None, ioc_type_col: Optional[str] = None, query_type: Optional[str] = None, **kwargs) → pandas.core.frame.DataFrame

Lookup collection of IoC observables.

Parameters

  • data (Union[pd.DataFrame, Dict[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in obs_col parameter) 2. Dict of observable, IoCType 3. Iterable of observables - IoCTypes will be inferred

  • obs_col (str, optional) – DataFrame column to use for observables, by default None

  • ioc_type_col (str, optional) – DataFrame column to use for IoCTypes, by default None

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

DataFrame of results.

Return type

pd.DataFrame

parse_results(response: msticpy.sectools.tiproviders.ti_provider_base.LookupResult) → Tuple[bool, msticpy.sectools.tiproviders.ti_provider_base.TISeverity, Any]

Return the details of the response.

Parameters

response (LookupResult) – The returned data response

Returns

bool = positive or negative hit TISeverity = enumeration of severity Object with match details

Return type

Tuple[bool, TISeverity, Any]

static resolve_ioc_type(observable: str) → str

Return IoCType determined by IoCExtract.

Parameters

observable (str) – IoC observable string

Returns

IoC Type (or unknown if type could not be determined)

Return type

str

property supported_types: List[str]

Return list of supported IoC types for this provider.

Returns

List of supported type names

Return type

List[str]

classmethod usage()

Print usage of provider.

msticpy.sectools.tiproviders.greynoise module

GreyNoise Provider.

Input can be a single IoC observable or a pandas DataFrame containing multiple observables. Processing may require a an API key and processing performance may be limited to a specific number of requests per minute for the account type that you have.

class msticpy.sectools.tiproviders.greynoise.GreyNoise(**kwargs)

Bases: msticpy.sectools.tiproviders.http_base.HttpProvider

GreyNoise Lookup.

Initialize a new instance of the class.

property ioc_query_defs: Dict[str, Any]

Return current dictionary of IoC query/request definitions.

Returns

IoC query/requist definitions keyed by IoCType

Return type

Dict[str, Any]

classmethod is_known_type(ioc_type: str) → bool

Return True if this a known IoC Type.

Parameters

ioc_type (str) – IoCType string to test

Returns

True if known type.

Return type

bool

is_supported_type(ioc_type: Union[str, msticpy.sectools.iocextract.IoCType]) → bool

Return True if the passed type is supported.

Parameters

ioc_type (Union[str, IoCType]) – IoC type name or instance

Returns

True if supported.

Return type

bool

lookup_ioc(ioc: str, ioc_type: str = None, query_type: str = None, **kwargs) → msticpy.sectools.tiproviders.ti_provider_base.LookupResult

Lookup a single IoC observable.

Parameters

  • ioc (str) – IoC observable

  • ioc_type (str, optional) – IocType, by default None (type will be inferred)

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

The lookup result: result - Positive/Negative, details - Lookup Details (or status if failure), raw_result - Raw Response reference - URL of IoC

Return type

LookupResult

Raises

NotImplementedError – If attempting to use an HTTP method or authentication protocol that is not supported.

Notes

Note: this method uses memoization (lru_cache) to cache results for a particular observable to try avoid repeated network calls for the same item.

lookup_iocs(data: Union[pandas.core.frame.DataFrame, Dict[str, str], Iterable[str]], obs_col: Optional[str] = None, ioc_type_col: Optional[str] = None, query_type: Optional[str] = None, **kwargs) → pandas.core.frame.DataFrame

Lookup collection of IoC observables.

Parameters

  • data (Union[pd.DataFrame, Dict[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in obs_col parameter) 2. Dict of observable, IoCType 3. Iterable of observables - IoCTypes will be inferred

  • obs_col (str, optional) – DataFrame column to use for observables, by default None

  • ioc_type_col (str, optional) – DataFrame column to use for IoCTypes, by default None

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

DataFrame of results.

Return type

pd.DataFrame

parse_results(response: msticpy.sectools.tiproviders.ti_provider_base.LookupResult) → Tuple[bool, msticpy.sectools.tiproviders.ti_provider_base.TISeverity, Any]

Return the details of the response.

Parameters

response (LookupResult) – The returned data response

Returns

bool = positive or negative hit TISeverity = enumeration of severity Object with match details

Return type

Tuple[bool, TISeverity, Any]

static resolve_ioc_type(observable: str) → str

Return IoCType determined by IoCExtract.

Parameters

observable (str) – IoC observable string

Returns

IoC Type (or unknown if type could not be determined)

Return type

str

property supported_types: List[str]

Return list of supported IoC types for this provider.

Returns

List of supported type names

Return type

List[str]

classmethod usage()

Print usage of provider.

msticpy.sectools.tiproviders.http_base module

HTTP TI Provider base.

Input can be a single IoC observable or a pandas DataFrame containing multiple observables. Processing may require a an API key and processing performance may be limited to a specific number of requests per minute for the account type that you have.

class msticpy.sectools.tiproviders.http_base.HttpProvider(**kwargs)

Bases: msticpy.sectools.tiproviders.ti_provider_base.TIProvider

HTTP TI provider base class.

Initialize a new instance of the class.

property ioc_query_defs: Dict[str, Any]

Return current dictionary of IoC query/request definitions.

Returns

IoC query/requist definitions keyed by IoCType

Return type

Dict[str, Any]

classmethod is_known_type(ioc_type: str) → bool

Return True if this a known IoC Type.

Parameters

ioc_type (str) – IoCType string to test

Returns

True if known type.

Return type

bool

is_supported_type(ioc_type: Union[str, msticpy.sectools.iocextract.IoCType]) → bool

Return True if the passed type is supported.

Parameters

ioc_type (Union[str, IoCType]) – IoC type name or instance

Returns

True if supported.

Return type

bool

lookup_ioc(ioc: str, ioc_type: str = None, query_type: str = None, **kwargs) → msticpy.sectools.tiproviders.ti_provider_base.LookupResult

Lookup a single IoC observable.

Parameters

  • ioc (str) – IoC observable

  • ioc_type (str, optional) – IocType, by default None (type will be inferred)

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

The lookup result: result - Positive/Negative, details - Lookup Details (or status if failure), raw_result - Raw Response reference - URL of IoC

Return type

LookupResult

Raises

NotImplementedError – If attempting to use an HTTP method or authentication protocol that is not supported.

Notes

Note: this method uses memoization (lru_cache) to cache results for a particular observable to try avoid repeated network calls for the same item.

lookup_iocs(data: Union[pandas.core.frame.DataFrame, Dict[str, str], Iterable[str]], obs_col: Optional[str] = None, ioc_type_col: Optional[str] = None, query_type: Optional[str] = None, **kwargs) → pandas.core.frame.DataFrame

Lookup collection of IoC observables.

Parameters

  • data (Union[pd.DataFrame, Dict[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in obs_col parameter) 2. Dict of observable, IoCType 3. Iterable of observables - IoCTypes will be inferred

  • obs_col (str, optional) – DataFrame column to use for observables, by default None

  • ioc_type_col (str, optional) – DataFrame column to use for IoCTypes, by default None

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

DataFrame of results.

Return type

pd.DataFrame

abstract parse_results(response: msticpy.sectools.tiproviders.ti_provider_base.LookupResult) → Tuple[bool, msticpy.sectools.tiproviders.ti_provider_base.TISeverity, Any]

Return the details of the response.

Parameters

response (LookupResult) – The returned data response

Returns

bool = positive or negative hit TISeverity = enumeration of severity Object with match details

Return type

Tuple[bool, TISeverity, Any]

static resolve_ioc_type(observable: str) → str

Return IoCType determined by IoCExtract.

Parameters

observable (str) – IoC observable string

Returns

IoC Type (or unknown if type could not be determined)

Return type

str

property supported_types: List[str]

Return list of supported IoC types for this provider.

Returns

List of supported type names

Return type

List[str]

classmethod usage()

Print usage of provider.

class msticpy.sectools.tiproviders.http_base.IoCLookupParams(path: str = '', verb: str = 'GET', full_url: bool = False, headers: Dict[str, str] = NOTHING, params: Dict[str, str] = NOTHING, data: Dict[str, str] = NOTHING, auth_type: str = '', auth_str: List[str] = NOTHING, sub_type: str = '')

Bases: object

IoC HTTP Lookup Params definition.

Method generated by attrs for class IoCLookupParams.

auth_str: List[str]
auth_type: str
data: Dict[str, str]
full_url: bool
headers: Dict[str, str]
params: Dict[str, str]
path: str
sub_type: str
verb: str

msticpy.sectools.tiproviders.ibm_xforce module

IBM XForce Provider.

Input can be a single IoC observable or a pandas DataFrame containing multiple observables. Processing may require a an API key and processing performance may be limited to a specific number of requests per minute for the account type that you have.

class msticpy.sectools.tiproviders.ibm_xforce.XForce(**kwargs)

Bases: msticpy.sectools.tiproviders.http_base.HttpProvider

IBM XForce Lookup.

Initialize a new instance of the class.

property ioc_query_defs: Dict[str, Any]

Return current dictionary of IoC query/request definitions.

Returns

IoC query/requist definitions keyed by IoCType

Return type

Dict[str, Any]

classmethod is_known_type(ioc_type: str) → bool

Return True if this a known IoC Type.

Parameters

ioc_type (str) – IoCType string to test

Returns

True if known type.

Return type

bool

is_supported_type(ioc_type: Union[str, msticpy.sectools.iocextract.IoCType]) → bool

Return True if the passed type is supported.

Parameters

ioc_type (Union[str, IoCType]) – IoC type name or instance

Returns

True if supported.

Return type

bool

lookup_ioc(ioc: str, ioc_type: str = None, query_type: str = None, **kwargs) → msticpy.sectools.tiproviders.ti_provider_base.LookupResult

Lookup a single IoC observable.

Parameters

  • ioc (str) – IoC observable

  • ioc_type (str, optional) – IocType, by default None (type will be inferred)

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

The lookup result: result - Positive/Negative, details - Lookup Details (or status if failure), raw_result - Raw Response reference - URL of IoC

Return type

LookupResult

Raises

NotImplementedError – If attempting to use an HTTP method or authentication protocol that is not supported.

Notes

Note: this method uses memoization (lru_cache) to cache results for a particular observable to try avoid repeated network calls for the same item.

lookup_iocs(data: Union[pandas.core.frame.DataFrame, Dict[str, str], Iterable[str]], obs_col: Optional[str] = None, ioc_type_col: Optional[str] = None, query_type: Optional[str] = None, **kwargs) → pandas.core.frame.DataFrame

Lookup collection of IoC observables.

Parameters

  • data (Union[pd.DataFrame, Dict[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in obs_col parameter) 2. Dict of observable, IoCType 3. Iterable of observables - IoCTypes will be inferred

  • obs_col (str, optional) – DataFrame column to use for observables, by default None

  • ioc_type_col (str, optional) – DataFrame column to use for IoCTypes, by default None

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

DataFrame of results.

Return type

pd.DataFrame

parse_results(response: msticpy.sectools.tiproviders.ti_provider_base.LookupResult) → Tuple[bool, msticpy.sectools.tiproviders.ti_provider_base.TISeverity, Any]

Return the details of the response.

Parameters

response (LookupResult) – The returned data response

Returns

bool = positive or negative hit TISeverity = enumeration of severity Object with match details

Return type

Tuple[bool, TISeverity, Any]

static resolve_ioc_type(observable: str) → str

Return IoCType determined by IoCExtract.

Parameters

observable (str) – IoC observable string

Returns

IoC Type (or unknown if type could not be determined)

Return type

str

property supported_types: List[str]

Return list of supported IoC types for this provider.

Returns

List of supported type names

Return type

List[str]

classmethod usage()

Print usage of provider.

msticpy.sectools.tiproviders.intsights module

Intsights Provider.

Input can be a single IoC observable or a pandas DataFrame containing multiple observables. Processing may require a an API key and processing performance may be limited to a specific number of requests per minute for the account type that you have.

class msticpy.sectools.tiproviders.intsights.IntSights(**kwargs)

Bases: msticpy.sectools.tiproviders.http_base.HttpProvider

IntSights Lookup.

Initialize a new instance of the class.

property ioc_query_defs: Dict[str, Any]

Return current dictionary of IoC query/request definitions.

Returns

IoC query/requist definitions keyed by IoCType

Return type

Dict[str, Any]

classmethod is_known_type(ioc_type: str) → bool

Return True if this a known IoC Type.

Parameters

ioc_type (str) – IoCType string to test

Returns

True if known type.

Return type

bool

is_supported_type(ioc_type: Union[str, msticpy.sectools.iocextract.IoCType]) → bool

Return True if the passed type is supported.

Parameters

ioc_type (Union[str, IoCType]) – IoC type name or instance

Returns

True if supported.

Return type

bool

lookup_ioc(ioc: str, ioc_type: str = None, query_type: str = None, **kwargs) → msticpy.sectools.tiproviders.ti_provider_base.LookupResult

Lookup a single IoC observable.

Parameters

  • ioc (str) – IoC observable

  • ioc_type (str, optional) – IocType, by default None (type will be inferred)

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

The lookup result: result - Positive/Negative, details - Lookup Details (or status if failure), raw_result - Raw Response reference - URL of IoC

Return type

LookupResult

Raises

NotImplementedError – If attempting to use an HTTP method or authentication protocol that is not supported.

Notes

Note: this method uses memoization (lru_cache) to cache results for a particular observable to try avoid repeated network calls for the same item.

lookup_iocs(data: Union[pandas.core.frame.DataFrame, Dict[str, str], Iterable[str]], obs_col: Optional[str] = None, ioc_type_col: Optional[str] = None, query_type: Optional[str] = None, **kwargs) → pandas.core.frame.DataFrame

Lookup collection of IoC observables.

Parameters

  • data (Union[pd.DataFrame, Dict[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in obs_col parameter) 2. Dict of observable, IoCType 3. Iterable of observables - IoCTypes will be inferred

  • obs_col (str, optional) – DataFrame column to use for observables, by default None

  • ioc_type_col (str, optional) – DataFrame column to use for IoCTypes, by default None

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

DataFrame of results.

Return type

pd.DataFrame

parse_results(response: msticpy.sectools.tiproviders.ti_provider_base.LookupResult) → Tuple[bool, msticpy.sectools.tiproviders.ti_provider_base.TISeverity, Any]

Return the details of the response.

Parameters

response (LookupResult) – The returned data response

Returns

bool = positive or negative hit TISeverity = enumeration of severity Object with match details

Return type

Tuple[bool, TISeverity, Any]

static resolve_ioc_type(observable: str) → str

Return IoCType determined by IoCExtract.

Parameters

observable (str) – IoC observable string

Returns

IoC Type (or unknown if type could not be determined)

Return type

str

property supported_types: List[str]

Return list of supported IoC types for this provider.

Returns

List of supported type names

Return type

List[str]

classmethod usage()

Print usage of provider.

msticpy.sectools.tiproviders.kql_base module

Kql TI Provider base.

Input can be a single IoC observable or a pandas DataFrame containing multiple observables. Processing may require a an API key and processing performance may be limited to a specific number of requests per minute for the account type that you have.

class msticpy.sectools.tiproviders.kql_base.KqlTIProvider(**kwargs)

Bases: msticpy.sectools.tiproviders.ti_provider_base.TIProvider

KQL TI provider base class.

Initialize a new instance of the class.

property ioc_query_defs: Dict[str, Any]

Return current dictionary of IoC query/request definitions.

Returns

IoC query/requist definitions keyed by IoCType

Return type

Dict[str, Any]

classmethod is_known_type(ioc_type: str) → bool

Return True if this a known IoC Type.

Parameters

ioc_type (str) – IoCType string to test

Returns

True if known type.

Return type

bool

is_supported_type(ioc_type: Union[str, msticpy.sectools.iocextract.IoCType]) → bool

Return True if the passed type is supported.

Parameters

ioc_type (Union[str, IoCType]) – IoC type name or instance

Returns

True if supported.

Return type

bool

lookup_ioc(ioc: str, ioc_type: str = None, query_type: str = None, **kwargs) → msticpy.sectools.tiproviders.ti_provider_base.LookupResult

Lookup a single IoC observable.

Parameters

  • ioc (str) – IoC observable

  • ioc_type (str, optional) – IocType, by default None (type will be inferred)

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

The lookup result: result - Positive/Negative, details - Lookup Details (or status if failure), raw_result - Raw Response reference - URL of IoC

Return type

LookupResult

Raises

LookupError – If a query could not be found for the ioc_type.

Notes

Note: this method uses memoization (lru_cache) to cache results for a particular observable to try avoid repeated network calls for the same item.

lookup_iocs(data: Union[pandas.core.frame.DataFrame, Dict[str, str], Iterable[str]], obs_col: Optional[str] = None, ioc_type_col: Optional[str] = None, query_type: Optional[str] = None, **kwargs) → pandas.core.frame.DataFrame

Lookup collection of IoC observables.

Parameters

  • data (Union[pd.DataFrame, Dict[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in obs_col parameter) 2. Dict of observable, IoCType 3. Iterable of observables - IoCTypes will be inferred

  • obs_col (str, optional) – DataFrame column to use for observables, by default None

  • ioc_type_col (str, optional) – DataFrame column to use for IoCTypes, by default None

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

DataFrame of results.

Return type

pd.DataFrame

abstract parse_results(response: msticpy.sectools.tiproviders.ti_provider_base.LookupResult) → Tuple[bool, msticpy.sectools.tiproviders.ti_provider_base.TISeverity, Any]

Return the details of the response.

Parameters

response (LookupResult) – The returned data response

Returns

bool = positive or negative hit TISeverity = enumeration of severity Object with match details

Return type

Tuple[bool, TISeverity, Any]

static resolve_ioc_type(observable: str) → str

Return IoCType determined by IoCExtract.

Parameters

observable (str) – IoC observable string

Returns

IoC Type (or unknown if type could not be determined)

Return type

str

property supported_types: List[str]

Return list of supported IoC types for this provider.

Returns

List of supported type names

Return type

List[str]

classmethod usage()

Print usage of provider.

msticpy.sectools.tiproviders.open_page_rank module

Open Page Rank Provider.

Input can be a single IoC observable or a pandas DataFrame containing multiple observables. Processing may require a an API key and processing performance may be limited to a specific number of requests per minute for the account type that you have.

class msticpy.sectools.tiproviders.open_page_rank.OPR(**kwargs)

Bases: msticpy.sectools.tiproviders.http_base.HttpProvider

Open PageRank Lookup.

Initialize a new instance of the class.

property ioc_query_defs: Dict[str, Any]

Return current dictionary of IoC query/request definitions.

Returns

IoC query/requist definitions keyed by IoCType

Return type

Dict[str, Any]

classmethod is_known_type(ioc_type: str) → bool

Return True if this a known IoC Type.

Parameters

ioc_type (str) – IoCType string to test

Returns

True if known type.

Return type

bool

is_supported_type(ioc_type: Union[str, msticpy.sectools.iocextract.IoCType]) → bool

Return True if the passed type is supported.

Parameters

ioc_type (Union[str, IoCType]) – IoC type name or instance

Returns

True if supported.

Return type

bool

lookup_ioc(ioc: str, ioc_type: str = None, query_type: str = None, **kwargs) → msticpy.sectools.tiproviders.ti_provider_base.LookupResult

Lookup a single IoC observable.

Parameters

  • ioc (str) – IoC observable

  • ioc_type (str, optional) – IocType, by default None (type will be inferred)

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

The lookup result: result - Positive/Negative, details - Lookup Details (or status if failure), raw_result - Raw Response reference - URL of IoC

Return type

LookupResult

Raises

NotImplementedError – If attempting to use an HTTP method or authentication protocol that is not supported.

Notes

Note: this method uses memoization (lru_cache) to cache results for a particular observable to try avoid repeated network calls for the same item.

lookup_iocs(data: Union[pandas.core.frame.DataFrame, Dict[str, str], Iterable[str]], obs_col: Optional[str] = None, ioc_type_col: Optional[str] = None, query_type: Optional[str] = None, **kwargs) → pandas.core.frame.DataFrame

Lookup collection of IoC observables.

Parameters

  • data (Union[pd.DataFrame, Dict[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in obs_col parameter) 2. Dict of observable, IoCType 3. Iterable of observables - IoCTypes will be inferred

  • obs_col (str, optional) – DataFrame column to use for observables, by default None

  • ioc_type_col (str, optional) – DataFrame column to use for IoCTypes, by default None

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

DataFrame of results.

Return type

pd.DataFrame

parse_results(response: msticpy.sectools.tiproviders.ti_provider_base.LookupResult) → Tuple[bool, msticpy.sectools.tiproviders.ti_provider_base.TISeverity, Any]

Return the details of the response.

Parameters

response (LookupResult) – The returned data response

Returns

bool = positive or negative hit TISeverity = enumeration of severity Object with match details

Return type

Tuple[bool, TISeverity, Any]

static resolve_ioc_type(observable: str) → str

Return IoCType determined by IoCExtract.

Parameters

observable (str) – IoC observable string

Returns

IoC Type (or unknown if type could not be determined)

Return type

str

property supported_types: List[str]

Return list of supported IoC types for this provider.

Returns

List of supported type names

Return type

List[str]

classmethod usage()

Print usage of provider.

msticpy.sectools.tiproviders.riskiq module

msticpy.sectools.tiproviders.ti_provider_base module

Module for TILookup classes.

Input can be a single IoC observable or a pandas DataFrame containing multiple observables. Processing may require a an API key and processing performance may be limited to a specific number of requests per minute for the account type that you have.

class msticpy.sectools.tiproviders.ti_provider_base.LookupResult(ioc: str, ioc_type: str, safe_ioc: str = '', query_subtype: Optional[str] = None, provider: Optional[str] = None, result: bool = False, severity: int = 0, details: Optional[Any] = None, raw_result: Optional[Union[str, dict]] = None, reference: Optional[str] = None, status: int = 0)

Bases: object

Lookup result for IoCs.

Method generated by attrs for class LookupResult.

classmethod column_map()

Return a dictionary that maps fields to DF Names.

details: Any
ioc: str
ioc_type: str
provider: Optional[str]
query_subtype: Optional[str]
raw_result: Optional[Union[str, dict]]
property raw_result_fmtd

Print raw results of the Lookup Result.

reference: Optional[str]
result: bool
safe_ioc: str
set_severity(value: Any)

Set the severity from enum, int or string.

Parameters

value (Any) – The severity value to set

severity: int
property severity_name: str

Return text description of severity score.

Returns

Severity description.

Return type

str

status: int
property summary

Print a summary of the Lookup Result.

class msticpy.sectools.tiproviders.ti_provider_base.SanitizedObservable(observable, status)

Bases: tuple

Create new instance of SanitizedObservable(observable, status)

count(value, /)

Return number of occurrences of value.

index(value, start=0, stop=9223372036854775807, /)

Return first index of value.

Raises ValueError if the value is not present.

property observable

Alias for field number 0

property status

Alias for field number 1

class msticpy.sectools.tiproviders.ti_provider_base.TILookupStatus(value)

Bases: enum.Enum

Threat intelligence lookup status.

bad_format = 2
not_supported = 1
ok = 0
other = 10
query_failed = 3
class msticpy.sectools.tiproviders.ti_provider_base.TIPivotProvider

Bases: abc.ABC

A class which provides pivot functions and a means of registering them.

abstract register_pivots(pivot_reg: PivotRegistration, pivot: Pivot)

Register pivot functions for the TI Provider.

Parameters
class msticpy.sectools.tiproviders.ti_provider_base.TIProvider(**kwargs)

Bases: abc.ABC

Abstract base class for Threat Intel providers.

Initialize the provider.

property ioc_query_defs: Dict[str, Any]

Return current dictionary of IoC query/request definitions.

Returns

IoC query/requist definitions keyed by IoCType

Return type

Dict[str, Any]

classmethod is_known_type(ioc_type: str) → bool

Return True if this a known IoC Type.

Parameters

ioc_type (str) – IoCType string to test

Returns

True if known type.

Return type

bool

is_supported_type(ioc_type: Union[str, msticpy.sectools.iocextract.IoCType]) → bool

Return True if the passed type is supported.

Parameters

ioc_type (Union[str, IoCType]) – IoC type name or instance

Returns

True if supported.

Return type

bool

abstract lookup_ioc(ioc: str, ioc_type: Optional[str] = None, query_type: Optional[str] = None, **kwargs) → msticpy.sectools.tiproviders.ti_provider_base.LookupResult

Lookup a single IoC observable.

Parameters

  • ioc (str) – IoC Observable value

  • ioc_type (str, optional) – IoC Type, by default None (type will be inferred)

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

The returned results.

Return type

LookupResult

lookup_iocs(data: Union[pandas.core.frame.DataFrame, Dict[str, str], Iterable[str]], obs_col: Optional[str] = None, ioc_type_col: Optional[str] = None, query_type: Optional[str] = None, **kwargs) → pandas.core.frame.DataFrame

Lookup collection of IoC observables.

Parameters

  • data (Union[pd.DataFrame, Dict[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in obs_col parameter) 2. Dict of observable, IoCType 3. Iterable of observables - IoCTypes will be inferred

  • obs_col (str, optional) – DataFrame column to use for observables, by default None

  • ioc_type_col (str, optional) – DataFrame column to use for IoCTypes, by default None

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

DataFrame of results.

Return type

pd.DataFrame

abstract parse_results(response: msticpy.sectools.tiproviders.ti_provider_base.LookupResult) → Tuple[bool, msticpy.sectools.tiproviders.ti_provider_base.TISeverity, Any]

Return the details of the response.

Parameters

response (LookupResult) – The returned data response

Returns

bool = positive or negative hit TISeverity = enumeration of severity Object with match details

Return type

Tuple[bool, TISeverity, Any]

static resolve_ioc_type(observable: str) → str

Return IoCType determined by IoCExtract.

Parameters

observable (str) – IoC observable string

Returns

IoC Type (or unknown if type could not be determined)

Return type

str

property supported_types: List[str]

Return list of supported IoC types for this provider.

Returns

List of supported type names

Return type

List[str]

classmethod usage()

Print usage of provider.

class msticpy.sectools.tiproviders.ti_provider_base.TISeverity(value)

Bases: enum.Enum

Threat intelligence report severity.

high = 2
information = 0
classmethod parse(value) → msticpy.sectools.tiproviders.ti_provider_base.TISeverity

Parse string or numeric value to TISeverity.

Parameters

value (Any) – TISeverity, str or int

Returns

TISeverity instance.

Return type

TISeverity

unknown = -1
warning = 1
msticpy.sectools.tiproviders.ti_provider_base.entropy(input_str: str) → float

Compute entropy of input string.

msticpy.sectools.tiproviders.ti_provider_base.generate_items(data: Any, obs_col: Optional[str] = None, ioc_type_col: Optional[str] = None) → Iterable[Tuple[Optional[str], Optional[str]]]

Generate item pairs from different input types.

Parameters

  • data (Any) – DataFrame, dictionary or iterable

  • obs_col (Optional[str]) – If data is a DataFrame, the column containing the observable value.

  • ioc_type_col (Optional[str]) – If data is a DataFrame, the column containing the observable type.

Return type

Iterable[Tuple[Optional[str], Optional[str]]]] - a tuple of Observable/Type.

msticpy.sectools.tiproviders.ti_provider_base.get_schema_and_host(url: str, require_url_encoding: bool = False) → Tuple[Optional[str], Optional[str], Optional[str]]

Return URL scheme and host and cleaned URL.

Parameters

  • url (str) – Input URL

  • require_url_encoding (bool) – Set to True if url needs encoding. Defualt is False.

Returns

Tuple of URL, scheme, host

Return type

Tuple[Optional[str], Optional[str], Optional[str]

msticpy.sectools.tiproviders.ti_provider_base.preprocess_observable(observable, ioc_type, require_url_encoding: bool = False) → msticpy.sectools.tiproviders.ti_provider_base.SanitizedObservable

Preprocesses and checks validity of observable against declared IoC type.

param observable

the value of the IoC

param ioc_type

the IoC type

msticpy.sectools.tiproviders.tor_exit_nodes module

Tor Exit Nodes Provider.

Input can be a single IoC observable or a pandas DataFrame containing multiple observables. Processing may require a an API key and processing performance may be limited to a specific number of requests per minute for the account type that you have.

class msticpy.sectools.tiproviders.tor_exit_nodes.Tor(**kwargs)

Bases: msticpy.sectools.tiproviders.ti_provider_base.TIProvider

Tor Exit Nodes Lookup.

Instantiate Tor class.

property ioc_query_defs: Dict[str, Any]

Return current dictionary of IoC query/request definitions.

Returns

IoC query/requist definitions keyed by IoCType

Return type

Dict[str, Any]

classmethod is_known_type(ioc_type: str) → bool

Return True if this a known IoC Type.

Parameters

ioc_type (str) – IoCType string to test

Returns

True if known type.

Return type

bool

is_supported_type(ioc_type: Union[str, msticpy.sectools.iocextract.IoCType]) → bool

Return True if the passed type is supported.

Parameters

ioc_type (Union[str, IoCType]) – IoC type name or instance

Returns

True if supported.

Return type

bool

lookup_ioc(ioc: str, ioc_type: Optional[str] = None, query_type: Optional[str] = None, **kwargs) → msticpy.sectools.tiproviders.ti_provider_base.LookupResult

Lookup a single IoC observable.

Parameters

  • ioc (str) – IoC Observable value

  • ioc_type (str, optional) – IoC Type, by default None (type will be inferred)

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

The returned results.

Return type

LookupResult

lookup_iocs(data: Union[pandas.core.frame.DataFrame, Dict[str, str], Iterable[str]], obs_col: Optional[str] = None, ioc_type_col: Optional[str] = None, query_type: Optional[str] = None, **kwargs) → pandas.core.frame.DataFrame

Lookup collection of IoC observables.

Parameters

  • data (Union[pd.DataFrame, Dict[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in obs_col parameter) 2. Dict of observable, IoCType 3. Iterable of observables - IoCTypes will be inferred

  • obs_col (str, optional) – DataFrame column to use for observables, by default None

  • ioc_type_col (str, optional) – DataFrame column to use for IoCTypes, by default None

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

DataFrame of results.

Return type

pd.DataFrame

parse_results(response: msticpy.sectools.tiproviders.ti_provider_base.LookupResult) → Tuple[bool, msticpy.sectools.tiproviders.ti_provider_base.TISeverity, Any]

Return the details of the response.

Parameters

response (LookupResult) – The returned data response

Returns

bool = positive or negative hit TISeverity = enumeration of severity Object with match details

Return type

Tuple[bool, TISeverity, Any]

static resolve_ioc_type(observable: str) → str

Return IoCType determined by IoCExtract.

Parameters

observable (str) – IoC observable string

Returns

IoC Type (or unknown if type could not be determined)

Return type

str

property supported_types: List[str]

Return list of supported IoC types for this provider.

Returns

List of supported type names

Return type

List[str]

classmethod usage()

Print usage of provider.

msticpy.sectools.tiproviders.virustotal module

VirusTotal Provider.

Input can be a single IoC observable or a pandas DataFrame containing multiple observables. Processing may require a an API key and processing performance may be limited to a specific number of requests per minute for the account type that you have.

class msticpy.sectools.tiproviders.virustotal.VirusTotal(**kwargs)

Bases: msticpy.sectools.tiproviders.http_base.HttpProvider

VirusTotal Lookup.

Initialize a new instance of the class.

property ioc_query_defs: Dict[str, Any]

Return current dictionary of IoC query/request definitions.

Returns

IoC query/requist definitions keyed by IoCType

Return type

Dict[str, Any]

classmethod is_known_type(ioc_type: str) → bool

Return True if this a known IoC Type.

Parameters

ioc_type (str) – IoCType string to test

Returns

True if known type.

Return type

bool

is_supported_type(ioc_type: Union[str, msticpy.sectools.iocextract.IoCType]) → bool

Return True if the passed type is supported.

Parameters

ioc_type (Union[str, IoCType]) – IoC type name or instance

Returns

True if supported.

Return type

bool

lookup_ioc(ioc: str, ioc_type: str = None, query_type: str = None, **kwargs) → msticpy.sectools.tiproviders.ti_provider_base.LookupResult

Lookup a single IoC observable.

Parameters

  • ioc (str) – IoC observable

  • ioc_type (str, optional) – IocType, by default None (type will be inferred)

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

The lookup result: result - Positive/Negative, details - Lookup Details (or status if failure), raw_result - Raw Response reference - URL of IoC

Return type

LookupResult

Raises

NotImplementedError – If attempting to use an HTTP method or authentication protocol that is not supported.

Notes

Note: this method uses memoization (lru_cache) to cache results for a particular observable to try avoid repeated network calls for the same item.

lookup_iocs(data: Union[pandas.core.frame.DataFrame, Dict[str, str], Iterable[str]], obs_col: Optional[str] = None, ioc_type_col: Optional[str] = None, query_type: Optional[str] = None, **kwargs) → pandas.core.frame.DataFrame

Lookup collection of IoC observables.

Parameters

  • data (Union[pd.DataFrame, Dict[str, str], Iterable[str]]) – Data input in one of three formats: 1. Pandas dataframe (you must supply the column name in obs_col parameter) 2. Dict of observable, IoCType 3. Iterable of observables - IoCTypes will be inferred

  • obs_col (str, optional) – DataFrame column to use for observables, by default None

  • ioc_type_col (str, optional) – DataFrame column to use for IoCTypes, by default None

  • query_type (str, optional) – Specify the data subtype to be queried, by default None. If not specified the default record type for the IoC type will be returned.

Returns

DataFrame of results.

Return type

pd.DataFrame

parse_results(response: msticpy.sectools.tiproviders.ti_provider_base.LookupResult) → Tuple[bool, msticpy.sectools.tiproviders.ti_provider_base.TISeverity, Any]

Return the details of the response.

Parameters

response (LookupResult) – The returned data response

Returns

bool = positive or negative hit TISeverity = enumeration of severity Object with match details

Return type

Tuple[bool, TISeverity, Any]

static resolve_ioc_type(observable: str) → str

Return IoCType determined by IoCExtract.

Parameters

observable (str) – IoC observable string

Returns

IoC Type (or unknown if type could not be determined)

Return type

str

property supported_types: List[str]

Return list of supported IoC types for this provider.

Returns

List of supported type names

Return type

List[str]

classmethod usage()

Print usage of provider.

Module contents

TI Providers sub-package.