msticpy.data.drivers.azure_kusto_driver module

Kusto Driver subclass.

class msticpy.data.drivers.azure_kusto_driver.AuthParams(method: str, params: Dict[str, Any], uri: str)

Bases: NamedTuple

NamedTuple for auth parameters.

Create new instance of AuthParams(method, params, uri)

count(value, /)

Return number of occurrences of value.

index(value, start=0, stop=9223372036854775807, /)

Return first index of value.

Raises ValueError if the value is not present.

method: str

Alias for field number 0

params: Dict[str, Any]

Alias for field number 1

uri: str

Alias for field number 2

class msticpy.data.drivers.azure_kusto_driver.AzureKustoDriver(connection_str: str | None = None, **kwargs)

Bases: DriverBase

Kusto Driver class to execute kql queries for Azure Data Explorer.

Instantiate KustoDriver.

Parameters:
  • connection_str (str, optional) – Connection string

  • debug (bool) – print out additional diagnostic information.

  • timeout (int) – Query timeout in seconds, default is 240 seconds (4 minutes) Maximum is 3600 seconds (1 hour). (can be set here or in connect and overridden in query methods)

  • proxies (Dict[str, str]) – Proxy settings for Kusto queries. Dictionary format is {protocol: proxy_url} Where protocol is https, http, etc. and proxy_url can contain optional authentication information in the format “https://username:password@proxy_host:port” If you have a proxy configuration in msticpyconfig.yaml and you do not want to use it, set this to an empty dictionary. (can be overridden in connect method)

add_query_filter(name: str, query_filter: str | Iterable)

Add an expression to the query attach filter.

property cluster_config_name: str

Return current cluster URI.

property cluster_name: str

Return current cluster URI.

property cluster_uri: str

Return current cluster URI.

property configured_clusters: Dict[str, KustoConfig]

Return current Kusto config settings.

connect(connection_str: str | None = None, **kwargs)

Connect to data source.

Either a connection string or a cluster name must be specified. The cluster name can be a short name or a full URI. If a short name, the cluster must be defined in the msticpyconfig.yaml file. In this case, the short name can be either the key of the cluster definition the host name part of the cluster URI.

Parameters:
  • connection_str (str, optional) – Connect to a data source

  • cluster (str, optional) – Short name or URI of cluster to connect to.

  • database (str, optional) – Name to set the default database to.

  • tenant_id (str, optional) – Azure tenant ID for the cluster.

  • connection_str – Kusto connection string, including authentication credentials.

  • auth_types (Union[str, list], optional) – Credential type or types to use for authentication. Use msticpy.auth.azure_auth.list_auth_methods() to get a list of available methods.

  • mp_az_auth (Union[bool, str, list, None], optional) – Deprecated parameter to use MSTICPy Azure authentication. Values can be: True or “default”: use the settings in msticpyconfig.yaml ‘Azure’ section str: single auth method name List[str]: list of acceptable auth methods

  • mp_az_tenant_id (str, optional) – alias for tenant_id.

  • timeout (int) – Query timeout in seconds, default is 240 seconds (4 minutes) Maximum is 3600 seconds (1 hour). (can be overridden in query methods)

See also

msticpy.auth.azure_auth.list_auth_methods

property connected: bool

Return true if at least one connection has been made.

Returns:

True if a successful connection has been made.

Return type:

bool

Notes

This does not guarantee that the last data source connection was successful. It is a best effort to track whether the provider has made at least one successful authentication.

property current_connection: str | None

Return current connection string or URI.

property driver_queries: Iterable[Dict[str, Any]]

Return queries retrieved from the service after connecting.

Returns:

List of Dictionary of query_name, query_text. Name of container to add queries to.

Return type:

List[Dict[str, str]]

get_database_names() List[str]

Get a list of database names from the connected cluster.

get_database_schema(database: str | None = None) Dict[str, Dict[str, str]]

Get table names and schema from the connected cluster/database.

Parameters:

database (Optional[str]) – Name of the database to get schema for. The default is the last connected database.

Returns:

Dictionary of table names, each with a dictionary of column names and types.

Return type:

Dict[str, Dict[str, str]]

Raises:
  • ValueError : – No database name specified or set as the default.

  • MsticpyNotConnectedError : – Not connected to a cluster.

  • MsticpyDataQueryError : – Error querying the cluster.

get_driver_property(name: str) Any

Return value or KeyError from driver properties.

static get_http_timeout(**kwargs)

Get http timeout from settings or kwargs.

property instance: str | None

Return instance name, if one is set.

Returns:

The name of driver instance or None if the driver does not support multiple instances

Return type:

Optional[str]

property loaded: bool

Return true if the provider is loaded.

Returns:

True if the provider is loaded.

Return type:

bool

Notes

This is not relevant for some providers.

query(query: str, query_source: QuerySource | None = None, **kwargs) DataFrame | Any

Execute query string and return DataFrame of results.

Parameters:
  • query (str) – The query to execute

  • query_source (QuerySource) – The query definition object

  • database (str, Optional) – Supply or override the Kusto database name

  • timeout (int) – Query timeout in seconds, default is 240 seconds (4 minutes) Maximum is 3600 seconds (1 hour).

Returns:

A DataFrame (if successful) or the underlying provider result if an error.

Return type:

Union[pd.DataFrame, results.ResultSet]

property query_attach_spec: Dict[str, Set[str]]

Parameters that determine whether a query is relevant for the driver.

query_usable(query_source) bool

Return True if query source is valid for current cluster.

query_with_results(query: str, **kwargs) Tuple[DataFrame | None, Any]

Return query results as a DataFrame and the result status.

Parameters:

query (str) – The query string

Returns:

DataFrame of results and the result status.

Return type:

Tuple[Optional[pd.DataFrame], Any]

Raises:
property schema: Dict[str, Dict]

Return schema for current database.

property service_queries: Tuple[Dict[str, str], str]

Return queries retrieved from the service after connecting.

Returns:

Dictionary of query_name, query_text. Name of container to add queries to.

Return type:

Tuple[Dict[str, str], str]

set_cluster(cluster: str)

Set the current cluster to cluster and connect.

set_database(database: str)

Set the default database to database.

set_driver_property(name: str, value: Any)

Set an item in driver properties.

msticpy.data.drivers.azure_kusto_driver.KFields

alias of ConfigFields

class msticpy.data.drivers.azure_kusto_driver.KustoConfig(name: str, cluster: str, alias: str, path: str, args: ~msticpy.common.provider_settings.ProviderArgs = <factory>, tenant_id: str | None = None, integrated_auth: bool = False, cluster_groups: ~typing.List[str] = <factory>)

Bases: object

Kusto configuration class.

class ConfigFields

Bases: object

Kusto configuration fields.

ARGS = 'Args'
CERTIFICATE = 'Certificate'
CLIENT_ID = 'ClientId'
CLIENT_SEC = 'ClientSecret'
CLUSTER = 'Cluster'
CLUSTER_GROUPS = 'ClusterGroups'
DEFAULTS = 'ClusterDefaults'
INTEG_AUTH = 'IntegratedAuth'
TENANT_ID = 'TenantId'
alias: str
args: ProviderArgs
cluster: str
cluster_groups: List[str]
property default_db

Return default database for this cluster.

integrated_auth: bool = False
name: str
path: str
tenant_id: str | None = None
class msticpy.data.drivers.azure_kusto_driver.QuerySourceFields

Bases: object

Kusto query source/yaml query fields.

CLUSTER = 'cluster'
CLUSTERS = 'clusters'
CLUSTER_GROUPS = 'cluster_groups'
DATA_ENVS = 'data_environments'
DATA_FAMILIES = 'data_families'
msticpy.data.drivers.azure_kusto_driver.get_cluster_name(cluster_uri)

Return the cluster name from the cluster uri.