msticpy.data.drivers.azure_kusto_driver module
Kusto Driver subclass.
- class msticpy.data.drivers.azure_kusto_driver.AuthParams(method: str, params: Dict[str, Any], uri: str)
Bases:
NamedTuple
NamedTuple for auth parameters.
Create new instance of AuthParams(method, params, uri)
- count(value, /)
Return number of occurrences of value.
- index(value, start=0, stop=9223372036854775807, /)
Return first index of value.
Raises ValueError if the value is not present.
- method: str
Alias for field number 0
- params: Dict[str, Any]
Alias for field number 1
- uri: str
Alias for field number 2
- class msticpy.data.drivers.azure_kusto_driver.AzureKustoDriver(connection_str: str | None = None, **kwargs)
Bases:
DriverBase
Kusto Driver class to execute kql queries for Azure Data Explorer.
Instantiate KustoDriver.
- Parameters:
connection_str (str, optional) – Connection string
debug (bool) – print out additional diagnostic information.
timeout (int) – Query timeout in seconds, default is 240 seconds (4 minutes) Maximum is 3600 seconds (1 hour). (can be set here or in connect and overridden in query methods)
proxies (Dict[str, str]) – Proxy settings for Kusto queries. Dictionary format is {protocol: proxy_url} Where protocol is https, http, etc. and proxy_url can contain optional authentication information in the format “https://username:password@proxy_host:port” If you have a proxy configuration in msticpyconfig.yaml and you do not want to use it, set this to an empty dictionary. (can be overridden in connect method)
- add_query_filter(name: str, query_filter: str | Iterable)
Add an expression to the query attach filter.
- property cluster_config_name: str
Return current cluster URI.
- property cluster_name: str
Return current cluster URI.
- property cluster_uri: str
Return current cluster URI.
- property configured_clusters: Dict[str, KustoConfig]
Return current Kusto config settings.
- connect(connection_str: str | None = None, **kwargs)
Connect to data source.
Either a connection string or a cluster name must be specified. The cluster name can be a short name or a full URI. If a short name, the cluster must be defined in the msticpyconfig.yaml file. In this case, the short name can be either the key of the cluster definition the host name part of the cluster URI.
- Parameters:
connection_str (str, optional) – Connect to a data source
cluster (str, optional) – Short name or URI of cluster to connect to.
database (str, optional) – Name to set the default database to.
tenant_id (str, optional) – Azure tenant ID for the cluster.
connection_str – Kusto connection string, including authentication credentials.
auth_types (Union[str, list], optional) – Credential type or types to use for authentication. Use msticpy.auth.azure_auth.list_auth_methods() to get a list of available methods.
mp_az_auth (Union[bool, str, list, None], optional) – Deprecated parameter to use MSTICPy Azure authentication. Values can be: True or “default”: use the settings in msticpyconfig.yaml ‘Azure’ section str: single auth method name List[str]: list of acceptable auth methods
mp_az_tenant_id (str, optional) – alias for tenant_id.
timeout (int) – Query timeout in seconds, default is 240 seconds (4 minutes) Maximum is 3600 seconds (1 hour). (can be overridden in query methods)
See also
msticpy.auth.azure_auth.list_auth_methods
- property connected: bool
Return true if at least one connection has been made.
- Returns:
True if a successful connection has been made.
- Return type:
bool
Notes
This does not guarantee that the last data source connection was successful. It is a best effort to track whether the provider has made at least one successful authentication.
- property current_connection: str | None
Return current connection string or URI.
- property driver_queries: Iterable[Dict[str, Any]]
Return queries retrieved from the service after connecting.
- Returns:
List of Dictionary of query_name, query_text. Name of container to add queries to.
- Return type:
List[Dict[str, str]]
- get_database_names() List[str]
Get a list of database names from the connected cluster.
- get_database_schema(database: str | None = None) Dict[str, Dict[str, str]]
Get table names and schema from the connected cluster/database.
- Parameters:
database (Optional[str]) – Name of the database to get schema for. The default is the last connected database.
- Returns:
Dictionary of table names, each with a dictionary of column names and types.
- Return type:
Dict[str, Dict[str, str]]
- Raises:
ValueError : – No database name specified or set as the default.
MsticpyNotConnectedError : – Not connected to a cluster.
MsticpyDataQueryError : – Error querying the cluster.
- get_driver_property(name: str) Any
Return value or KeyError from driver properties.
- static get_http_timeout(**kwargs)
Get http timeout from settings or kwargs.
- property instance: str | None
Return instance name, if one is set.
- Returns:
The name of driver instance or None if the driver does not support multiple instances
- Return type:
Optional[str]
- property loaded: bool
Return true if the provider is loaded.
- Returns:
True if the provider is loaded.
- Return type:
bool
Notes
This is not relevant for some providers.
- query(query: str, query_source: QuerySource | None = None, **kwargs) DataFrame | Any
Execute query string and return DataFrame of results.
- Parameters:
query (str) – The query to execute
query_source (QuerySource) – The query definition object
database (str, Optional) – Supply or override the Kusto database name
timeout (int) – Query timeout in seconds, default is 240 seconds (4 minutes) Maximum is 3600 seconds (1 hour).
- Returns:
A DataFrame (if successful) or the underlying provider result if an error.
- Return type:
Union[pd.DataFrame, results.ResultSet]
- property query_attach_spec: Dict[str, Set[str]]
Parameters that determine whether a query is relevant for the driver.
- query_usable(query_source) bool
Return True if query source is valid for current cluster.
- query_with_results(query: str, **kwargs) Tuple[DataFrame | None, Any]
Return query results as a DataFrame and the result status.
- Parameters:
query (str) – The query string
- Returns:
DataFrame of results and the result status.
- Return type:
Tuple[Optional[pd.DataFrame], Any]
- Raises:
MsticpyNotConnectedError – If there is no connection to the data source.
MsticpyDataQueryError – If no database is specified in the query or parameters and there is no default database.
- property schema: Dict[str, Dict]
Return schema for current database.
- property service_queries: Tuple[Dict[str, str], str]
Return queries retrieved from the service after connecting.
- Returns:
Dictionary of query_name, query_text. Name of container to add queries to.
- Return type:
Tuple[Dict[str, str], str]
- set_cluster(cluster: str)
Set the current cluster to cluster and connect.
- set_database(database: str)
Set the default database to database.
- set_driver_property(name: str, value: Any)
Set an item in driver properties.
- msticpy.data.drivers.azure_kusto_driver.KFields
alias of
ConfigFields
- class msticpy.data.drivers.azure_kusto_driver.KustoConfig(name: str, cluster: str, alias: str, path: str, args: ~msticpy.common.provider_settings.ProviderArgs = <factory>, tenant_id: str | None = None, integrated_auth: bool = False, cluster_groups: ~typing.List[str] = <factory>)
Bases:
object
Kusto configuration class.
- class ConfigFields
Bases:
object
Kusto configuration fields.
- ARGS = 'Args'
- CERTIFICATE = 'Certificate'
- CLIENT_ID = 'ClientId'
- CLIENT_SEC = 'ClientSecret'
- CLUSTER = 'Cluster'
- CLUSTER_GROUPS = 'ClusterGroups'
- DEFAULTS = 'ClusterDefaults'
- INTEG_AUTH = 'IntegratedAuth'
- TENANT_ID = 'TenantId'
- alias: str
- args: ProviderArgs
- cluster: str
- cluster_groups: List[str]
- property default_db
Return default database for this cluster.
- integrated_auth: bool = False
- name: str
- path: str
- tenant_id: str | None = None
- class msticpy.data.drivers.azure_kusto_driver.QuerySourceFields
Bases:
object
Kusto query source/yaml query fields.
- CLUSTER = 'cluster'
- CLUSTERS = 'clusters'
- CLUSTER_GROUPS = 'cluster_groups'
- DATA_ENVS = 'data_environments'
- DATA_FAMILIES = 'data_families'
- msticpy.data.drivers.azure_kusto_driver.get_cluster_name(cluster_uri)
Return the cluster name from the cluster uri.