msticpy.init.azure_synapse_tools module

Help functions for Synapse pipelines notebooks.

class msticpy.init.azure_synapse_tools.LinkedService(**kwargs)

Bases: object

Azure Synapse Linked Service settings.

Initialize Linked Service instance.

class ServiceTypes

Bases: object

Enumeration of linked service type names.

AzureBlobFS = 'AzureBlobFS'

AzureBlobStorage = 'AzureBlobStorage'

AzureFileStorage = 'AzureFileStorage'

AzureKeyVault = 'AzureKeyVault'

AzureSqlDW = 'AzureSqlDW'

property azure_name

Return Azure resource name.

property svc_type: str

Return type of linked service.

class msticpy.init.azure_synapse_tools.MPSparkUtils(mount_point=None, container=None)

Bases: object

MSTICPy Spark Synapse Utility class.

Initialize MPSparkUtils class.

Parameters:

• mount_point (str | None)

• container (str | None)

property config_path

Return mount path for MSTICPy config.

property fs_mounts: dict[str, str]

Return a dictionary of mount points and targets.

get_all_services_of_type(svc_type)

Return list of Linked services of svc_type.

Parameters:

svc_type (str) – Service Type (AzureBlobFS, AzureBlobStorage, AzureSqlDW, AzureKeyVault)

Returns:

List of LinkedService instances of type svc_type.

Return type:

List[LinkedService]

get_kv_secret(secret_name)

Return secret from linked service.

Parameters:

secret_name (str)

Return type:

str

get_service(svc_name)

Return named linked service.

Returns:

Named service service.

Return type:

Optional[LinkedService]

Parameters:

svc_name (str)

get_service_of_type(svc_type)

Return the first linked service of specific svc_type.

Parameters:

svc_type (str) – Service Type (AzureBlobFS, AzureBlobStorage, AzureSqlDW, AzureKeyVault)

Returns:

LinkedService instance for the svc_type.

Return type:

Optional[LinkedService]

get_storage_service(linked_svc_name=None)

Return linked storage service (named) or default storage.

Parameters:

linked_svc_name (str | None)

Return type:

LinkedService

get_ws_default_storage()

Return default storage linked service.

Returns:

Default storage service.

Return type:

Optional[LinkedService]

property job_id

Return current job ID.

property workspace_name: str

Return the Synapse workspace name.

class msticpy.init.azure_synapse_tools.SparkUtilsPlaceHolder

Bases: object

Placeholder for non-spark environments.

class msticpy.init.azure_synapse_tools.SynapseName(workspace_id)

Bases: object

Name mapping to default values.

Initialize the Synapse Name class.

Parameters:

workspace_id (str)

container = 'sentinelfiles'

property key_vault: str

Return default Key Vault name.

key_vault_name_prefix = 'kvforsentinel'

kv_linked_service = 'Akvlink'

sp_client_id_name = 'clientid'

sp_client_sec_name = 'clientsecret'

property storage_account: str

Return default storage account name.

storage_account_prefix = 'adlsforsentinel'

msticpy.init.azure_synapse_tools.current_mounts()

Return dictionary of current Synapse mount points.

Return type:

dict[str, str]

msticpy.init.azure_synapse_tools.init_synapse(identity_type='service_principal', storage_svc_name=None, tenant_id=None, cloud='global')

Initialize Synapse Spark notebook pipeline.

Parameters:

• identity_type (IdentityType, optional) – The authentication type to be used by MSTICPy, by default “managed”. Options are “managed”, “service_principal”

• storage_svc_name (Optional[str], optional) – Override the default storage linked service name, by default None

• tenant_id (Optional[str], optional) – Override the default tenant_id for the Azure authentication, by default None

• cloud (str) – Azure cloud - default is “global”.

Raises:

RuntimeError – No suitable linked storage service found. Could not mount configuration data container. Could not authenticate to Azure. Could not retrieve secret from Key Vault.

msticpy.init.azure_synapse_tools.is_in_synapse()

Return True if running in Synapse Pipeline.

msticpy.init.azure_synapse_tools.mount_container(store_acct_name, container, mount_path, linked_service, folder='', cloud='global')

Mount Azure file container to Synapse file system.

Parameters:

• store_acct_name (str) – Storage account

• container (str) – Container name to mount

• mount_path (str) – Local path to mount

• linked_service (str) – Storage linked service name

• folder (str, optional) – Subfolder of container, if any, by default “”

• cloud (str, optional) – Azure Cloud, by default “global”

Returns:

True if mounting successful or if mount point already connected. False if mounting failed or mount point connected to a different storage location.

Return type:

bool

Raises:

NotImplementedError – If using a cloud other than Azure Global (public cloud).