Adding Pivot functions

See Pivot Functions for more details on use of pivot functions.

The pivot library supports adding functions as pivot functions from any importable Python library. Not all functions will be wrappable. Currently Pivot supports functions that take input parameters as either scalar values (I’m including strings in this although that isn’t exactly correct), iterables or DataFrames with column specifications.

You can create a persistent pivot function definition (that can be loaded from a yaml file) or an ad hoc definition that you can create in code. The next two sections describe creating a persistent function definition. Programmatically creating pivots follows this.

Information needed to define a pivot function

If you have a library function that you want to expose as a pivot function you need to gather a bit of information about it.

This table describes the configuration parameters needed to create a pivot function (most are optional).






The src_module to containing the class or function



The class containing function



The name of the function to wrap



Rename the function


input type

The input type that the wrapped function expects (DataFrame iterable value)



Mapping of entity and attribute used for function



The param name that the function uses as input param for DataFrame

If DF input


The param name that function uses to identify the input column name

If DF input


Name of the column in the output DF to use as a key to join

If DF output


dict of static name/value params always sent to the function



Name of the param that the wrapped function uses for its input value

If not DF input


True if the function supports being called multiple times




The name of the container in the entity where the func will appear



The entity_map item specifies which entity or entities the pivot function will be added to. Each entry requires an Entity name (see entities) and an entity attribute name. The attribute name is only used if you want to use an instance of the entity as a parameter to the function. If you don’t care about this you can pick any attribute.

For IpAddress in the (partial) example below, the pivot function will try to extract the value of the Address attribute when an instance of IpAddress is used as a function parameter.

  IpAddress: Address
  Host: HostName
  Account: Name

This means that you can specify different attributes of the same entity for different functions (or even for two instances of the same function)

The func_df_param_name and func_df_col_param_name are needed only if the source function (i.e. the function to be wrapped as a pivot function) takes a DataFrame and column name as input parameters.

func_out_column_name is needed if the source function returns a DataFrame. In order to join input data with output data this needs to be the column in the output that has the same value as the function input For example, if your function processes IP addresses and returns the IP in a column named “ip_addr”, put “ip_addr” as the value of func_out_column_name.

Adding ad hoc pivot functions in code

You can also add ad hoc functions as pivot functions in code.

To do this use the Pivot method add_pivot_function

You can supply the pivot registration parameters as keyword arguments to this function:

def my_func(input: str):
    return input.upper()

    entity_map={"Host": "HostName"},

Alternatively, you can create a PivotRegistration object and supply that (along with the func parameter), to add_pivot_function.

from msticpy.init.pivot_core.pivot_register import PivotRegistration

def my_func(input: str):
    return input.upper()

piv_reg = PivotRegistration(
    entity_map={"Host": "HostName"},

Pivot.add_pivot_function(my_func, piv_reg, container="change_case")

The function parameters and PivotRegistration attributes are described in the previous section Information needed to define a pivot function.

Creating a persistent pivot function definition

You can also add pivot definitions to yaml files and load your pivots from this.

The top-level element of the file is pivot_providers.

Example from the MSTICPy ip_utils who_is function

   src_module: msticpy.context.ip_utils
   src_func_name: get_whois_df
   func_new_name: whois
   input_type: dataframe
     IpAddress: Address
   func_df_param_name: data
   func_df_col_param_name: ip_column
   func_out_column_name: query
     all_columns: True
     show_progress: False
   func_input_value_arg: ip_address


the library also support creating pivots from ad hoc functions created in the current notebook (see below).

You can also put this function into a Python module. If your module is in the current directory and is called my_new_module, the value you specify for src_module will be “my_new_module”.

Once you have your yaml definition file you can call register_pivot_providers



The pivot functions created will not persist across notebook sessions. You will need to call register_pivot_providers with your yaml files each time you start a new session. Currently there is no option to automate this via msticpyconfig.yaml but this would be easy to add - please open an issue in the MSTICPy repo if you would like to see this.

Creating and deleting shortcut pivot functions

If you are adding pivot functions of your own, you can add shortcuts (i.e. direct methods of the entity, rather than methods in sub-containers) to those functions.

Every entity class has the class method make_pivot_shortcut. You can use this to add a shortcut to an existing pivot function on that entity. Note that you must call this method on the entity class and not on an instance of that Entity.

The parameters that you must supply are func_name and target. The former is the relative path to the pivot function that you want to make the shortcut to, e.g. for IpAddress.util.whois you would use the string “util.whois”. target is the string name that you want the shortcut function to be called. This should be a valid Python identifier - a string starting with a letter or underscore, followed by any combination of letters, digits and underscores. If you supply a string that is not a valid identifier, the function will try to transform it into one.

>>> IpAddress.make_pivot_shortcut(func_name="util.whois", target="my_whois")
>>> IpAddress.my_whois("")
ip_column    AsnDescription    whois_result   NA                {'nir': None, 'asn_registry': 'arin', ...

If the shortcut function already exists, you will get an error (AttributeError). You can force overwriting of an existing shortcut by adding overwrite=True.

To delete a shortcut use del_pivot_shortcut, giving the single parameter func_name with the name of the shortcut function you want to remove.

Removing pivot functions from an entity or all entities

Although not a common operation you can remove all pivot functions from an entity or from all entities.

See remove_pivot_funcs for more details.