analyzer.coffea_patches.helper

Classes

nonserializable_attribute	Generalizing the container for non-serializable objects.
container_converter	Running over the all arguments of arbitrary function inputs (*args,
numpy_call_wrapper	Wrapper for awkward.to_numpy evaluations for dask_awkward array inputs.

Module Contents

class analyzer.coffea_patches.helper.nonserializable_attribute(nonserial_list: list[str])

Generalizing the container for non-serializable objects.

For a given unserializable object of name “obj”, on container “x” initialization, it will create a dummy instance of the “x._obj = None”. On the first time that “x.obj” is called, the contents of “x._obj” will be replaced by the return value of “x._create_obj()” method. The corresponding “_create_obj” method must be able to return the correct object without additional function arguments, so all arguments required to create the object of interest needs to be stored in the container instance.

Parameters

nonserial_listList[str]

A list of string for the names of the unserializable objects.

__getattr__(name)

The method __getattr__ is only invoke if the attribute cannot be found with conventional method (such as with the not-explicitly defined method nonserializable_attribute.att_name with no “_” prefix)

__getstate__()

Explicitly setting all the unserializable objects to be hidden from getstate requests. This ensures that wrapper objects can be pickled regardless of whether evaluations of the various objects has been carried out or not.

__setstate__(d: dict)

Due to the overloading the of the __getattr__ and __getstate__ method, we need to explicitly define the __setstate__ method. Notice that due to the design of the __getstate__ method, this should never receive a dictionary that initializes the unserializable objects under normal operations.

class analyzer.coffea_patches.helper.container_converter(method_map, default_conv=None)

Running over the all arguments of arbitrary function inputs (*args, **kwargs), iterating through the base python containers and running a conversion function on each of the leaf entries.

The method_map has types as the map key and a callable as the value, the callable will only be used on the matching type.

A default conversion callable can also be provided to convert objects that is not explicitly listed. By default, if a type is not explicitly listed, an exception is raised.

method_map

default_conv = None

convert(arg, maybe_backends=None)

__call__(*args, **kwargs) → tuple

static no_action(x)

static unrecognized(x)

class analyzer.coffea_patches.helper.numpy_call_wrapper

Bases: abc.ABC

Wrapper for awkward.to_numpy evaluations for dask_awkward array inputs.

For tools outside the coffea package (like for ML inference), the inputs typically expect a numpy-like input. This class wraps up the user-level awkward->numpy data mangling and the underlying numpy evaluation calls to recognizable to dask.

For the class to be fully functional, the user must overload these methods:

• numpy_call: How the evaluation using all-numpy tool be performed

• prepare_awkward: How awkward arrays should be translated to a numpy format that is compatible with the numpy_call

Additionally, the following helper functions can be omitted, but will help assist with either code debugging or for data mangling niceties.

• validate_numpy_input: makes sure the computation routine understand the input.

• postprocess_awkward: Additional translation to convert numpy outputs to awkward (defaults to a simple awkward.from_numpy conversion)

validate_numpy_input(*args, **kwargs) → None

Validating that the numpy-like input arguments are compatible with the underlying evaluation calls. This function should raise an exception if invalid input values are found. The base method performs no checks but raises a warning that no checks were performed.

abstractmethod numpy_call(*args, **kwargs)

Underlying numpy-like evaluation. This method should be reimplemented by the user or by tool-specialized classes.

abstractmethod prepare_awkward(*args, **kwargs) → tuple

Converting awkward-array like inputs into be numpy-compatible awkward-arrays compatible with the numpy_call method. The actual conversion to numpy is handled automatically. The return value should be (*args, **kwargs) pair that is compatible with the numpy_call.

Consult the following documentation to find the awkward operations needed. https://awkward-array.org/doc/main/user-guide/how-to-restructure-pad.html

get_awkward_lib(*args, **kwargs)

postprocess_awkward(return_array, *args, **kwargs)

Additional conversion from the numpy_call output back to awkward arrays. This method does not need to need to be overloaded, but can make the data-mangling that occurs outside the class cleaner (ex: additional awkward unflatten calls). To ensure that the data mangling can occur, the unformatted awkward-like inputs are also passed to this function.

For the base method, we will simply iterate over the containers and call the default awkward.from_numpy conversion

__call__(*args, **kwargs)

Highest level abstraction to be directly called by the user. Checks whether the inputs has any awkward arrays or dask_awkward arrays, and call the corresponding function if they are found. If no dask awkward or awkward arrays are found, calling the underlying _call_numpy method.