analyzer.postprocessing.running

Attributes

logger

Classes

PostprocessorConfig
LoadStyles	Interface to extend the Worker

Functions

initProcess()
determineFileGroups(→ set[frozenset])
makeApproxEqualSubgroups(groups, target_num_groups[, ...])
maximalSubgroups(→ set[frozenset])
loadPostprocessor(path[, variable_name])
runResults(postprocessor, results[, parallel, prefix])
runPostprocessors(path, input_files[, parallel, ...])

Module Contents

analyzer.postprocessing.running.logger

class analyzer.postprocessing.running.PostprocessorConfig

processors: list[analyzer.postprocessing.basic_histograms.BasePostprocessor]

default_style_set: analyzer.postprocessing.style.StyleSet

default_plot_config: analyzer.postprocessing.plots.common.PlotConfiguration

drop_sample_pattern: analyzer.utils.querying.BasePattern | None = None

do_merge_and_scale: bool = True

keepPatterns()

analyzer.postprocessing.running.initProcess()

class analyzer.postprocessing.running.LoadStyles

Bases: distributed.WorkerPlugin

Interface to extend the Worker

A worker plugin enables custom code to run at different stages of the Workers’ lifecycle.

A plugin enables custom code to run at each of step of a Workers’s life. Whenever such an event happens, the corresponding method on this class will be called. Note that the user code always runs within the Worker’s main thread.

To implement a plugin:

1. inherit from this class

2. override some of its methods

3. register the plugin using Client.register_plugin.

The idempotent attribute is used to control whether or not the plugin should be ignored upon registration if a worker plugin with the same name already exists. If True, the plugin is ignored, otherwise the existing plugin is replaced. Defaults to False.

Examples

>>> class ErrorLogger(WorkerPlugin):
...     def __init__(self, logger):
...         self.logger = logger
...
...     def setup(self, worker):
...         self.worker = worker
...
...     def transition(self, key, start, finish, *args, **kwargs):
...         if finish == 'error':
...             ts = self.worker.tasks[key]
...             exc_info = (type(ts.exception), ts.exception, ts.traceback)
...             self.logger.error(
...                 "Error during computation of '%s'.", key,
...                 exc_info=exc_info
...             )

>>> import logging
>>> plugin = ErrorLogger(logging)
>>> client.register_plugin(plugin)

setup(worker)

Run when the plugin is attached to a worker. This happens when the plugin is registered and attached to existing workers, or when a worker is created after the plugin has been registered.

teardown(worker)

Run when the worker to which the plugin is attached is closed, or when the plugin is removed.

analyzer.postprocessing.running.determineFileGroups(postprocessors, results) → set[frozenset]

analyzer.postprocessing.running.makeApproxEqualSubgroups(groups, target_num_groups, size_func=lambda x: ...)

analyzer.postprocessing.running.maximalSubgroups(groups: set[frozenset]) → set[frozenset]

analyzer.postprocessing.running.loadPostprocessor(path, variable_name=None)

analyzer.postprocessing.running.runResults(postprocessor, results, parallel=None, prefix=None)

analyzer.postprocessing.running.runPostprocessors(path, input_files, parallel=None, prefix=None, target_load_size: int | None = None, include_sidecar: bool = False)