analyzer.core.event_collection¶

Attributes¶

logger

Classes¶

`SourceDescription`	Helper class that provides a standard way to create an ABC using
`FileInfo`
`DasCollection`	Helper class that provides a standard way to create an ABC using
`FileListCollection`	Helper class that provides a standard way to create an ABC using
`FileSet`
`FileChunk`

Functions¶

`getDatasets`(query, client)
`getReplicas`(dataset, client)
`getDasFileSet`(dataset)
`decideFile`(possible_files[, location_priorities])
`buildMissingFileset`(→ FileSet)	Two cases:
`chunkN`(nevents, chunk_size)
`getFileInfo`(file_path, tree_name, **kwargs)
`configureConverter`(conv)

Module Contents¶

analyzer.core.event_collection.logger[source]¶

class analyzer.core.event_collection.SourceDescription[source]¶

Bases: abc.ABC

Helper class that provides a standard way to create an ABC using inheritance.

abstractmethod getFileSet() → FileSet[source]¶

abstractmethod getMissing(fs: FileSet, **kwargs) → FileSet[source]¶

analyzer.core.event_collection.getDatasets(query, client)[source]¶

analyzer.core.event_collection.getReplicas(dataset, client)[source]¶

analyzer.core.event_collection.getDasFileSet(dataset)[source]¶

analyzer.core.event_collection.decideFile(possible_files, location_priorities=None)[source]¶

class analyzer.core.event_collection.FileInfo[source]¶

file_path: str[source]¶

nevents: int | None = None[source]¶

tree_name: str | None = None[source]¶

schema_name: str | None = None[source]¶

chunks: set[tuple[int, int]] | None = None[source]¶

target_chunk_size: int | None = None[source]¶

checkCompatible(other: FileInfo)[source]¶

__iadd__(other: FileInfo)[source]¶

__isub__(other: FileInfo)[source]¶

__add__(other: FileInfo)[source]¶

__sub__(other: FileInfo)[source]¶

iChunk(chunk_size)[source]¶

chunked(chunk_size)[source]¶

intersects(other)[source]¶

intersection(other)[source]¶

property chunked_events[source]¶

property empty[source]¶

property is_chunked[source]¶

toFileChunks()[source]¶

analyzer.core.event_collection.buildMissingFileset(source_description: SourceDescription, file_set: FileSet) → FileSet[source]¶

Two cases:

The file never got preprocessed or none of the chunks ran. In this case, we add the empty file to the return set.
The file was preprocesses but not all the chunks successfully ran. In this case, we compute the chunking based on the target chunk size and number of events, then take the difference.

class analyzer.core.event_collection.DasCollection[source]¶

Bases: SourceDescription

Helper class that provides a standard way to create an ABC using inheritance.

das_path: str[source]¶

tree_name: str = 'Events'[source]¶

schema_name: str | None = None[source]¶

getFileSet(location_priorities=None) → FileSet[source]¶

getMissing(file_set: FileSet, location_priorities=None)[source]¶

class analyzer.core.event_collection.FileListCollection[source]¶

Bases: SourceDescription

Helper class that provides a standard way to create an ABC using inheritance.

files: list[str][source]¶

tree_name: str = 'Events'[source]¶

schema_name: str | None = None[source]¶

getFileSet(**kwargs) → FileSet[source]¶

getMissing(file_set: FileSet) → FileSet[source]¶

analyzer.core.event_collection.chunkN(nevents, chunk_size)[source]¶

class analyzer.core.event_collection.FileSet[source]¶

files: dict[str, FileInfo][source]¶

property chunked_events[source]¶

property total_file_events[source]¶

static fromChunk(chunk)[source]¶

asMaximal()[source]¶

updateInfoFromOther(other: FileSet)[source]¶

updateFileInfo(file_info)[source]¶

getNeededUpdatesFuncs()[source]¶

updateFromCache()[source]¶

intersection(other)[source]¶

__iadd__(other)[source]¶

__isub__(other)[source]¶

__add__(other)[source]¶

__sub__(other)[source]¶

property materialized[source]¶

property empty[source]¶

justChunked()[source]¶

justUnchunked()[source]¶

applySliceChunks()[source]¶

splitFiles(files_per_set)[source]¶

iterChunks()[source]¶

toChunked(chunk_size)[source]¶

analyzer.core.event_collection.getFileInfo(file_path, tree_name, **kwargs)[source]¶

class analyzer.core.event_collection.FileChunk[source]¶

file_path: str[source]¶

event_start: int | None = None[source]¶

event_stop: int | None = None[source]¶

tree_name: str = 'Events'[source]¶

target_chunk_size: int | None = None[source]¶

schema_name: str | None = None[source]¶

file_nevents: int | None = None[source]¶

property nevents[source]¶

loadEvents(backend='coffea-virtual', view_kwargs=None)[source]¶

overlaps(other: FileChunk)[source]¶

toFileSet()[source]¶

analyzer.core.event_collection.configureConverter(conv)[source]¶