activations

Activation Index

diagnnose.activations.activation_index.activation_index_len(activation_index: Union[int, slice, List[int], Tensor]) → int

Returns the number of items in an activation index.

diagnnose.activations.activation_index.activation_index_to_iterable(activation_index: Union[int, slice, List[int], Tensor], stop_index: Optional[int] = None) → Iterable

Transforms an activation index into an iterable object.

Activation Reader

class diagnnose.activations.activation_reader.ActivationReader(activations_dir: Optional[str] = None, activation_dict: Optional[Dict[Tuple[int, str], Tensor]] = None, activation_names: Optional[List[Tuple[int, str]]] = None, activation_ranges: Optional[List[Tuple[int, int]]] = None, selection_func: Optional[Callable[[int, Example], bool]] = None, store_multiple_activations: bool = False, cat_activations: bool = False)

Bases: object

Reads in pickled activations that have been extracted.

An ActivationReader can also be created directly from an ActivationDict, in which case the corresponding ActivationRanges and SelectionFunc should be provided too.

Parameters

• activations_dir (str, optional) – Directory containing the extracted activations.

• activation_dict (ActivationDict, optional) – If activations have not been extracted to disk, the activation_dict containing all extracted activations can be provided directly as well.

• activation_names (ActivationNames, optional) – Activation names, provided as a list of (layer, name) tuples. If not provided the index to __getitem__() must always contain the activation_name that is being requested, as the ActivationReader can not infer it automatically.

• activation_ranges (ActivationRanges, optional) – ActivationRanges dictionary that should be provided if activation_dict is passed directly.

• selection_func (SelectionFunc, optional) – SelectionFunc that was used for extraction and that should be passed if activation_dict is passed directly.

• store_multiple_activations (bool, optional) – Set to true to store multiple activation arrays in RAM at once. Defaults to False, meaning that only one activation type will be stored in the class.

• cat_activations (bool, optional) – Toggle to concatenate the activations returned by __getitem__(). Otherwise the activations will be split into a tuple with each each tuple item containing the activations of one sentence.

__getitem__(key: Union[int, slice, List[int], Tensor, Tuple[Union[int, slice, List[int], Tensor], Tuple[int, str]]]) → Union[Tensor, Iterator[Tensor]]

Allows for concise and efficient indexing of activations.

The key argument should be either an ActivationIndex (i.e. an iterable that can be used to index a tensor), or a (index, activation_name) tuple. An activation_name is a tuple of shape (layer, name).

If multiple activation_names have been extracted the activation_name must be provided, otherwise it can be left out.

The return value is a generator of tensors, with each tensor of shape (sen_len, nhid), or a concatenated tensor if self.cat_activations is set to True.

Example usage:

activation_reader = ActivationReader(
    dir, activation_names=[(0, "hx"), (1, "hx")], **kwargs
)

# activation_name must be passed because ActivationReader
# contains two activation_names.
activations_first_sen = activation_reader[0, (1, "hx")]
all_activations = activation_reader[:, (1, "hx")]


activation_reader2 = ActivationReader(
    dir, activation_names=[(1, "hx")], **kwargs
)

# activation_name can be left implicit.
activations_first_10_sens = activation_reader2[:10]

Parameters

key (ActivationKey) – ActivationIndex or (index, activation_name), as explained above.

Returns

• split_activations (Tensor | Iterator[Tensor, …]) – Tensor, if self.cat_activations is set to True. Otherwise a Generator of tensors, with each item corresponding to the extracted activations of a specific sentence.

• .. automethod:: __getitem__

__len__() → int

Returns the total number of extracted activations.

property activation_ranges: List[Tuple[int, int]]

activations(activation_name: Tuple[int, str]) → Tensor

static get_item_generator(ranges, activations) → Iterator[Tensor]

property selection_func: Callable[[int, Example], bool]

to(device: str) → None

Cast activations to a different device.

Activation Writer

class diagnnose.activations.activation_writer.ActivationWriter(activations_dir: str)

Bases: object

Writes activations to file, using an ExitStack.

Parameters

activations_dir (str, optional) – Directory to which activations will be written

concat_pickle_dumps(overwrite: bool = True) → None

Concatenates a sequential pickle dump and pickles to file .

Note that this overwrites the sequential pickle dump by default.

Parameters

overwrite (bool, optional) – Set to True to overwrite the file containing the sequential pickle dump, otherwise creates a new file. Defaults to True.

create_output_files(stack: ExitStack, activation_names: List[Tuple[int, str]]) → None

Opens a file for each to-be-extracted activation.

dump_activations(activations: Dict[Tuple[int, str], Tensor]) → None

Dumps the generated activations to a list of opened files

Parameters

activations (PartialArrayDict) – The Tensors for each activation that was specifed by self.activation_names at initialization.

dump_meta_info(activation_ranges: List[Tuple[int, int]], selection_func: Callable[[int, Example], bool]) → None

Dumps activation_ranges and selection_func to disk.

Selection Funcs

diagnnose.activations.selection_funcs.final_sen_token(w_idx: int, item: Example) → bool

Only returns the final token of a sentence.

diagnnose.activations.selection_funcs.final_token(sen_column: str = 'sen') → Callable[[int, Example], bool]

Only returns the final token of a sentence.

Wrapper allows a different sen_column to be set, that indicates the sen attribute of a corpus item that is being processed.

diagnnose.activations.selection_funcs.first_n(n: int) → Callable[[int, Example], bool]

Wrapper that creates a selection_func that only returns True for the first n items of a corpus.

diagnnose.activations.selection_funcs.in_sen_ids(sen_ids: List[int]) → Callable[[int, Example], bool]

Wrapper that creates a selection_func that only returns True for a sen_id if it is part of the provided list of sen_ids.

diagnnose.activations.selection_funcs.intersection(selection_funcs: Iterable[Callable[[int, Example], bool]]) → Callable[[int, Example], bool]

Returns the intersection of an iterable of selection_funcs.

diagnnose.activations.selection_funcs.negate(selection_func: Callable[[int, Example], bool]) → Callable[[int, Example], bool]

Returns the negation of a selection_func.

diagnnose.activations.selection_funcs.no_special_tokens(tokenizer: transformers.PreTrainedTokenizer, sen_column: str = 'sen') → Callable[[int, Example], bool]

diagnnose.activations.selection_funcs.nth_token(n: int) → Callable[[int, Example], bool]

Wrapper that creates a selection_func that only returns True for the n^{th} token of a sentence.

diagnnose.activations.selection_funcs.only_mask_token(mask_token: str, sen_column: str = 'sen') → Callable[[int, Example], bool]

diagnnose.activations.selection_funcs.return_all(_w_idx: int, _item: Example) → bool

Always returns True for every token.

diagnnose.activations.selection_funcs.union(selection_funcs: Iterable[Callable[[int, Example], bool]]) → Callable[[int, Example], bool]

Returns the union of an iterable of selection_funcs.