attribute

Decomposer

class diagnnose.attribute.decomposer.ContextualDecomposer(model: LanguageModel, num_samples: Optional[int] = None, tensor_type: str = 'ShapleyTensor')[source]

Bases: Decomposer

A ContextualDecomposer propagates each input feature contribution individually, set out against the contributions of all other features combined.

This idea has been proposed in Murdocht et al., (2018): https://arxiv.org/abs/1801.05453

An input sequence of \(n\) features will be transformed into a ShapleyTensor containing \(2\) feature contributions: one containing the contributions of the feature of interest (\(\beta\)), and one containing the contributions of all other features (\(\gamma\)).

Concretely: if we have an input tensor \(X\) of shape: (num_features, input_dim) we can express this as a sum of features: \(X = \beta^i + \gamma^i\), where both \(\beta\) and \(\gamma\) are also of shape (num_features, input_dim), with \(\beta^i_j = \begin{cases}X_j&i=j\\0&\textit{otherwise}\end{cases}\) and \(\gamma^i_j = \begin{cases}X_j&i\neq j\\0&\textit{otherwise}\end{cases}\)

This way of partitioning scales polynomially in the number of input features, but requires a separate forward pass for each individual feature contribution \(\beta^i\).

decompose(batch_encoding: transformers.BatchEncoding) ShapleyTensor[source]
wrap_inputs_embeds(input_ids: Tensor) List[ShapleyTensor][source]
class diagnnose.attribute.decomposer.Decomposer(model: LanguageModel, num_samples: Optional[int] = None, tensor_type: str = 'ShapleyTensor')[source]

Bases: ABC

Abstract base class for Decomposer classes.

A Decomposer takes care of dividing the input features into the desired partition of contributions.

abstract decompose(batch_encoding: transformers.BatchEncoding) ShapleyTensor[source]
abstract wrap_inputs_embeds(input_ids: Tensor) ShapleyTensor[source]
class diagnnose.attribute.decomposer.ShapleyDecomposer(model: LanguageModel, num_samples: Optional[int] = None, tensor_type: str = 'ShapleyTensor')[source]

Bases: Decomposer

A ShapleyDecomposer propagates all input feature contributions simultaneously.

That is, an input sequence of \(n\) features will be transformed into a ShapleyTensor containing \(n\) feature contributions.

Concretely: if we have an input tensor \(X\) of shape: (num_features, input_dim) we can express this as a sum of features: \(X = \sum_i^n \phi^i\), where \(\phi^i\) is also of shape (num_features, input_dim), with \(\phi^i_j = \begin{cases}X_j&i=j\\0&\textit{otherwise}\end{cases}\)

Without approximations this way of partitioning scales exponentially in the number of input features, quickly becoming infeasible when \(n > 10\).

decompose(batch_encoding: transformers.BatchEncoding) ShapleyTensor[source]
wrap_inputs_embeds(input_ids: Tensor) ShapleyTensor[source]

Explainer

class diagnnose.attribute.explainer.Explainer(decomposer: Decomposer, tokenizer: transformers.PreTrainedTokenizer)[source]

Bases: object

Generates an explanation for a specific input.

explain(input_tokens: Union[str, List[str]], output_tokens: List[str])[source]
print_attributions(full_probs: Tensor, contribution_probs: List[Tensor], input_tokens: Union[str, List[str]], output_tokens: List[str])[source]

GCD Tensor

class diagnnose.attribute.gcd_tensor.GCDTensor(data: Tensor, contributions: Optional[List[Tensor]] = None, shapley_factors: Optional[List[Tuple[List[int], int]]] = None, num_samples: Optional[int] = None, validate: bool = False, baseline_partition: int = 0)[source]

Bases: ShapleyTensor

mul_contributions(*args, **kwargs)[source]

Shapley Tensor

class diagnnose.attribute.shapley_tensor.ShapleyTensor(data: Tensor, contributions: Optional[List[Tensor]] = None, shapley_factors: Optional[List[Tuple[List[int], int]]] = None, num_samples: Optional[int] = None, validate: bool = False, baseline_partition: int = 0)[source]

Bases: object

A ShapleyTensor wraps a torch Tensor. It allows the tensor to be decomposed into a sum of tensors, that each define the contribution of a feature to the tensor.

ShapleyTensors can be passed to any type of torch model. For each operation in the model the intermediate Shapley values are calculated for the list of contributions. This is done using __torch_function__, that allows to override tensor operations.

Parameters
  • data (Tensor) – Input tensor that is decomposed into a sum of contributions.

  • contributions (List[Tensor]) – List of contributions that should sum up to data.

  • shapley_factors (List[Tuple[List[int], int]], optional) – Shapley factors that are calculated with calc_shapley_factors. To prevent unnecessary compute these factors are passed on to subsequent ShapleyTensors.

  • num_samples (int, optional) – Number of feature permutation samples. Increasing the number of samples will reduce the variance of the approximation. If not provided the exact Shapley values will be computed.

  • validate (bool, optional) – Toggle to validate at each step whether contributions still sums up to data. Defaults to False.

  • baseline_partition (int, optional) – Index of the contribution partition to which the baseline fn(0) will be added. If we do not add this baseline the contributions won’t sum up to the full output. Defaults to 0.

__iter__()[source]

Allows a ShapleyTensor to be unpacked directly as:

data, contributions = shapley_tensor
add_contributions(*args, **kwargs)[source]

Non-ShapleyTensors are added to the baseline partition.

cat_contributions(*args, **kwargs)[source]
dim(*args, **kwargs)[source]
dropout2d_contributions(*args, **kwargs)[source]
dropout3d_contributions(*args, **kwargs)[source]
dropout_contributions(*args, **kwargs)[source]

In principle dropout should be disabled when calculating Shapley contributions, but we should still take care of it.

We determine the dropout mask by looking at the difference between the new output data and the input.

linear_contributions(*args, **kwargs)[source]
matmul_contributions(*args, **kwargs)[source]
mul_contributions(*args, **kwargs)[source]
property num_features: int
size(*args, **kwargs)[source]
static split_contributions(*args, **kwargs)[source]

Utils

diagnnose.attribute.utils.calc_exact_shapley_values(fn: Callable, num_features: int, shapley_factors: List[Tuple[List[int], int]], new_data: Union[Tensor, Sequence[Tensor]], baseline_partition: int, *args, **kwargs) Union[List[Tensor], Sequence[List[Tensor]]][source]

Calculates the exact Shapley values for some function fn.

Note that this procedure grows exponentially in the number of features, and should be handled with care.

Parameters
  • fn (Callable) – Torch function for which the Shapley values will be computed.

  • num_features (int) – Number of features for which contributions will be computed.

  • shapley_factors (List[Tuple[List[int], int]]) – List of Shapley factors that is computed using calc_shapley_factors.

  • new_data (Tensor | Sequence[Tensor]) – The output tensor that is currently being decomposed into its contributions. We pass this so we can instantiate the contribution tensors with correct shape beforehand.

  • baseline_partition (int) – Index of the contribution partition to which the baseline fn(0) will be added. If we do not add this baseline the contributions won’t sum up to the full output.

diagnnose.attribute.utils.calc_sample_shapley_values(fn: Callable, num_features: int, num_samples: int, new_data: Union[Tensor, Sequence[Tensor]], baseline_partition: int, *args, **kwargs) Union[List[Tensor], Sequence[List[Tensor]]][source]

Calculates the approximate Shapley values for some function fn.

This procedure is based on that of Castro et al. (2008), and approximates Shapley values in polynomial time.

Parameters
  • fn (Callable) – Torch function for which the Shapley values will be computed.

  • num_features (int) – Number of features for which contributions will be computed.

  • num_samples (int) – Number of feature permutation samples. Increasing the number of samples will reduce the variance of the approximation.

  • new_data (Tensor | Sequence[Tensor]) – The output tensor that is currently being decomposed into its contributions. We pass this so we can instantiate the contribution tensors with correct shape beforehand.

  • baseline_partition (int) – Index of the contribution partition to which the baseline fn(0) will be added. If we do not add this baseline the contributions won’t sum up to the full output.

diagnnose.attribute.utils.calc_shapley_factors(num_features: int) List[Tuple[List[int], int]][source]

Creates the normalization factors for each subset of features.

These factors are based on the original Shapley formulation: https://en.wikipedia.org/wiki/Shapley_value

If, for instance, we were to compute these factors for item \(a\) in the set \(N = \{a, b, c\}\), we would pass \(|N|\). This returns the list \([([], 2), ([0], 1), ([1], 1), ([0, 1], 2])]\). The first item of each tuple should be interpreted as the indices for the set \(N\setminus\{a\}: (0 \Rightarrow b, 1 \Rightarrow c)\), mapped to their factors: \(|ids|! \cdot (n - |ids|)!\).

Parameters

num_features (int) – Number of features for which Shapley values will be computed.

Returns

shapley_factors – Dictionary mapping a tuple of indices to its corresponding normalization factor.

Return type

List[Tuple[List[int], int]]

diagnnose.attribute.utils.init_contributions(new_data: Union[Tensor, Sequence[Tensor]], num_features: int) Union[List[Tensor], Sequence[List[Tensor]]][source]
diagnnose.attribute.utils.monkey_patch()[source]

Not all torch functions correctly implement __torch_function__ yet (i.e. in torch v1.5), as is discussed here: https://github.com/pytorch/pytorch/issues/34294

We override the __torch_function__ behaviour for torch.cat, torch.stack, Tensor.expand_as, and Tensor.type_as.

diagnnose.attribute.utils.normalize_contributions(contributions: Union[List[Tensor], Sequence[List[Tensor]]], factor: int, output_is_sequential: bool) None[source]
diagnnose.attribute.utils.perm_generator(num_features: int, num_samples: int) Iterable[List[int]][source]

Generator for feature index permutations.

diagnnose.attribute.utils.sum_contributions(contributions: List[Tensor], coalition: List[int]) Tensor[source]

Sums the contributions that are part of the provided coalition.

diagnnose.attribute.utils.unwrap(args: Any, attr: str = 'data', coalition: Optional[List[int]] = None) Any[source]

Unwraps a list of args that might contain ShapleyTensors.

Can be used to retrieve: 1. The full tensor of each ShapleyTensor, 2. The list of contributions, or 3. The sum of contributions for a specific coalition.

Unwrapping is performed recursively. Non-ShapleyTensors are left unchanged.

Parameters
  • args (Any) – Either the full list of args, or an individual element of that list, as unwrapping is performed recursively.

  • attr (str, optional) – The ShapleyTensor attribute that should be returned, either data or contributions.

  • coalition (List[int], optional) – Optional list of coalition indices. If provided the contributions at the indices of the coalition are summed up and returned, instead of the full list of contributions.

diagnnose.attribute.utils.update_contributions(contributions: Union[List[Tensor], Sequence[List[Tensor]]], f_idx: int, output_is_sequential: bool, data_with: Union[Tensor, Sequence[Tensor]], data_wo: Optional[Union[Tensor, Sequence[Tensor]]] = None, factor: float = 1.0) None[source]