attribute¶
Decomposer¶
- class diagnnose.attribute.decomposer.ContextualDecomposer(model: LanguageModel, num_samples: Optional[int] = None, tensor_type: str = 'ShapleyTensor')[source]¶
Bases:
Decomposer
A ContextualDecomposer propagates each input feature contribution individually, set out against the contributions of all other features combined.
This idea has been proposed in Murdocht et al., (2018): https://arxiv.org/abs/1801.05453
An input sequence of \(n\) features will be transformed into a ShapleyTensor containing \(2\) feature contributions: one containing the contributions of the feature of interest (\(\beta\)), and one containing the contributions of all other features (\(\gamma\)).
Concretely: if we have an input tensor \(X\) of shape:
(num_features, input_dim)
we can express this as a sum of features: \(X = \beta^i + \gamma^i\), where both \(\beta\) and \(\gamma\) are also of shape(num_features, input_dim)
, with \(\beta^i_j = \begin{cases}X_j&i=j\\0&\textit{otherwise}\end{cases}\) and \(\gamma^i_j = \begin{cases}X_j&i\neq j\\0&\textit{otherwise}\end{cases}\)This way of partitioning scales polynomially in the number of input features, but requires a separate forward pass for each individual feature contribution \(\beta^i\).
- decompose(batch_encoding: transformers.BatchEncoding) ShapleyTensor [source]¶
- wrap_inputs_embeds(input_ids: Tensor) List[ShapleyTensor] [source]¶
- class diagnnose.attribute.decomposer.Decomposer(model: LanguageModel, num_samples: Optional[int] = None, tensor_type: str = 'ShapleyTensor')[source]¶
Bases:
ABC
Abstract base class for Decomposer classes.
A Decomposer takes care of dividing the input features into the desired partition of contributions.
- abstract decompose(batch_encoding: transformers.BatchEncoding) ShapleyTensor [source]¶
- abstract wrap_inputs_embeds(input_ids: Tensor) ShapleyTensor [source]¶
- class diagnnose.attribute.decomposer.ShapleyDecomposer(model: LanguageModel, num_samples: Optional[int] = None, tensor_type: str = 'ShapleyTensor')[source]¶
Bases:
Decomposer
A ShapleyDecomposer propagates all input feature contributions simultaneously.
That is, an input sequence of \(n\) features will be transformed into a ShapleyTensor containing \(n\) feature contributions.
Concretely: if we have an input tensor \(X\) of shape:
(num_features, input_dim)
we can express this as a sum of features: \(X = \sum_i^n \phi^i\), where \(\phi^i\) is also of shape(num_features, input_dim)
, with \(\phi^i_j = \begin{cases}X_j&i=j\\0&\textit{otherwise}\end{cases}\)Without approximations this way of partitioning scales exponentially in the number of input features, quickly becoming infeasible when \(n > 10\).
- decompose(batch_encoding: transformers.BatchEncoding) ShapleyTensor [source]¶
- wrap_inputs_embeds(input_ids: Tensor) ShapleyTensor [source]¶
Explainer¶
- class diagnnose.attribute.explainer.Explainer(decomposer: Decomposer, tokenizer: transformers.PreTrainedTokenizer)[source]¶
Bases:
object
Generates an explanation for a specific input.
GCD Tensor¶
- class diagnnose.attribute.gcd_tensor.GCDTensor(data: Tensor, contributions: Optional[List[Tensor]] = None, shapley_factors: Optional[List[Tuple[List[int], int]]] = None, num_samples: Optional[int] = None, validate: bool = False, baseline_partition: int = 0)[source]¶
Bases:
ShapleyTensor
Shapley Tensor¶
- class diagnnose.attribute.shapley_tensor.ShapleyTensor(data: Tensor, contributions: Optional[List[Tensor]] = None, shapley_factors: Optional[List[Tuple[List[int], int]]] = None, num_samples: Optional[int] = None, validate: bool = False, baseline_partition: int = 0)[source]¶
Bases:
object
A ShapleyTensor wraps a torch Tensor. It allows the tensor to be decomposed into a sum of tensors, that each define the contribution of a feature to the tensor.
ShapleyTensors can be passed to any type of torch model. For each operation in the model the intermediate Shapley values are calculated for the list of contributions. This is done using __torch_function__, that allows to override tensor operations.
- Parameters
data (Tensor) – Input tensor that is decomposed into a sum of contributions.
contributions (List[Tensor]) – List of contributions that should sum up to data.
shapley_factors (List[Tuple[List[int], int]], optional) – Shapley factors that are calculated with calc_shapley_factors. To prevent unnecessary compute these factors are passed on to subsequent ShapleyTensors.
num_samples (int, optional) – Number of feature permutation samples. Increasing the number of samples will reduce the variance of the approximation. If not provided the exact Shapley values will be computed.
validate (bool, optional) – Toggle to validate at each step whether contributions still sums up to data. Defaults to False.
baseline_partition (int, optional) – Index of the contribution partition to which the baseline fn(0) will be added. If we do not add this baseline the contributions won’t sum up to the full output. Defaults to 0.
- __iter__()[source]¶
Allows a ShapleyTensor to be unpacked directly as:
data, contributions = shapley_tensor
- dropout_contributions(*args, **kwargs)[source]¶
In principle dropout should be disabled when calculating Shapley contributions, but we should still take care of it.
We determine the dropout mask by looking at the difference between the new output data and the input.
- property num_features: int¶
Utils¶
- diagnnose.attribute.utils.calc_exact_shapley_values(fn: Callable, num_features: int, shapley_factors: List[Tuple[List[int], int]], new_data: Union[Tensor, Sequence[Tensor]], baseline_partition: int, *args, **kwargs) Union[List[Tensor], Sequence[List[Tensor]]] [source]¶
Calculates the exact Shapley values for some function fn.
Note that this procedure grows exponentially in the number of features, and should be handled with care.
- Parameters
fn (Callable) – Torch function for which the Shapley values will be computed.
num_features (int) – Number of features for which contributions will be computed.
shapley_factors (List[Tuple[List[int], int]]) – List of Shapley factors that is computed using
calc_shapley_factors
.new_data (Tensor | Sequence[Tensor]) – The output tensor that is currently being decomposed into its contributions. We pass this so we can instantiate the contribution tensors with correct shape beforehand.
baseline_partition (int) – Index of the contribution partition to which the baseline fn(0) will be added. If we do not add this baseline the contributions won’t sum up to the full output.
- diagnnose.attribute.utils.calc_sample_shapley_values(fn: Callable, num_features: int, num_samples: int, new_data: Union[Tensor, Sequence[Tensor]], baseline_partition: int, *args, **kwargs) Union[List[Tensor], Sequence[List[Tensor]]] [source]¶
Calculates the approximate Shapley values for some function fn.
This procedure is based on that of Castro et al. (2008), and approximates Shapley values in polynomial time.
- Parameters
fn (Callable) – Torch function for which the Shapley values will be computed.
num_features (int) – Number of features for which contributions will be computed.
num_samples (int) – Number of feature permutation samples. Increasing the number of samples will reduce the variance of the approximation.
new_data (Tensor | Sequence[Tensor]) – The output tensor that is currently being decomposed into its contributions. We pass this so we can instantiate the contribution tensors with correct shape beforehand.
baseline_partition (int) – Index of the contribution partition to which the baseline fn(0) will be added. If we do not add this baseline the contributions won’t sum up to the full output.
- diagnnose.attribute.utils.calc_shapley_factors(num_features: int) List[Tuple[List[int], int]] [source]¶
Creates the normalization factors for each subset of features.
These factors are based on the original Shapley formulation: https://en.wikipedia.org/wiki/Shapley_value
If, for instance, we were to compute these factors for item \(a\) in the set \(N = \{a, b, c\}\), we would pass \(|N|\). This returns the list \([([], 2), ([0], 1), ([1], 1), ([0, 1], 2])]\). The first item of each tuple should be interpreted as the indices for the set \(N\setminus\{a\}: (0 \Rightarrow b, 1 \Rightarrow c)\), mapped to their factors: \(|ids|! \cdot (n - |ids|)!\).
- Parameters
num_features (int) – Number of features for which Shapley values will be computed.
- Returns
shapley_factors – Dictionary mapping a tuple of indices to its corresponding normalization factor.
- Return type
List[Tuple[List[int], int]]
- diagnnose.attribute.utils.init_contributions(new_data: Union[Tensor, Sequence[Tensor]], num_features: int) Union[List[Tensor], Sequence[List[Tensor]]] [source]¶
- diagnnose.attribute.utils.monkey_patch()[source]¶
Not all torch functions correctly implement
__torch_function__
yet (i.e. in torch v1.5), as is discussed here: https://github.com/pytorch/pytorch/issues/34294We override the
__torch_function__
behaviour fortorch.cat
,torch.stack
,Tensor.expand_as
, andTensor.type_as
.
- diagnnose.attribute.utils.normalize_contributions(contributions: Union[List[Tensor], Sequence[List[Tensor]]], factor: int, output_is_sequential: bool) None [source]¶
- diagnnose.attribute.utils.perm_generator(num_features: int, num_samples: int) Iterable[List[int]] [source]¶
Generator for feature index permutations.
- diagnnose.attribute.utils.sum_contributions(contributions: List[Tensor], coalition: List[int]) Tensor [source]¶
Sums the contributions that are part of the provided coalition.
- diagnnose.attribute.utils.unwrap(args: Any, attr: str = 'data', coalition: Optional[List[int]] = None) Any [source]¶
Unwraps a list of args that might contain ShapleyTensors.
Can be used to retrieve: 1. The full tensor of each ShapleyTensor, 2. The list of contributions, or 3. The sum of contributions for a specific coalition.
Unwrapping is performed recursively. Non-ShapleyTensors are left unchanged.
- Parameters
args (Any) – Either the full list of args, or an individual element of that list, as unwrapping is performed recursively.
attr (str, optional) – The ShapleyTensor attribute that should be returned, either data or contributions.
coalition (List[int], optional) – Optional list of coalition indices. If provided the contributions at the indices of the coalition are summed up and returned, instead of the full list of contributions.