Vanilla Saliency and SmoothGrad

• Vanilla Saliency

• SmoothGrad

class tf_keras_vis.saliency.Saliency(model, model_modifier=None, clone=True)

Bases: ModelVisualization

Vanilla Saliency and Smooth-Grad

References

• Vanilla Saliency: Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps (https://arxiv.org/pdf/1312.6034)

• SmoothGrad: removing noise by adding noise (https://arxiv.org/pdf/1706.03825)

Parameters:

• model – A tf.keras.Model instance. When model_modifier is NOT None, this model will be cloned with tf.keras.models.clone_model function and then will be modified by model_modifier according to needs.

• model_modifier – A tf_keras_vis.utils.model_modifiers.ModelModifier instance, a function or a list of them. We recommend to apply tf_keras_vis.utils.model_modifiers.ReplaceToLinear to all visualizations (except tf_keras_vis.scorecam.Scorecam) when the model output is softmax. Defaults to None.

• clone – A bool that indicates whether or not it clones the model. When False, the model won’t be cloned. Note that, although when True, the model won’t be clone if model_modifier is None. Defaults to True.

__call__(score, seed_input, smooth_samples=0, smooth_noise=0.20, keepdims=False, gradient_modifier=lambda grads: ..., training=False, normalize_map=True, unconnected_gradients=tf.UnconnectedGradients.NONE)

Generate an attention map that appears how output value changes with respect to a small change in input image pixels.

Parameters:

• score –

  A tf_keras_vis.utils.scores.Score instance, function or a list of them. For example of the Score instance to specify visualizing target:

  scores = CategoricalScore([1, 294, 413])
  

  The code above means the same with the one below:

  score = lambda outputs: (outputs[0][1], outputs[1][294], outputs[2][413])
  

  When the model has multiple outputs, you MUST pass a list of Score instances or functions. For example:

  from tf_keras_vis.utils.scores import CategoricalScore, InactiveScore
  score = [
      CategoricalScore([1, 23]),  # For 1st model output
      InactiveScore(),            # For 2nd model output
      ...
  ]
  

• seed_input – A tf.Tensor, numpy.ndarray or a list of them to input in the model. That’s when the model has multiple inputs, you MUST pass a list of tensors.

• smooth_samples (int, optional) – The number of calculating gradients iterations. When over zero, this method will work as SmoothGrad. When zero, it will work as Vanilla Saliency. Defaults to 0.

• smooth_noise – Noise level. Defaults to 0.20.

• keepdims – A bool that indicates whether or not to keep the channels-dimension. Defaults to False.

• gradient_modifier – A function to modify gradients. Defaults to None.

• training – A bool that indicates whether the model’s training-mode on or off. Defaults to False.

• normalize_map (bool, optional) – When True, saliency map will be normalized. Defaults to True.

• unconnected_gradients – Specifies the gradient value returned when the given input tensors are unconnected. Defaults to tf.UnconnectedGradients.NONE.

Returns:

An numpy.ndarray or a list of them. They are the saliency maps that indicate the seed_input regions whose change would most contribute the score value.

Raises:

ValueError – When there is any invalid arguments.

Return type:

ndarray | list