What’s new with Explainable AI in Julia?

An invitation to the Julia-XAI ecosystem

Adrian Hill
hill@tu-berlin.de

Technische Universität Berlin

2024-07-11

Introduction

Invitation to the Julia-XAI ecosystem

  • JuliaCon 2022: ExplainableAI.jl
  • JuliaCon 2024: Julia-XAI organization

What kind of explanations?

Post-hoc, local input-space explanations of black-box models:

“Which part of the input is responsible for the model’s output?”

Related work: Taija ecosystem by Patrick Altmeyer et al.

Motivating example

using Flux, Metalhead
model = VGG(19; pretrain=true).layers
testmode!(model)

prediction = model(input)
class = argmax(prediction) # class 484: castle
 Downloading artifact: vgg19-IMAGENET1K_V1
CartesianIndex(484, 1)

Pretrained model predicts class “castle”. But why?

What’s new: methods

  • ExplainableAI.jl – applicable on any differentiable classifier
    • Gradient
    • InputTimesGradient
    • SmoothGrad Smilkov et al. (2017)
    • IntegratedGradients Sundararajan, Taly, and Yan (2017)
    • 🆕 GradCAM Selvaraju et al. (2017)
  • 🆕 RelevancePropagation.jl – applicable on Flux.jl classifiers
    • LRP Bach et al. (2015)
      • 🆕 support for ResNets
      • 🆕 support for Transformers Ali et al. (2022), Achtibat et al. (2024)
    • 🆕 CRP Achtibat et al. (2023)

Methods: input sensitivity

using ExplainableAI
using VisionHeatmaps

analyzer = Gradient(model)
expl = analyze(input, analyzer)
heatmap(expl)

Methods: GradCAM

using ExplainableAI
using VisionHeatmaps

analyzer = GradCAM(model[1], model[2])
expl = analyze(input, analyzer)
heatmap_overlay(expl, img)

Methods: LRP

using RelevancePropagation
using VisionHeatmaps

analyzer = LRP(model, EpsilonPlusFlat())
explanation = analyze(input, analyzer)
heatmap(explanation)

What’s new: Julia-XAI ecosystem

What’s new: VisionHeatmaps.jl

heatmap(explanation)
heatmap(explanation; colorscheme=:inferno, rangescale=:extrema)

What’s new: TextHeatmaps.jl

Visualize explanations on language models

using TextHeatmaps

words = split("I loved the concert but not the opening act")
val = [0.1, 2.5, 0.0, 0.3, -0.6, -1.4, 0.0, 0.1, -0.1]
TextHeatmaps.heatmap(val, words)
WARNING: using TextHeatmaps.heatmap in module Notebook conflicts with an existing identifier.

Ilovedtheconcertbutnottheopeningact

TextHeatmaps.heatmap(val, words; rangescale=:extrema)

Ilovedtheconcertbutnottheopeningact

using ColorSchemes
TextHeatmaps.heatmap(val, words; colorscheme=:inferno)

Ilovedtheconcertbutnottheopeningact

What’s new: XAIBase.jl

Interface definition:

  1. method has to be a subtype of AbstractXAIMethod
  2. method has to implement analyze function
  3. method has to return Explanation

Benefits

Acknowledgements

Adrian Hill acknowledges support by the Federal Ministry of Education and Research (BMBF) for the Berlin Institute for the Foundations of Learning and Data (BIFOLD) (01IS18037A).

References

Achtibat, Reduan, Maximilian Dreyer, Ilona Eisenbraun, Sebastian Bosse, Thomas Wiegand, Wojciech Samek, and Sebastian Lapuschkin. 2023. “From Attribution Maps to Human-Understandable Explanations Through Concept Relevance Propagation.” Nature Machine Intelligence 5 (9): 1006–19. https://doi.org/10.1038/s42256-023-00711-8.
Achtibat, Reduan, Sayed Mohammad Vakilzadeh Hatefi, Maximilian Dreyer, Aakriti Jain, Thomas Wiegand, Sebastian Lapuschkin, and Wojciech Samek. 2024. “Attnlrp: Attention-Aware Layer-Wise Relevance Propagation for Transformers.” arXiv Preprint arXiv:2402.05602.
Ali, Ameen, Thomas Schnake, Oliver Eberle, Grégoire Montavon, Klaus-Robert Müller, and Lior Wolf. 2022. “XAI for Transformers: Better Explanations Through Conservative Propagation.” In International Conference on Machine Learning, 435–51. PMLR.
Bach, Sebastian, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. 2015. “On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation.” PloS One 10 (7): e0130140.
Selvaraju, Ramprasaath R, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. “Grad-Cam: Visual Explanations from Deep Networks via Gradient-Based Localization.” In Proceedings of the IEEE International Conference on Computer Vision, 618–26.
Smilkov, Daniel, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. 2017. “Smoothgrad: Removing Noise by Adding Noise.” arXiv Preprint arXiv:1706.03825.
Sundararajan, Mukund, Ankur Taly, and Qiqi Yan. 2017. “Axiomatic Attribution for Deep Networks.” In International Conference on Machine Learning, 3319–28. PMLR.

github.com/Julia-XAI