Intel® Explainable AI Tools v1.3.0

What's Changed

• Updated Python Dependencies to the latest compatible versions to support Python versions 3.10 and 3.11.
• Added support for five more datasets in the Toxicity Detection Model Benchmarking Script on BeaverTails, OpenAI Moderation, SurgeAI Toxicity, ToxicChat, ToxiGen, and XSTest
  datasets with additional support of Xeon CPU along with the previous Gaudi 2 accelerator support.
• Replace Gaudi trainer with Habana torch for Gaudi 2 accelerator inference in Toxicity Detection Benchmarking Script.
• Separated targets with Separate virtual environments in Makefile to prevent dependency conflicts and streamline the package installation during usage of Poetry.

Jupyter Notebooks

• Jupyter notebook for running Model Card generator for Image Classification tasks image-classification-model-card.ipynb
• Jupyter notebook for running Model Card generator for Multimodal Classification tasks multimodal-classification-model-card.ipynb

Bug fixes

• Enhanced model path compatibility with Keras updates and optimized dimension swapping in shap_values for DeepExplainer and GradientExplainer, including updated unittests.

Validated configuration

• Ubuntu 22.04/24.04 LTS
• Python 3.10, 3.11
• PyTorch 2.5.1
• TensorFlow-cpu 2.17.1
• Torchvision 0.20.1
• TensorFlow Hub 0.16.1

Known limitations

Intel® Explainable AI Tools is only supported on Linux

GitHub pages:

https://intel.github.io/intel-xai-tools/v1.3.0/

Intel® Explainable AI Tools v1.2.0

What's Changed

• Added instructions and detailed examples in placeholder for Model Card Generator UI fields to improve user experience.
• Docker Compose infra for Model Card Generator UI deployments.
• Added HTML template support for Model Card Generator.
• Added functionality to plot individual graphs in the Model Card for each class in multiclass scenarios using Model Card Generator.
• Added Toxicity Detection Model Benchmarking Script on Jigsaw Unintended Bias and ToxicChat datasets with Gaudi 2 accelerator support.

Jupyter Notebooks

• Jupyter notebook for running Model Card generator on Hugging Face model for multiclass classification multiclass-classification-model-card.ipynb

Bug fixes

• Improved error handling in case of metric file uploads in Model Card Generator UI.
• Added unit tests for benchmarking toxicity detection models.
• Resize Images in the Model Card based on the window size.

Validated configuration

• Ubuntu 22.04 LTS
• Python 3.9, 3.10
• PyTorch 2.2.0
• Intel® Optimization for TensorFlow 2.14.0
• Torchvision 0.17.0
• TensorFlow Hub 0.15.0

Known limitations

Intel® Explainable AI Tools is only supported on Linux

GitHub pages:

https://intel.github.io/intel-xai-tools/v1.2.0/

New Contributors

• @qgao007

Intel® Explainable AI Tools v1.1.0

What's Changed

• Generate Model Cards independent of AI Framework (e.g. PyTorch, TensorFlow, etc) by removing TFMA hard dependence.
• Add NeuralChat SUT to ModelGauge for running MLCommons v0.5 Standard Safety Benchmark
• Provide User Interface for creating Model Cards Model Card Generator UI
• Docker Compose infra for Explainer and Model Card Generator deployments
• Adding non-root user to all containers
• Fuzzing support with tests for Explainer and ModelCardGen
• Added Python code styler checker to CICD

Jupyter Notebooks

• Jupyter notebook for running model card generator on hugging face model hugging-face-model-card.ipynb

Validated configuration

• Ubuntu 22.04 LTS
• Python 3.9, 3.10
• PyTorch 2.2.0
• Intel® Optimization for TensorFlow 2.14.0
• Torchvision 0.17.0
• TensorFlow Hub 0.15.0

Known limitations

Intel® Explainable AI Tools is only supported on Linux

GitHub pages:

https://intel.github.io/intel-xai-tools/v1.1.0/

New Contributors

• @etcylfleet
• @mhbuehler

Intel® Explainable AI Tools v1.0.0

New Features

• Rearchitected project to provide plugin-based approach via Poetry dependency manager
• LLM Explainer: Hugging Face attributions plugin that uses SHAP to explain generative text LLM model output
• Dockerfiles for explainer and model card generator with Jupyter interface and example notebooks

Jupyter Notebooks

• HugingFace_Gen_LLM_Interpret.ipynb

Bug fixes

• Removed Pipeline explainer
• Added unit test for new LLM Explainer
• All notebooks and unit tests updated for latest API

Validated configuration

• Ubuntu 22.04 LTS
• Python 3.9, 3.10
• Intel® Optimization for TensorFlow 2.14.0
• PyTorch 2.2.0
• Torchvision 0.17.0
• TensorFlow Hub 0.15.0

Known limitations

• Intel® Explainable AI Tools is only supported on Linux

GitHub pages:

https://intelai.github.io/intel-xai-tools/v1.0.0/

Latest binaries published here https://storage.googleapis.com/public-artifacts/xai/intel_ai_safety-1.0.0-py3-none-any.whl

Intel® Explainable AI Tools v0.6.0

New Features

• Added link to repo on all model card generations

Jupyter Notebooks

• Added EigenCAM notebook

Bug fixes

• Added unit tests for EigenCAM and refactored the EigenCAM class to be consistent with explainer class structures
• Updated versions of dependencies in order to work on python 3.8 - 3.10
• Updated package requirements for notebooks

Validated configuration

• Ubuntu 22.04 LTS
• Python 3.8, 3.9, 3.10
• Intel® Optimization for TensorFlow 2.13.0
• PyTorch 2.0.1
• Torchvision 0.15.2
• TensorFlow Hub 0.14.0

Known limitations

• Intel® Explainable AI Tools is only supported on Linux

GitHub pages:

https://intelai.github.io/intel-xai-tools/v0.6.0/

Intel® Explainable AI Tools v0.5.0

New Features

• ShapUI: a user interface to explore and compare impact scores of model predictions for each record of a tabular data set and discover insights of a model's behavior.
• Added info panel feature with text descriptions designed to help user with interpreting graphs

Jupyter Notebooks

• Added notebook to benchmark PartitionExplainer() in AI Kit environment against basic Python environment.

Bug fixes

• Improved consistency of code between explainer visualize methods
• Split TensorFlow* implementations from PyTorch implementation for both Explainer and Model Card Generator
• Fixes to links in documentation
• Improve test coverage for Explainer's attributions module
• Documented how to use Model Card Generator with multiple TF records
• Improved compatibility for dependencies on Python 3.9
• Moved Model Card Generator Notebooks to common directory as Explainer
• Simplified directory structure
• Fixed tests dependence on UCI Machine Learning dataset URL

Validated configuration

• Ubuntu 22.04 LTS
• Python 3.9
• Intel® Optimization for TensorFlow 2.12.0
• PyTorch 1.13.1
• Torchvision 0.14.1
• TensorFlow Hub 0.13.0

Known limitations

• Intel® Explainable AI Tools is only supported on Python 3.9

GitHub pages:

https://intelai.github.io/intel-xai-tools/v0.5.0/

Intel® Explainable AI Tools v0.4.0

New Features:

• ShapUI: a user interface to explore and compare impact scores of model predictions for each record of a tabular data set and discover insights of a model's behavior.
• Added info panel feature with text descriptions designed to help user with interpreting graphs
• Experimental support for Python 3.10

Validated configuration

​

• Ubuntu 22.04 LTS
• Python 3.9, 3.10
• Intel® Optimization for TensorFlow 2.11.0
• PyTorch 1.13.1
• Torchvision 0.14.1
• TensorFlow Hub 0.12.0
  ​

Known limitations

• Model Card Generator is only supported on Python 3.9 and did not get packaged as part of installer wheel

Intel® Explainable AI Tools v0.3.0

New Features:

• Single installer for both Model card generator and Explainers

Explainers:

• Unified Explainers' APIs
• CAM explainer which utilizes XGradCAM, the SOTA CAM method
• EigenCAM explainer for object detection model (FasterRCNN, YOLO)
• Compatibility support for Frozen models introduced by SciPy 1.10

Jupyter Notebooks

• ResNet50 ImageNet Classification using the CAM Explainer
• Custom CNN MNIST Classification using the Attributions Explainer
• Custom NN NewsGroups Classification using the Attributions Explainer
• Custom CNN CIFAR-10 Classification using the Attributions Explainer
• Multimodal Breast Cancer Detection Explainability
• Fine Tuned Text Classifier with PyTorch using the Intel® Explainable AI API
• Custom Neural Network Heart Disease Classification using the Attributions Explainer

Bug fixes:

• Many documentation improvements
• Improve test coverage for both Explainer and Model card generator-

Validated configuration

• Ubuntu 20.04 LTS
• Python 3.9
• Intel® Optimization for TensorFlow 2.11.0
• PyTorch 1.13.1
• Torchvision 0.14.1
• TensorFlow Hub 0.12.0

Known limitations

• Intel® Explainable AI Tools in only supported on Python 3.9

Intel® Explainable AI Tools v0.2

New Features:

Model Card Generator:

• Support for general model overview plots visualize performance as a function of threshold score.
• Support for interactive plots to visualize fairness metrics across data groupings.
• Added support for Model Card generation for PyTorch models.
• Added support for Model Cards for multiple datasets.

Explainer:

• Allows injection of XAI methods into Python workflows/notebooks without requiring version compatibility of resident packages in the active python environment.
• Supports 3 explainable plugin methods:
  • feature attributions: Explains a model’s predictions based on how the model has weighted features it’s been trained on
  • metrics: calculates and plots the standard base metrics used to evaluate model performance
  • language model explanations: explains transformer based language models by visualizing input token importance, hidden state contributions, sequence embeddings and attention heads
• An interactive CLI allows the user to install each plugin. Provides a simple solution to create new plugins and expand on existing plugins.
• Complete documentation with notebooks examples in the natural language, computer vision, and data frame domain.

Bug fixes:

Model Card Generator:

• N/A

Explainer:

• N/A, Initial public release

Supported Configurations

Intel® Explainable AI Tools v0.2.0 is validated on the following environment:

• Ubuntu 20.04 LTS
• Python 3.9

Intel® Explainable AI Tools v0.0.1

Supported Frameworks

- TensorFlow

New features

• Model Card Generator:

Allows users to create interactive HTML reports of containing model performance and fairness metrics.
Supports general model overview plots visualize performance as a function of threshold score.
Supports interactive plots to visualize fairness metrics across data groupings.

Bug fixes:

• N/A

Supported Configurations

Intel® Explainable AI Tools v0.0.1 is validated on the following environment:

• Ubuntu 20.04 LTS
• Python 3.8, 3.9