Captum

Captum is an open-source library for model interpretability built on PyTorch. It provides algorithms to attribute predictions to input features across modalities such as vision, text, and more. Captum supports most PyTorch models with minimal code changes and includes well-known methods like Integrated Gradients. Its extensible design facilitates research, benchmarking, and implementation of new interpretability algorithms.

Key Features

• Multi-Modal: Supports interpretability for vision, text, and other modalities.
• Built on PyTorch: Works with most PyTorch models and requires minimal modification.
• Extensible: Open-source and generic, enabling easy implementation of new algorithms.

Quickstart

Install via conda or pip, define a model, and apply an algorithm like Integrated Gradients to obtain feature attributions.