Model interpretability and explainability using SHAP (SHapley Additive exPlanations). Use this skill when explaining machine learning model predictions, computing feature importance, generating SHAP plots (waterfall, beeswarm, bar, scatter, force, heatmap), debugging models, analyzing model bias or fairness, comparing models, or implementing explainable AI. Works with tree-based models (XGBoost, LightGBM, Random Forest), deep learning (TensorFlow, PyTorch), linear models, and any black-box model.
SKILL.md
SHAP
Use SHAP to describe how a fitted predictive model maps inputs to outputs. Work from the modern shap.Explanation API, make the explained output and background distribution explicit, and validate every explanation before interpreting it.
This skill is aligned with SHAP 0.52.0 (released 2026-05-28). That release requires Python 3.12 or newer.
Operating Rules
Explain a fixed, evaluated model; do not use SHAP as a substitute for predictive validation.
Use held-out or clearly labeled analysis rows for explanations. Choose background rows only from an appropriate training or reference population.
State the explained output: regression value, raw margin, probability, log loss, logit, or another model method.
Keep explanations as shap.Explanation objects. Call explainer(X); use .shap_values(X) only when maintaining legacy code.
For multi-output models, select one output before using tabular plots: explanation[..., output_index].
Check base_values + values.sum(...) against the exact model output being explained.
Treat SHAP as a description of model behavior under a masking/background choice. It does not establish causality, fairness, recourse, or scientific mechanism.
Never silence an additivity failure until input shape, preprocessing, model version, output space, and row ordering have been checked.
Do not load untrusted pickle, joblib, model, or explainer artifacts; those formats can execute code during deserialization.
Install
Create an isolated environment and pin the documented release:
shap[plots] installs the plotting dependencies. Add the fitted model's package at a version compatible with the project. For older Python compatibility, read references/migration.md instead of silently installing a different SHAP release.
Confirm the environment before debugging an API mismatch:
For classifiers, decide whether the task needs raw margins or probabilities. Defaults differ by model family; never infer units from the plot color or sign.
2. Select an explainer and masker
Start with shap.Explainer(model, masker) when automatic dispatch is sufficient. Instantiate a specialized explainer when its assumptions or output controls matter.
Situation
Preferred choice
Important constraint
Supported tree ensemble
TreeExplainer
model_output="probability" and "log_loss" require interventional masking and background data
Linear model
LinearExplainer
The masker determines interventional versus correlation-aware behavior
Small feature space
ExactExplainer
Cost grows quickly with unconstrained feature count
General tabular callable
PermutationExplainer
Budget at least one full forward/reverse permutation
Hierarchical feature groups, text, or image
PartitionExplainer
The partition tree changes the cooperative game
Differentiable neural network
DeepExplainer or GradientExplainer
Framework support, output shape, and background choice require testing
feature_perturbation="auto" uses interventional semantics when background data is supplied and tree-path-dependent semantics otherwise;
probability and log-loss output modes are supported only with interventional semantics;
pass approximate=True to explainer(X, approximate=True) if deliberately using the lower-fidelity tree approximation; do not pass it to the constructor.
5. Use a model-agnostic callable deliberately
Pass the exact callable whose outputs will be interpreted:
Increase max_evals to average over more permutations when estimates are unstable. Keep the seed, background sample, and evaluation budget in the report.
6. Visualize the question, not merely the available plot
Question
Plot
Which features have the largest average attribution magnitude?
shap.plots.bar(exp)
How do direction, magnitude, and observed values vary globally?
shap.plots.beeswarm(exp)
Why did one prediction differ from its baseline?
shap.plots.waterfall(exp[i])
How does one feature's attribution vary over its values?
shap.plots.scatter(exp[:, feature])
Do explanations form sample-level patterns?
shap.plots.heatmap(exp)
How do predefined cohorts differ descriptively?
shap.plots.bar(exp.cohorts(labels).abs.mean(0))
Which tokens or image regions contribute to an output?
additivity error or applicable approximation diagnostics;
known correlated/grouped features;
whether results are local, aggregated, or cohort-specific;
a clear non-causal statement.
Common Tasks
Global and local analysis
Use global plots to locate important patterns, scatter plots to inspect those patterns, and local plots to investigate selected rows. Do not select only visually dramatic rows without documenting the selection rule.
Multiclass models
Set output_names where possible, inspect explanation.output_names, and slice an output before plotting:
class_exp = explanation[..., "class_name"]
# or
class_exp = explanation[..., class_index]
Never average signed attributions across classes. For cross-class comparison, preserve the same model, rows, background, output space, and aggregation.
Cohorts, subgroup analysis, and fairness
SHAP can compare how a model uses features across cohorts, but this is not a fairness test. A protected feature with small SHAP magnitude does not rule out proxy discrimination, and removing a protected feature does not establish fairness. Pair attribution analysis with performance, calibration, error-rate, and domain-appropriate fairness metrics.
See references/workflows.md for cohort construction, model comparison, error analysis, log-loss explanations, monitoring, and production records.
Text and images
Use domain maskers rather than treating tokens or pixels as ordinary independent columns:
shap.maskers.Text(tokenizer) with PartitionExplainer for token groups;
shap.maskers.Image(...) with PartitionExplainer for image regions;
restrict expensive multi-output models with outputs=....
Print Python, SHAP, model-library, NumPy, and framework versions.
Verify the model receives exactly the same transformed columns, order, dtype, and missing-value representation used during fitting.
Print values.shape, base_values.shape, data.shape, feature_names, and output_names.
Confirm the selected output and output units.
Recompute predictions on the same rows in the same order.
Test a smaller batch and representative background.
Only then investigate package-specific compatibility or approximation settings.
Use references/troubleshooting.md for additivity failures, shape mismatches, categorical features, pipelines, deep-learning frameworks, plotting, and performance.
Bundled Script
Run a deterministic, self-contained tabular example that writes importance data, metadata, and plots:
The script does not download data or deserialize models. Read it as a template, then replace the built-in dataset and model while preserving output selection and additivity validation.
