shap

SHAP values attribute model outputs to input features by computing Shapley values from game theory. Load a trained model, initialize the appropriate explainer (TreeExplainer for XGBoost/LightGBM/CatBoost/Random Forest, DeepExplainer for TensorFlow/PyTorch, or KernelExplainer for black-box models), pass background data, compute SHAP values, then visualize with waterfall, beeswarm, bar, or force plots to see feature contributions.

Waterfall plots display how each feature pushes a single prediction from the baseline toward the final output, showing both magnitude and direction. Beeswarm plots reveal feature value distributions across many predictions. For a single instance, waterfall provides the clearest local explanation; for cohorts, use beeswarm or bar plots to identify patterns.

Yes. Compute SHAP values separately for different cohorts or demographics, then compare feature attributions and prediction patterns across groups using heatmap or scatter plots. This reveals whether the model relies on protected attributes or treats similar cases differently, enabling bias detection and fairness auditing.

Yes. SHAP includes DeepExplainer for neural networks; it computes attributions using expected gradients. For tree-based models, use TreeExplainer. For any other black-box model or when you lack white-box access, use KernelExplainer, which works model-agnostically but requires more computation.

Prepare a trained model and background data—a representative sample of your training or validation set used to establish baseline predictions. Larger background datasets improve SHAP value stability but increase computation time. Ensure features are in the same format and scale as the model expects.

Generate a waterfall plot for the problematic prediction to see which features drove the decision and their magnitudes. Compare against waterfall plots for similar correct predictions using force or scatter plots. This reveals feature interactions, data issues, or model biases causing the error.