model-evaluation

Compare Bayesian models with ArviZ 1.1 LOO, ELPD, and Bayes factors.

76|10|Updated Feb 5, 2026

Search Tags:#model comparison #model evaluation #arviz #bayesian modeling #loo-cv #model averaging

Ready to run

npx skills add https://github.com/pymc-labs/python-analytics-skills --skill model-evaluation-pymc-labs

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: model-evaluation
Source: https://github.com/pymc-labs/python-analytics-skills/tree/main/skills/model-evaluation
Command: npx skills add https://github.com/pymc-labs/python-analytics-skills --skill model-evaluation-pymc-labs

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires arviz, arviz.stats, arviz.plots, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps users compare and evaluate Bayesian models using ArviZ 1.1 metrics and diagnostics, providing insights into model performance and reliability.

Core Features & Use Cases

Model Comparison: Compare multiple models on predictive accuracy using LOO, ELPD, and Bayes factors.
LOO-CV: Perform leave-one-out cross-validation with PSIS for efficient computation.
Pareto k Diagnostics: Assess the reliability of PSIS approximation for each observation.
Moment Matching: Refit problematic observations using moment matching for improved PSIS estimates.
K-Fold CV: Use exact k-fold cross-validation when PSIS fails.
Model Averaging: Combine predictions from multiple models for better predictive performance.
Use Case: Suppose you have three different models for predicting house prices. Use this Skill to compare their predictive accuracy and select the best model.

Quick Start

Run the model-evaluation skill to compare the predictive performance of two models using LOO-CV and ELPD metrics.

Frequently Asked Questions about model-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I compare Bayesian models using leave-one-out cross-validation?▼

Compare Bayesian models by computing expected log predictive density (ELPD) and LOO metrics with Pareto smoothed importance sampling (PSIS) to evaluate and rank predictive performance accurately.

What does a high Pareto k diagnostic value mean in LOO-CV?▼

A high Pareto k diagnostic value indicates the PSIS approximation is unreliable for that observation. Use moment matching to refit problematic observations or switch to exact k-fold cross-validation for reliable Bayesian model evaluation.

Do I need ArviZ 1.1 to compute Bayes factors for model selection?▼

Yes, ArviZ 1.1 is required to compute Bayes factors and execute model diagnostics. The skill relies on ArviZ 1.1 stats to calculate ELPD, perform LOO-CV, and compare Bayesian predictive performance.

Can I combine predictions from multiple Bayesian models?▼

Combine predictions from multiple Bayesian models through model averaging. This merges predictive performance from different models to yield more robust and accurate overall predictions.

When should I use k-fold cross-validation instead of PSIS for model evaluation?▼

Use exact k-fold cross-validation instead of PSIS when the Pareto k diagnostics indicate that PSIS approximation fails. K-fold CV provides a robust alternative for computing model predictive performance in Bayesian statistics.