skills.rest
matthiaskloft
matthiaskloftCOMMUNITY
Explore all skills by this author →
GitHub

bayesflow-validation

Validate BayesFlow models with Simulation-Based Calibration and coverage analysis.

Updated Mar 5, 2026
Search Tags:#validation#calibration#coverage#bayesflow#sbc#model assessment
One-click install
npx skills add https://github.com/matthiaskloft/claude-skills --skill bayesflow-validation
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: bayesflow-validation
Source: https://github.com/matthiaskloft/claude-skills/tree/main/bayesflow/skills/bayesflow-validation
Command: npx skills add https://github.com/matthiaskloft/claude-skills --skill bayesflow-validation

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill addresses the critical need to validate the performance and calibration of BayesFlow models, ensuring they accurately represent the underlying data-generating process.

Core Features & Use Cases

  • Simulation-Based Calibration (SBC): Verifies that posterior credible intervals achieve nominal coverage, a key indicator of model correctness.
  • Condition Grid Validation: Tests model performance across the entire design space, not just training conditions, to prevent overfitting.
  • Coverage Metrics & Quality Gates: Provides quantitative metrics like calibration error and C2ST deviation, enabling automated pass/fail criteria for training.
  • Use Case: After training a new posterior estimator, use this Skill to run an SBC analysis, check coverage metrics against predefined thresholds, and ensure the model is reliable before deployment.

Quick Start

Use the bayesflow-validation skill to run the validation pipeline on the current workflow with the pre-computed validation data.

Frequently Asked Questions about bayesflow-validation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate a trained BayesFlow posterior estimator?▼

Validate a trained BayesFlow posterior estimator by running Simulation-Based Calibration (SBC) to verify that posterior credible intervals achieve nominal coverage and accurately represent the underlying data-generating process.

What is simulation-based calibration for model assessment?▼

Simulation-based calibration is a model assessment technique that verifies posterior credible intervals achieve nominal coverage, serving as a key indicator of model correctness and calibration in scientific computing workflows.

How do I check if my BayesFlow model is overfitting to training conditions?▼

Check for overfitting to training conditions by performing condition grid validation to test model performance across the entire design space rather than just the training conditions, preventing overfitting.

Can I set automated pass/fail quality gates for posterior calibration?▼

Set automated pass/fail quality gates for posterior calibration by applying quantitative coverage metrics like calibration error and C2ST deviation against predefined threshold criteria.

Do I need pre-computed validation data to run SBC analysis?▼

Running the SBC analysis validation pipeline requires pre-computed validation data to evaluate coverage metrics, generate diagnostic plots, and estimate calibration floors.

Related Skills

Explore More Data & Analytics Skills→
Godimas101Godimas101

space-engineers

Guide Space Engineers modding across SBC/XML, C#, PB scripts, and asset pipelines.

Community
Advanced
jg-youjg-you
3

Visualization

Generate ArviZ diagnostic and substantive figures for Bayesian analyses.

Community
Intermediate
Learning-Bayesian-StatisticsLearning-Bayesian-Statistics
161

amortized-workflow

Implement a guarded amortized Bayesian workflow for simulation-based inference with BayesFlow.

Official
Advanced
fuzzy-dynamicsfuzzy-dynamics
1

pymc

Build and infer Bayesian models with MCMC sampling and variational inference.

Official
Advanced