eos-model-bench-scenario-audit

Auditing think call-sites involves reading the canonical BUCKETS from apps/model-bench/app.py and grepping Python code under apps/ and plugins/ for self.think, self.think_stream, and self.think_compare to generate a per-file audit table and bucket roll-up.

The BUCKETS taxonomy is a classification structure defined in apps/model-bench/app.py that records id, domain, task_shape, and description to categorize think-related call-sites across EmptyOS.

Finding missing or orphan buckets requires mapping all self.think, self.think_stream, and self.think_compare call-sites against the canonical BUCKETS taxonomy, then flagging unmatched entries and proposing changes to refresh scenario coverage.

Yes, you can audit Python plugins and apps for think call-sites without external dependencies by reading the BUCKETS definition locally and grepping the codebase to capture domain, task_shape, and system prompts.

The Model Bench scenario audit report includes a per-file audit table of think call-sites, a bucket roll-up, and flags for missing or orphan buckets along with suggested changes to align with the taxonomy.

You should run a Model Bench scenario audit when you need to verify that all think-related call-sites across your apps and plugins correctly map to the BUCKETS taxonomy and ensure no orphan or missing buckets exist.