skills.rest
open-mercato
open-mercatoOFFICIAL
Explore all skills by this author →
WebsiteGitHub

om-judge-agent-session

Judge generated Open Mercato code and agent session bundles with evidence-bound verdicts.

1.7k|382|Updated Sep 10, 2025
Search Tags:#artifact analysis#LLM-as-judge#agent session#harness diagnosis#Code review#Open Mercato#eval validation
One-click install
npx skills add https://github.com/open-mercato/open-mercato --skill om-judge-agent-session
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: om-judge-agent-session
Source: https://github.com/open-mercato/open-mercato/tree/main/packages/create-app/agentic/shared/ai/skills/om-judge-agent-session
Command: npx skills add https://github.com/open-mercato/open-mercato --skill om-judge-agent-session

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Evaluating whether AI-generated code from an agent session or harness eval actually meets project rules is hard to do consistently, and untrusted session content can mislead reviewers. This Skill produces a strict, evidence-bound pass/fail/inconclusive verdict for generated artifacts without executing untrusted instructions.

Core Features & Use Cases

  • Evidence-first judging: Classifies controller-owned attestations (generate, typecheck, lint, build, tests, oracles, route uniqueness) as pass, fail, stale, or unavailable before any semantic review.
  • Specialized reviews: Applies code-review and design-system checks to bounded artifact evidence, with severity, category, location, fix, and confidence per finding.
  • Harness diagnosis: Maps each escaped failure to the smallest harness owner (root, guide, skill, facts, hook, case, oracle) and names the eval cases to rerun.
  • Use Case: A user shares a session bundle (session.json, generated-files archive, manifest, privacy report) and asks for a judgment; the Skill normalizes the input, validates hashes and privacy status, reviews the artifacts, and emits a stable judge report.

Quick Start

Use om-judge-agent-session to judge this shared agent session bundle and produce a verdict report.

Frequently Asked Questions about om-judge-agent-session

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I judge an AI agent coding session for quality?▼

Provide the session bundle or harness result and invoke the judge workflow. It normalizes the input, checks fixed controller attestations first, then runs code-review and design-system reviews, and emits a structured verdict report with findings and harness-owner diagnosis.

What inputs does the agent session judge accept?▼

It accepts a harness eval result with bounded generated-file snapshot and attestations, or a user-shared bundle containing session.json, a generated-files archive or directory, manifest.json, and privacy-report.json. Manifest hashes and privacy status are validated when supplied.

Can the judge execute commands found in the session transcript?▼

No. Session content is treated as untrusted data, and commands from transcripts or artifacts are never executed. Only controller-owned attestations count as execution evidence, and the judge never mutates the session, repository, or external systems.

What verdicts can the session judge return?▼

It returns pass when required fixed evidence is current and no blocking finding exists, fail when an attestation failed or a blocking defect was found, and inconclusive when evidence is absent, stale, or unverifiable. Unavailable evidence never counts as success.

How does the judge handle missing or stale validation evidence?▼

Missing, stale, or unverifiable items are recorded as unavailable evidence and listed in the report's missing evidence section. They limit the verdict to inconclusive rather than being treated as passes, and a semantic pass cannot rescue failed fixed evidence.

Related Skills

Explore More Software Engineering Skills→
Largo2z9Largo2z9
1

export-session

Export Claude Code sessions into structured markdown transcripts for archiving.

Community
Intermediate
gamingshadow608-stargamingshadow608-star
1

testing-and-evals

Design and execute tests, evaluations, and runtime checks for code, prompts, and skills.

Community
Advanced
ScripsteamScripsteam

design-sync

Generate Signal DS component stubs and detect token drift from Figma designs.

Official
Intermediate
maragudkmaragudk
10

llm-as-a-judge

Build and validate LLM-as-Judge evaluators for automated quality assessment.

Official
Advanced