rubric-forecasting

What problem does it solve?

Rubric Forecasting prevents LLMs from “hand-calculating” probabilities by separating semantic judgment from deterministic scoring, ensuring auditable numeric results with explicit evidence tracking.

Core Features & Use Cases

• Topic-specific rubric design: dynamically defines rubric dimensions and weights for each forecasting question.
• Deterministic script-based scoring: computes evidence quality, dependency penalties, option aggregation, softmax normalization, and sensitivity analysis without model math in chat.
• Audit-friendly structured outputs: returns numeric scores plus reasoning artifacts (evidence ledger, sensitivity deltas, and multidimensional narrative) suitable for review or downstream submission.
• Use Case: forecast a binary event (e.g., “Will X happen by date Y?”) using structured evidence where each item maps to options, stance, strength, and rubric dimension scores.

Quick Start

Provide a JSON input matching the skill contract (question, options, resolution_rule, evidence, and optional rubric_dimensions) and ask the system to run the rubric forecasting engine on it to produce final_answer, normalized_scores, evidence_ledger, and sensitivity.