What problem does it solve?
Rubric Forecasting prevents LLMs from “hand-calculating” probabilities by separating semantic judgment from deterministic scoring, ensuring auditable numeric results with explicit evidence tracking.
Core Features & Use Cases
- Topic-specific rubric design: dynamically defines rubric dimensions and weights for each forecasting question.
- Deterministic script-based scoring: computes evidence quality, dependency penalties, option aggregation, softmax normalization, and sensitivity analysis without model math in chat.
- Audit-friendly structured outputs: returns numeric scores plus reasoning artifacts (evidence ledger, sensitivity deltas, and multidimensional narrative) suitable for review or downstream submission.
- Use Case: forecast a binary event (e.g., “Will X happen by date Y?”) using structured evidence where each item maps to options, stance, strength, and rubric dimension scores.
Quick Start
Provide a JSON input matching the skill contract (question, options, resolution_rule, evidence, and optional rubric_dimensions) and ask the system to run the rubric forecasting engine on it to produce final_answer, normalized_scores, evidence_ledger, and sensitivity.