rubric-forecasting

Generate auditable probability forecasts by scoring evidence against topic-specific rubrics.

92|21|Updated Apr 17, 2026
One-click install
npx skills add https://github.com/NeoSoul-AI/rubric-prediction-skill --skill rubric-forecasting
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: rubric-forecasting
Source: https://github.com/NeoSoul-AI/rubric-prediction-skill/tree/main
Command: npx skills add https://github.com/NeoSoul-AI/rubric-prediction-skill --skill rubric-forecasting

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Rubric Forecasting prevents LLMs from “hand-calculating” probabilities by separating semantic judgment from deterministic scoring, ensuring auditable numeric results with explicit evidence tracking.

Core Features & Use Cases

  • Topic-specific rubric design: dynamically defines rubric dimensions and weights for each forecasting question.
  • Deterministic script-based scoring: computes evidence quality, dependency penalties, option aggregation, softmax normalization, and sensitivity analysis without model math in chat.
  • Audit-friendly structured outputs: returns numeric scores plus reasoning artifacts (evidence ledger, sensitivity deltas, and multidimensional narrative) suitable for review or downstream submission.
  • Use Case: forecast a binary event (e.g., “Will X happen by date Y?”) using structured evidence where each item maps to options, stance, strength, and rubric dimension scores.

Quick Start

Provide a JSON input matching the skill contract (question, options, resolution_rule, evidence, and optional rubric_dimensions) and ask the system to run the rubric forecasting engine on it to produce final_answer, normalized_scores, evidence_ledger, and sensitivity.

Frequently Asked Questions about rubric-forecasting

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate structured forecasts with auditable probability calibration?

To generate structured forecasts with auditable probability calibration, provide a JSON input with your question, options, resolution rule, and evidence. The engine designs a topic-specific rubric and scores evidence deterministically into normalized probability outputs.

What is rubric scoring for forecasting and how does it prevent LLM math errors?

Rubric scoring for forecasting separates semantic judgment from deterministic computation, preventing LLMs from hand-calculating probabilities. It maps evidence to rubric dimensions and computes scores via a Python engine, yielding auditable numeric results with explicit evidence tracking.

How do I perform sensitivity analysis on forecasted probabilities?

To perform sensitivity analysis on forecasted probabilities, the engine computes sensitivity deltas during the scoring process. It evaluates evidence quality, applies dependency penalties, and aggregates scores to check the robustness of your normalized probability outputs.

Can I use JSON input to track evidence for binary event forecasting?

Yes, you can use JSON input to track evidence for binary event forecasting. The input contract accepts structured evidence where each item maps to options, stance, strength, and rubric dimension scores, returning results with an evidence ledger.

What is the best way to normalize forecast scores across multiple options?

The best way to normalize forecast scores across multiple options is using softmax normalization. The deterministic engine aggregates weighted dimension scores for each option and applies softmax to produce normalized, auditable probability distributions.

Why does my multi-option forecast require a YAML-defined entrypoint?

Your multi-option forecast requires a YAML-defined entrypoint to establish the JSON input contract and invoke the Python engine. This ensures deterministic computation of evidence-to-option attribution, weighted scoring, and sensitivity analysis without relying on model math.