ce-reject-policy

Integrate configurable reject and defer policies into ML prediction workflows.

78|15|Updated May 1, 2023
One-click install
npx skills add https://github.com/Moffran/calibrated_explanations --skill ce-reject-policy
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ce-reject-policy
Source: https://github.com/Moffran/calibrated_explanations/tree/main/.claude/skills/ce-reject-policy
Command: npx skills add https://github.com/Moffran/calibrated_explanations --skill ce-reject-policy

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Configures and integrates reject and defer decision policies into production prediction and explanation flows, addressing uncertainty by optionally skipping, flagging, or deferring cases.

Core Features & Use Cases

  • Implements four policies: NONE, FLAG, ONLY_REJECTED, ONLY_ACCEPTED, plus deprecation-mapped legacy names.
  • Provides a RejectResult envelope exposing prediction, explanation, rejection mask, policy, and telemetry metadata.
  • Supports per-call policy overrides and explainer-level defaults, enabling flexible governance in live systems.
  • Demonstrates integration patterns with calibration outcomes and how to inspect init_error and rejection metrics.

Quick Start

Run an end-to-end example using the FLAG policy to flag uncertain predictions and inspect the RejectResult metadata.

Frequently Asked Questions about ce-reject-policy

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I configure reject policies for uncertain model predictions in production pipelines?

Reject policies for uncertain model predictions are configured using NONE, FLAG, ONLY_REJECTED, or ONLY_ACCEPTED modes. You can apply these as explainer-level defaults or per-call overrides within your prediction and explanation workflows to govern live ML systems.

What is a RejectResult envelope in machine learning explanation flows?

A RejectResult envelope is a data structure that exposes the prediction, explanation, rejection mask, applied policy, and telemetry metadata. It wraps the output of explanation flows to standardize how deferred or flagged uncertainty cases are handled in production pipelines.

How do I flag uncertain predictions instead of dropping them entirely?

To flag uncertain predictions instead of dropping them, apply the FLAG policy to your prediction workflow. This configuration marks uncertain cases within the RejectResult envelope while still returning the prediction and explanation data for downstream review.

Can I override the default rejection policy for individual prediction calls?

Yes, you can override the default rejection policy for individual prediction calls. The system supports per-call policy overrides, allowing you to switch between NONE, FLAG, ONLY_REJECTED, and ONLY_ACCEPTED modes on the fly without changing explainer-level defaults.

Do I need calibration outputs to apply defer and reject decision policies?

Yes, defined policy enums, per-call overrides, and calibration outputs are required to function correctly. The reject and defer decision policies integrate directly with calibration outcomes to assess uncertainty and determine whether to skip, flag, or defer cases.

How do I inspect rejection rates and init failure handling in ML pipelines?

You inspect rejection rates and init failure handling by examining the telemetry metadata within the RejectResult envelope. This metadata tracks rejection metrics and exposes init_error states, allowing you to monitor policy effectiveness and initialization issues.