proactive-task-validator

Validate proactive assistant dataset tasks against hidden and dependency intents.

58|1|Updated May 13, 2026
One-click install
npx skills add https://github.com/Simplified-Reasoning/Pi-Bench --skill proactive-task-validator
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: proactive-task-validator
Source: https://github.com/Simplified-Reasoning/Pi-Bench/tree/main/data/pharmacist/skills/proactive-task-validator
Command: npx skills add https://github.com/Simplified-Reasoning/Pi-Bench --skill proactive-task-validator

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pyyaml, python3, json, pathlib, argparse, and includes scripts (resource) components.

What problem does it solve?

This Skill helps you validate proactive personal-assistant dataset tasks end-to-end, ensuring that checklists can be satisfied only with the intended hidden intent and dependency intent context—without leaking checklist information into candidate generation.

Core Features & Use Cases

  • Checklist-safe proactive validation: Verifies three-step pass/fail logic so Step 1 and Step 2 satisfy checklist items, while Step 3 (prompt-only) should fail.
  • Dependency-group isolation: Builds validation packets per dependency group to prevent checklist leakage across tasks.
  • Two-stage workflow orchestration: Generates isolated candidate outputs via subagents and performs centralized, itemized checklist judging.
  • Dataset audit reporting: Produces a reviewed Markdown report with structural warnings, per-task verdicts, and repair guidance for weak tasks.

Quick Start

Use the proactive-task-validator skill to validate a dataset rooted at a folder containing episode.yaml and tasks/<task_id>/task.yaml by asking it to run on that dataset root and produce a reviewed report with filled verdicts.

Frequently Asked Questions about proactive-task-validator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate proactive assistant tasks for checklist leakage in a dataset?

Validating proactive assistant tasks for checklist leakage requires checking whether Step 1 and Step 2 satisfy checklist items using hidden and dependency intents, while Step 3 prompt-only outputs fail. This Skill orchestrates isolated candidate generation and centralized judging to verify this three-step logic.

How does dependency-group isolation work when auditing dataset task.yaml files?

Dependency-group isolation works by building per-task validation packets that prevent checklist leakage across tasks. It isolates candidate outputs via subagents for each dependency group before centrally judging Pass/Fail/N/A verdicts with itemized checklist coverage.

What is the best way to generate an audit report for long-horizon proactive workflows?

Generating an audit report for long-horizon proactive workflows involves running a two-stage validation process on episode.yaml datasets. It produces a reviewed Markdown report containing structural warnings, per-task verdicts, and repair guidance for weak tasks.

Do I need Python and PyYAML to run dataset validation on episode.yaml files?

Yes, you need Python3 and PyYAML to run dataset validation on episode.yaml files. The Skill also depends on standard libraries including json, pathlib, and argparse to parse task structures and execute validation scripts.

Can I use this validation approach for datasets without dependency intents?

This validation approach is specifically designed for datasets structured as episode.yaml with proactive and dependency-heavy evaluations. Datasets without dependency intents or hidden context lack the required structure to generate per-task dependency-group validation packets.

Why does prompt-only candidate generation fail during checklist validation?

Prompt-only candidate generation fails during checklist validation because the logic intentionally verifies that checklist items cannot be satisfied without hidden intent and dependency intent context. This ensures prompt-only information is insufficient to pass the evaluation.