skills.rest
tnunamak
tnunamakCOMMUNITY
Explore all skills by this author →
GitHub

autoresearch-tasks

Generates autonomous-agent task packages with anti-cheating safeguards for software repositories.

Updated Mar 22, 2018
Search Tags:#autonomous agents#task generation#behavior preservation#repo analysis#autoresearch#test oracle#goodhart prevention
One-click install
npx skills add https://github.com/tnunamak/dotfiles --skill autoresearch-tasks
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: autoresearch-tasks
Source: https://github.com/tnunamak/dotfiles/tree/main/ai/skills/local/autoresearch-tasks
Command: npx skills add https://github.com/tnunamak/dotfiles --skill autoresearch-tasks

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

It helps you discover autonomous-agent-friendly work for a repository while minimizing the risk of the agent “cheating” by deleting or shrinking the system to game the evaluation.

Core Features & Use Cases

  • Oracle-first task design: Produces tasks around building or leveraging robust, behavior-preserving test oracles instead of vague metrics that are easy to Goodhart.
  • Repo-aware workflow: Instructs the agent to inspect common repo signals (README/docs, build manifests, CI, devcontainers, routes/CLIs, schema/migrations, and existing tests).
  • Tiered oracle strategy: Classifies repositories into Tier 1/2/3 based on oracle strength and then generates task packages appropriate to each tier (including oracle bootstrapping for Tier 2 and safe inventory for Tier 3).
  • Anti-Goodhart constraints: Enforces guardrails like locking observable surface manifests, preventing deletion-based shortcuts, flagging large negative diffs, and requiring objective evaluator descriptions.

Quick Start

Ask an agent: “Use autoresearch-tasks to analyze this repository and produce ranked task packages with robust, behavior-preserving oracles and explicit anti-Goodhart safeguards.”

Frequently Asked Questions about autoresearch-tasks

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate autonomous agent tasks for a software repository without the agent cheating?▼

To generate safe autonomous agent tasks, you need oracle-backed task packages that enforce anti-Goodhart constraints, preventing the agent from succeeding by deleting, narrowing, or disabling the system to game the evaluation.

What is a behavior-preserving test oracle for autonomous coding?▼

A behavior-preserving test oracle is a robust verification mechanism used to generate autonomous-agent-friendly tasks, ensuring agents cannot game vague metrics by shrinking the system or taking deletion-based shortcuts.

How do I assess if my repository is ready for autonomous coding agents?▼

You can assess auto-researchability by inspecting repository signals like READMEs, build manifests, CI, and existing tests to classify the repo into Tier 1, 2, or 3 based on oracle strength.

How do I bootstrap test oracles for repositories with weak verification signals?▼

For repositories classified as Tier 2 or Tier 3 based on oracle strength, generate task packages that include oracle bootstrapping and safe inventory creation to establish robust, behavior-preserving verification.

What anti-Goodhart constraints should I apply to autonomous agent task generation?▼

Effective anti-Goodhart constraints for task generation include locking observable surface manifests, flagging large negative diffs, preventing deletion-based shortcuts, and requiring objective evaluator descriptions.

Can I use repo analysis to plan long-horizon autonomous coding work?▼

Yes, repo analysis inspects common signals like routes, CLIs, schema, and migrations to output complete task packages with measurable success criteria tailored for planning long-horizon autonomous coding work.

Related Skills

Explore More Software Engineering Skills→
mashharukimashharuki

repo-documentation

Generate README-style documentation from repository files and directory structure.

Community
Intermediate
openaiopenai
24.7k

define-goal

Define measurable goals with success criteria and quantitative validation.

Official
Intermediate
misty-stepmisty-step
13

oracle

Bundle prompts and repository files for browser-based ChatGPT consultation.

Official
Intermediate
markusstrassermarkusstrasser

researcher

Orchestrate MCP tools to gather, verify, and synthesize research evidence.

Community
Advanced