analyzing-task-runs

Analyze PostHog task run logs for inefficiencies and report evidence-backed findings.

713|118|Updated Aug 11, 2020
One-click install
npx skills add https://github.com/PostHog/posthog-foss --skill analyzing-task-runs
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: analyzing-task-runs
Source: https://github.com/PostHog/posthog-foss/tree/main/products/tasks/skills/analyzing-task-runs
Command: npx skills add https://github.com/PostHog/posthog-foss --skill analyzing-task-runs

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Agent task runs often waste time and tokens on environment failures, missing CLI tools, verbose command output, and redundant retries, but these inefficiencies are buried inside tens of megabytes of raw JSONL logs that nobody reviews. This Skill turns a completed run log into a small set of measured, evidence-backed findings filed through the report_insight tool.

Core Features & Use Cases

  • Bounded log querying: Detects the log format (pi or ACP) and applies verified jq recipes that cap output with head and string slicing, so multi-megabyte logs never flood the context.
  • Finding taxonomy: Classifies inefficiencies into eight categories such as environment_failure, missing_tool, verbose_output, redundant_work, and wasted_retry, with measured wasted effort in tool calls, seconds, tokens, and output bytes.
  • Verified reporting: Submits each finding via report_insight with verbatim evidence quotes that the tool checks against the raw log, plus concrete suggested fixes with checkable done-when conditions.
  • Use Case: After an agent run felt slow, attach its run-log.jsonl and ask for an analysis; the Skill locates the log, finds that the test suite failed twice because Postgres was not running, measures 14 wasted tool calls and 210 seconds, and files a finding recommending Postgres be pre-started in the sandbox.

Quick Start

Analyze the attached run log for inefficiencies and report each finding with report_insight.

Frequently Asked Questions about analyzing-task-runs

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I analyze an agent task run log for inefficiencies?

Locate the attached run-log.jsonl under .posthog/attachments, detect whether it uses the pi or ACP format, then run the bounded jq recipes for overview, failed calls, and largest outputs. Report each confirmed finding through report_insight with verbatim evidence quotes.

What log formats does the run analysis support?

Two formats are supported: pi format, where events are wrapped as pi_event objects, and ACP format, where events are JSON-RPC session/update notifications. If the log matches neither schema, the failure protocol applies instead of reverse-engineering it.

Why should I never cat or fully read a run log?

Run logs can be tens of megabytes, so unfiltered reads flood the context window. All queries must cap row listings with head and slice large strings, while aggregate censuses are allowed because they emit only small fixed results.

How is wasted effort measured in a run analysis?

Wasted effort is measured, never estimated: bracket the wasted span with start and end line numbers, then count distinct tool calls, subtract timestamps for seconds, sum completed-turn tokens, and sum tool-output bytes. Omit any dimension the log cannot support.

What happens if the run log has no findings or an unknown format?

Make exactly one report_insight call with a no_findings_reason of run_was_efficient, too_short_to_judge, or insufficient_visibility, then call finish. Zero findings is a valid complete analysis and unknown formats must not be reverse-engineered.