analyze-latest-policy-sweep

Compare policy analysis runs from JSONL and JSON files into a markdown report.

7|4|Updated Feb 12, 2024

Search Tags:#report generation #diagnostics #benchmark #performance metrics #policy analysis #robot navigation

Ready to run

npx skills add https://github.com/ll7/robot_sf_ll7 --skill analyze-latest-policy-sweep

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: analyze-latest-policy-sweep
Source: https://github.com/ll7/robot_sf_ll7/tree/main/.codex/skills/analyze_latest_policy_sweep
Command: npx skills add https://github.com/ll7/robot_sf_ll7 --skill analyze-latest-policy-sweep

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires ffmpeg, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill automates the analysis of benchmark runs for robot navigation policies, providing a consolidated report of key metrics and diagnostics.

Core Features & Use Cases

Comparative Analysis: Compares multiple policy analysis runs (e.g., PPO, ORCA, planner) by examining episode metrics and summary data.
Report Generation: Creates a concise markdown report detailing performance, including aggregate metrics, collision scenarios, and problem episodes.
Diagnostic Insights: Provides specific diagnostic information like path-efficiency saturation and low-speed filter behavior.
Use Case: After running several different navigation algorithms, use this Skill to quickly understand which performed best by generating a comparative report with visualizations of critical scenarios.

Quick Start

Use the analyze-latest-policy-sweep skill to generate a markdown report comparing recent policy analysis runs.

Frequently Asked Questions about analyze-latest-policy-sweep

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I compare robot navigation policy performance across multiple benchmark runs?▼

Comparing navigation policy performance involves processing episode and summary metrics from JSONL and JSON files to generate a markdown report detailing aggregate metrics and worst-case scenarios for benchmarked policies.

What is the best way to generate diagnostic reports for PPO and ORCA navigation benchmarks?▼

The best way to generate diagnostic reports for PPO and ORCA benchmarks is to process episode metrics and produce markdown detailing path-efficiency saturation, low-speed filter behavior, and collision scenarios.

Do I need ffmpeg to analyze navigation policy sweep runs?▼

You need ffmpeg only for optional frame extraction from video artifacts during navigation policy sweep analysis; the core report generation and metrics comparison process does not require it.

Can I extract video frames from problem episodes during policy analysis?▼

Yes, you can optionally extract video frames from problem episodes during policy analysis if ffmpeg is installed, allowing you to visualize critical collision scenarios and worst-case navigation failures.

How do I identify worst-case scenarios and problem episodes from robot navigation metrics?▼

You identify worst-case scenarios from robot navigation metrics by parsing JSONL episode data to isolate collision instances, path-efficiency saturation, and low-speed filter behavior into a consolidated diagnostic report.

What file formats are required for analyzing robot navigation sweep data?▼

Analyzing robot navigation sweep data requires episode and summary metrics stored in JSONL and JSON files to generate comparative markdown reports detailing aggregate performance and diagnostic information.