paper-benchmark-execution

Orchestrate paper benchmark execution for RGB, multichannel, DataLoader, and video benchmarks.

95|3|Updated Oct 25, 2024

Search Tags:#planning #video #benchmark #dataloader #paper #rgb

Ready to run

npx skills add https://github.com/albumentations-team/benchmark --skill paper-benchmark-execution

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: paper-benchmark-execution
Source: https://github.com/albumentations-team/benchmark/tree/main/.cursor/skills/paper-benchmark-execution
Command: npx skills add https://github.com/albumentations-team/benchmark --skill paper-benchmark-execution

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This plan coordinates and executes the paper benchmark workflow for RGB, multichannel, DataLoader, and video benchmarks, enabling reproducible experiments and consistent result recording.

Core Features & Use Cases

Orchestrates the paper benchmark execution using the canonical sources at _internal/plans/paper_benchmark_execution_plan.md and the policy and matrix definitions in benchmark.
Validates and applies checked-in YAML configs under configs/paper to drive CPU and GPU runs across RGB, 9-channel, and video scenarios.
Supports re-running, result validation, and comparison across libraries (e.g., AlbumentationsX, torchvision, Kornia) within the prescribed plan.

Quick Start

Run the paper benchmark execution workflow by selecting the appropriate config from configs/paper and invoking the plan and run commands as documented.

Frequently Asked Questions about paper-benchmark-execution

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run reproducible image and video augmentation benchmarks across CPU and GPU configurations?▼

You can run reproducible image and video augmentation benchmarks by selecting a checked-in YAML config under configs/paper and invoking the plan and run commands to orchestrate CPU and GPU scenarios.

What is the canonical source for validating the paper benchmark execution plan?▼

The canonical source for validating the paper benchmark execution plan is the internal plan document, alongside benchmark/matrix.py for plan support and benchmark/policy.py for slow-transform thresholds and defaults.

Can I compare image and video augmentation libraries like AlbumentationsX, torchvision, and Kornia within the benchmark plan?▼

Yes, the benchmark plan supports re-running, result validation, and comparison across image and video augmentation libraries including AlbumentationsX, torchvision, and Kornia within the prescribed execution workflow.

Does this benchmark execution plan support multichannel image scenarios in addition to standard RGB?▼

Yes, the benchmark execution plan supports multichannel image scenarios, specifically validating and applying YAML configs to drive CPU and GPU runs across RGB, 9-channel, and video scenarios.

What limitations or constraints apply when using slow-transform thresholds in benchmark runs?▼

Slow-transform thresholds and defaults are governed by benchmark/policy.py, meaning your benchmark runs must conform to these prescribed policy definitions to maintain reproducible and consistent result recording.