skills.rest
youdotcom-oss
youdotcom-ossOFFICIAL
Explore all skills by this author →
Website𝕏@youdotcomGitHub

web-search-agent-evals

Run controlled web search agent evaluations across CLI agents in Docker containers.

5|Updated Jan 20, 2026
Search Tags:#automation#cli#docker#benchmark#web-search#agent-eval
One-click install
npx skills add https://github.com/youdotcom-oss/web-search-agent-evals --skill web-search-agent-evals
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-search-agent-evals
Source: https://github.com/youdotcom-oss/web-search-agent-evals/tree/main/.plaited/skills/web-search-agent-evals
Command: npx skills add https://github.com/youdotcom-oss/web-search-agent-evals --skill web-search-agent-evals

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

The Web Search Agent Evaluations skill coordinates automated testing of multiple CLI web search agents across isolated Docker containers to enable reproducible benchmarking and insights.

Core Features & Use Cases

  • Headless adapters and harness-driven evaluation pipelines for 4 agents (Claude Code, Gemini, Droid, Codex) with 2 search providers (builtin and MCP) to generate 8 experiment pairings.
  • Type-safe configuration constants, MCP server definitions, and a Bun-based entrypoint to orchestrate runs in containers.
  • Rich results, summaries, and prompts-management tooling to support iterative analysis, comparisons, and publication-quality reports.

Quick Start

Run bun run to start the evaluation workflow across all agent/tool combinations.

Frequently Asked Questions about web-search-agent-evals

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate web search agent benchmarks across multiple CLI tools?▼

Automate web search agent benchmarks by coordinating controlled evaluations across multiple CLI agents in isolated Docker containers to generate reproducible results and comparison summaries.

What is the best way to compare web search agents like Claude Code, Gemini, Droid, and Codex?▼

Compare web search agents by running headless adapters and harness-driven evaluation pipelines across four agents and two search providers to generate eight experiment pairings for analysis.

Can I run reproducible web search agent evaluations using Docker and Bun?▼

Run reproducible web search agent evaluations using a Bun-based CLI workflow and TypeScript components to orchestrate isolated Docker containers, ensuring controlled testing environments and consistent benchmark results.

How do I manage prompts for iterative agent testing and comparison?▼

Manage prompts for iterative agent testing using the included prompts-management tooling, which supports iterative analysis, comparisons, and publication-quality reports for web search agent experiments.

Does this evaluation harness support builtin and MCP search providers?▼

The evaluation harness supports both builtin and MCP search providers, allowing you to generate eight experiment pairings by combining these two tools with the four supported CLI web search agents.

What limitations exist when running headless web search agent evaluations in containers?▼

Limitations include relying strictly on Docker orchestration for container isolation and requiring a Bun-based environment, meaning evaluations are constrained to supported CLI agents and configured search providers.

Related Skills

Explore More Software Engineering Skills→
matteobortolazzomatteobortolazzo
1

setup

Verify cenci binary availability and build Docker or Podman container images.

Community
Intermediate
databricks-solutionsdatabricks-solutions
5

genie-benchmark-generator

Generate and validate benchmark questions with SQL execution and MLflow synchronization.

Official
Advanced
harnessharness
80

enforce-slsa

Add an SLSA Verification step to Harness pipelines for provenance attestations.

Official
Advanced
withkynamwithkynam
1.1k

vc-scout

Scout codebases to locate files and gather task context across directories.

Community
Advanced