skills.rest
microsoft
microsoftOFFICIAL
Explore all skills by this author →
Website𝕏@OpenAtMicrosoftGitHub

agent-host-e2e-tests

Write, record, and troubleshoot agent host end-to-end tests using AHP protocol replay fixtures.

190k|41.9k|Updated Sep 3, 2015
Search Tags:#E2E Testing#VSCode#Integration Tests#agent host#replay proxy#ahp protocol#fixture recording
One-click install
npx skills add https://github.com/microsoft/vscode --skill agent-host-e2e-tests
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: agent-host-e2e-tests
Source: https://github.com/microsoft/vscode/tree/main/.github/skills/agent-host-e2e-tests
Command: npx skills add https://github.com/microsoft/vscode --skill agent-host-e2e-tests

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Writing and maintaining end-to-end tests for the VS Code agent host is hard because tests must drive real provider SDKs (Claude, Copilot, Codex) over the AHP protocol without burning tokens or hitting flaky network behavior. This Skill enforces a deterministic replay-based workflow so tests stay fast, offline, and reliable.

Core Features & Use Cases

  • Cross-provider test authoring: Add tests under src/vs/platform/agentHost/test/node/e2e/suites/ that run against Claude, Copilot, and Codex with shared fixtures.
  • Fixture recording and re-recording: Record YAML fixtures via AGENT_HOST_REPLAY_RECORD=1 after SDK bumps or test renames, with strict redaction of secrets and paths.
  • Non-deterministic test gating: Gate streaming, abort, or POSIX-only tests precisely instead of loosening timeouts, and track every disabled variant in KNOWN_ISSUES.md.
  • Use Case: After bumping the bundled Copilot CLI, re-record all Copilot fixtures, review the diff for leaked tokens or paths, and confirm replay-mode tests stay green before merging.

Quick Start

Use the agent-host-e2e-tests skill to add a new cross-provider test for tool-call streaming and record fixtures for Claude, Copilot, and Codex.

Frequently Asked Questions about agent-host-e2e-tests

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a new end-to-end test for the VS Code agent host?▼

Add a test under src/vs/platform/agentHost/test/node/e2e/suites using hostOnlyTest for host-only behavior or test for cross-provider coverage. Drive turns with dispatchTurn and waitForNotification, then record fixtures per provider with AGENT_HOST_REPLAY_RECORD=1.

How do I re-record agent host e2e fixtures after an SDK bump?▼

Export GITHUB_TOKEN or use gh auth token, then run AGENT_HOST_REPLAY_RECORD=1 ./scripts/test-integration.sh --run against each provider's integrationTest.ts. Review the git diff for leaked secrets or paths before committing.

Why does my agent host e2e test fail with a cache miss?▼

A cache miss means the test issued a request not present in the committed YAML fixture. Either the fixture is stale after an SDK bump, the test was renamed without re-recording, or the test is making a real network call that should be gated or stubbed in capiStubs.ts.

Can agent host e2e tests run on Windows?▼

Most tests replay deterministically across platforms, but POSIX-specific behaviors like shell tools, pwd, and git worktrees must be gated with !isWindows or per-provider flags. Each gated variant must be documented in e2e/KNOWN_ISSUES.md.

What is the difference between hostOnlyTest and test in agent host e2e suites?▼

hostOnlyTest registers a test that must never cross the model boundary and shares the strict captures/empty.yaml fixture. Regular test allows model-backed turns and requires a per-test fixture named ${provider}-${slug}.yaml derived from the test title.

Related Skills

Explore More Software Engineering Skills→
jaredcroxtonjaredcroxton

github-auth

Configure GitHub authentication via HTTPS tokens, SSH keys, or gh CLI.

Community
Intermediate
pyoclawpyoclaw

ci-cd-and-automation

Automate CI/CD quality gates for lint, type checks, tests, build, and security audits.

Community
Advanced
tankpkgtankpkg
1

@tank/token-redaction

Detect and redact API tokens, keys, bearer values, and JWTs from SQLite session stores.

Official
Advanced
lushly-devlushly-dev
4

write-tests

Guide unit, integration, and E2E test authoring with TDD workflows.

Official
Intermediate