skills.rest
microsoft
microsoftOFFICIAL
Explore all skills by this author →
Website𝕏@OpenAtMicrosoftGitHub

validate-ui-scenario

Drive VS Code through scripted UI scenarios and capture captioned video evidence.

190k|41.9k|Updated Sep 3, 2015
Search Tags:#playwright#ui testing#Automation#bug reproduction#VSCode#video evidence#scenario runner
One-click install
npx skills add https://github.com/microsoft/vscode --skill validate-ui-scenario
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: validate-ui-scenario
Source: https://github.com/microsoft/vscode/tree/main/.github/skills/validate-ui-scenario
Command: npx skills add https://github.com/microsoft/vscode --skill validate-ui-scenario

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires ffmpeg, ffprobe, playwright.

What problem does it solve?

Reproducing a UI bug or verifying a fix in VS Code usually means clicking through the editor by hand, hoping to hit the right state, and pasting screenshots into an issue. This Skill automates that loop end to end so the reproduction is deterministic, the evidence is captioned, and the report can be attached to a bug or pull request.

Core Features & Use Cases

  • Scenario runner: Executes a JavaScript scenario file against an installed Insiders build, a dev build from the current checkout, a pinned install, or the web build, with isolated profiles and extensions.
  • Evidence capture: Records a captioned MP4 video, per-step screenshots, a Playwright trace, window and server logs, and an HTML report with a step table and outcome.
  • Step semantics: Each step returns a validation string, throws to fail, or calls skip(reason, { needs }) to mark it as needing a human or a missing harness capability.
  • Use Case: A user reports that searching chat confirm in the Settings editor does not surface the Max Requests setting. Write a scenario that opens Settings, runs the search, asserts the setting is present, and attach the captioned recording to the issue.

Quick Start

Use the validate-ui-scenario skill to reproduce issue 250159 by writing a scenario that opens the Settings editor, searches for chat confirm, and asserts that Max Requests appears in the results.

Frequently Asked Questions about validate-ui-scenario

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I reproduce a VS Code UI bug and capture evidence automatically?▼

Write a CommonJS scenario file exporting id, title, source, workspacePath, and an array of steps, then run it with node test/scenario/out/runScenario.js. The runner launches VS Code, records video, captures per-step screenshots, and writes report.html and manifest.json under .build/vscode-playwright-mcp/evidence/<run-id>/.

What is the difference between validate-ui-scenario and smoke tests in VS Code?▼

validate-ui-scenario is for one-off, issue-derived validation that produces captioned video evidence for a bug report or pull request. Smoke tests are deterministic regression coverage that runs on every build and live in the smoke-tests skill.

Does the scenario runner work with VS Code Insiders, Stable, and dev builds?▼

Yes. With no flag it uses installed Insiders and falls back to Stable. Use --dev to validate an unmerged change from the current checkout, --build <app-root> to pin a specific install, and --web --headless for browser-only behavior.

Why is my scenario video missing step captions?▼

Caption rendering requires ffmpeg and ffprobe on PATH or pointed to by FFMPEG_PATH and FFPROBE_PATH. Install them with winget install Gyan.FFmpeg on Windows, brew install ffmpeg on macOS, or sudo apt install ffmpeg on Linux, then re-render with node test/scenario/out/renderEvidenceChapters.js <run-dir>.

How do I mark a step that cannot be automated in a VS Code scenario?▼

Call ctx.skip(reason, { needs }) inside the step. Use needs: 'human' when a person is required, such as physical hardware or a sign-in that cannot be scripted, and needs: 'infrastructure' when the harness lacks the capability. Skipped steps appear in a Needs attention section of report.html.

Related Skills

Explore More Software Engineering Skills→
pcasarettopcasaretto

bug-reproduction-validator

Reproduce reported software issues and classify them as bugs, environmental problems, or user errors.

Community
Advanced
AlejandroFiginiAlejandroFigini

qa-only

Generates structured bug reports from web application QA testing without code changes.

Community
Intermediate
AscendAscend
31

msverl-daily-regression-triage

Parse msverl logs to classify daily run status and rank suspect commits.

Official
Advanced
aist-ai-sastaist-ai-sast

context-extractor-mcp-audit

Extract contextual information from MCP findings to generate regression tests.

Official
Intermediate