skills.rest
nearai
nearaiOFFICIAL
Explore all skills by this author →
WebsiteGitHub

railway-test

Runs exact-head browser acceptance tests on Railway PR previews and publishes evidence comments.

12.6k|1.5k|Updated Feb 3, 2026
Search Tags:#Browser Testing#railway#acceptance testing#preview deployment#gitHub cli#QA evidence#Pull request
One-click install
npx skills add https://github.com/nearai/ironclaw --skill railway-test
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: railway-test
Source: https://github.com/nearai/ironclaw/tree/main/.claude/skills/railway-test
Command: npx skills add https://github.com/nearai/ironclaw --skill railway-test

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Verifying that a pull request actually delivers its promised behavior on a live Railway preview deployment is error-prone: stale builds, substitute providers, and superficial click-throughs produce misleading PASS results. This Skill enforces rigorous exact-head browser QA with a mandatory PASS/FAIL/BLOCKED status gate and publishes auditable evidence directly on the pull request.

Core Features & Use Cases

  • Exact-head build confirmation: Polls commit-scoped Railway status via scripts/preview_state.sh so tests never run against a stale or partial deployment, and re-verifies the head SHA before publishing.
  • PR-derived test matrix: Builds a Given/When/Then matrix from the PR diff with Required/Supplemental classifications, using recipes in references/test-recipes.md and streaming checks in references/streaming-cadence.md.
  • Evidence publication: Posts a deduplicated, head-marked QA comment on the PR for PASS, FAIL, and BLOCKED outcomes, with guarded cleanup of test data.
  • Use Case: Before merging a PR that changes chat streaming, invoke the skill to confirm the Railway preview matches the PR head, measure visible DOM streaming growth in a browser, and publish the resulting evidence comment for reviewers.

Quick Start

Use $railway-test to run exact-head browser acceptance tests on the Railway preview for PR 123 and publish the QA evidence as a comment on that pull request.

Frequently Asked Questions about railway-test

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test a pull request on a Railway preview deployment?▼

Invoke the skill with a PR number or URL; it reads the PR diff, builds a Given/When/Then test matrix, confirms the exact head build is live via commit-scoped Railway status, then runs browser tests and posts evidence on the PR.

How does the skill avoid testing a stale Railway build?▼

It queries commit-scoped status and check-run endpoints bound to the PR head SHA, polls until the Railway state reports success, and re-reads headRefOid before opening the preview and before publishing. Any head change restarts confirmation or yields BLOCKED.

What does a BLOCKED result mean in Railway preview QA?▼

BLOCKED means a required test case could not be executed against the intended contract, such as a missing provider, wrong role, or inconclusive result. Supplemental passes or healthy deployments can never upgrade BLOCKED to PASS.

Can I test a repository other than nearai/ironclaw?▼

Yes, but you must supply the repository explicitly because the preview URL fallback is repo-specific. For other repos, provide the preview URL directly or ensure the Railway status details contain the deployment hostname.

How is the preview bearer token handled during testing?▼

The token is typed only into the preview login UI and is never persisted, printed, or included in scripts, logs, screenshots, PR comments, or the final response. An authenticated browser session is reused when possible.

What happens if the PR head changes during testing?▼

The skill re-reads headRefOid before publishing; if it differs from the tested head, the browser observations are discarded and the evidence is rebuilt as BLOCKED with the old and new SHAs, never publishing a stale PASS or FAIL.

Related Skills

Explore More Software Engineering Skills→
paperclipaipaperclipai
79.8k

prcheckloop

Automate the resolution of failing GitHub pull request status checks and CI workflows.

Official
Intermediate
MoonLadderStudiosMoonLadderStudios
12

pr-resolver

Orchestrate GitHub pull request resolution with bounded retries and structured artifacts.

Official
Advanced
OM-bit-hubOM-bit-hub

ci-test-pipeline

Automate CI/CD pipeline setup with parallel test runs and coverage reporting.

Community
Advanced
cybersadercybersader
5

docs-testing

Automate end-to-end testing of the Crosswalker documentation site with Playwright.

Community
Intermediate