skills.rest
Aradotso
AradotsoOFFICIAL
Explore all skills by this author →
Website𝕏@aradotsoGitHub

replay-railway-preview

Deploys nightly build to preview and executes timed test suite with health checks and automated cleanup, generating a final report and optionally triggering CI/CD pipeline.

Updated Apr 17, 2026
Search Tags:#testing#deployment#railway#braintrust#replay#website-agent#healthz
One-click install
npx skills add https://github.com/Aradotso/ara.engineer --skill replay-railway-preview
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: replay-railway-preview
Source: https://github.com/Aradotso/ara.engineer/tree/main/skills/replay-railway-preview
Command: npx skills add https://github.com/Aradotso/ara.engineer --skill replay-railway-preview

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Automates end-to-end validation of a website-agent fix by deploying the current branch to a persistent Railway preview environment and validating the deployment health.

Core Features & Use Cases

  • Deploys a branch to a stable Railway preview environment and verifies the deployment health via /healthz.
  • Executes a focused webhook replay against the preview to exercise the fix and collect metrics (outcome, rounds, tools, and reply).
  • Generates a Braintrust permalink and offers a guided cleanup workflow after tests.

Quick Start

Push your branch, deploy to the preview environment, and run the replay script with the preview URL.

Frequently Asked Questions about replay-railway-preview

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate a website-agent fix in a Railway preview environment?▼

Trigger a signed webhook replay against the Railway preview URL to exercise the website-agent fix, then poll the health status to collect metrics including outcome, rounds, tools, and reply data.

What is end-to-end replay for website-agent previews?▼

End-to-end replay for website-agent previews is an automated validation process that deploys a branch to Railway, triggers a focused webhook replay, and verifies results in Braintrust to ensure production-like functionality before merging.

How do I get a Braintrust permalink after testing a website-agent?▼

You get a Braintrust permalink after testing a website-agent by executing a focused webhook replay against the deployed preview environment, which automatically generates a structured result with a permalink to the test metrics.

Can I use Railway preview deployments for production-like verification?▼

Yes, you can use Railway preview deployments for production-like verification by deploying your current branch to a persistent preview environment and validating the deployment health prior to merging changes.

How do I clean up a Railway preview environment after webhook replay tests?▼

You clean up a Railway preview environment after webhook replay tests by following the guided cleanup workflow provided after the replay execution and health validation are completed.

Related Skills

Explore More Software Engineering Skills→
GaryOcean428GaryOcean428

deployment-readiness

Validate environment variables, migrations, and health endpoints for production deployment readiness.

Community
Advanced
tfunk1030tfunk1030
1

braintrust-analyze

Analyze Braintrust tracing data to surface patterns, issues, and insights from Claude Code sessions.

Community
Intermediate
yzavyasyzavyas
7

build-eval

Design and run evaluation harnesses for LLM agents and systems.

Community
Advanced
yolopunkyolopunk

land-and-deploy

Coordinates automated release workflow from PR merge to canary health validation.

Community
Advanced