skills.rest
jjackson
jjacksonCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

ocs-chatbot-qa

Capture structured QA transcripts from ACE OCS chatbot interactions.

1|2|Updated Apr 1, 2026
Search Tags:#automation#testing#qa#evaluation#chatbot#transcripts#ocs-chatbot
One-click install
npx skills add https://github.com/jjackson/ace --skill ocs-chatbot-qa
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ocs-chatbot-qa
Source: https://github.com/jjackson/ace/tree/main/skills/ocs-chatbot-qa
Command: npx skills add https://github.com/jjackson/ace --skill ocs-chatbot-qa

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automates capturing structured QA transcripts from ACE OCS chatbot interactions and records them for evaluation by ocs-chatbot-eval.

Core Features & Use Cases

  • Supports three modes (--quick, --deep, --monitor) to cover smoke tests, pre-launch checks, and recurring validation.
  • Writes transcripts to ACE/<opp-name>/qa-captures/YYYY-MM-DD-ocs-chat-<mode>.md, enabling both opp-specific testing and golden-template no-opp runs.
  • Produces per-entry data for the evaluator including response content, citations, tags, and structural pass/fail indicators.

Quick Start

Run the OCS Chatbot QA workflow to capture a structured transcript for a given opp and mode.

Frequently Asked Questions about ocs-chatbot-qa

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate capturing QA transcripts from chatbot interactions?▼

Automating chatbot QA transcripts involves resolving embed credentials, building mode-specific test prompts, and chatting via the anonymous widget API to write standardized transcripts for downstream evaluation.

What is the difference between quick, deep, and monitor modes for chatbot testing?▼

Quick, deep, and monitor modes cover different testing needs: quick mode runs smoke tests, deep mode performs pre-launch checks, and monitor mode handles recurring validation of chatbot interactions.

How are chatbot QA transcripts structured for evaluation?▼

Chatbot QA transcripts are written as standardized markdown files containing per-entry data such as response content, citations, tags, and deterministic structural pass/fail indicators for the evaluator.

Do I need specific credentials to test an OCS chatbot via the anonymous widget API?▼

Yes, testing an OCS chatbot requires resolving embed credentials before building test prompts and chatting via the anonymous widget API to generate structured QA transcripts.

Where are chatbot QA transcript files saved during automated testing?▼

Chatbot QA transcript files are saved to specific markdown paths under ACE/opp-name/qa-captures/YYYY-MM-DD-ocs-chat-mode.md, enabling both opportunity-specific testing and golden-template no-opp runs.

Related Skills

Explore More Software Engineering Skills→
jjacksonjjackson
1

ocs-agent-setup

Automate OCS chatbot provisioning by cloning templates, uploading documents, and publishing versions.

Community
Advanced
dimagi-internaldimagi-internal
1

ocs-chatbot-eval

Grade OCS chatbot transcripts against LLM-as-Judge rubrics.

Official
Advanced
jjacksonjjackson
1

ocs-chatbot-eval

Processes chestnut images and generates a PDF report of findings.

Community
Advanced
hwj123hwjhwj123hwj
4

wizard

Guide users through interactive bash procedures with configurable stages.

Community
Intermediate