chat-quality

Converts live chat thumbs feedback into golden-set eval cases and fixes on a local branch.

1|Updated Feb 14, 2026
One-click install
npx skills add https://github.com/robritacca-dotcom/design-system --skill chat-quality-robritacca-dotcom
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: chat-quality
Source: https://github.com/robritacca-dotcom/design-system/tree/main/.claude/skills/chat-quality
Command: npx skills add https://github.com/robritacca-dotcom/design-system --skill chat-quality-robritacca-dotcom

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Visitor thumbs-down verdicts on a site's AI chat accumulate in Redis but are never read, so real answer failures never become regression tests or fixes. This Skill closes that feedback loop on a biweekly cadence. ## Core Features & Use Cases - Feedback triage: Reads thumbs verdicts from the live Upstash Redis (read-only), joins them to the logged exchanges, and identifies genuine answer failures without manufacturing cases from fine answers. - Golden-set growth: Encodes each real failure as a paraphrased golden-set case with required facts and assertions per the eval spec, protecting visitor privacy by keeping verbatim questions out of commits. - Eval-driven fixes: Runs the answer-quality eval once, fixes corpus sources, persona, or chat lookup tools, and lands everything on a local branch with a plain-English report for approval — never pushing or deploying. - Use Case: Run the loop every two weeks to turn the last window of disliked chat answers into new regression cases, verify they fail before and pass after the fix, and hand the user a branch to merge or reject. ## Quick Start Ask the AI to run the chat quality loop for the feedback received since the last report.

Frequently Asked Questions about chat-quality

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I turn user chat feedback into eval test cases?

Run the chat quality loop: it reads thumbs verdicts from the live Redis, joins them to logged exchanges, and encodes each genuine failure as a paraphrased golden-set case with required facts and the cheapest assertion that would have caught it.

How do I run the chat quality loop?

Invoke it by asking to run the chat quality loop or via /chat-quality. It pulls verdicts since the last report, triages failures, updates the golden set, runs npm run eval:chat once, and delivers fixes on a local branch.

Does the chat quality loop modify the live Redis data?

No. It is strictly read-only against the live Redis, using only SCAN, GET, LRANGE, and LLEN on chat:* keys. It never writes, deletes, or expires anything, and it never pushes, merges, or deploys changes.

How does the loop protect visitor privacy in chat logs?

Visitor questions may appear verbatim only in the local report, never in committed diffs. Golden-set cases use paraphrases that preserve the failure, and visitor hashes never leave the local report.

What happens when there are no negative chat verdicts?

An empty or all-up window is a valid outcome. The loop writes a short report noting the window and verdict counts, then stops without manufacturing cases from answers that were actually fine.