Evaluate generated text against predefined rubrics using the Anthropic Messages API.
npx skills add https://github.com/ramboz/servo --skill content-fidelity
Please help me install this Agent Skill. Skill: content-fidelity Source: https://github.com/ramboz/servo/tree/main/hosts/codex/plugins/servo/skills/content-fidelity Command: npx skills add https://github.com/ramboz/servo --skill content-fidelity
This Skill ensures that generated text aligns with specific intent or specifications, eliminating inconsistencies and ensuring quality in text outputs.
Author a content fidelity component for your project and run the init command.
High-intent search queries and answers about installing and using this skill.
To evaluate text fidelity, apply a rubric to the generated text and use model-based judgment to score its alignment with your predefined criteria for tone, content, and style.
Rubric-based assessment is a method to ensure text generation consistency by judging AI outputs against a defined spec, eliminating inconsistencies and verifying that content matches the desired intent.
Yes, you can use the Anthropic Messages API or Claude CLI to score generated text consistency, provided your environment has Python 3.9 or higher installed to run the evaluation scripts.
You check alignment by applying a project rubric that specifies tone and intent, then using model-based judgment to evaluate if the generated text meets those defined specifications.
Text fidelity evaluation requires setting up a Python 3.9+ environment to run the scripts, along with access to the Anthropic Messages API or Claude CLI to execute the model-based scoring.
Use rubric-based text evaluation when consistency with intent is critical, specifically to ensure AI-generated content matches your desired tone and style specifications before finalizing outputs.
rambozAutomates creation of rubrics, reference sets, and evaluation components for text-based acceptance criteria.
illogicalEvaluate LLM outputs with structured prompts and rubrics into JSON scores.
hoanghd218Analyzes video scripts using storytelling techniques and generates scored Markdown improvement reports.
apassuelloEvaluate LLM outputs with automated metrics, human judgments, and benchmarking.
