rag-evaluation-review

What problem does it solve?

This Skill assesses the retrieval and grounding evaluation of RAG-based GenAI systems, ensuring they meet regulatory and operational standards in a financial services context.

Core Features & Use Cases

• RAG Evaluation: Conducts an in-depth review of a retrieval-augmented GenAI system, ensuring its corpus, retrieval quality, and grounding are appropriate for the intended purpose.
• Risk Mitigation: Identifies potential failure modes, assesses mitigation measures, and documents ongoing monitoring.
• Sector Overlay: Provides tailored analysis based on the sector of the institution using the system, such as banking, insurance, or capital markets.

Quick Start

To start a RAG evaluation, execute the 'evaluate-rag-system' script with the appropriate arguments: use-case ID, model-card record, validation-plan record, and other relevant documents.