Audit AI product explainability against Doshi-Velez & Kim's tiered evaluation.
npx skills add https://github.com/haabe/mycelium --skill xai-check
Please help me install this Agent Skill. Skill: xai-check Source: https://github.com/haabe/mycelium/tree/main/plugins/mycelium/skills/xai-check Command: npx skills add https://github.com/haabe/mycelium --skill xai-check
This Skill addresses the need for an in-depth explainability audit for AI products, ensuring they meet defensible XAI standards.
Run /mycelium:xai-check to initiate the Explainability Audit for your AI product.
High-intent search queries and answers about installing and using this skill.
An AI explainability audit evaluates product components against tiered evaluation, stakeholder matrices, and fidelity checks to ensure XAI compliance. This process verifies system cards and recourse paths for AI Act adherence using the Mycelium framework.
Doshi-Velez & Kim tiered evaluation is a framework assessing AI system explainability across multiple levels of abstraction. The audit uses this methodology to validate that rationales generated by AI systems are faithful and answerable to stakeholders.
Yes, the Mycelium framework is required to run this AI Act compliance check. It provides the foundational environment needed to detect AI components and execute the explainability audit scripts.
To verify LLM fidelity and system cards, the audit reviews LLM-generated rationales for faithfulness to system outputs and checks the publication of system cards detailing AI features and limitations. This ensures full transparency and regulatory compliance.
Testing recourse paths involves ensuring a clear human review process exists for AI decisions. The audit performs a recourse path test to validate that users have actionable mechanisms to challenge and review automated AI outcomes.
A stakeholder matrix is needed during AI risk assessment to evaluate the answerability of systems for various stakeholders. It maps explainability requirements to specific user groups, ensuring AI components meet defensible XAI standards across different risk tiers.
haabeAudits canvas files for staleness, missing fields, inconsistent evidence types, and orphaned references.
DTMC-marketplaceGenerate structured model cards documenting AI model details and EU AI Act compliance.
PrepLabsAISimulate responsible AI interview scenarios with rubric-based scoring.
JeffallanDesign and optimize LLM prompts with structured outputs and evaluation frameworks.
