skills.rest
BSteffaniak
BSteffaniakCOMMUNITY
Explore all skills by this author →
GitHub

ai-conversation-audit

Audit persisted SQLite AI conversations for tool usage and data gaps.

Updated Feb 9, 2026
Search Tags:#prompt engineering#data quality#conversation analysis#tool usage#ai audit#ai quality assurance
One-click install
npx skills add https://github.com/BSteffaniak/crime-map --skill ai-conversation-audit
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ai-conversation-audit
Source: https://github.com/BSteffaniak/crime-map/tree/main/.opencode/skills/ai-conversation-audit
Command: npx skills add https://github.com/BSteffaniak/crime-map --skill ai-conversation-audit

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This Skill provides a structured process to systematically audit AI conversations, identifying and rectifying issues in tool usage, prompt engineering, and data coverage to enhance AI assistant performance.

Core Features & Use Cases

  • Conversation Analysis: Review AI-tool interactions, parameter correctness, and result interpretation.
  • Issue Identification: Pinpoint problems in tool selection, parameter accuracy, result comprehension, answer quality, and data gaps.
  • Improvement Suggestions: Generate actionable recommendations for system prompt enhancements, tool definition updates, and tool implementation fixes.
  • Use Case: A product manager uses this skill to review a week's worth of user-AI interactions, identifies that the AI frequently misinterprets date ranges, and then suggests a specific update to the system prompt to improve accuracy.

Quick Start

Begin an audit session by listing recent conversations using cargo conversations list.

Frequently Asked Questions about ai-conversation-audit

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I audit AI conversations for tool usage and prompt engineering issues?▼

To audit AI conversations, you systematically review persisted SQLite interactions to evaluate tool selection, parameter accuracy, and result interpretation, generating actionable feedback for system prompt and tool implementation enhancements.

What is AI conversation analysis and how does it improve assistant performance?▼

AI conversation analysis reviews user-AI interactions to pinpoint problems in tool usage and answer quality. It improves assistant performance by providing actionable recommendations for system prompt updates and tool definition fixes.

How do I identify data coverage gaps in AI conversation logs?▼

You identify data coverage gaps in AI conversation logs by systematically analyzing persisted SQLite conversations to detect missing incident data regions, ensuring comprehensive context for tool selection and result interpretation.

Can I review a week's worth of user-AI interactions to find parameter accuracy issues?▼

Yes, you can review a week's worth of user-AI interactions by listing recent conversations and analyzing them to identify parameter accuracy issues, suggesting specific system prompt updates to improve tool usage.

What are the limitations of auditing AI conversations using persisted SQLite logs?▼

Auditing AI conversations using persisted SQLite logs is limited to analyzing historical text interactions, meaning it cannot evaluate real-time tool execution failures or fix data coverage gaps without separate system prompt updates.

Does ai-conversation-audit work with SQLite conversation databases?▼

Yes, ai-conversation-audit works with SQLite conversation databases by systematically analyzing persisted interactions to provide actionable feedback for system prompt, tool definition, and tool implementation enhancements.

Related Skills

Explore More Product & Management Skills→
letta-ailetta-ai
134

train-fasttext

Train FastText text classification models under size and accuracy constraints.

Official
Advanced
sergicalsergical
1

system-prompts

Generate agent definition templates with XML tag hierarchy and RFC 2119 norms.

Community
Advanced
pe-menezespe-menezes
2

audit

Audit code against Definition of Done and project patterns with test suite execution.

Community
Advanced
ramirlmramirlm

ai-usage-monitor

Monitor AI tool usage and estimate costs across billing cycles.

Community
Intermediate