log-analyst

What problem does it solve?

It helps you interpret JSONL logs from an agentic system and quickly identify the high-impact mismatches and performance issues that explain why the agent seems wrong.

Core Features & Use Cases

• Retrieval mismatch detection: Verifies whether retrieved property IDs match the requested property IDs within each session.
• Confidence and scoring pattern analysis: Analyzes top_score distributions to flag low-confidence retrievals or suspiciously high scores tied to wrong outputs.
• Latency and completeness auditing: Flags slow LLM calls (p95 thresholds) and missing response events that indicate broken execution traces.
• Cross-session anomaly spotting: Detects repeated retrieved IDs across multiple borrowers that may indicate a filter or isolation bug.

Quick Start

Paste the JSONL logs from a borrower-agent run into the skill input and ask for the sessions, events, and fields that indicate retrieval mismatches, latency spikes, or hallucination-related patterns.