hunt-llm-ai

Validates data boundaries and tool authorization in LLM chat and agent applications.

10|3|Updated Aug 10, 2026
One-click install
npx skills add https://github.com/baiqigo/baiqi-redteam-lab --skill hunt-llm-ai-baiqigo
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: hunt-llm-ai
Source: https://github.com/baiqigo/baiqi-redteam-lab/tree/main/.agents/skills/hunt-llm-ai
Command: npx skills add https://github.com/baiqigo/baiqi-redteam-lab --skill hunt-llm-ai-baiqigo

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? LLM-powered chat and agent applications can leak data across tenants, invoke tools without proper authorization, or render model output unsafely, and these risks are hard to verify without a structured, evidence-based testing method. ## Core Features & Use Cases - Trust Boundary Modeling: Maps user input, system prompts, retrieval sources, tool declarations, authorization identity, approval points, and output rendering to separate what the model says from what the application actually executes. - Minimal Safe Validation: Uses self-owned test documents, test tenants, and harmless canaries to verify cross-tenant isolation, tool-call authorization and confirmation, source isolation, and safe output rendering. - Oracle and False-Positive Rules: Defines strict validity criteria so that model claims, prompt-injection text echoes, or simulated tool returns are not mistaken for real findings. - Use Case: A security researcher testing an AI assistant with RAG ingestion plants a canary in one tenant's documents and checks whether another tenant's session ever retrieves it, capturing conversation hashes and tool audit logs as evidence. ## Quick Start Ask the agent to run the hunt-llm-ai route against my authorized LLM chat application to verify tenant isolation and tool-call authorization using test canaries.

Frequently Asked Questions about hunt-llm-ai

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test an LLM application for cross-tenant data leakage?

Plant a harmless canary string in one test tenant's documents, then query from a separate clean tenant session and check whether the canary ever appears. Only an actual cross-tenant canary return counts as a valid finding, not model claims or prompt echoes.

How to verify agent tool calling requires user authorization?

Probe whether the application invokes sensitive tools without current-user authorization or explicit confirmation, and capture audit logs as evidence. Simulated tool returns or the model claiming access do not constitute a valid finding.

What counts as a false positive in LLM security testing?

Hallucinated claims of access, refusal-behavior changes, prompt-injection text appearing in output, and mocked tool responses are all false positives. Findings require observable application behavior such as actual unauthorized tool execution with audit evidence.

When should LLM security testing stop immediately?

Stop when testing touches real secrets, customer documents, external connectors, automated actions, or irreversible side effects. WAF blocks, captchas, timeouts, and one-off anomalies are recorded as inconclusive rather than exploited.

Does this testing approach work with RAG and vector search applications?

Yes, document ingestion and retrieval sources are part of the trust boundary model, and canary tests verify retrieval isolation. Findings specific to vector retrieval are routed to a dedicated RAG/vector testing route for deeper analysis.