ai-answer-trace

Trace search queries, retrieved pages, and cited sources behind Claude, ChatGPT, and Gemini answers.

15|3|Updated Feb 23, 2026
One-click install
npx skills add https://github.com/XPOZpublic/xpoz-agent-skills --skill ai-answer-trace-xpozpublic
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ai-answer-trace
Source: https://github.com/XPOZpublic/xpoz-agent-skills/tree/main/skills/ai-answer-trace
Command: npx skills add https://github.com/XPOZpublic/xpoz-agent-skills --skill ai-answer-trace-xpozpublic

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires anthropic, openai, google-genai, google-auth.

What problem does it solve? AI assistants answer questions with hidden web searches, making it impossible to know which sources shaped their recommendations. This Skill captures the full evidence trail behind each answer so you can measure and improve your brand's visibility in AI-generated responses (GEO). ## Core Features & Use Cases - Full Answer Tracing: Runs a question through Claude, ChatGPT, and Gemini with live web search enabled, recording the answer text, the search queries each engine ran, the pages retrieved, and the URLs actually cited with character-level spans. - Cited Domain Aggregation: Counts which domains each engine cites across multiple samples, separating stable citation signals from one-off noise. - Near-Miss Analysis: Identifies pages that engines retrieved but never cited (Claude and ChatGPT), revealing content the engines considered and rejected. - Use Case: Ask "Why do AI assistants recommend my competitor?" and get a report showing which domains win citations for your category, which surfaces carry your brand, and where you are absent entirely. ## Quick Start Ask the agent to trace what ChatGPT, Claude, and Gemini answer for "best social listening tool for startups" and show which sources each engine cites.

Frequently Asked Questions about ai-answer-trace

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I see which sources ChatGPT cites in its answers?

Run the trace-openai.py script with your question and an OpenAI API key. It calls the Responses API with web search enabled and include=["web_search_call.action.sources"], returning the answer text, search queries, retrieved pages, and cited URLs with character spans.

How to trace AI answer citations across Claude, ChatGPT, and Gemini?

Run each engine's trace script (trace-claude.py, trace-openai.py, trace-gemini.py) 2-3 times per question, saving JSON output per sample. Then run cited-domains.py on all trace files to count cited domains per engine and compare citation patterns.

What API keys do I need to trace AI engine answers?

Each engine runs independently, so one key is enough to start: ANTHROPIC_API_KEY for Claude, OPENAI_API_KEY for ChatGPT, and GEMINI_API_KEY for Gemini. Gemini also supports Vertex AI credentials via GOOGLE_APPLICATION_CREDENTIALS plus VERTEX_PROJECT_ID.

Why do AI answers give different citations each time?

AI engines are nondeterministic, so a single run is not a reliable signal. Run each question 2-3 times per engine and treat sources that recur across samples as the stable citation signal; one-sample citations are noise until they repeat.

Can I see pages an AI engine retrieved but did not cite?

Yes, for Claude and ChatGPT: compare searches[].results against cited_urls to find retrieved-but-not-cited near-misses. This does not work for Gemini, whose retrieved set is essentially its cited set, and you must strip the ?utm_source=openai suffix from ChatGPT URLs before matching.

What are the limitations of tracing Gemini grounding citations?

Gemini pools all fan-out queries into a single search entry without per-query attribution, and its grounding URLs are Google redirect links that expire within days. The script resolves redirects to real destinations at capture time, falling back to the bare domain when resolution fails.