rag_architecture

Load documents with a DocumentLoader, split them with a TextSplitter, embed them into a vector store like Chroma, and connect a RetrievalQA chain to an LLM. Use semantic chunking and hybrid search combining BM25 with vector similarity for better retrieval quality.

Pass a memory object such as ConversationBufferMemory to initialize_agent with a memory_key. For long conversations use ConversationSummaryMemory, for sliding windows use ConversationBufferWindowMemory, and for semantic recall use VectorStoreRetrieverMemory.

Hybrid search combines keyword-based BM25 retrieval with vector cosine similarity, merging results via Reciprocal Rank Fusion. It outperforms vector-only search, especially for queries containing exact terms or technical vocabulary.

Use ConversationSummaryMemory to compress older messages into summaries, or ConversationBufferWindowMemory to keep only the last N messages. Plain ConversationBufferMemory suits short conversations under roughly ten messages.

Hallucination occurs when the system prompt lacks strict grounding instructions or retrieved context is irrelevant. Instruct the model to answer only from provided context, add citation requirements, and measure groundedness with the RAG Triad using Ragas or TruLens.

Measure the RAG Triad: context relevance, groundedness, and answer relevance, using frameworks like Ragas or TruLens. Also track retrieval latency under 200ms and log negative user feedback as evaluation examples.