ai-engineer

Combine dense vector similarity with BM25 keyword matching, then apply a cross-encoder reranker before passing results to the LLM. Use vector databases like Pinecone, Qdrant, Weaviate, or pgvector, and tune chunking strategy and retrieval limits per query.

LangGraph suits complex stateful workflows, CrewAI fits role-based multi-agent collaboration, and AutoGen handles conversational multi-agent systems. Choose based on state management needs, tool integration requirements, and team familiarity.

Pinecone offers managed scaling, Qdrant and Weaviate support hybrid search and self-hosting, and pgvector integrates with existing PostgreSQL. Selection depends on scale, indexing strategy like HNSW or IVF, and operational constraints.

Apply semantic caching and response memoization, route requests to smaller models when possible, and compress retrieved context to reduce token usage. Add rate limiting, quota management, and monitoring to track spend per feature.

Add input guardrails that detect injection patterns, filter PII, and enforce content policies before prompts reach the model. Combine this with output moderation, structured outputs, and adversarial testing during validation.

Avoid LLMs for pure data science or traditional ML tasks without generative requirements, and for trivial UI changes unrelated to AI features. Deterministic rules or classical models are often cheaper and more predictable in those cases.