spring-ai

To add LLM chat and RAG to a Spring Boot application, configure the Spring AI BOM, set up a ChatClient, and use a VectorStore like pgvector for similarity search over indexed documents to ground responses.

Spring AI handles structured output by parsing LLM responses directly into typed JSON records. This mechanism ensures reliable downstream logic execution without manual string parsing or fragile glue code between your chat client and business logic.

Yes, you can use pgvector for document similarity search with Spring Boot. The integration requires vector store configuration including schema and migrations to enable retrieval-augmented generation grounded in your indexed document data.

The best way to track token usage in Spring AI chat workflows is using the framework's built-in observability features. This provides cost visibility by monitoring token consumption during ChatClient calls without extra external tracking libraries.

You can test AI components in Spring Boot without calling real LLMs by applying testable patterns provided by the Spring AI framework. This approach allows you to validate chat workflows and structured output parsing safely without incurring API costs.