solr-pdf-indexing

You can index PDFs in Solr for hybrid search by using Tika for full-text extraction, parsing filesystem metadata, and generating vector embeddings to support keyword, semantic, and RRF-merged queries. This requires Solr 9.x and an external embeddings service.

Hybrid search combines traditional keyword matching with semantic vector similarity to locate relevant PDF documents. It merges full-text extraction results with vector embeddings using Reciprocal Rank Fusion (RRF), enabling more accurate queries over large libraries.

Yes, you need an external embeddings service to generate vector representations for semantic search. The Skill processes PDF text and metadata extracted by Tika, but relies on this external service to produce the embeddings required for similarity queries.

Yes, the indexing process parses metadata such as author, title, and year directly from the filesystem paths. This extracted metadata is combined with the full-text content obtained through Solr's Tika integration to enrich the searchable document index.

The best approach is to index the research papers using Tika for text extraction and an embeddings service for vector generation, allowing RRF-merged hybrid queries. This supports robust searching across keywords, authors, and semantic similarity simultaneously.

This approach requires Solr 9.x with Tika extraction enabled and a separate external embeddings service. It is designed for large PDF libraries where robust search is needed, but may not suit smaller collections due to the infrastructure overhead required.