librarian-index

To build a semantic vector index, the skill chunks library content by markdown headings or line windows, embeds chunks locally using FastEmbed, and upserts embeddings with metadata into a local Qdrant collection for deterministic offline semantic search.

Local semantic search handles re-indexing deterministically by generating content hashes that produce stable vector IDs. Re-indexing overwrites existing Qdrant entries instead of duplicating them, ensuring the local collection remains idempotent and accurate.

The best way to make git-backed libraries searchable offline is registering them in a libraries.yaml file and running an indexer. It embeds text and code locally with FastEmbed, storing vectors in a local Qdrant database without external API dependencies.

Yes, you can index specific file types by applying configurable file filters and extensions. The indexer processes registered libraries, skipping large files while chunking allowed markdown and source code before embedding and upserting into Qdrant.

Markdown chunking splits documents by headings, whereas code chunking uses configurable line windows. Both methods prepare content for local FastEmbed embedding before upserting the resulting vectors into a local Qdrant collection.

Limitations of local FastEmbed indexing include skipping large files to manage memory and relying on local compute resources. It is designed for registered libraries in libraries.yaml, so unregistered or oversized files outside configured filters will be ignored.