skills.rest
qte77
qte77COMMUNITY
Explore all skills by this author →
WebsiteGitHub

implementing-document-indexing

Index documents into heading-aware chunks with sentence-transformers and FAISS.

2|1|Updated Feb 22, 2026
Search Tags:#rag#search#chunking#embedding#vector-store#faiss#document-indexing
One-click install
npx skills add https://github.com/qte77/claude-code-plugins --skill implementing-document-indexing
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: implementing-document-indexing
Source: https://github.com/qte77/claude-code-plugins/tree/main/plugins/rag-core/skills/implementing-document-indexing
Command: npx skills add https://github.com/qte77/claude-code-plugins --skill implementing-document-indexing

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill enables efficient, structured document search by converting long documents into a hierarchy-aware set of chunks, embedding them, storing them in a vector index, and enabling hybrid retrieval that combines semantic ranking with structural filtering.

Core Features & Use Cases

  • Heading-boundary chunking preserves document structure for coherent, context-rich retrieval
  • Embedding with sentence-transformers and FAISS vector store for fast similarity search
  • Hybrid retrieval that combines vector search with PageIndex-style tree filtering for precise results
  • Use cases include building RAG pipelines, document search, and memory layers in knowledge-heavy applications

Quick Start

Ingest a Markdown document and run the hybrid retrieval pipeline to fetch the most relevant sections.

Frequently Asked Questions about implementing-document-indexing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I chunk Markdown documents for RAG while preserving heading structure?▼

Heading-boundary chunking splits documents at heading markers to preserve hierarchical structure, producing coherent, context-rich chunks for embedding in a RAG pipeline. This maintains structural context during semantic search and retrieval.

How does FAISS IndexFlatIP work with sentence-transformers for document retrieval?▼

Sentence-transformers embed heading-aware document chunks into vectors, and FAISS IndexFlatIP stores them for fast inner-product similarity search. This combination enables rapid semantic ranking across large document repositories.

What's the best way to filter RAG search results by document hierarchy?▼

Hybrid retrieval combines FAISS vector search with PageIndex-style tree filtering to constrain results by document hierarchy. This structural filtering narrows semantic matches to specific sections, improving retrieval precision in knowledge-heavy applications.

Can I use this document indexing approach for building memory layers in applications?▼

Yes, this heading-aware embedding pipeline supports memory layers by indexing hierarchical content into a FAISS vector store. Applications can retrieve relevant context structurally, making it suitable for knowledge-heavy persistent memory systems.

Do I need a separate database to store embeddings when using FAISS for document search?▼

No, FAISS IndexFlatIP serves as the vector store directly, holding embedded chunks generated by sentence-transformers. You ingest Markdown documents, and the pipeline handles chunking, embedding, and indexing without requiring an external database.

When should I not use heading-based chunking for document indexing?▼

Heading-based chunking is less effective for documents lacking clear hierarchical structure or headings, such as flat plain-text files. Without heading boundaries, the structural filtering benefit is lost, making standard fixed-size chunking more appropriate.

Related Skills

Explore More Software Engineering Skills→
iterationlayeriterationlayer
2

convert-document-for-rag-ingestion

Convert PDFs and web pages into clean Markdown for RAG ingestion.

Official
Intermediate
floflo777floflo777
33

rag-eval

Evaluate RAG systems using retrieval, generation, and latency metrics.

Community
Intermediate
pepecczpepeccz

msia-rag

Automate RAG-based regulatory document QA with citations from uploaded texts.

Community
Advanced
rizaldiemrizaldiem

chunking-strategy

Break large documents into semantically meaningful segments for retrieval and embedding.

Community
Advanced