abd-chunk-markdown

You can chunk Markdown for RAG by applying a context_chunking_spec.yaml to guide structure-aware splitting. This produces memory chunks with YAML front matter containing chunk_id, section_path, and taxonomy defaults ready for downstream embedding and retrieval.

Structure-aware Markdown chunking splits documents using section_boundaries, headings, and chapter markers. Each resulting chunk retains its section_path and evidence metadata in YAML front matter, preserving document hierarchy for accurate downstream retrieval.

Adding evidence metadata to Markdown chunks requires injecting YAML front matter into each segment during the chunking process. The system automatically tags each chunk with a chunk_id, section_path, and taxonomy defaults specified by your context_chunking_spec.yaml file.

Yes, you can draft and apply a memory/context_chunking_spec.yaml to govern chunk sizes, taxonomy, and front matter. This specification file guides the structure-aware chunking process to ensure the output meets your specific retrieval and memory workflow requirements.

The best way to prepare converted Markdown for embedding is to chunk it into retrieval-sized segments with evidence labels. Applying a context_chunking_spec.yaml ensures each chunk includes YAML front matter with section_path metadata for efficient search.

Yes, python-dotenv is a required dependency for running the Markdown chunking scripts. You need this environment setup to execute the workflow that processes your topic folders and produces memory chunks with front matter.