wiki-chunk

Chunk transcripts into semantic topic segments with Gemini and output JSON.

2|1|Updated Mar 18, 2026

Search Tags:#rag #search #nlp #gemini #transcripts #chunking #embedding

Ready to run

npx skills add https://github.com/cdeistopened/skill-stack-skills --skill wiki-chunk

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: wiki-chunk
Source: https://github.com/cdeistopened/skill-stack-skills/tree/main/wiki-pipeline/wiki-chunk
Command: npx skills add https://github.com/cdeistopened/skill-stack-skills --skill wiki-chunk

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Chunk transcripts into semantic topic segments to enable fast search, structured retrieval, and efficient downstream embedding for retrieval-augmented workflows.

Core Features & Use Cases

Semantic chunking: split transcripts into topic-based chunks with timestamps, topic types, and key entities.
Gemini-powered: leverages Gemini to generate coherent topic boundaries and context.
Output format: emits data/chunks/{episode_id}.json with a structured array of chunks for embedding and indexing.
Use case: empower RAG pipelines and search over large transcript corpora to surface relevant topic segments quickly.

Quick Start

Run the wiki-chunk pipeline to generate topic-based JSON chunks from transcripts.

Frequently Asked Questions about wiki-chunk

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I chunk transcripts into semantic topic segments for RAG pipelines?▼

To chunk transcripts for RAG pipelines, you segment pre-transcription text into topic-based chunks with timestamps and key entities. This process outputs structured JSON files suitable for embedding and indexing to enable fast search and relevance ranking.

Can I use Gemini for topic segmentation of podcast transcripts?▼

Yes, you can use Gemini for topic segmentation of transcripts. The pipeline leverages Gemini to generate coherent topic boundaries and context, splitting transcripts into semantic segments with timestamps and topic types for structured retrieval.

What is the best way to structure transcript chunks for embedding and indexing?▼

The best way to structure transcript chunks for embedding is semantic topic segmentation. This approach splits transcripts into topic-based chunks with timestamps, topic types, and key entities, emitting structured per-episode JSON arrays for efficient downstream embedding.

Do I need a GEMINI_API_KEY to generate topic-based JSON chunks from transcripts?▼

Yes, you need a GEMINI_API_KEY set in your environment to generate topic-based JSON chunks. The pipeline requires this API key, along with dependencies like google-genai and pyyaml, and a wiki.yaml configuration file to process transcripts.

How does semantic chunking improve search relevance over large transcript corpora?▼

Semantic chunking improves search relevance by splitting transcripts into coherent topic segments with context. This allows retrieval-augmented generation pipelines to surface relevant topic segments quickly, enabling fast search and structured retrieval over large corpora.

What format does the chunking pipeline output for downstream indexing?▼

The chunking pipeline outputs per-episode JSON files containing a structured array of chunks. These files are emitted into a chunks directory and are formatted specifically for downstream embedding and indexing in retrieval-augmented workflows.