book-sft-pipeline

Name: book-sft-pipeline
Availability: InStock
Author: shakudo

Convert ePub books into LoRA-ready SFT datasets with JSONL output.

Updated Feb 14, 2026

Search Tags:#lora #epub #style-transfer #instruction-generation #sft #dataset-generation

Ready to run

npx skills add https://github.com/Shakudo-io/opencode-skills --skill book-sft-pipeline-shakudo-io

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: book-sft-pipeline
Source: https://github.com/Shakudo-io/opencode-skills/tree/main/context-optimization/examples/book-sft-pipeline
Command: npx skills add https://github.com/Shakudo-io/opencode-skills --skill book-sft-pipeline-shakudo-io

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Book SFT Pipeline automates turning books into structured fine-tuning data and training workflows by extracting text from ePub, segmenting into coherent chunks, and generating diverse instruction prompts for LoRA-style models.

Core Features & Use Cases

Extracts text from ePub books and cleans front/back matter to form a clean corpus.
Performs intelligent segmentation into 150-400 word chunks with overlap to maximize style-learning signals.
Generates diverse instructions using multiple templates to teach a single author’s voice without memorizing exact passages.
Constructs JSONL-ready datasets for Tinker-style fine-tuning, including system/user/assistant prompt structures.
Facilitates end-to-end training with LoRA on lightweight bases and evaluates style transfer quality.
Provides standalone case studies (e.g., Gertrude Stein) as references for validation and experimentation.

Quick Start

Process a public-domain book to produce an SFT dataset and a LoRA-ready training plan.

Frequently Asked Questions about book-sft-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I create an SFT dataset from an ePub book for LoRA training?▼

To create an SFT dataset from an ePub, you extract the text, segment it into 150-400 word overlapping chunks, and generate diverse instruction prompts. This produces a JSONL-ready dataset structured with system, user, and assistant prompts for LoRA fine-tuning.

What is the best way to fine-tune a model to mimic a specific author's writing style?▼

Fine-tuning for author-style transfer involves generating diverse instruction prompts from segmented text chunks of the author's work. This teaches the model the author's voice using LoRA adaptation without memorizing exact passages from the book.

How does paragraph-aware segmentation improve dataset generation for instruction tuning?▼

Paragraph-aware segmentation splits text into coherent 150-400 word chunks with overlap to maximize style-learning signals. This ensures the instruction generation process captures contextual continuity, producing higher quality training data for SFT.

Can I use this SFT dataset generation pipeline with Tinker for end-to-end training?▼

Yes, the pipeline constructs JSONL-formatted datasets compatible with Tinker-style fine-tuning. It facilitates end-to-end training with LoRA on lightweight base models and provides reference case studies for evaluating style transfer quality.

Why generate multiple instruction templates for a single text chunk during SFT data preparation?▼

Generating multiple instruction templates for a single text chunk teaches the model the author's voice without memorizing exact passages. This diverse prompt generation ensures robust style transfer during LoRA fine-tuning.