book-sft-pipeline

To fine-tune an LLM for author style replication, you can automate the creation of Supervised Fine-Tuning datasets from books. This pipeline extracts ePub texts, segments them into coherent chunks, generates diverse instructions, and prepares JSONL data for LoRA training.

Creating an SFT dataset for style transfer involves extracting book text and splitting it into semantically coherent chunks of 150-400 words. The pipeline then uses LLM prompting with multiple templates to generate diverse instructions, preventing memorization and ensuring genuine style learning.

Yes, you can train LLMs on ePub files using LoRA. The pipeline handles text extraction from ePub formats, constructs a JSONL dataset, and runs LoRA training on platforms like Tinker, validating the results through modern scenario testing and originality checks.

To prevent memorization during LLM training, the pipeline employs diverse instruction generation using multiple prompt and system templates. This approach ensures the model learns the author's style transfer rather than simply replicating the original text chunks.

The pipeline explicitly supports LoRA training on platforms like Tinker. It outputs standard JSONL datasets, allowing integration with other LoRA training platforms that accept this format.