data-ingestion-pipeline

To automate math content extraction, this pipeline reads markdown textbooks and uses Claude Code to write structured concepts, exercises, and Manim animations directly to PostgreSQL and Neo4j via TutorDataServiceWriter. It ensures deterministic IDs and self-correction throughout the data ingestion process.

Generating personalized math exercises is supported by rewriting extracted textbook concepts to target specific student interests like basketball or gaming. The pipeline leverages Claude Code as the LLM to tailor educational content and practice problems for various grade levels during ingestion.

Manim animation code generation works by having Claude Code process extracted math concepts from markdown and automatically write Python scripts for visual explanations. This happens within the ingestion pipeline, creating animations for each section of the ingested educational content.

No external API calls are needed because the pipeline leverages Claude Code as the primary LLM to perform all content creation tasks. This approach ensures consistency across generated concepts, personalized content, and exercises while reducing reliance on external services.

The best way to structure textbook markdown is through an end-to-end pipeline that extracts core concepts, generates grade-specific content, and writes directly to PostgreSQL and Neo4j. This creates a queryable data service layer enabling personalized learning experiences with deterministic IDs.

Ingesting an Algebra 1 chapter involves pointing the pipeline to the markdown file path, such as curriculum/algebra1/textbooks/chapter_07_factoring.md. Claude Code then extracts concepts, generates personalized content and Manim animations, and writes to PostgreSQL and Neo4j with deterministic IDs.