llm-extraction-pipeline

Build schema-driven LLM extraction pipelines converting unstructured documents into structured claims.

3|Updated Mar 17, 2026

Search Tags:#data extraction #llm #jsonl #json parsing #schema prompting #claim normalization #robust recovery

Ready to run

npx skills add https://github.com/transreal/claudecode --skill llm-extraction-pipeline

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: llm-extraction-pipeline
Source: https://github.com/transreal/claudecode/tree/main/Claude%20Directives/skills/llm-extraction-pipeline
Command: npx skills add https://github.com/transreal/claudecode --skill llm-extraction-pipeline

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

LLM-based information extraction often returns malformed or truncated JSON, causing fragile pipelines and wasted time when building structured datasets.

Core Features & Use Cases

Schema-driven prompt construction: Generates prompts from a structured schema to enforce consistent output shape and field semantics.
Robust 5-stage JSON parsing: Recovers data from markdown fences, prose-wrapped JSON, and truncated responses using fence stripping, raw JSON import, bracket-block extraction, and partial array recovery.
JSON-safe normalization for Mathematica types: Sanitizes unsupported values (e.g., Missing, Automatic, DateObject) into JSON-compatible representations before writing JSONL.
Verbose diagnostics for LLM calls: Provides timing and response-length visibility to quickly distinguish timeouts, early truncation, and parse failures.
Claim post-processing hooks: Supports normalization/validation steps such as required-field checks, ID generation, and content hashing before persisting records.

Quick Start

Use the llm-extraction-pipeline skill to extract a list of claims from your source text by supplying a schema and requesting JSON-only output from the LLM.

Frequently Asked Questions about llm-extraction-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I parse truncated JSON from LLM outputs into structured claims?▼

Parsing malformed JSON from LLM outputs requires a multi-stage recovery pipeline that strips markdown fences, extracts raw bracket blocks, and repairs partial arrays to produce valid structured claims.

What is the best way to extract structured data from unstructured text using schema prompting?▼

Schema-driven prompt construction generates prompts from a defined schema to enforce consistent output shape and field semantics, turning unstructured documents into reliable structured claims.

Why does my LLM return prose-wrapped JSON instead of valid JSONL?▼

LLMs often wrap JSON in markdown fences or prose. A bracket-aware parser with multi-stage recovery extracts valid JSON blocks and sanitizes non-serializable values to produce clean JSONL output.

Can I normalize non-serializable values like Missing or Automatic before writing JSONL?▼

JSON-safe normalization sanitizes unsupported values like Missing, Automatic, and DateObject into JSON-compatible representations before writing records to JSONL, ensuring valid persistent data.

How do I debug LLM timeouts and early JSON truncation during data extraction?▼

Verbose diagnostics provide call timing and response-length visibility to quickly distinguish network timeouts, early token truncation, and structural parse failures during LLM data extraction.

Does this JSON parsing approach support required-field checks and ID generation for extracted claims?▼

Claim post-processing hooks support normalization and validation steps including required-field checks, ID generation, and content hashing, executing before persisting records to ensure dataset completeness.