databricks-unstructured-pdf-generation

Generate synthetic PDF documents with JSON metadata for RAG evaluation.

Updated Sep 9, 2017

Search Tags:#pdf #llm #rag #document-generation #data-ingestion #synthetic-data

Ready to run

npx skills add https://github.com/mirakui/dotfiles --skill databricks-unstructured-pdf-generation-mirakui

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: databricks-unstructured-pdf-generation
Source: https://github.com/mirakui/dotfiles/tree/main/claude/skills/databricks-unstructured-pdf-generation
Command: npx skills add https://github.com/mirakui/dotfiles --skill databricks-unstructured-pdf-generation-mirakui

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill enables generation of synthetic PDF documents to support RAG workflows and unstructured data testing, reducing manual document creation effort and enabling realistic evaluation datasets.

Core Features & Use Cases

Content generation: LLM-driven production of realistic PDF content tailored to a topic.
Structured artifacts: accompanying JSON files with questions and evaluation guidelines for RAG testing.
Automation & storage: automatic output to Unity Catalog volumes and organized folder structures for easy retrieval.

Quick Start

Generate 10 synthetic PDFs describing system architecture and API references using the MCP tool with catalog my_catalog, schema my_schema, count 10, and default volume/folder settings.

Frequently Asked Questions about databricks-unstructured-pdf-generation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I generate synthetic PDFs for RAG evaluation testing?▼

You can generate synthetic PDFs for RAG evaluation by using an LLM-driven process to create realistic document content tailored to specific topics. This produces reproducible PDF outputs alongside structured JSON metadata files containing questions and evaluation guidelines.

What is the best way to create test datasets of unstructured PDF documents?▼

Creating test datasets of unstructured PDF documents is best handled by deterministic generation pipelines that output organized folder structures. This approach ensures your synthetic data is reproducible and automatically accompanied by metadata for easy retrieval.

Can I automatically save generated PDF documents to Unity Catalog volumes?▼

Yes, you can automatically save generated PDF documents directly to Unity Catalog volumes. By configuring your MCP tool with specific catalog and schema settings, the synthetic outputs are stored automatically for immediate downstream access.

How do I get accompanying metadata files when generating synthetic documents for RAG?▼

To get accompanying metadata files when generating synthetic documents for RAG, the generation process outputs structured JSON artifacts alongside the PDFs. These JSON files contain relevant questions and evaluation guidelines designed specifically for testing retrieval workflows.

Do I need to manually organize synthetic PDF outputs for data ingestion pipelines?▼

No, you do not need to manually organize synthetic PDF outputs because the generation process automates storage into organized folder structures. This automated output ensures easy retrieval and immediate integration into your data ingestion pipelines.

Are deterministically generated PDFs suitable for unstructured data demos?▼

Deterministically generated PDFs are highly suitable for unstructured data demos because they provide reproducible outputs. This ensures consistent evaluation datasets across multiple test runs and reliable demonstrations of document processing workflows.