skills.rest
Kaelio
KaelioOFFICIAL
Explore all skills by this author →
GitHub

live_database_ingest

Converts live database schema snapshots into semantic-layer sources and wiki pages.

1.6k|101|Updated May 10, 2026
Search Tags:#data warehouse#sql validation#database ingestion#wiki documentation#Semantic Layer#identifier verification#schema snapshot
One-click install
npx skills add https://github.com/Kaelio/ktx --skill live-database-ingest
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: live_database_ingest
Source: https://github.com/Kaelio/ktx/tree/main/packages/cli/src/skills/live_database_ingest
Command: npx skills add https://github.com/Kaelio/ktx --skill live-database-ingest

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Turning raw database schema snapshots into accurate semantic-layer definitions and documentation is error-prone, especially when identifiers, joins, and business meanings must be verified against the actual warehouse rather than invented.

Core Features & Use Cases

  • Semantic-layer source generation: Reads raw table JSON, connection metadata, and foreign-key files from a live database sync, then creates or updates a semantic-layer source with correct table names, grains, and column types.
  • Identifier verification protocol: Confirms every schema.table and column reference via discovery, entity details, and SQL probes before writing it into wiki pages or semantic-layer definitions.
  • Grounded documentation: Preserves database comments as descriptions and writes wiki pages only for durable business meaning actually present in the snapshot.
  • Use Case: After a warehouse sync captures the public.orders table with foreign keys to customers, use this Skill to produce a validated semantic-layer source with verified joins and documented columns.

Quick Start

Ingest the raw files under raw-sources/<connectionId>/live-database/<syncId>/ and generate a validated semantic-layer source for each table.

Frequently Asked Questions about live_database_ingest

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate a semantic-layer source from a database schema snapshot?▼

Read the raw table JSON and connection.json from the sync directory, then create or update a semantic-layer source using the physical table name, primary key as grain, and mapped column types. Run sl_validate on the source before completing the work unit.

How do I verify table and column identifiers before documenting them?▼

Use discover_data to find existing pages, then entity_details to confirm each identifier resolves with correct types and keys. For literal values like status codes, check sampleValues or run a SELECT DISTINCT SQL probe against the warehouse connection.

When should joins be added to a semantic-layer source?▼

Add joins only when the foreign-keys.json file explicitly names both sides of the relationship. Joins that are not present in the snapshot files must not be invented.

What database types map to semantic-layer column types?▼

Map database types to string, number, time, or boolean in the semantic-layer source. When a database type is ambiguous, default to string.

What are the limitations of schema-snapshot-based ingestion?▼

The snapshot is structural evidence only, so measures, segments, and business definitions not present in table or column comments cannot be created. Sample values come solely from the scan record and must not be fabricated.

Related Skills

Explore More Data & Analytics Skills→
PostHogPostHog
58

setting-up-a-data-warehouse-source

Connect external data warehouse sources to PostHog with per-table sync methods.

Official
Advanced
AradotsoAradotso
7

ktx-ai-data-agents

Build an agent context layer from metrics and wiki definitions for data warehouse queries.

Official
Advanced
MaartenKestersMaartenKesters
3

powerbi-documentation

Generate Word documentation and Mermaid diagrams from Power BI semantic models.

Community
Advanced
AradotsoAradotso
7

ktx-data-agent-context-layer

Build a semantic context layer for AI agents querying data warehouses.

Official
Advanced