skills.rest
b08x
b08xCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

data-engineer

Design schema-first data infrastructure for Ruby AI systems with Sequel migrations.

Updated Mar 29, 2026
Search Tags:#schema#pipeline#ruby#data-engineering#rrf#sfl#vector-embedding
One-click install
npx skills add https://github.com/b08x/syncopated-context --skill data-engineer-b08x
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-engineer
Source: https://github.com/b08x/syncopated-context/tree/main/plugins/rubysmithing/skills/data-engineer
Command: npx skills add https://github.com/b08x/syncopated-context --skill data-engineer-b08x

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides a schema-first foundation and tooling necessary to design, migrate, and operate robust data infrastructure for Ruby AI systems, enabling reliable embedding, retrieval, and metadata management.

Core Features & Use Cases

  • Schema-driven migrations (DDL + JSONB metadata) for disciplined data governance and traceability.
  • Hybrid vector/SQL search using RRF and HNSW for performant retrieval across clauses, tokens, and documents.
  • SFL-based clause-level data modeling and NLP annotation to support semantic search and reasoning.
  • Document ingestion and multi-format extraction pipelines (PDF/DOCX) with robust metadata capture and idempotent persistence.

Quick Start

Activate rubysmithing-context and run the data-engineer workflow to scaffold the production-ready schema.

Frequently Asked Questions about data-engineer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up schema-first data infrastructure for Ruby AI pipelines?▼

Schema-first data infrastructure for Ruby AI pipelines is established using Sequel migrations, dry-schema validation, and JSONB metadata to enforce disciplined data governance and traceability across relational, graph, and in-memory stores.

What is the best way to implement hybrid vector search in Ruby using RRF and HNSW?▼

Hybrid vector search in Ruby is implemented using Reciprocal Rank Fusion (RRF) and HNSW indexes to perform performant retrieval across clauses, tokens, and documents combined with 384-dimensional vector embeddings.

How do I build a document ingestion pipeline for extracting and embedding PDFs in Ruby?▼

Document ingestion pipelines for PDF and DOCX extraction in Ruby use multi-format processing with robust metadata capture, clause-level embedding, and idempotent persistence to ensure reliable vector storage and retrieval.

Does this data engineering approach support semantic search and NLP annotation?▼

Semantic search and NLP annotation are supported through SFL-based clause-level data modeling, enabling neuro-symbolic retrieval and reasoning across ingested documents and their corresponding 384-dim vector embeddings.

Do I need dry-schema and Sequel to manage migrations and validation for AI data stores?▼

Yes, dry-schema and Sequel are required to manage strict validation and DDL migrations, providing the schema-first foundation necessary for operating robust data infrastructure across relational and graph stores.

When should I use 384-dimensional vector embeddings with HNSW indexes for document retrieval?▼

384-dimensional vector embeddings with HNSW indexes should be used when your document retrieval pipeline requires performant hybrid search combining semantic similarity with SQL filtering across large datasets.

Related Skills

Explore More Software Engineering Skills→
agnaldo4jagnaldo4j
1

db-migrations

Manage PostgreSQL schema lifecycles with versioned Flyway migrations and HikariCP pooling.

Community
Advanced
haxlyshaxlys

hybrid-search-implementation

Combine vector similarity with keyword matching using RRF or linear fusion.

Community
Advanced
wentoraiwentorai
850

iyeque-pdf-reader

Extract plain text and metadata from PDF files using PyMuPDF.

Community
Basic
cxnaivecxnaive

chroma

Store and retrieve vector embeddings with metadata filtering for RAG applications.

Community
Intermediate