skills.rest
constructorfabric
constructorfabricOFFICIAL
Explore all skills by this author →
WebsiteGitHub

connector

Create, test, validate, and deploy Airbyte-based data connectors for the Insight ingestion platform.

10|9|Updated May 22, 2026
Search Tags:#kubernetes#dbt#data ingestion#etl#connector#clickhouse#airbyte
One-click install
npx skills add https://github.com/constructorfabric/insight --skill connector-constructorfabric
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: connector
Source: https://github.com/constructorfabric/insight/tree/main/.claude/skills/connector
Command: npx skills add https://github.com/constructorfabric/insight --skill connector-constructorfabric

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Building and operating data connectors for the Insight platform involves many failure-prone steps: authoring Airbyte declarative manifests, wiring descriptors, dbt models, K8s Secrets, and repo-wide registries. A single missed entry (wrong semver version, missing bootstrap config, mismatched silver column types) passes code review and only breaks after merge. This Skill encodes the full connector lifecycle with the wiring invariants and guard checks that prevent those silent post-merge failures. ## Core Features & Use Cases - Connector creation: Scaffold nocode (declarative YAML) or CDK (Python) connector packages with manifest, descriptor, K8s Secret example, README, dbt models, and identity-resolution chains. - Testing and validation: Run the L0 static / L1 mock / L2 live test ladder, strict Builder-UI manifest validation, per-stream read smoke tests, and the repo-wide wiring guard (connector_wiring.py). - Deployment and operations: Register connectors in Airbyte, create connections, reconcile Argo CronWorkflows, run end-to-end syncs, inspect logs, and reset connectors after breaking schema changes. - Use Case: Add a new HR data source by asking the Skill to create the connector package, test it against a real tenant, validate it against the spec, and deploy it to Airbyte and Argo with a verified first sync. ## Quick Start Ask the assistant to create a new connector named jira under the task-tracking category and then run its test workflow.

Frequently Asked Questions about connector

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new Airbyte declarative connector for the Insight platform?▼

Run the create workflow with a category, connector name, API base URL, and auth type. It scaffolds connector.yaml, descriptor.yaml, a K8s Secret example, README, and dbt models following Builder-UI-compatible patterns with granular $ref linking.

How do I test an Airbyte connector before deploying it?▼

Run the test workflow, which validates the manifest, executes the mock-server pytest suite, then runs check, discover, and read against a real tenant. It verifies every record carries tenant_id, source_id, and unique_key, and that all cursor fields exist in the schema.

Why does my connector pass review but break after merging to main?▼

Common causes are a non-semver descriptor version that aborts the CI bump-descriptors job, a missing entry in bootstrap-db connectors-config.yaml, or mismatched silver class column types. Run scripts/ci/connector_wiring.py before opening the PR to catch all three.

Why does my Airbyte sync show succeeded but commit zero records?▼

The Airbyte job status is unreliable in both directions. Check per-job aggregatedStats.recordsCommitted and bronze table freshness via max(_airbyte_extracted_at) instead; green-but-empty syncs often mean the replication pod failed or was OOM-killed.

Can I use whole-object $ref in an Airbyte declarative manifest?▼

No. The Airbyte Builder strict validator only accepts granular leaf-field $ref into definitions.linked components. Whole-object refs to definitions or streams load in the CDK runtime but cannot be opened in the Builder UI.

When should I use a CDK Python connector instead of a nocode YAML one?▼

Use CDK when the source API requires logic the declarative manifest cannot express, such as custom pagination or complex record transformation. CDK connectors require a Dockerfile, a descriptor images block, and a Docker build via build-connector.sh.

Related Skills

Explore More Software Engineering Skills→
databrickslabsdatabrickslabs
60

authenticate-source

Generate connector specifications, collect credentials, and validate data source authentication.

Official
Intermediate
clawdataclawdata
24

dbt-model-gen

Generate dbt models across bronze, silver, gold, and semantic layers from source definitions.

Official
Intermediate
tonone-aitonone-ai
69

flux-health

Detect data quality issues and pipeline health across databases and warehouses.

Official
Intermediate
nrakownrakow
1

ingestion-strategy

Select and configure data ingestion strategies for Fivetran, Airbyte, or custom pipelines.

Community
Intermediate