dataset-catalog-designer

Design identity-aware dataset catalogs with metamodels, glossaries, lineage, and agent-consumable search APIs.

Updated Jul 16, 2026
One-click install
npx skills add https://github.com/Cloud-Byte-Consulting/plugins --skill dataset-catalog-designer-cloud-byte-consulting
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: dataset-catalog-designer
Source: https://github.com/Cloud-Byte-Consulting/plugins/tree/main/research-data-platform/skills/dataset-catalog-designer
Command: npx skills add https://github.com/Cloud-Byte-Consulting/plugins --skill dataset-catalog-designer-cloud-byte-consulting

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Research teams cannot find the datasets they need because catalogs either duplicate stale metadata centrally or mirror pipelines and org charts, making search fail. This Skill guides the design of a dataset catalog or marketplace where researchers and agents go from a need statement to a pinned, accessible dataset version without leaking unauthorized metadata. ## Core Features & Use Cases - Metamodel and domain design: Define domains from knowledge areas rather than pipelines or org charts, with vertical, horizontal (lineage-to-checkpoint), and relational browse dimensions. - Catalog vs marketplace architecture decision: Choose push-based catalog, pull-based marketplace, or hybrid using a decision table covering self-registration, freshness, and incentives. - Governance and search specification: Produce role models, three-tier glossary governance (folksonomy, taxonomy, thesaurus), recall/precision benchmarks, and identity-aware search that filters before retrieval. - Use Case: A research division wants agents to discover training data via MCP-style APIs. Use this Skill to specify the agent-consumable surface: permanent URIs, structured metadata schemas, policy-partitioned vectorized summaries, and certification status inline in results. ## Quick Start Use the dataset-catalog-designer skill to design a dataset catalog for our research division, including the domain map, marketplace architecture decision, role assignments, and a benchmark query set for measuring search recall and precision.

Frequently Asked Questions about dataset-catalog-designer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design a data catalog that researchers can actually search?

Structure domains by knowledge areas rather than pipelines or org charts, and serve three browse dimensions: vertical domains, horizontal lineage, and relational graphs. Measure search quality with a benchmark set of about 20 real queries tracking recall and precision.

Data catalog vs data marketplace: which architecture should I choose?

Choose the marketplace pattern when datasets expose self-describing discovery APIs, since it pulls live metadata and avoids duplication. Start catalog-style for passive sources but adopt self-registration and summaries-plus-links immediately; hybrid is typical mid-transition.

How do I prevent a data catalog from leaking metadata to unauthorized users?

Authenticate before retrieval, filtering, ranking, autocomplete, and graph traversal, and partition indexes or embeddings by policy domain. Apply identity-aware filters before candidate generation so unauthorized users never learn asset names, summaries, lineage, or existence.

What glossary structure works best for dataset discovery?

Run three glossary types simultaneously: a free folksonomy for emergent tags, a domain taxonomy for formal hierarchies and facets, and a global thesaurus for cross-domain synonym control. Folksonomy tags feed taxonomy evolution over time.

How do I make a data catalog consumable by AI agents?

Expose everything the UI shows through machine-readable APIs with stable schemas, give every asset a permanent URI, and provide structured standards-based metadata. Bind vectorized summaries to enforceable access policies and include certification and SLO status inline in results.

Why does data catalog search return poor results even with good data?

Search works on metadata quality, not data quality, so pristine data with thin metadata finds nothing. Block index-everything-now plans, assign owners and stewards to every asset, and log zero-result queries as the backlog for glossary work.