representation-pipeline

What problem does it solve?

This Skill prevents misleading evaluation by ensuring that representation quality metrics are computed in the exact transformed feature space where the downstream task geometry is defined.

Core Features & Use Cases

• Progressive representation refinement protocol: Guides pipelines across normalization, relational/affinity transforms, metric learning embeddings, and retrieval/ranking/diversity layers.
• Metric alignment principle: Enforces “measure in-space” so BSS/TSS, silhouette, and retrieval metrics are computed against the representation each stage actually produced.
• Layerwise hyperparameter thinking: Helps define input/output contracts and tunables for each stage, including practical guidance for layered tuning (e.g., Optuna-style layerwise approaches).
• Bootstrap loop for mutual dependence: Provides a controlled weak-label → metric learning → re-cluster workflow with convergence stopping criteria.
• Layered architecture exemplar (GIST-style retrieval): Shows how multi-layer retrieval systems distribute budgets and reranking steps while keeping evaluation consistent.
• MemoRAG placement integration: Clarifies when and where to add a global memory recall layer upstream of retrieval to improve ranking metrics.

Quick Start

Use the representation-pipeline Skill to design or diagnose a multi-stage retrieval or clustering pipeline by transforming representations layer-by-layer and evaluating each layer using the metric computed in its own space.