tda-representation-diagnostics

Diagnose embedding stability, PCA loadings, UMAP/t-SNE projections, and representation drift in TDL datasets.

1|Updated Dec 13, 2025
One-click install
npx skills add https://github.com/stephendor/TDL --skill tda-representation-diagnostics
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: tda-representation-diagnostics
Source: https://github.com/stephendor/TDL/tree/main/.agents/skills/tda-representation-diagnostics
Command: npx skills add https://github.com/stephendor/TDL --skill tda-representation-diagnostics

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires scikit-learn, umap-learn, shap, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides diagnostic tools for checking embedding stability, scaler/PCA/loadings behavior, UMAP or t-SNE projections, observed/null coordinate-frame alignment, representation drift, and SHAP/feature explanations in TDL.

Core Features & Use Cases

  • Embedding Stability: Check for stability in embeddings across different runs.
  • Scaler/PCA/Loadings Behavior: Diagnose behavior of scalers, PCA, and loadings matrices.
  • UMAP/t-SNE Projections: Validate UMAP or t-SNE projections for accurate representation.
  • Coordinate Frame Alignment: Ensure observed and null data share a common fitted frame.
  • Representation Drift: Detect changes in representation over time.
  • SHAP/Feature Explanations: Provide associational explanations without causation claims.
  • Use Case: When analyzing a complex dataset, use this Skill to diagnose issues in embeddings and representations, ensuring data accuracy and reliability.

Quick Start

Run the tda-representation-diagnostics skill on the dataset 'dataset.json' to check for representation drift and SHAP explanations.

Frequently Asked Questions about tda-representation-diagnostics

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check embedding stability and detect representation drift in my dataset?

You can check embedding stability and representation drift by running diagnostic scripts that compare embeddings across runs and detect temporal changes in data representations, ensuring long-term reliability.

How do I validate UMAP or t-SNE projections for accurate representation?

Validate UMAP or t-SNE projections by diagnosing scaler, PCA, and loadings behavior alongside observed and null coordinate-frame alignment to ensure your dimensionality reduction preserves meaningful variance.

Can I use SHAP explanations to diagnose feature behavior in TDL datasets?

Yes, you can generate SHAP explanations to provide associational feature importance diagnostics for TDL datasets, though these diagnostics do not imply causation between features and outcomes.

Do I need scikit-learn and umap-learn installed to diagnose PCA loadings and projections?

Yes, diagnosing PCA loadings behavior and UMAP projections requires specific Python libraries including scikit-learn, umap-learn, and shap to run the statistical diagnostics and visualizations.

What is coordinate frame alignment and why does it matter for representation accuracy?

Coordinate frame alignment ensures observed and null data share a common fitted frame, which is critical because misaligned frames can distort UMAP or t-SNE projections and compromise representation accuracy.

What are the limitations of using SHAP explanations for representation diagnostics?

SHAP explanations in representation diagnostics provide associational insights only and explicitly avoid causation claims, meaning they can identify feature correlations but cannot prove causal relationships in the data.