tda-representation-diagnostics

Diagnose embedding stability, PCA loadings, UMAP/t-SNE projections, and representation drift in TDL datasets.

1|Updated Dec 13, 2025

Search Tags:#data analysis #diagnostics #representation #embedding stability #SHAP explanations

Ready to run

npx skills add https://github.com/stephendor/TDL --skill tda-representation-diagnostics

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: tda-representation-diagnostics
Source: https://github.com/stephendor/TDL/tree/main/.agents/skills/tda-representation-diagnostics
Command: npx skills add https://github.com/stephendor/TDL --skill tda-representation-diagnostics

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires scikit-learn, umap-learn, shap, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides diagnostic tools for checking embedding stability, scaler/PCA/loadings behavior, UMAP or t-SNE projections, observed/null coordinate-frame alignment, representation drift, and SHAP/feature explanations in TDL.

Core Features & Use Cases

Embedding Stability: Check for stability in embeddings across different runs.
Scaler/PCA/Loadings Behavior: Diagnose behavior of scalers, PCA, and loadings matrices.
UMAP/t-SNE Projections: Validate UMAP or t-SNE projections for accurate representation.
Coordinate Frame Alignment: Ensure observed and null data share a common fitted frame.
Representation Drift: Detect changes in representation over time.
SHAP/Feature Explanations: Provide associational explanations without causation claims.
Use Case: When analyzing a complex dataset, use this Skill to diagnose issues in embeddings and representations, ensuring data accuracy and reliability.

Quick Start

Run the tda-representation-diagnostics skill on the dataset 'dataset.json' to check for representation drift and SHAP explanations.

Frequently Asked Questions about tda-representation-diagnostics

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I check embedding stability and detect representation drift in my dataset?▼

You can check embedding stability and representation drift by running diagnostic scripts that compare embeddings across runs and detect temporal changes in data representations, ensuring long-term reliability.

How do I validate UMAP or t-SNE projections for accurate representation?▼

Validate UMAP or t-SNE projections by diagnosing scaler, PCA, and loadings behavior alongside observed and null coordinate-frame alignment to ensure your dimensionality reduction preserves meaningful variance.

Can I use SHAP explanations to diagnose feature behavior in TDL datasets?▼

Yes, you can generate SHAP explanations to provide associational feature importance diagnostics for TDL datasets, though these diagnostics do not imply causation between features and outcomes.

Do I need scikit-learn and umap-learn installed to diagnose PCA loadings and projections?▼

Yes, diagnosing PCA loadings behavior and UMAP projections requires specific Python libraries including scikit-learn, umap-learn, and shap to run the statistical diagnostics and visualizations.

What is coordinate frame alignment and why does it matter for representation accuracy?▼

Coordinate frame alignment ensures observed and null data share a common fitted frame, which is critical because misaligned frames can distort UMAP or t-SNE projections and compromise representation accuracy.

What are the limitations of using SHAP explanations for representation diagnostics?▼

SHAP explanations in representation diagnostics provide associational insights only and explicitly avoid causation claims, meaning they can identify feature correlations but cannot prove causal relationships in the data.