What problem does it solve? Broken data at scale cannot be fixed row-by-row, and sending PII to cloud LLMs violates compliance. This Skill compresses thousands of anomalous rows into semantic clusters, generates deterministic fix logic with air-gapped local models via Ollama, and enforces a mathematical zero-data-loss guarantee on every batch. ## Core Features & Use Cases - Semantic Anomaly Compression: Embeds suspect rows with sentence-transformers and clusters them in ChromaDB or FAISS, turning 50,000 errors into a dozen pattern families. - Air-Gapped Fix Generation: Prompts local SLMs (Phi-3, Llama-3, Mistral) to output only validated Python lambdas, keeping PII entirely inside the network perimeter. - Zero-Data-Loss Reconciliation: Enforces Source_Rows == Success_Rows + Quarantine_Rows on every batch, with low-confidence fixes routed to human review and a full audit trail per row. - Use Case: A pipeline flags 2 million rows with inconsistent date formats. The Skill clusters them into 47 patterns, generates 47 validated lambdas locally, applies them vectorized, and reconciles every row before staging promotion. ## Quick Start Ask the agent to analyze the anomalous rows tagged NEEDS_AI from your pipeline, cluster them semantically, and generate safe fix lambdas using a local Ollama model.
