distill-classifier

Distill frontier classifiers into lightweight open-weight students using teacher-labeler hard-label distillation.

10|5|Updated Jun 3, 2026

Search Tags:#classification #machine-learning #distillation #SFT #teacher-student #consensus-labeling

Ready to run

npx skills add https://github.com/understudylabs/understudy-agent-tools --skill distill-classifier

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: distill-classifier
Source: https://github.com/understudylabs/understudy-agent-tools/tree/main/skills/distill-classifier
Command: npx skills add https://github.com/understudylabs/understudy-agent-tools --skill distill-classifier

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Distill Classifier replaces expensive frontier classifiers with a fine-tuned, open-weight student using teacher-as-labeler hard-label distillation. It enables multi-teacher consensus labeling, data-driven SFT design, and a gate-based verdict system to control deployment cost.

Core Features & Use Cases

Replaces frontier classifiers with a lightweight student trained on consensus labels.
Supports multi-teacher majority voting, data curation for SFT, and deployment gating.
Use Case: when high accuracy is needed but latency and cost must be reduced for classification tasks (binary, multi-class, multi-label, and structured extraction).

Quick Start

Train a lightweight student on consensus-labeled data and evaluate its performance against the best teacher.

Frequently Asked Questions about distill-classifier

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How does teacher-labeler hard-label distillation reduce classifier cost?▼

Teacher-labeler hard-label distillation reduces classifier cost by using multiple frontier models to generate consensus labels, which then train a lightweight open-weight student model. This approach maintains high accuracy for knowledge-bound classification tasks while significantly lowering inference latency and operational deployment costs.

What classification task formats are supported by multi-teacher consensus labeling?▼

Multi-teacher consensus labeling supports binary, multi-class, and multi-label classification tasks, alongside structured extraction. This workflow leverages multiple teacher models to curate high-quality datasets for supervised fine-tuning of a lightweight student model.

Can I evaluate a distilled student classifier against its baseline teacher model?▼

Yes, you can evaluate a distilled student classifier against its baseline teacher model using the built-in AB test framework. This comparison measures the student's performance and informs the deployment gating system to ensure accuracy is maintained before replacing the frontier classifier.

When should I use a four-way verdict system for classifier deployment?▼

You should use the four-way verdict system—PROMOTE, SHADOW, COLLECT, or STOP—when deciding whether to deploy a distilled student classifier. It provides a data-driven gate based on AB test results to control deployment costs and ensure the lightweight model meets accuracy requirements.

Do I need multiple frontier models to build a teacher-student distillation workflow?▼

Yes, building a teacher-student distillation workflow requires multiple frontier models to perform consensus labeling. These teacher models generate the majority voting labels needed to curate the supervised fine-tuning data for training the lightweight open-weight student classifier.

What is the best way to replace expensive frontier classifiers for structured extraction?▼

The best way to replace expensive frontier classifiers for structured extraction is through hard-label distillation. By training a lightweight open-weight student on consensus-labeled data from multiple teachers, you achieve comparable accuracy with reduced latency and lower inference cost.