skills.rest
Kiln-AI
Kiln-AIOFFICIAL
Explore all skills by this author →
WebsiteGitHub

kiln-check-finetune-deprecation

Audit Kiln's fine-tunable model list for deprecated or unsupported base models across providers.

5.0k|376|Updated Jul 23, 2024
Search Tags:#audit#fine-tuning#vertex ai#fireworks#Model Registry#together ai#model deprecation
One-click install
npx skills add https://github.com/Kiln-AI/Kiln --skill kiln-check-finetune-deprecation
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: kiln-check-finetune-deprecation
Source: https://github.com/Kiln-AI/Kiln/tree/main/.agents/skills/kiln-check-finetune-deprecation
Command: npx skills add https://github.com/Kiln-AI/Kiln --skill kiln-check-finetune-deprecation

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Fine-tuning providers silently drop support for base models over time, leaving Kiln's fine-tune dropdown offering models that can no longer actually be fine-tuned. This Skill audits every fine-tunable model entry against live provider APIs and documentation to find stale or deprecated entries before users hit failures.

Core Features & Use Cases

  • Static Model Audit: Extracts every provider_finetune_id from ml_model_list.py and validates each against Together AI docs and the Vertex AI publisher models API.
  • Fireworks Dynamic Cross-Reference: Compares Fireworks' live supervisedLoraTunable/supervisedFullParameterTunable API fields against the canonical FIREWORKS_SUPPORTED_FINETUNE_MODELS allowlist in both directions, flagging stale entries and missing additions.
  • Guided Remediation: Reports findings in a structured table and recommends concrete fixes (update the fine-tune ID, remove it, or set deprecated=True), then verifies changes with the finetune API test suite.
  • Use Case: Before a Kiln release, run the audit to discover that two Together AI Llama reference models are no longer fine-tunable, then update the model list and confirm tests pass.

Quick Start

Ask the AI to check Kiln's fine-tunable models for deprecated or unsupported base models and report which entries need updating.

Frequently Asked Questions about kiln-check-finetune-deprecation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check if fine-tune base models are deprecated in Kiln?▼

Run the check_finetune.py script with the 'static' or 'all' mode from the repo root after sourcing your .env file. It extracts every provider_finetune_id from ml_model_list.py and validates each against the provider's current fine-tuning model list, outputting JSON results.

How to audit Fireworks fine-tunable models against an allowlist?▼

Run the script in 'fireworks' mode with FIREWORKS_API_KEY set. It fetches models marked supervisedLoraTunable or supervisedFullParameterTunable from the Fireworks API and cross-references them against FIREWORKS_SUPPORTED_FINETUNE_MODELS in fireworks_finetune.py, flagging stale and missing entries in both directions.

Which providers does the fine-tune deprecation check support?▼

The audit covers OpenAI, Together AI, Vertex AI, and Fireworks AI. Together AI is checked by scraping its docs page, Vertex AI via the publisher models API with gcloud authentication, and Fireworks via its live tunable-models API.

Why does the Vertex AI fine-tune check get skipped?▼

The Vertex check is skipped when VERTEX_PROJECT_ID is not set or gcloud authentication fails. Run gcloud auth login and confirm gcloud auth print-access-token works before rerunning the audit.

Does the skill automatically remove deprecated fine-tune models?▼

No, the skill is non-destructive and only reports findings. It recommends remediation such as updating the provider_finetune_id, removing it, or setting deprecated=True, but always asks for user confirmation before making code changes.

Related Skills

Explore More Software Engineering Skills→
rhpdsrhpds

01-vertex-setup

Configure Claude Code to use Google Cloud Vertex AI as the backend provider.

Official
Advanced
Kiln-AIKiln-AI
5.0k

kiln-add-model

Add LLM models to Kiln's registry and draft Discord release posts.

Official
Advanced
ml-exploreml-explore
771

swift-mlx-lm

Run local LLM and VLM inference and fine-tuning on Apple Silicon with MLX Swift.

Official
Advanced
519lab519lab
2

unsloth

Accelerate LoRA and QLoRA fine-tuning with reduced VRAM usage.

Official
Intermediate