skills.rest
terrene-foundation
terrene-foundationOFFICIAL
Explore all skills by this author →
WebsiteGitHub

kailash-align

Enforce evaluation-before-serving gates for TRL-based LLM fine-tuning workflows.

Updated Mar 31, 2026
Search Tags:#dpo#llm alignment#evaluation gating#lora adapters#ollama deployment#vllm serving#trl training
One-click install
npx skills add https://github.com/terrene-foundation/kailash-coc-claude-rb --skill kailash-align-terrene-foundation
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: kailash-align
Source: https://github.com/terrene-foundation/kailash-coc-claude-rb/tree/main/.claude/skills/35-kailash-align
Command: npx skills add https://github.com/terrene-foundation/kailash-coc-claude-rb --skill kailash-align-terrene-foundation

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill prevents unsafe or unverified fine-tuning by enforcing an evaluation-before-serving gate for TRL-based alignment workflows.

Core Features & Use Cases

  • Eval-before-serve enforcement: Blocks deployment unless the fine-tuned adapter passes comparison against the base model and required safety checks.
  • Training across 12 alignment methods: Supports offline (SFT/DPO/CPO/ORPO), unpaired (KTO/BCO), and online RL-style methods with correct dataset expectations.
  • Adapter lifecycle and chaining: Manages LoRA adapter versioning (draft → evaluated → active → archived) including two-stage flows like sft_then_dpo.

Quick Start

Install kailash-align with the appropriate extras, then train an adapter with the AlignmentPipeline, run AlignmentEvaluator on the trained adapter_id, and only pass that eval_result into AlignmentServing.deploy() for Ollama or vLLM.

Frequently Asked Questions about kailash-align

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I enforce an evaluation gate before serving fine-tuned LLM adapters?▼

You can enforce an evaluation-before-serving gate by running an AlignmentEvaluator on the trained adapter_id and only passing the resulting eval_result into a serving deploy function, which blocks deployment unless safety checks pass.

What alignment methods does TRL support for unpaired LLM fine-tuning?▼

TRL supports unpaired alignment methods like KTO and BCO, alongside offline methods such as SFT, DPO, CPO, and ORPO, each requiring correctly formatted dataset inputs for successful fine-tuning.

Can I chain LoRA adapters for multi-stage LLM alignment like sft_then_dpo?▼

Yes, you can chain LoRA adapters for multi-stage alignment workflows like sft_then_dpo, managing the adapter lifecycle from draft to evaluated to active and archived versions sequentially.

Do I need Python to train and align LLMs with TRL?▼

Yes, TRL-based LLM fine-tuning and alignment requires Python-only training, utilizing strict config validation with immutable dataclasses and numeric NaN/Inf checks to ensure safe execution.

Does kailash-align support Ollama and vLLM for serving fine-tuned models?▼

Yes, kailash-align supports both Ollama and vLLM serving, allowing you to deploy fine-tuned LLMs only after the adapter passes the mandatory evaluation gate against the base model.

Why is trust_remote_code disabled for TRL alignment workflows?▼

trust_remote_code is disabled to prevent unsafe or unverified fine-tuning, ensuring that TRL-based alignment workflows maintain strict security boundaries during training and serving.

Related Skills

Explore More Software Engineering Skills→
ovachieverovachiever
52

openrlhf-training

Train large language models with RLHF using Ray and vLLM acceleration.

Community
Advanced
ScientiaCapitalScientiaCapital
1

model-deployment

Export fine-tuned models to GGUF and deploy with vLLM or Ollama.

Community
Advanced
FFOO6866FFOO6866

kailash-align

Automate LLM fine-tuning with TRL methods and enforce evaluation before serving.

Community
Advanced
ever-oliever-oli
3

fine-tuning-with-trl

Fine-tune HuggingFace Transformers with TRL for RLHF alignment.

Community
Advanced