skills.rest
patchy631
patchy631COMMUNITY
Explore all skills by this author →
𝕏@akshay_pachaarGitHub

grpo-finetune

Fine-tune Qwen3-8B with GRPO reinforcement learning on Fireworks GPUs from a dataset and reward function.

37.3k|6.2k|Updated Oct 21, 2024
Search Tags:#fine-tuning#grpo#fireworks#JSON extraction#reward function#Qwen3#Reinforcement Learning
One-click install
npx skills add https://github.com/patchy631/ai-engineering-hub --skill grpo-finetune
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: grpo-finetune
Source: https://github.com/patchy631/ai-engineering-hub/tree/main/grpo-finetuning-qwen3/agent-skill/grpo-finetune
Command: npx skills add https://github.com/patchy631/ai-engineering-hub --skill grpo-finetune

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires openai, python-dotenv, jsonschema, fireworks, nest_asyncio.

What problem does it solve?

Training a model to reliably produce structured outputs (like JSON matching a schema) normally requires deep RL infrastructure expertise. This Skill turns a plain-English task description plus a JSONL dataset into a complete GRPO fine-tuning run on Fireworks-managed GPUs, ending with a deployed, evaluated model.

Core Features & Use Cases

  • Reward Function Validation: Generates and validates a schema-based reward.py (score contract: 1.0 valid schema, 0.5 valid JSON wrong shape, 0.0 not JSON) before any GPU spend, using self-tests that must print PASS.
  • End-to-End GRPO Pipeline: Uploads the dataset, runs GRPO training on Qwen3-8B via Fireworks training shapes, deploys the resulting model, and evaluates accuracy on an eval set.
  • Live Agent Demo: Runs the fine-tuned model on sample invoices with a terminal UI showing per-field extraction results and schema-match percentage.
  • Use Case: You have 200 invoice prompts in JSONL and want a model that always returns {vendor, date, amount, currency} as valid JSON. The Skill writes the reward, trains, deploys, and reports final accuracy.

Quick Start

Fine-tune a model on my train.jsonl and eval.jsonl to extract vendor, date, amount, and currency from invoices using GRPO.

Frequently Asked Questions about grpo-finetune

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune a model with GRPO on Fireworks?▼

Provide a train.jsonl and eval.jsonl dataset plus a task description. The Skill writes a schema-based reward.py, validates it with self-tests, then runs run_pipeline.py to upload the dataset, train Qwen3-8B with GRPO on Fireworks GPUs, deploy the model, and report eval accuracy.

What is the reward function contract for GRPO training?▼

The reward module must define score(completion, row=None) returning a float in [0,1]. The default schema-only reward returns 1.0 for valid JSON matching the schema, 0.5 for valid JSON with wrong shape, and 0.0 for non-JSON output.

What API keys and environment variables does GRPO fine-tuning need?▼

The pipeline loads FIREWORKS_API_KEY, FIREWORKS_ACCOUNT_ID, and OPENROUTER_API_KEY from a .env file in the current directory via python-dotenv. No additional setup is required once these keys are present.

How long does GRPO training on Fireworks take?▼

Training typically takes 30 to 60 minutes or more for 200 prompts with one epoch. The Skill instructs using a timeout of at least 7200 seconds rather than the default 10-minute timeout when running the pipeline.

Why does reward validation fail before training starts?▼

Validation fails if reward.py is missing, lacks a callable score function, crashes on probe inputs, returns values outside [0,1], or fails any SELF_TESTS entries. Fix the reward module until the validator prints PASS before proceeding.

Can I customize what the GRPO model is trained to optimize?▼

Yes. After the Skill shows the generated reward.py, you can edit it to reflect your own definition of a good output before training begins. The pipeline then uses your edited reward for both training and post-training evaluation.

Related Skills

Explore More Software Engineering Skills→
castform-aicastform-ai
49

generate-data

Create train and eval JSONL datasets for Castform runs.

Official
Advanced
shafibabarshafibabar
1

canary-deployment

Coordinate Linkerd traffic splits and Argo Rollouts gates for canary deployments.

Community
Advanced
voxel51voxel51
37

fiftyone-model-evaluation

Evaluate model predictions against ground truth using FiftyOne evaluation protocols.

Official
Intermediate
VYRE-StudiosVYRE-Studios

fine-tuning-with-trl

Fine-tune HuggingFace Transformers language models with TRL for RLHF alignment.

Official
Advanced