skills.rest
dennisvink
dennisvinkCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

aws-ai-ml

Fine-tune, evaluate, and deploy AI models on Amazon SageMaker.

Updated Sep 8, 2026
Search Tags:#aws#llm#fine-tuning#model deployment#bedrock#mlflow#sagemaker
One-click install
npx skills add https://github.com/dennisvink/yolomancer --skill aws-ai-ml-dennisvink
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: aws-ai-ml
Source: https://github.com/dennisvink/yolomancer/tree/main/skills/aws/core-skills/aws-ai-ml
Command: npx skills add https://github.com/dennisvink/yolomancer --skill aws-ai-ml-dennisvink

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires boto3, pandas, and includes scripts (resource) and references (resource) components.

What problem does it solve? Customizing and deploying foundation models on AWS involves many decisions — choosing a base model, picking a fine-tuning technique, validating datasets, running training jobs, evaluating quality, and deploying to endpoints. This Skill guides you through the entire SageMaker model customization lifecycle with validated workflows, so you avoid misconfigured training jobs, incompatible datasets, and failed deployments. ## Core Features & Use Cases - Model Selection & Planning: Discover available models in SageMaker Hub, define use case specifications, and generate a step-by-step project plan (fine-tuning, evaluate-first, or deploy-base-model paths). - Fine-Tuning & Dataset Workflows: Generate training code for SFT, DPO, RLVR, and RLAIF techniques, validate dataset formats, and transform datasets into required JSONL schemas. - Evaluation, Deployment & Diagnostics: Evaluate models with LLM-as-Judge or Custom Scorers, deploy to SageMaker endpoints or Bedrock, manage SageMaker Managed MLflow apps, and diagnose endpoint failures with CloudWatch metrics and container logs. - Use Case: You want to fine-tune a Llama model on your customer support data. The Skill helps you define the use case, select the model, validate your dataset, generate the SFT training code, evaluate the result, and deploy it to a SageMaker endpoint. ## Quick Start Ask the assistant to help you fine-tune or deploy a model on Amazon SageMaker, describing your use case and whether you already have a dataset or trained model.

Frequently Asked Questions about aws-ai-ml

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune a model on Amazon SageMaker?▼

Select a base model from SageMaker Hub, choose a fine-tuning technique (SFT, DPO, RLVR, or RLAIF), validate your JSONL training dataset, and generate training code from the provided templates. The workflow then runs a SageMaker Serverless Model Customization training job.

How do I choose between SFT, DPO, RLVR, and RLAIF fine-tuning?▼

The technique selection guide recommends a method based on your use case: SFT for supervised instruction tuning, DPO for preference alignment, RLVR for verifiable rewards, and RLAIF for AI-feedback-based training. The chosen technique is validated against the selected model's supported recipes.

Can I deploy a fine-tuned model to Amazon Bedrock?▼

Yes, OSS and Nova models fine-tuned through SageMaker Serverless Model Customization can be deployed to either SageMaker endpoints or Bedrock. JumpStart base models deploy only to SageMaker real-time endpoints.

What dataset format does SageMaker fine-tuning require?▼

All training and evaluation datasets must be serialized as JSONL, with one JSON object per line, matching the schema for your model type and fine-tuning technique. Non-JSONL files like parquet or CSV must be transformed first, even if their columns match.

Why is my SageMaker endpoint failing or slow?▼

The endpoint diagnostics workflow collects endpoint status via DescribeEndpoint, CloudWatch metrics for invocations, errors, latency, and utilization, plus recent container logs. The agent interprets these facts and references the official AWS troubleshooting guide.

What is not covered by SageMaker model customization workflows?▼

Validated workflows do not cover Ground Truth labeling, Feature Store, HyperPod training, full fine-tuning (FFT), BYO containers, traditional ML models like XGBoost, or general-purpose AWS infrastructure. Best-effort guidance is offered for these cases.

Related Skills

Explore More Software Engineering Skills→
awslabsawslabs
881

sdk-getting-started

Validates SageMaker SDK version, AWS region, and execution role before ML operations.

Official
Intermediate
frankxaifrankxai
2

AWS AI Services Expert

Coordinate Bedrock, SageMaker, and AWS AI services to automate enterprise AI workflows.

Community
Advanced
Albatross679Albatross679

mlflow-monitor

Launch the MLflow UI to monitor and compare training experiment metrics.

Community
Intermediate
awslabsawslabs
881

model-deployment

Generates code to deploy LoRA fine-tuned SageMaker models to SageMaker endpoints or Bedrock.

Official
Advanced