skills.rest
martinholovsky
martinholovskyCOMMUNITY
Explore all skills by this author →
𝕏@martinholovskyGitHub

sota-llm-engineering

Provide guidelines for engineering and auditing LLM-powered features.

12|2|Updated Jun 17, 2026
Search Tags:#prompt engineering#agent#LLM#RAG#retrieval#tool#AI feature
One-click install
npx skills add https://github.com/martinholovsky/SOTA-skills --skill sota-llm-engineering
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sota-llm-engineering
Source: https://github.com/martinholovsky/SOTA-skills/tree/main/skills/sota-llm-engineering
Command: npx skills add https://github.com/martinholovsky/SOTA-skills --skill sota-llm-engineering

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sota-code-security, sota-sandboxing, sota-privacy-compliance, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill ensures the engineering and auditing of LLM-powered features are done according to the latest 2026 standards, enhancing build quality and security.

Core Features & Use Cases

  • LLM Feature Engineering: Defines best practices for building LLM features with measurable, grounded, bounded, and observable outcomes.
  • Audit Mode: Identifies and reports quality violations with severity ratings.
  • Eval Suite Development: Provides guidelines for creating comprehensive eval suites to ensure feature quality.
  • Prompt and Context Engineering: Offers strategies for structuring prompts and managing context budgets.
  • RAG and Retrieval: Details how to design and evaluate RAG systems for reliable information retrieval.
  • Agent and Tool Use: Establishes guidelines for designing and managing agents and tools used in LLM workflows.
  • Production Engineering: Covers best practices for model selection, cost engineering, and observability in production environments.
  • Data Lifecycle: Provides guidelines for dataset curation, feedback loops, and embedding migration.
  • Use Case: Imagine you are developing a chatbot that provides financial advice. Use this Skill to ensure that the chatbot's advice is grounded in relevant information, that the system prompts are secure and effective, and that the model selection is appropriate for the task.

Quick Start

Run the sota-llm-engineering skill to start building or auditing an LLM feature.

Frequently Asked Questions about sota-llm-engineering

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What are the best practices for engineering LLM-powered features to ensure grounded and observable outcomes?▼

Engineering LLM-powered features requires following state-of-the-art 2026 practices to ensure outcomes are measurable, grounded, bounded, and observable. This involves strict prompt and context engineering, comprehensive eval suite development, and robust production engineering.

How do I design and evaluate a RAG system for reliable information retrieval in an LLM application?▼

Designing and evaluating a RAG system requires applying specific retrieval design guidelines to ensure reliable information retrieval. This includes managing the data lifecycle through dataset curation, establishing feedback loops, and handling embedding migration effectively.

How do I audit an existing LLM feature for quality and security violations?▼

Auditing an LLM feature involves running an audit mode that identifies and reports quality violations with severity ratings. This process evaluates prompt and context engineering, agent and tool use, and production engineering against state-of-the-art 2026 standards.

Do I need prior knowledge of LLM architecture and programming to use these prompt engineering guidelines?▼

Yes, applying these prompt engineering and agent design guidelines requires existing knowledge of LLM architecture and programming. The Skill provides advanced state-of-the-art 2026 practices for engineering and auditing features, not introductory programming concepts.

What is the best way to manage context budgets and structure prompts for an LLM agent workflow?▼

Managing context budgets and structuring prompts requires applying specific prompt and context engineering strategies. For agent workflows, this includes following established guidelines for designing and managing agents and tools used in LLM operations.

How do I set up an eval suite to ensure the quality of a production LLM feature?▼

Setting up an eval suite requires following provided guidelines to create comprehensive evaluation suites that ensure LLM feature quality. This integrates with production engineering best practices for model selection, cost engineering, and observability.

Related Skills

Explore More Software Engineering Skills→
haxlyshaxlys

k8s-security-policies

Generate and enforce Kubernetes NetworkPolicy, PodSecurityPolicy, and RBAC security policies.

Community
Advanced
jokken79jokken79
1

langfuse

Trace and monitor LLM applications across LangChain, OpenAI, and custom pipelines.

Community
Advanced
smarmen86smarmen86

guard

Intercept destructive shell commands and restrict file writes to a defined directory.

Community
Advanced
ryasrkryasrk

prompt-engineering

Design system prompts, RAG pipelines, and tool-calling workflows for LLM providers.

Community
Advanced