smith

Automate OPA policy creation, testing, and iterative refinement for AI code agents.

16|2|Updated Jun 19, 2026
One-click install
npx skills add https://github.com/IBM/smith --skill smith-ibm
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: smith
Source: https://github.com/IBM/smith/tree/main
Command: npx skills add https://github.com/IBM/smith --skill smith-ibm

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires openai, python-dotenv, numpy, pandas, pydantic, PyYAML, networkx, pydot, matplotlib, seaborn, nltk, scikit-learn, sentence-transformers, typer, rich, termcolor, httpx, requests, tqdm, tenacity, jinja2, mcp[cli], and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill automates the lifecycle management of Open Policy Agent (OPA) policies and test generation for AI code agents, streamlining policy creation, testing, and refinement.

Core Features & Use Cases

  • Policy Creation: Create OPA policies from natural language guidance and an agent description.
  • Test Generation: Generate synthetic legitimate and adversarial test cases using LLM-based fuzzing and red-teaming tools.
  • Policy Testing: Test policies against generated and custom test suites.
  • Policy Refinement: Automatically refine policies through iterative feedback loops.
  • Use Case: Imagine you have a set of access control policies that need to be enforced. Use this Skill to automatically generate test cases, test the policies, and refine them based on feedback.

Quick Start

Use the smith skill to create an OPA policy for your access control requirements and generate test cases to validate it.

Frequently Asked Questions about smith

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate OPA policy lifecycle management for AI agents?

Automate OPA policy lifecycle management by generating policies from natural language, creating synthetic test cases, testing, and iteratively refining them. This handles access control policy creation and validation for AI code agents.

What is the best way to generate test cases for Open Policy Agent policies?

Generate test cases for Open Policy Agent policies using LLM-based fuzzing and red-teaming tools. This approach creates both legitimate and adversarial synthetic inputs to thoroughly validate access control rules.

Do I need Python and OPA to create and test policies from natural language?

Yes, you need Python and OPA, along with Regal, ARES, and Promptfoo, to create and test policies from natural language. These dependencies provide the environment for policy generation and evaluation.

Can I refine OPA policies automatically through iterative feedback loops?

You can refine OPA policies automatically through iterative feedback loops. The policy testing process evaluates results against generated test suites and adjusts rules based on the feedback.

How does LLM-based fuzzing work for OPA policy test generation?

LLM-based fuzzing for OPA policy test generation works by leveraging natural language processing and machine learning to create synthetic test cases. It generates both legitimate and adversarial inputs for comprehensive policy validation.

Why should I use automated policy refinement instead of manually testing OPA rules?

Automated policy refinement replaces manually testing OPA rules by streamlining the iterative feedback loop. It automatically adjusts policies based on test results, reducing manual effort and improving rule accuracy.