skills.rest
ao-agent-ops
ao-agent-opsOFFICIAL
Explore all skills by this author →
GitHub

ao

Record agent traces as dataflow graphs and edit node inputs for testing.

2|Updated May 8, 2025
Search Tags:#debugging#agent development#observability#a/b testing#dataflow#llmops
One-click install
npx skills add https://github.com/ao-agent-ops/ao-dev --skill ao
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ao
Source: https://github.com/ao-agent-ops/ao-dev/tree/main
Command: npx skills add https://github.com/ao-agent-ops/ao-dev --skill ao

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pyyaml, dill, debugpy, psutil, psycopg2-binary, python-dotenv, fastapi, flatten_dict, flatten_json, openai, anthropic, openai-agents, google-genai, hydra-core, e2b_code_interpreter, together, langchain, langchain-anthropic, langchain-openai, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps you develop and maintain adaptable agentic systems by providing integrated observability, accelerated A/B testing, a curated design guide, and dynamic runtime lesson injection.

Core Features & Use Cases

  • Integrated Observability: Record agent traces as dataflow graphs with zero code changes.
  • Accelerated A/B Testing: Edit node inputs/outputs and rerun to see how changes propagate.
  • Lessons: Inject learned lessons into agent context dynamically at runtime.
  • Design Guide: Query curated agent development techniques for problems you are encountering.
  • Use Case: When debugging an agent that is not performing as expected, use ao-record to visualize its execution flow, identify bottlenecks, and then use ao-tool edit-and-rerun to test potential fixes.

Quick Start

Use ao-tool record to run your python script.

Frequently Asked Questions about ao

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I debug an LLM agent that is not performing as expected?▼

Debug an underperforming LLM agent by recording its execution traces as dataflow graphs to visualize execution flow, identify bottlenecks, and dynamically inject learned lessons into agent context at runtime.

What is the best way to run A/B testing on agent dataflow graphs?▼

Run A/B testing on agent dataflow graphs by editing node inputs and outputs, then rerunning the flow to observe how changes propagate through the system without modifying underlying code.

Can I use this with my existing LangChain and OpenAI agents?▼

Yes, it supports LangChain, OpenAI, Anthropic, and Google GenAI dependencies, allowing you to record agent traces and integrate with Claude Code for enhanced development workflows.

How do I record agent traces without changing my Python code?▼

Record agent traces with zero code changes by using the command-line tool to run your Python script, automatically capturing the execution dataflow graph for observability.

Does agent observability support injecting lessons dynamically at runtime?▼

Agent observability supports dynamic runtime lesson injection by inserting learned lessons directly into agent context, enabling the system to adapt based on previous execution outcomes.

When do I need to query a design guide for agent development?▼

Query a design guide for agent development when encountering specific problems during development, allowing you to retrieve curated agent development techniques to solve them.

Related Skills

Explore More Software Engineering Skills→
BleveneBlevene
4

giga-research

Orchestrate multi-provider research across Claude, OpenAI, and Gemini with citation validation.

Community
Advanced
tenstorrenttenstorrent
74

graph-break-analysis

Diagnose graph breaks in TT-XLA compilation logs and generate fix reports.

Official
Advanced
HouseGarofaloHouseGarofalo
3

ai-engineer-agent

Develops LLM apps, RAG systems, and agentic workflows with OpenAI, Anthropic, and LangChain.

Official
Advanced
BedaBeda-GrowthBedaBeda-Growth

Fix Together AI Source

Configure Together AI as a Craft Agent source with bearer token authentication.

Official
Intermediate