skills.rest
orizuai
orizuaiOFFICIAL
Explore all skills by this author →
WebsiteGitHub

orizu-cli

Evaluate and optimize LLM applications using human feedback and auto-evaluation.

Updated Feb 20, 2026
Search Tags:#optimization#machine learning#evaluation#LLM#AI
One-click install
npx skills add https://github.com/orizuai/orizu-cli --skill orizu-cli
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: orizu-cli
Source: https://github.com/orizuai/orizu-cli/tree/main/plugins/claude-code/skills/orizu-cli
Command: npx skills add https://github.com/orizuai/orizu-cli --skill orizu-cli

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires node.js, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenge of improving Large Language Model (LLM) applications by providing a comprehensive platform for evaluation and optimization, streamlining the process of building evals and optimizing against them.

Core Features & Use Cases

  • Human Evaluation: Collect and manage human feedback on LLM outputs.
  • Auto Evaluation: Automate evaluation using machine learning models.
  • Optimization: Use evals to hill climb and optimize LLM performance.
  • Use Case: If you have a set of LLM outputs and you want to improve their accuracy, you can use this Skill to collect human feedback, create evals, and optimize the model against them.

Quick Start

Run the orizu setup command to initialize your local workspace and install the necessary skills.

Frequently Asked Questions about orizu-cli

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate and optimize LLM outputs using human feedback?▼

You can evaluate and optimize LLM outputs by collecting human feedback to build evals, then hill climbing the model's performance against those evals. This workflow streamlines improving accuracy through human evaluation.

What is the best way to automate LLM application evaluation?▼

Automating LLM application evaluation uses machine learning models for auto-evaluation. This approach automates the assessment of LLM outputs, enabling subsequent optimization without relying solely on manual human review.

Do I need Node.js to run LLM evaluation and optimization workflows?▼

Yes, you need Node.js installed to run LLM evaluation and optimization workflows. The setup requires initializing a local workspace via command line, which depends on the Node.js environment.

Can I optimize my machine learning model by building evals first?▼

Yes, you can optimize machine learning models by building evals first. The workflow focuses on creating evaluations against which the LLM is subsequently optimized to achieve better accuracy.

How do I set up my local workspace for LLM optimization?▼

To set up your local workspace for LLM optimization, run the setup command to initialize the environment and install necessary skills. This requires Node.js and an Orizu account.

Why focus on building evals before optimizing LLM applications?▼

Building evals before optimizing LLM applications establishes a baseline for performance. By defining metrics first, the optimization process can effectively hill climb and enhance model accuracy against those targets.

Related Skills

Explore More Software Engineering Skills→
MakisuoMakisuo
1.6k

clickhousectl-local-dev

Install and configure a local ClickHouse environment with clickhousectl.

Community
Intermediate
viktorbezdekviktorbezdek
10

agent-evaluation

Evaluate LLM agent outputs using multi-dimensional rubrics and bias mitigation.

Community
Advanced
LuisSambranoLuisSambrano
7

llm-eval

Evaluate LLM applications using automated metrics and human feedback.

Community
Advanced
Zhow01Zhow01
52

llm-evaluation

Assess and benchmark LLM outputs across automated metrics, human judgments, and safety checks.

Community
Advanced