testing-heterogeneous-agents

Runs the live LobeHub official-provider compatibility matrix across external CLI agents.

Updated Jan 27, 2026
One-click install
npx skills add https://github.com/SmallAi-API/smaihub --skill testing-heterogeneous-agents-smallai-api
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: testing-heterogeneous-agents
Source: https://github.com/SmallAi-API/smaihub/tree/main/.agents/skills/testing-heterogeneous-agents
Command: npx skills add https://github.com/SmallAi-API/smaihub --skill testing-heterogeneous-agents-smallai-api

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? Verifying that every server-advertised official model actually completes through each supported external CLI agent (Claude Code, Codex, Grok Build, Kimi Code, Pi, TRAE) requires tedious manual testing across the real Desktop renderer, IPC, and relay path. This Skill automates that compatibility matrix with an executable harness and Acceptance-compatible evidence reporting. ## Core Features & Use Cases - Live Matrix Discovery: Reads the live getServerDefaultHeterogeneousCapability response and probes installed CLI binaries without making model calls, so the plan always matches the server's advertised agents and models. - Gated Live Execution: Runs each agent/model cell sequentially through the real server-default provider binding only after explicit --confirm-live approval, verifying a unique marker round trip plus authoritative relay, provider, and model attestation. - Acceptance-Compatible Reporting: Writes result.json, per-cell text evidence, and a structured matrix table under .records/reports/ for publishing as an immutable Acceptance round. - Use Case: Before a LobeHub release, invoke the skill to prove that every official model completes through Claude Code, Codex, and the other supported CLIs, then publish the resulting compatibility matrix as release evidence. ## Quick Start Ask the AI to run the testing-heterogeneous-agents compatibility matrix against the signed-in Electron instance and approve the discovered agent and model scope before any live requests are made.

Frequently Asked Questions about testing-heterogeneous-agents

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test CLI agent compatibility with LobeHub official providers?

Invoke the skill manually, then run the harness with `list --json` to discover the live agent/model matrix and `run --confirm-live` to execute each cell through the real Desktop IPC path. Each cell verifies a unique marker round trip plus relay attestation.

Which CLI agents does the compatibility matrix cover?

The matrix covers Claude Code, Codex, Grok Build, Kimi Code, Pi, and TRAE, using the models returned by the live getServerDefaultHeterogeneousCapability response. It does not cover user-defined providers or user-managed API keys.

Can the harness run automatically during development or CI?

No. The skill is manual-invocation only and must not be auto-loaded, scheduled, or triggered by other skills. Live runs additionally require explicit scope and cost approval before passing --confirm-live.

What happens when a required CLI is not installed?

Missing CLIs are reported as blocked cells, not failures. The harness never invokes installers, updaters, or sign-in flows, so you must install and authenticate the CLI separately before rerunning.

Why did a cell fail even though the CLI returned the marker?

A cell passes only when the marker round trip completes and the relay invocation attestation matches the expected agent, ingress, model, operation, and lobehub provider. A CLI completing without that authoritative relay proof is recorded as a failure.