skills.rest
biocypher
biocypherOFFICIAL
Explore all skills by this author →
WebsiteGitHub

karenina-adapter-test

Validates new karenina adapters through conformance, mocked, and live API test suites.

13|3|Updated Jun 27, 2025
Search Tags:#pytest#mcp#mocking#usage tracking#llm agents#adapter testing#conformance suite
One-click install
npx skills add https://github.com/biocypher/karenina --skill karenina-adapter-test-biocypher
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: karenina-adapter-test
Source: https://github.com/biocypher/karenina/tree/main/skills/karenina-adapter-test
Command: npx skills add https://github.com/biocypher/karenina --skill karenina-adapter-test-biocypher

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? New karenina adapters can appear to work while silently ignoring configuration, dropping token usage, or failing on real filesystem access. This Skill provides a systematic test strategy that validates every capability an adapter claims to support through observable behavior rather than SDK internals. ## Core Features & Use Cases - Cold tests (mocked): Protocol conformance, message roundtrip, trace format, registration integrity, configuration passthrough, error mapping, usage aggregation, and streaming/timeout partial recovery. - Hot tests (live API): End-to-end checks for workspace file access, tool-use traces, turn limits, LLM invocation, structured parsing, usage tracking, and error propagation across all three ports. - Use Case: After implementing a new adapter (e.g., for a custom agent SDK), run this Phase 4 workflow to catch silent bugs like zero-token usage reporting or agents that generate synthetic data instead of reading real workspace files. ## Quick Start Run the karenina-adapter-test skill to validate my newly implemented adapter against the conformance suite and hot tests.

Frequently Asked Questions about karenina-adapter-test

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test a new karenina adapter?▼

Run the cold tests first with pytest against the conformance suite and adapter-specific tests, then run hot tests with a live API after user approval. Cold tests cover protocol conformance, config passthrough, and usage extraction; hot tests verify real workspace access, tool traces, and structured parsing.

What is the difference between cold tests and hot tests for adapters?▼

Cold tests run fully mocked with no API calls, validating protocol conformance, error mapping, and usage aggregation. Hot tests call live LLM APIs to verify end-to-end behavior like workspace file access, tool-use traces, and turn limits that mocks cannot catch.

Why does my adapter pass tests but ignore AgentConfig settings?▼

This is the silent failure the C5 configuration passthrough test catches. Mock the SDK's agent creation function to capture its arguments, then verify system_prompt, max_turns, and workspace_path actually reach the SDK rather than being dropped by the adapter.

Does the adapter test suite require an MCP server?▼

Only for tool-loop adapters without built-in tools, such as langchain. The skill provides a FastMCP test server with read_file and list_directory tools on port 8321. Adapters like claude_agent_sdk and langchain_deep_agents have built-in filesystem tools and skip this.

Why does my adapter report zero tokens in usage results?▼

The C7 usage extraction test catches this by mocking multi-turn responses with known token counts and verifying aggregation across all turns. Common causes are reading only the last turn's usage, missing usage_metadata fields, or dropping the model name.

What is the minimum viable test set for a karenina adapter?▼

The minimum set is C1, C1d, C4, C5, C7, C8 (streaming adapters only), C9, H1, H2, H5, H6, and H8. These cover all three ports, streaming and timeout recovery, usage tracking, and the most critical silent failure modes.

Related Skills

Explore More Software Engineering Skills→
ndthanhdevndthanhdev
54

hexagonal-inversify-di

Wire Hexagonal architecture ports and adapters with InversifyJS across a monorepo.

Community
Advanced
vlad-ryzhkovvlad-ryzhkov
25

auditor

Audits code changes in pull requests for user-facing quality, test coverage, and documentation.

Community
Advanced
FMurrayFMurray

querying-mlflow-metrics

Fetches aggregated trace metrics from MLflow tracking servers.

Community
Intermediate
meistro57meistro57
1

laravel:ports-and-adapters

Implement Ports and Adapters architecture with PHP interfaces and provider-specific adapters.

Community
Intermediate