biocypher
Official@biocypher
BioCypher's karenina skill suite enables rigorous evaluation, benchmarking, and verification of language model outputs via templates, rubrics, scenarios, and adapters.
Agent Skills by biocypher
Showing 17 vetted skills indexed across 1 GitHub repositories.
biocypherkarenina-verification
Configure and run karenina's LLM verification pipeline with models, adapters, guards, and presets.
biocyphertesting-gate
Runs the karenina live pytest battery gating verification engine, adapter, and scenario changes.
biocypherkarenina-cli
Run karenina benchmark verifications from the terminal with presets, feature flags, and resumable runs.
biocypherkarenina-scenarios
Build and run multi-turn scenario evaluations for conversational LLMs with karenina.
biocypherkarenina-manual
Replay pre-recorded LLM responses through karenina's full verification pipeline using ManualAdapter.
biocypherkarenina-adapter-design
Generates a design spec for a new karenina adapter from a gathered context document.
biocypherkarenina-rubric-authoring
Create karenina evaluation rubrics with LLM, regex, callable, metric, and agentic traits.
biocypherkarenina-adapter-test
Validates new karenina adapters through conformance, mocked, and live API test suites.
biocypherkarenina-qa
Build and run single-turn QA benchmarks for LLM evaluation with karenina.
biocypherkarenina-task-eval
Evaluate pre-recorded LLM outputs against templates and rubrics using karenina's TaskEval API.
biocypherkarenina-results
Load, analyze, and export karenina verification results as DataFrames.
biocypherkarenina-adapter-implement
Implements a new karenina adapter package file-by-file following a design spec.
biocypherkarenina-adapter-gather-context
Gathers SDK documentation and capability requirements for designing new karenina adapters.
biocypherusing-karenina
Routes LLM evaluation requests to karenina workflow skills with shared API references and gotchas.
biocypherkarenina-adapter-create
Guides creation of karenina adapters implementing AgentPort, LLMPort, and ParserPort interfaces.
biocypherkarenina-template-authoring
Create and validate BaseAnswer templates for karenina LLM evaluation pipelines.
biocypherkarenina-adapter-review
Reviews new karenina adapters for code quality, convention compliance, and correctness before merge.
Frequently Asked Questions About biocypher
FAQPage SchemaWhat tasks can I accomplish with biocypher's karenina skills?▼
You can build single-turn QA benchmarks, run multi-turn scenario evaluations with branching dialogue and sycophancy checks, author answer templates and rubrics with five trait types, evaluate pre-recorded outputs via TaskEval, configure verification pipelines with guards and presets, and analyze or export scored results as DataFrames.
Who should use the karenina skill suite?▼
Evaluation engineers, model quality researchers, and developers benchmarking language model outputs against structured criteria. It also serves engineers extending karenina itself, since dedicated skills cover the full adapter creation lifecycle from requirements gathering through design, implementation, conformance testing, and final review.
How do I run karenina evaluations from the terminal?▼
Use the karenina-cli skill, which covers environment setup, preset management, and running karenina verify with flags. It supports progressive save and resume for long runs, model comparison across checkpoints, and result export, providing the fastest path from a saved checkpoint to scored evaluation results.
How are karenina changes tested before merging?▼
The testing-gate skill runs the live test battery under karenina/tests/live/, including async adapter tests B1 through B11, QA and scenario progressive save/resume checks, and a docker-backed test. Opt in with KARENINA_LIVE_TESTS=1; the docker test additionally requires KARENINA_LIVE_DOCKER_TESTS=1 and a reachable daemon.
What is required to add support for a new model provider?▼
Follow the five-phase adapter lifecycle: gather SDK documentation and map capabilities to karenina ports, produce a design spec with registry integration plan, implement guided file-by-file code, run cold mocked and hot live conformance tests, then complete a final review for quality and convention compliance before merge.