skills.rest
av
avCOMMUNITY
Explore all skills by this author →
Website𝕏@EverlierGitHub

test-boost-module

Live-test Harbor Boost modules by sending prompts through llamacpp and validating output.

3.2k|224|Updated Jul 27, 2024
Search Tags:#QA#Integration Testing#lLM#prompt testing#harbor#llamacpp#boost module
One-click install
npx skills add https://github.com/av/harbor --skill test-boost-module
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: test-boost-module
Source: https://github.com/av/harbor/tree/main/.agents/skills/test-boost-module
Command: npx skills add https://github.com/av/harbor --skill test-boost-module

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Verifying that a Harbor Boost module actually changes LLM output requires manual, error-prone testing. This Skill provides a repeatable live integration test that runs a real prompt through a module and compares it against a control run.

Core Features & Use Cases

  • Live Integration Testing: Sends real prompts through a running Boost module via harbor launch and pi, rather than relying on unit tests.
  • Control Comparison: Runs the same prompt with and without the module to confirm the module's effect is visible and correct.
  • Prompt Strategy Guidance: Maps module types (style, reasoning, retrieval, transform, guard, injection) to prompts that make their effects obvious.
  • Use Case: After writing a new Boost module like a reasoning-chain module, run this Skill to confirm it produces visible multi-step reasoning on a math prompt and passes the validation checklist.

Quick Start

Test the boost module named g1 by running a math prompt through it with harbor launch and pi, then compare the output against a control run without the module.

Frequently Asked Questions about test-boost-module

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test a Harbor Boost module with a real prompt?▼

Run harbor launch with --workflow set to the module name and --model set to a loaded llamacpp model, followed by pi with -p, --no-tools, and --no-session. Compare the output against a control run without the --workflow flag.

What prompt should I use to verify a Boost module works?▼

Pick a prompt that makes the module's effect obvious in the output. For style modules ask for a detailed explanation, for reasoning modules use a math or logic problem, and for retrieval modules ask about recent developments in a topic.

Why does my Boost module output look identical to the control run?▼

Identical output usually means the module did not activate or the prompt does not exercise its behavior. Restart Boost with harbor down boost && harbor up boost, wait for health checks, and check docker logs harbor.boost for errors.

What services must be running before testing a Boost module?▼

Both llamacpp and boost must be running via harbor up, and pi must be installed on the host. Wait for health checks to pass before sending test prompts.

Why does the Boost module test hang or return empty output?▼

Hangs usually mean the model is too large or generation looped; retry with a smaller model or a timeout. Empty output suggests the module swallowed the completion, so inspect docker logs harbor.boost for errors.

Related Skills

Explore More Software Engineering Skills→
avav
3.2k

boost-modules

Develop Python modules for custom processing of LLM chat completions in Harbor Boost.

Community
Advanced
xsw632xsw632
2

simulate

Automates Verilator simulation runs and structured pass/fail reporting from logs.

Community
Intermediate
theodrosyimertheodrosyimer

plan-feature

Generate a detailed implementation plan with TDD-aligned test cases for a proposed software feature.

Community
Advanced
89jobrien89jobrien

rust-unit-tests

Automate writing and running unit tests for Warp Rust crates.

Community
Intermediate