explore-to-sdk-evals

Provide the Explore-generated test cases and let the Skill emit one @mcpjam/sdk eval test per case using the exact prompt text. It detects Jest or Vitest from your repo and generates matching imports, assertions, and lifecycle hooks.

Connect with MCPClientManager over HTTP or stdio, fetch tools via getToolsForAiSdk, and run prompts through HostRunner. Assert results with hasToolCall, matchNoToolCalls, or argument matchers like matchToolCallWithPartialArgs.

Yes, the Skill inspects package.json and config files for Jest or Vitest and emits the correct imports. If neither framework is present, it defaults to Vitest and notes the dependency to add.

Explore cases often show illustrative placeholder arguments like empty strings, which fail when passed to matchToolCallWithPartialArgs. Prefer hasToolCall unless the case documents real literal values, or use matchToolArgumentWith predicates.

Module-level env checks run before dotenv loads if dotenv is only configured in vitest.config.ts, since config and test workers run in different contexts. Add import "dotenv/config" as the first import or use a setupFiles entry that loads .env.

Yes, createEvalRunReporter uploads results when given an MCPJAM_API_KEY, and passing the connected MCPClientManager attaches serverReplayConfigs for HTTP servers. Call finalize before disconnecting, and wrap it in try/catch if upload failures should not fail the run.