mat-bench-client

To run materials science agent evaluations, you need a tool that coordinates question retrieval, data download, task submission, and grading. This client automates the entire loop by applying your authenticated session to fetch prompts, upload multipart outputs, and poll the benchmark server for weighted scores.

Yes, you can submit benchmark scoring results asynchronously via REST API. The client supports multipart file upload for produced outputs and handles asynchronous polling, waiting until the benchmark server returns available checkpoints and final weighted scores for the submitted agent tasks.

Authenticating an agent evaluation session requires a pre-created server session using a provided API token. Once configured, this session persists, allowing you to run multiple materials science questions across domains like structure retrieval, workflow orchestration, and safety scenarios without needing to reauthenticate.

Yes, mat-bench-client supports scientific analysis and execution contract validation. It orchestrates structured materials science tasks including batch processing, data diagnosis, and safety/refusal scenarios, producing auditable and scored runs to validate the execution contracts of your AI agents.

To retrieve question prompts and data files for benchmark scoring, you request the server to list questions by capability or domain. After fetching a specific question, the client automatically downloads the associated data files required to execute the materials science agent workflow.

Using pyyaml for workflow orchestration in agent evaluation limits dependencies to YAML parsing only. The client requires a pre-existing authenticated server session and relies entirely on external REST API endpoints for task execution, meaning it cannot function offline or without valid server credentials.