ground-truth-design

Design ground-truth queries to measure search quality for Oak curriculum content.

7|3|Updated Jul 28, 2025

Search Tags:#benchmarking #curriculum #semantic-search #ground-truth #search-quality #uk-teacher

Ready to run

npx skills add https://github.com/oaknational/oak-open-curriculum-ecosystem --skill ground-truth-design

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: ground-truth-design
Source: https://github.com/oaknational/oak-open-curriculum-ecosystem/tree/main/.cursor/skills/ground-truth-design
Command: npx skills add https://github.com/oaknational/oak-open-curriculum-ecosystem --skill ground-truth-design

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Ground-truth queries establish known-answer tests for Oak's semantic search, enabling deterministic evaluation of retrieval quality.

Core Features & Use Cases

Ground-truth methodology for evaluating search results.
Works with bulk data and known-answer-first testing to validate curriculum content search.
Supports iterative refinement of queries to improve measurement of relevance.

Quick Start

Provide a ground-truth query and run a search to validate results against known answers.

Frequently Asked Questions about ground-truth-design

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

What is a ground-truth query for evaluating semantic search quality?▼

A ground-truth query establishes a known-answer test for semantic search, enabling deterministic evaluation of retrieval quality against curriculum content. It uses a known-answer-first approach to validate that search results align with expected educational materials.

How do I design ground-truth queries to benchmark curriculum search results?▼

Design ground-truth queries by following a structured process from content discovery to query evaluation, using bulk data and predefined relevance criteria. You run searches and validate top results against known answers using a standardized scoring system to measure search quality.

Can I use bulk data to assess search relevance for the Oak curriculum?▼

Yes, you can use bulk data to assess search relevance for the Oak curriculum. The workflow supports loading bulk data and applying predefined relevance criteria to evaluate top results, enforcing a structured process to validate search quality at scale.

Does this ground-truth methodology require UK teacher input to validate search results?▼

Yes, the methodology targets professional UK teachers to verify that search results align with curriculum content. It requires teacher input to apply predefined relevance criteria and use the standardized scoring system for evaluating top search results.

What's the best way to iteratively refine queries to improve search quality measurement?▼

The best way to iteratively refine queries is to use the structured workflow that moves from content discovery to query evaluation. By applying predefined relevance criteria and a standardized scoring system to top results, you can continuously adjust queries to improve measurement accuracy.

Why does my semantic search evaluation need a known-answer-first approach?▼

A known-answer-first approach is needed for semantic search evaluation because it establishes deterministic tests for retrieval quality. By defining ground-truth queries with known correct answers, you can accurately measure whether the search service returns relevant curriculum content.