skills.rest
JoaquinCampo
JoaquinCampoCOMMUNITY
Explore all skills by this author →
GitHub

sparse-retrieval-eval

Evaluate sparse retrieval models on BEIR, MIRACL, and mMARCO benchmarks.

3|Updated Dec 22, 2025
Search Tags:#retrieval#benchmark#idf#sparse#csr#IR-evaluation
One-click install
npx skills add https://github.com/JoaquinCampo/Skills --skill sparse-retrieval-eval
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-retrieval-eval
Source: https://github.com/JoaquinCampo/Skills/tree/main/sparse-retrieval-eval
Command: npx skills add https://github.com/JoaquinCampo/Skills --skill sparse-retrieval-eval

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Evaluate and compare sparse retrieval models on standard IR benchmarks to quantify performance across diverse datasets and metrics.

Core Features & Use Cases

  • Supports loading BEIR, MIRACL, and mMARCO benchmarks and computing full IR metrics (nDCG@k, Recall@k, MAP, MRR).
  • Handles sparse encoding of document representations with CSR matrices, IDF-weight weighted retrieval, and caching for reproducibility.
  • Interprets results with clear guidance for analysis and comparison across models and datasets.

Quick Start

Run the sparse retrieval evaluation workflow on your dataset to compare BEIR, MIRACL, and mMARCO benchmarks.

Frequently Asked Questions about sparse-retrieval-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate sparse retrieval models on BEIR or MIRACL benchmarks?▼

To evaluate sparse retrieval models on BEIR and MIRACL, you need a model with encode_documents and encode_queries methods, a corpus with doc_ids, queries with query_ids, and qrels to compute IR metrics like nDCG@k and MAP.

What IR metrics are used for evaluating sparse retrieval systems?▼

Standard IR metrics for sparse retrieval evaluation include nDCG@k, Recall@k, MAP, and MRR to quantify ranking quality and relevance across standard benchmarks like BEIR, MIRACL, and mMARCO.

How does IDF weighting work in sparse information retrieval evaluation?▼

IDF weighting in sparse retrieval scales term importance during CSR matrix encoding, down-weighting frequent terms and up-weighting rare ones to improve retrieval accuracy before computing metrics like nDCG@k.

Do I need qrels to compute nDCG and Recall for IR benchmarks?▼

Yes, qrels are required to compute nDCG@k, Recall@k, MAP, and MRR during sparse retrieval evaluation. Query IDs in qrels map to relevant document IDs, enabling accurate relevance scoring.

Can I cache CSR encoded sparse matrices for reproducible IR evaluation?▼

Yes, you can cache CSR encoded sparse matrices and IDF-weighted retrieval results to ensure reproducible benchmark evaluation across BEIR, MIRACL, and mMARRO datasets.

What is the best way to compare sparse retrieval models across multilingual datasets?▼

Compare sparse retrieval models across multilingual datasets by evaluating on mMARCO and MIRACL benchmarks. Standardized metrics like nDCG@k and MRR provide clear guidance for analyzing performance differences.

Related Skills

Explore More Data & Analytics Skills→
SciMate-AISciMate-AI
1

geniml

Train genomic-interval embeddings from BED data for region similarity and clustering.

Official
Advanced
Avi977Avi977

evaluate-rag

Evaluate RAG retrieval and generation with Recall@k, MRR, and NDCG@k metrics.

Community
Advanced
ccfccf

embedding-strategies

Select and optimize embedding models for semantic search and RAG applications.

Community
Intermediate
ruiyangruiyiruiyangruiyi

pinecone

Create and query serverless vector indexes with hybrid dense and sparse matching.

Community
Intermediate