sayt2

Index and search Python dictionaries with ngram substring and BM25 queries.

Updated Apr 25, 2022
One-click install
npx skills add https://github.com/MacHu-GWU/afwf_github-project --skill sayt2
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: sayt2
Source: https://github.com/MacHu-GWU/afwf_github-project/tree/main/.claude/skills/sayt2
Command: npx skills add https://github.com/MacHu-GWU/afwf_github-project --skill sayt2

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

sayt2 provides a Python library to build a full-text search index from a list of dictionaries and query it with substring matching (ngram), BM25 full-text search, fuzzy search, range queries, sorting, and more -- all through a single DataSet object.

Core Features & Use Cases

  • Full-text search with BM25, substring search via ngram, and fuzzy search across stored fields.
  • Sorting, range queries, and caching via diskcache, with validation via pydantic and a Rust-backed Tantivy engine.
  • Use cases include building search-as-you-type interfaces and fast data retrieval for Python dictionaries in various applications.

Quick Start

Create a DataSet with a fields definition and a downloader, then call search to see results.

Frequently Asked Questions about sayt2

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I implement search-as-you-type in Python using dictionaries?

To build search-as-you-type in Python, load a list of dictionaries into a DataSet object and query using ngram-based substring matching and BM25 ranking to retrieve fast, ranked matches.

What is the best way to do full-text search and substring matching on Python dictionaries?

Full-text search and substring matching on Python dictionaries is best handled by indexing the data with a Rust-backed Tantivy engine, enabling BM25 ranking and ngram fields for partial text matches.

How does ngram substring search work with BM25 ranking?

Ngram substring search works by breaking text into character sequences for partial matching, while BM25 calculates relevance scores to rank the overall full-text search results returned by the query.

Can I use Tantivy with Python for fuzzy search and range queries?

Yes, you can use Tantivy with Python through specific bindings to execute fuzzy search and range queries, while leveraging pydantic validation and diskcache-based caching for the search results.

Do I need external dependencies to run a local search index with caching?

Running a local search index with caching requires no external dependencies, as the environment integrates a Rust-backed Tantivy engine and diskcache to handle indexing, pydantic validation, and storage automatically.