libindex

Implement storage-backed indexing for JSONL data with batched writes.

1|1|Updated Aug 7, 2025
One-click install
npx skills add https://github.com/copilot-ld/copilot-ld --skill libindex
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: libindex
Source: https://github.com/copilot-ld/copilot-ld/tree/main/packages/libindex
Command: npx skills add https://github.com/copilot-ld/copilot-ld --skill libindex

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires @copilot-ld/libtype.

What problem does it solve?

Base index classes simplify building storage-backed indexing for JSONL data, providing reliable in-memory state and durable disk writes.

Core Features & Use Cases

  • IndexBase offers loadData, add, get, has, and queryItems with configurable filtering.
  • BufferedIndex adds high-volume writes with a configurable flush interval and automatic batching, including shutdown support.
  • Integration-ready: exportable base classes for VectorIndex, TraceIndex, and domain-specific indexes, with test coverage to ensure correctness.

Quick Start

Create a BufferedIndex with a storage backend and an index key, then add items and periodically flush to persist.

Frequently Asked Questions about libindex

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I batch write JSONL data to disk with in-memory lookups?

Batch write JSONL data by using a BufferedIndex, which provides configurable flush intervals and automatic batching for durable persistence while maintaining fast in-memory state for immediate lookups.

What is the best way to index high-volume JSONL records without blocking reads?

Indexing high-volume JSONL records without blocking reads is handled by BufferedIndex, which batches writes in the background with a configurable flush interval and shutdown support, keeping queries responsive.

Can I filter and query items from an in-memory JSONL index?

Yes, you can filter and query items from an in-memory JSONL index using the queryItems method, which supports configurable filtering alongside loadData, add, get, and has operations.

Does this JSONL indexing approach support custom domain-specific indexes?

Yes, the JSONL indexing approach supports custom domain-specific indexes by providing exportable base classes like IndexBase, allowing you to build specialized indexes such as VectorIndex or TraceIndex.

When do I need batched persistence for JSONL storage?

You need batched persistence for JSONL storage when handling high-volume writes, where BufferedIndex automatically batches items and periodically flushes them to disk to ensure durable writes without performance degradation.