Slice long text inputs into chunks for map/reduce pipelines.
npx skills add https://github.com/aufrank/agent-skills --skill slicing-long-contexts
Please help me install this Agent Skill. Skill: slicing-long-contexts Source: https://github.com/aufrank/agent-skills/tree/main/skills/slicing-long-contexts Command: npx skills add https://github.com/aufrank/agent-skills --skill slicing-long-contexts
This Skill tackles the challenge of processing extremely long or dense text inputs that exceed the context window limits of language models, enabling detailed analysis and summarization of large documents, codebases, or logs.
Use the slicing-long-contexts skill to summarize the provided document 'research_paper.pdf' into a concise bulleted list.
High-intent search queries and answers about installing and using this skill.
To process large texts that exceed context window limits, you can use a map/reduce pipeline to slice inputs into smaller chunks, process each with sub-LM calls, and aggregate the results. This makes extensive codebase analysis and log processing reproducible.
The best way to summarize large documents using map reduce is to slice the text into manageable sections, have a sub-LM summarize each section, and then combine those summaries into a concise overview. This handles dense inputs effectively.
Yes, you can analyze an entire codebase or terminal logs by slicing the dense text inputs into smaller chunks. The skill supports heading-based, marker-based, and fixed-size chunking strategies to process the data recursively.
Yes, the map reduce approach supports network access for LLM providers. It also includes options for retries and verification during the recursive sub-LM calls to ensure reliable aggregation of the sliced text chunks.
Fixed-size chunking for text processing may split related context or logical sections across multiple slices, potentially degrading sub-LM reasoning quality. Alternatives like heading-based or marker-based slicing help preserve structural boundaries for better results.
TabithaFannyVerify research paper claims against raw result files with a fresh reviewer.
Rycen7822Fetches AI-generated arXiv paper overviews from alphaxiv.org via URLs or IDs.
evanfang0054Split large documents into smaller chunks using LangChain.js text splitters.
christhz666Store and retrieve context data using the filesystem.
