skills.rest
freedom909
freedom909COMMUNITY
Explore all skills by this author →
GitHub

content-hash-cache-pattern

Cache file processing results using SHA-256 content hashes as keys.

Updated Mar 18, 2026
Search Tags:#caching#python#file processing#cache invalidation#pdf parsing#sha-256#content hash
One-click install
npx skills add https://github.com/freedom909/real-estate-saas --skill content-hash-cache-pattern-freedom909
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: content-hash-cache-pattern
Source: https://github.com/freedom909/real-estate-saas/tree/main/.trae/skills/content-hash-cache-pattern
Command: npx skills add https://github.com/freedom909/real-estate-saas --skill content-hash-cache-pattern-freedom909

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Expensive file processing operations like PDF parsing, OCR, and text extraction often run repeatedly on the same files, wasting compute time. Path-based caching breaks when files are moved or renamed, and stale results appear when content changes. ## Core Features & Use Cases - Content-Hash Cache Keys: Uses SHA-256 of file contents (chunked for large files) so cache survives renames and auto-invalidates on content change. - File-Based Storage: Stores each entry as {hash}.json for O(1) lookup with no index file, treating corruption as a graceful cache miss. - Service Layer Separation: Wraps pure processing functions with a cache layer, preserving single-responsibility design and supporting --cache/--no-cache CLI options. - Use Case: A batch pipeline re-processes hundreds of PDFs across runs; with this pattern, unchanged files hit the cache instantly while edited files are re-extracted automatically. ## Quick Start Add content-hash caching to my PDF text extraction function so repeated runs skip unchanged files.

Frequently Asked Questions about content-hash-cache-pattern

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I cache file processing results in Python?▼

Compute a SHA-256 hash of the file contents and use it as the cache key, storing results as `{hash}.json` files in a cache directory. Wrap your pure processing function in a service layer that checks the cache before running extraction.

Why use content hash instead of file path for caching?▼

Content-hash caching survives file moves and renames because the key depends on file contents, not location. It also auto-invalidates when content changes, eliminating stale cache entries without any manual index management.

How to hash large files without loading them into memory?▼

Read the file in chunks (e.g., 64KB) and update the SHA-256 hasher incrementally with each chunk. This keeps memory usage constant regardless of file size while producing the same content hash.

What happens when a cache file is corrupted?▼

The pattern treats JSON decode errors, missing keys, and invalid values as a cache miss, returning None. The file is then re-processed and the cache entry rewritten, so corruption never crashes the pipeline.

When should I not use content-hash caching?▼

Avoid it for data that must always be fresh, such as real-time feeds, or when results depend on parameters beyond file content like different extraction configs. Extremely large cache entries may also be better handled with streaming.

Related Skills

Explore More Software Engineering Skills→
cat-xierluocat-xierluo
557

piclist-upload

Uploads local images from Markdown files to cloud storage via PicList HTTP Server.

Community
Intermediate
dotlab-hqdotlab-hq

langchain-rag

Build RAG pipelines that ingest, chunk, embed, and retrieve documents for LLMs.

Official
Intermediate
qiuhaomemqiuhaomem
1

prompt-cache-strategy

Maximize prefix cache hits by enforcing stable prompts and batch requests.

Official
Advanced
mthang1801mthang1801

redis

Provide Redis-based caching and distributed locking for distributed services.

Community
Advanced