content-hash-cache

Cache expensive file-processing results by computing SHA-256 content hashes to identify files. If a file's content changes, the cache entry automatically invalidates, preventing unnecessary recomputation in repeated data pipeline runs.

Content-based caching using SHA-256 hashes identifies file content uniquely instead of relying on file paths. This ensures caches remain valid even if files are moved or renamed, and automatically invalidates when the actual file content changes.

Provide a deterministic processing function and a local .cache directory. The system computes each file's SHA-256 hash, stores per-file results in the cache, and retrieves them on subsequent runs to skip expensive OCR or image analysis steps.

No index files are needed. The caching mechanism relies solely on SHA-256 content hashes stored in a local .cache directory, creating a simple and resilient cache that automatically invalidates entries when underlying file content changes.

Use content hash caching when processing files like PDFs or images where content changes should invalidate caches. It prevents stale results from renamed or moved files and ensures expensive computations only run when actual content changes occur.

This caching approach requires a deterministic processing function to ensure consistent results for identical file contents. It also requires a local .cache directory, meaning cache storage is tied to the local filesystem rather than distributed storage.