content-hash-cache-pattern

Compute a SHA-256 hash of the file contents and use it as the cache key, storing results as {hash}.json files. Check the cache before processing and write results after a miss, keeping the processing function itself pure.

Content-hash caching survives file moves and renames because the key depends on file contents, not location. It also auto-invalidates when content changes, eliminating stale cache entries without an index file.

Read the file in chunks (e.g., 64KB) and feed each chunk to hashlib.sha256() incrementally. This keeps memory usage constant regardless of file size while producing the same digest.

Treat corrupted cache entries as cache misses by catching JSONDecodeError, ValueError, and KeyError during reads. The file is then reprocessed and the cache rewritten, so corruption never crashes the pipeline.

Avoid it for data that must always be fresh, extremely large cache entries better handled by streaming, or results that depend on parameters beyond file content such as different extraction configurations.