content-hash-cache-pattern

You can cache file processing results by generating a SHA-256 hash of the file content and storing the extracted output as a `{hash}.json` file. This allows subsequent runs to perform an O(1) lookup and skip expensive parsing if the hash already exists.

Yes, content hashing handles renames and moves gracefully. Because the cache key is derived from the SHA-256 hash of the file bytes rather than the file path, renamed files will still result in cache hits, while any content changes trigger automatic cache invalidation.

The best way is to use a service-layer wrapper that checks a file-based cache before calling your pure extraction function. It checks for a `{hash}.json` entry, returns the cached result if it exists, and writes new results to the cache after processing.

You keep the processing function pure by wrapping it with a separate service layer. This service handles the cache checking, file extraction, and cache writing, ensuring the underlying processing function remains focused solely on data extraction.

Corrupted cache files are treated as cache misses. This graceful corruption handling ensures the pipeline safely re-processes the original file and overwrites the damaged cache entry rather than failing or returning bad data.

You compute hashes for large files using chunked hashing. By streaming the file contents through the SHA-256 algorithm in chunks, you can generate the content hash without loading the entire file into memory.