hook-deduplication-guide

Content-based deduplication uses SHA256 hashing to identify and filter duplicate insights stored across hook sessions. Apply hash rotation with a persistent state store to track previously seen content and preserve only unique entries, reducing redundant outputs.

Follow a three-phase migration: normalize input data to minimize false positives, compute SHA256 hashes for each insight, and store hashes in versioned backups within CLAUDE.md. Test validation across multiple sessions to ensure deduplication works without data loss.

Yes. Deduplication applies to refactoring scenarios where insights accumulate across sessions and testing phases where multiple unique insights must be preserved. It works across both single-session and multi-session hook development contexts.

Yes. A persistent state store with hash rotation is required to track hashes across sessions and maintain versioned backups. This prevents reprocessing duplicate content while preserving state for recovery and audit purposes.

Normalize input by standardizing formatting, whitespace, and encoding before hashing. This minimizes false positives where identical content appears in slightly different formats, ensuring only genuinely unique insights are flagged as new.