skills.rest
EthanSK
EthanSKCOMMUNITY
Explore all skills by this author →
GitHub

repost-dedup

Detect duplicate social media posts using string and cross-platform checks.

Updated Mar 21, 2026
Search Tags:#social media#web scraping#deduplication#post management#content filtering
One-click install
npx skills add https://github.com/EthanSK/Repost-with-agent --skill repost-dedup
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: repost-dedup
Source: https://github.com/EthanSK/Repost-with-agent/tree/main/skills/repost-dedup
Command: npx skills add https://github.com/EthanSK/Repost-with-agent --skill repost-dedup

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps identify and avoid duplicate reposts on social media platforms, ensuring unique content sharing.

Core Features & Use Cases

  • Exact and Fuzzy Deduplication: Checks if a post has already been shared based on source item ID and string similarity.
  • Multi-layer Dedupe Pipeline: Combines local ledger, global ledger, and remote destination scrapes to ensure comprehensive deduplication.
  • Use Case: A social media manager wants to prevent reposting the same article on multiple platforms, saving time and avoiding spammy content.

Quick Start

Run the repost-dedup skill to evaluate if a given post should be blocked from reposting due to duplication.

Frequently Asked Questions about repost-dedup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I prevent duplicate social media posts across multiple platforms?▼

Preventing duplicate social media posts requires cross-platform checks using source item IDs and string similarity. This Skill compares content against local ledgers, global ledgers, and remote destination scrapes to block redundant reposts.

What is the best way to detect duplicate reposts before publishing content?▼

Detecting duplicate reposts is best handled by a multi-layer deduplication pipeline. This Skill evaluates posts through exact and fuzzy string matching, comparing against local, global, and remote data sources to catch redundancy.

Does web scraping work for cross-platform post deduplication in real-time?▼

Web scraping enables cross-platform post deduplication by verifying remote destination platforms for existing content. The Skill scrapes target platforms and processes JSON in-memory to make real-time deduplication decisions for content distribution.

Can I use this for content filtering when managing posts across different social media sites?▼

You can use this for content filtering to manage posts across multiple social media sites. It handles local, global, and remote verification scenarios to optimize content distribution workflows and maintain platform compliance.

What are the limitations of string-based deduplication for social media content?▼

String-based deduplication for social media content relies on exact source item IDs and fuzzy string similarity matching. Its effectiveness depends on consistent local and global ledgers plus accurate remote destination scrapes for reliable verification.

Related Skills

Explore More Content & Communication Skills→
masharrattmasharratt
14

CFN Marketing Social Publishing Skill

Automate cross-platform social media publishing via n8n workflows.

Community
Intermediate
clenisaclenisa

publish-social-posts

Publish Strapi social post drafts to platforms via n8n workflows.

Community
Advanced
framerslabframerslab

social-automation

Automate social media posting, analytics aggregation, and batch scheduling across platforms.

Official
Advanced
winsorllcwinsorllc

web-fetch

Fetch and parse web content from specified URLs.

Community
Basic