skills.rest
JoyJoin-Tech-Limited
JoyJoin-Tech-LimitedOFFICIAL
Explore all skills by this author →
GitHub

content-safety-abuse-detection

Implements server-side content filtering, violation escalation, and admin moderation workflows for the JoyJoin platform.

Updated Dec 24, 2025
Search Tags:#rate limiting#express#content moderation#audit log#abuse detection#profanity filter#user ban
One-click install
npx skills add https://github.com/JoyJoin-Tech-Limited/JoyJoin_app_v0.1 --skill content-safety-abuse-detection-joyjoin-tech-limited
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: content-safety-abuse-detection
Source: https://github.com/JoyJoin-Tech-Limited/JoyJoin_app_v0.1/tree/main/.github/skills/content-safety-abuse-detection
Command: npx skills add https://github.com/JoyJoin-Tech-Limited/JoyJoin_app_v0.1 --skill content-safety-abuse-detection-joyjoin-tech-limited

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? User-generated text on a social platform can contain profanity, spam, or severe policy violations, and abuse must be detected, escalated, and audited consistently across many server routes without duplicating fragile inline logic. ## Core Features & Use Cases - Two-tier content gating: validate user-input fields with validateContentSafe (deterministic exact-substring filter) or validateContentSafeAsync (adds a fail-open WeChat msgSecCheck Tier-1 call under a 250ms budget), with every block logged to content_filter_logs. - Violation escalation and rate limiting: recordViolation in abuseDetection.ts drives the warning → 1-hour AI freeze → 24-hour freeze → permanent ban ladder, while createRateLimiter protects AI, auth, payment, and webhook endpoints. - Admin moderation and review queue: ban/unban routes emit USER_BANNED/USER_UNBANNED audit entries, and PATCH /api/admin/content-filter/logs/:id supports an idempotent review workflow with missFlag false-positive feedback into keyword-list curation. - Use Case: When adding a new free-text field like an event feedback comment, wire it through validateContentSafeAsync before persistence so severe violations are blocked unconditionally, warning-tier hits respect the contentModerationSevereFailClosedEnabled flag, and the route keeps its own recordViolation call. ## Quick Start Ask the AI to gate a new user-input field such as a profile bio through validateContentSafeAsync before saving it, following the content-safety skill's escalation and audit rules.

Frequently Asked Questions about content-safety-abuse-detection

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add content filtering to a new user-input field in Express?▼

Call validateContentSafeAsync (or validateContentSafe for deterministic-only checks) from lib/contentSafety.ts before persisting the value, and return contentViolationResponse if a violation is found. The helper logs blocks to content_filter_logs automatically; keep the route's own recordViolation call for sync-returned violations.

How do I add a new sensitive-word category to a profanity filter?▼

Extend sensitiveWordLists in contentFilter.ts with a new ViolationType entry, assign warning or severe severity, and add the matching message in the messages record. Severity must not be hard-coded in route handlers.

How does the violation escalation ladder work for user bans?▼

recordViolation in abuseDetection.ts is the single enforcement path: warnings add +1 and severe violations add +2 to users.violationCount, progressing from a 1-hour AI freeze to a 24-hour freeze to a permanent ban. Thresholds live in abuseDetection.ts constants, not inline in routes.

Does the in-memory rate limiter work across multiple server instances?▼

No. rateLimiter.ts stores state in a local Map, so limits are not shared across horizontally scaled instances. For attacker-facing endpoints in production, migrate to Redis or gateway-level limiting and leave a TODO noting the limitation.

Why is a user seeing the AI feature frozen message?▼

Check users.aiFrozenUntil in the database, then trace recordViolation in abuseDetection.ts to see which threshold was crossed. The freeze is time-bounded and results from accumulated warning or severe violations.

When should I not use this content moderation skill?▼

Use other skills for auth session policy and route gating, admin RBAC matrices and audit-log obligations, pure observability or metrics work, and generic code review without a safety focus. This skill covers filtering, escalation, rate limiting, and moderation workflows only.

Related Skills

Explore More Software Engineering Skills→
taishan1994taishan1994
30

schema-exploration

Identify and describe relational database schemas with tables, columns, keys, and relationships.

Community
Intermediate
kody-wkody-w
13

antigaslighter

Analyze GitHub Actions logs and state files to verify true workflow outcomes.

Community
Advanced
PostHogPostHog
713

investigating-metric-anomalies

Diagnoses server metric anomalies by correlating metrics, logs, and traces to root causes.

Official
Advanced
global-osglobal-os

vercel-connect

Manage OAuth token acquisition for Slack, GitHub, and MCP servers via Vercel Connect.

Official
Advanced