input-binary-detection

Detect binary and non-UTF-8 input streams using magic numbers and control character analysis.

836|66|Updated Feb 3, 2025
One-click install
npx skills add https://github.com/kreuzberg-dev/html-to-markdown --skill input-binary-detection
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: input-binary-detection
Source: https://github.com/kreuzberg-dev/html-to-markdown/tree/main/.ai-rulez/domains/html-parsing/skills/input-binary-detection
Command: npx skills add https://github.com/kreuzberg-dev/html-to-markdown --skill input-binary-detection

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill prevents errors and potential security issues by detecting and rejecting binary or non-UTF-8 input before it can be processed by downstream systems.

Core Features & Use Cases

  • Early Detection: Identifies binary data using magic number signatures and heuristic analysis.
  • Encoding Validation: Checks for common binary indicators like null bytes and control characters.
  • Use Case: Ensure that a web service expecting text-based HTML input does not crash or expose vulnerabilities when a user accidentally uploads a binary file like an image or an executable.

Quick Start

Run the input-binary-detection skill to validate that the provided input is valid UTF-8 text.

Frequently Asked Questions about input-binary-detection

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I detect binary data in an input stream to prevent parsing errors?

Detect binary data in an input stream by applying magic number signatures and heuristic analysis to prevent downstream parsing errors. This validates data integrity early by identifying common binary indicators before processing begins.

Why does my data ingestion pipeline crash when receiving non-UTF-8 input?

Data ingestion pipelines crash with non-UTF-8 input because downstream text processors cannot parse invalid byte sequences. Validating UTF-8 encoding and rejecting binary streams early prevents these unexpected parsing failures and security risks.

What is the best way to validate user input for binary content and null bytes?

The best way to validate user input for binary content is checking for null bytes, control character ratios, and magic number signatures. This robust validation detects non-text payloads to prevent vulnerabilities in web services expecting text input.

Does input validation for binary detection support UTF-16 heuristics?

Yes, robust input validation for binary detection supports UTF-16 heuristics alongside magic number detection and control character ratio analysis. These combined checks accurately identify non-UTF-8 streams within data ingestion pipelines.

When should I use magic number detection for input validation in a web service?

Use magic number detection for input validation when a web service expects text-based input like HTML but risks receiving binary files. This early detection prevents crashes and security issues from accidental binary file uploads.

Can I check control character ratios to identify non-text data in ingestion pipelines?

Yes, you can check control character ratios to identify non-text data in ingestion pipelines. Analyzing the frequency of control characters alongside null byte detection provides robust validation against binary input streams.