skills.rest
cas-bigdatalab
cas-bigdatalabOFFICIAL
Explore all skills by this author →
WebsiteGitHub

clean_links_mapper

Removes or replaces HTTP, HTTPS, and FTP links in text content using py-data-juicer.

541|171|Updated May 3, 2018
Search Tags:#document processing#text processing#text cleaning#link removal#URL cleaning
One-click install
npx skills add https://github.com/cas-bigdatalab/piflow --skill clean-links-mapper
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: clean_links_mapper
Source: https://github.com/cas-bigdatalab/piflow/tree/main/workspace/skills/clean_links_mapper
Command: npx skills add https://github.com/cas-bigdatalab/piflow --skill clean-links-mapper

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires py-data-juicer, and includes scripts (resource) components.

What problem does it solve?

This Skill solves the issue of unwanted or unneeded links cluttering text, by providing an efficient way to remove or replace them from various documents.

Core Features & Use Cases

  • Link Removal: Automatically removes HTTP, HTTPS, and FTP links from text content.
  • Link Replacement: Replaces links with specified text if desired.
  • Use Case: Ideal for cleaning up user-generated content, such as comments, reviews, or documents where URLs are irrelevant or not desired.

Quick Start

Run the clean_links_mapper skill on your text with the --input_path and --output_path parameters.

Frequently Asked Questions about clean_links_mapper

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I remove HTTP, HTTPS, and FTP links from text documents?▼

To remove HTTP, HTTPS, and FTP links from text documents, use a text processing tool to automatically identify and delete or replace URLs, sanitizing user-generated content for document preparation.

Can I replace URLs with custom text instead of just deleting them?▼

Yes, URL cleaning tools can replace HTTP, HTTPS, and FTP links with specified text instead of deleting them, maintaining document readability while removing unwanted web addresses from user-generated content.

Does py-data-juicer support cleaning user-generated content like comments and reviews?▼

Yes, py-data-juicer supports cleaning user-generated content like comments and reviews by removing or replacing HTTP, HTTPS, and FTP links, making it suitable for text sanitization where URLs are irrelevant.

What is the best way to sanitize text by stripping out unwanted web addresses?▼

The best way to sanitize text by stripping unwanted web addresses is using an automated text cleaning script that targets and removes HTTP, HTTPS, and FTP links, efficiently preparing documents by eliminating URL clutter.

Do I need py-data-juicer to run text cleaning operations for link removal?▼

Yes, py-data-juicer is a required dependency for processing and executing the text cleaning scripts that remove or replace HTTP, HTTPS, and FTP links from documents during link removal operations.

How do I start processing text to clean links using a command line script?▼

To start processing text and clean links, run the text cleaning script with --input_path and --output_path parameters, executing the HTTP, HTTPS, and FTP link removal or replacement operations on your specified documents.

Related Skills

Explore More Content & Communication Skills→
cas-bigdatalabcas-bigdatalab
541

parquet_formatter

Convert Parquet files to JSONL format with configurable output paths.

Official
Intermediate
zinohomezinohome

pdf-extract

Extract text from PDF documents using the pdftotext utility.

Community
Basic
cas-bigdatalabcas-bigdatalab
541

clean_html_mapper

Convert HTML content to plain text with asterisk list formatting.

Official
Basic
sbussosbusso
189

add-pdf-reader

Extracts text from PDFs via pdftotext and pdfinfo for downstream processing.

Community
Advanced