skills.rest
wgzhao
wgzhaoCOMMUNITY
Explore all skills by this author →
GitHub

addax-project-knowledge

Guides development, configuration, and troubleshooting of Addax ETL jobs and plugins.

1.4k|342|Updated Jul 17, 2019
Search Tags:#etl#data synchronization#addax#job configuration#JAVA#Plugin Development#datax
One-click install
npx skills add https://github.com/wgzhao/Addax --skill addax-project-knowledge
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: addax-project-knowledge
Source: https://github.com/wgzhao/Addax
Command: npx skills add https://github.com/wgzhao/Addax --skill addax-project-knowledge

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Working on the Addax ETL codebase requires deep knowledge of its plugin architecture, Job JSON configuration model, Maven build conventions, and strict commit message rules. This Skill gives an AI assistant the domain context needed to write job configs, develop Reader/Writer plugins, debug sync tasks, and follow the project's contribution standards without reverse-engineering the repository.

Core Features & Use Cases

  • Job Configuration Guidance: Explains the Job JSON structure (settings, reader, writer, transformer), speed/errorLimit controls, dynamic parameters, and built-in time variables so you can author or review sync tasks correctly.
  • Plugin Development Support: Documents the Reader/Writer Job and Task interfaces, plugin.json conventions, Configuration path DSL, and split/schedule logic for building new data source plugins.
  • Operations & Debugging: Covers addax.sh CLI options, the HTTP server module for async job submission, remote JVM debugging, stats reporting, and dirty-data/error-limit troubleshooting.
  • Use Case: Ask the assistant to create a MySQL-to-PostgreSQL sync job with channel concurrency and error thresholds, or to scaffold a new writer plugin following the project's Conventional Commits and PR workflow.

Quick Start

Ask the assistant to write an Addax job JSON that syncs a MySQL table to PostgreSQL with 4 channels and a 1% error tolerance.

Frequently Asked Questions about addax-project-knowledge

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I write an Addax job configuration file?▼

Create a JSON file with a job object containing settings (speed.channel, errorLimit) and content (reader, writer, optional transformer). Run it with bin/addax.sh job/yourjob.json, and sample configs are available under core/src/main/job.

How do I develop a new Addax reader or writer plugin?▼

Create a Java module extending the Reader or Writer abstract class with inner Job and Task classes, plus a plugin.json whose name matches the plugin directory. The Job.split method divides work into Task configurations executed concurrently.

What is the difference between Addax and Alibaba DataX?▼

Addax is an actively maintained DataX fork adding YAML config support, password encryption, an HTTP server module, more transformers including Groovy, JDK 17 runtime with ZGC, and many new plugins such as Kafka, S3, Iceberg, and Paimon.

Does Addax support submitting jobs through an HTTP API?▼

Yes, the addax-server module exposes POST /api/submit to accept a job JSON and returns a taskId, and GET /api/status to query execution state. Concurrency is limited via the -p flag or ADDAX_SERVER_PARALLEL, defaulting to 30.

Why does my Addax job fail with too many error records?▼

Addax aborts a job when dirty records exceed errorLimit.record or errorLimit.percentage in job settings. Errors usually come from type conversion mismatches, so inspect the failing columns and adjust thresholds or add transformers.

What commit message format does the Addax project require?▼

Commits must follow Conventional Commits with a mandatory scope like plugin-mysqlwriter, and feat/fix/refactor commits need body lines prefixed with why:, what:, and impact:. CI validates messages via .github/scripts/lint-commit-msg.sh.

Related Skills

Explore More Software Engineering Skills→
awsaws
2.5k

ingesting-into-data-lake

Import data from S3, JDBC databases, Snowflake, BigQuery, and DynamoDB into AWS data lake tables.

Official
Advanced
mihaicrisan04mihaicrisan04

convex-cron-jobs

Schedule recurring Convex functions with interval or cron expressions.

Community
Intermediate
Datus-aiDatus-ai
1.5k

transfer-reconciliation

Verify post-transfer row count parity and target table accessibility.

Official
Intermediate
apacheapache
39

seatunnel-skill

Generate and debug Apache SeaTunnel job configurations in HOCON format.

Official
Intermediate