etl-dwd-to-dws

Trigger the QCM ETL 5-phase pipeline from biz_dwd to biz_dws via PostgreSQL.

Updated Mar 16, 2026
One-click install
npx skills add https://github.com/MJ-AgentLab/mj-agentlab-marketplace --skill etl-dwd-to-dws
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: etl-dwd-to-dws
Source: https://github.com/MJ-AgentLab/mj-agentlab-marketplace/tree/main/plugins/mj-ops/skills/mj-etl-dwd-to-dws
Command: npx skills add https://github.com/MJ-AgentLab/mj-agentlab-marketplace --skill etl-dwd-to-dws

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This skill bypasses the standard 5-minute polling wait for the QCM ETL pipeline, allowing for immediate updates to DWS metrics after DWD data is ready, or when DWS tables are empty or metrics are stale.

Core Features & Use Cases

  • Immediate ETL Triggering: Manually initiate the QCM ETL process to bypass scheduled waits.
  • DWS Metric Updates: Ensure DWS metrics are current when DWD data is updated.
  • Troubleshooting Stale Data: Resolve issues where DWS tables appear empty or metrics haven't refreshed.
  • Use Case: After a significant update to DWD data, you need to see the reflected DWS metrics immediately for a business review. Instead of waiting up to 10 minutes, you use this skill to trigger the ETL pipeline instantly.

Quick Start

Manually trigger the QCM ETL pipeline by running the command docker exec mj-system-postgres psql -U admin -d mj_system_db -c "SELECT * FROM biz_dws.run_qcm_etl_if_ready(0);".

Frequently Asked Questions about etl-dwd-to-dws

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I manually trigger an ETL pipeline from DWD to DWS?

To manually trigger an ETL pipeline from DWD to DWS, execute the QCM ETL function directly via Docker using the `docker exec mj-system-postgres psql` command to run `SELECT * FROM biz_dws.run_qcm_etl_if_ready(0);`.

Why are my DWS metrics not updating after DWD data is loaded?

DWS metrics may not be updating because the standard QCM ETL pipeline relies on a 5-minute polling interval. If DWS tables are empty or stale, manually triggering the pipeline bypasses this scheduled wait to refresh metrics immediately.

What is the best way to bypass the scheduled wait time for DWS metric updates?

The best way to bypass the scheduled wait for DWS metric updates is to manually trigger the QCM ETL 5-Phase parallel pipeline, which bypasses the silent windows and forces an immediate refresh of DWS tables from biz_dwd.

Do I need Docker to execute write operations for the DWD to DWS data processing pipeline?

Yes, you need Docker to execute write operations for the DWD to DWS data processing pipeline. Direct PostgreSQL execution via Docker is required for writes, while read-only checks can be performed using the `mcp__postgres-dev__query` tool.

Can I run read-only checks on DWS tables without triggering the full ETL pipeline?

Yes, you can run read-only checks on DWS tables without triggering the full ETL pipeline by using the `mcp__postgres-dev__query` tool to verify data availability or check if metrics are empty before deciding to manually execute the ETL process.