di-agent-query-optimization

Rewrite Substrait plans into compact full-pushdown SQL for DataStage execution.

3|Updated May 1, 2026
One-click install
npx skills add https://github.com/IBM/ibm-watsonx-data-integration-skills --skill di-agent-query-optimization
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: di-agent-query-optimization
Source: https://github.com/IBM/ibm-watsonx-data-integration-skills/tree/main/agent/skills/di-agent-query-optimization
Command: npx skills add https://github.com/IBM/ibm-watsonx-data-integration-skills --skill di-agent-query-optimization

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill rewrites Substrait plans into a compact full-pushdown form so data integration agents can execute more work inside the database instead of splitting logic across systems.

Core Features & Use Cases

  • Pushdown Classification: Distinguishes full SQL pushdown from partial pushdown and stops when the plan cannot be fully preserved.
  • Plan-to-SQL Validation: Converts the original plan to SQL, checks that every relation and read is preserved, and confirms same-connection eligibility.
  • Optimized Plan Generation: Produces a DataStage-ready Substrait structure with canonical output names, connection metadata, and a trailing SQL SELECT for downstream execution.
  • Use Case: A user provides a natural-language data flow request or an existing Substrait plan, and the Skill returns the optimized full-pushdown version that can feed the next DataStage workflow step.

Quick Start

Ask the skill to optimize the attached Substrait plan for full SQL pushdown and return the rewritten plan.

Frequently Asked Questions about di-agent-query-optimization

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I convert a Substrait plan to SQL pushdown for DataStage?

SQL pushdown optimization moves data transformation logic directly into the source database via a single SQL statement instead of splitting processing across systems. This Skill rewrites Substrait plans into a compact full-pushdown form so DataStage can execute more work inside the database.

How do I know if my Substrait plan supports full pushdown or partial pushdown?

Substrait plan pushdown classification checks whether every relation and read can be preserved in a single SQL statement with same-connection eligibility. If the plan cannot be fully preserved, the Skill stops rewriting and prevents partial pushdown execution.

What do I need to provide to optimize a Substrait plan for SQL pushdown?

You need to provide valid Substrait JSON, explicit connection metadata, and canonical output aliases. The Skill uses these inputs to validate same-connection eligibility and generate an optimized plan for downstream DataStage execution.

Does the optimized Substrait plan include connection metadata for DataStage?

Yes, the optimized Substrait plan includes explicit connection metadata, canonical output names, and a trailing SQL SELECT statement. This structure ensures the rewritten full-pushdown plan is ready for immediate downstream DataStage workflow execution.

Why does SQL pushdown optimization stop when a relation cannot be fully preserved?

SQL pushdown optimization stops when a relation cannot be fully preserved to prevent silent data loss or logic splitting across systems. This constraint ensures the rewritten Substrait plan only returns a compact structure when full same-connection SQL execution is verified.