audit-comet-expression

Audit Comet expressions for correctness and test coverage across Spark versions.

1.2k|344|Updated Jan 15, 2024
One-click install
npx skills add https://github.com/apache/datafusion-comet --skill audit-comet-expression
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: audit-comet-expression
Source: https://github.com/apache/datafusion-comet/tree/main/.claude/skills/audit-comet-expression
Command: npx skills add https://github.com/apache/datafusion-comet --skill audit-comet-expression

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This audit helps teams verify Comet expressions against Spark reference implementations, ensuring correctness, coverage, and alignment across versions.

Core Features & Use Cases

  • Spark version comparison across 3.4.3, 3.5.8, and 4.0.1
  • Compare Comet and DataFusion implementations to identify coverage gaps
  • Gap analysis and test recommendations
  • Optional: offer to implement missing tests

Quick Start

Audit the expression by running the audit against Spark versions 3.4.3, 3.5.8, and 4.0.1, then review results and suggested tests.

Frequently Asked Questions about audit-comet-expression

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I audit Comet expressions for correctness across multiple Spark versions?

You can audit Comet expressions by running a gap analysis against Spark versions 3.4.3, 3.5.8, and 4.0.1 to verify correctness, test coverage, and identify missing shims. The audit enumerates version differences and outputs concrete test recommendations.

What is the best way to compare Comet and DataFusion implementations for coverage gaps?

Comparing Comet and DataFusion implementations requires a gap analysis to identify coverage gaps and testing needs. This evaluates expression correctness against Spark reference implementations and outputs concrete test recommendations and implementation steps.

Does the Comet expression audit support Spark 3.4.3, 3.5.8, and 4.0.1?

Yes, the Comet expression audit supports Spark environments across versions 3.4.3, 3.5.8, and 4.0.1. It enumerates Spark version differences and documents existing Comet shims to ensure expression alignment across these specific releases.

How do I identify missing test coverage for Comet expressions in Spark?

To identify missing test coverage for Comet expressions, run the audit to compare implementations against Spark references. It documents existing Comet shims and outputs concrete test recommendations for uncovered expressions.

Can I implement missing tests after auditing Comet expressions?

Yes, after auditing Comet expressions and reviewing the identified coverage gaps, the process optionally offers to implement missing tests. It provides concrete test recommendations and implementation steps to ensure correctness across Spark versions.