Evaluate AI-generated SQL code for correctness, formatting, performance, and reliability.
npx skills add https://github.com/sanjogkadayat-web/eval-framework --skill sql-eval
Please help me install this Agent Skill. Skill: sql-eval Source: https://github.com/sanjogkadayat-web/eval-framework/tree/main/sql-eval Command: npx skills add https://github.com/sanjogkadayat-web/eval-framework --skill sql-eval
This Skill solves the challenge of objectively evaluating AI coding assistants by providing a deterministic, reproducible framework to score SQL query quality across correctness, formatting, performance, and reliability.
Run the sql-eval skill to evaluate all tasks and generate a comparative scorecard for the three supported models.
High-intent search queries and answers about installing and using this skill.
You can benchmark AI SQL coding assistants by running the sql-eval skill, which scores model-generated SQL queries across correctness, formatting, performance, and reliability using pre-recorded synthetic banking datasets. It evaluates 60 analytics tasks to generate a structured comparative scorecard.
Evaluating AI-generated SQL queries involves scoring them across correctness, formatting, performance, and reliability dimensions. The framework specifically assesses complex analytics engineering tasks like CTEs, window functions, and incremental loads using pre-recorded synthetic banking datasets.
Yes, the SQL evaluation framework supports comparing AI models like Claude, ChatGPT, and Gemini. It generates a comparative scorecard that allows analytics teams to evaluate performance on complex banking SQL tasks before integrating them into production pipelines.
Yes, you can use duckdb and pandas to process SQL benchmarking evaluation results. The framework requires pandas, sqlfluff, and duckdb to process the evaluation outputs and generate structured scorecards from the model-generated SQL queries.
The best way to automatically review SQL code for logic errors and schema fragility is using a structured critic review mechanism. The framework applies a specialized SQL Critic persona to automatically generate qualitative feedback and identify style issues.
Yes, you need sqlfluff to evaluate SQL formatting and style during AI benchmarking. It is a required dependency for processing evaluation results, ensuring that model-generated SQL queries are scored accurately across the formatting dimension.
crossxwillQuery Parquet data with DuckDB and convert results to pandas DataFrames.
sanjogkadayat-webEvaluate AI-generated Python code pipelines across correctness, formatting, performance, and reliability.
LincyawRun Black, isort, Flake8, and pytest checks on Python projects.
48Nauts-OperatorImplement LLM evaluation strategies with automated metrics and human feedback.
