Evaluate language model compatibility with ASE interpreter primitives.
npx skills add https://github.com/rse/ase --skill ase-meta-compat
Please help me install this Agent Skill. Skill: ase-meta-compat Source: https://github.com/rse/ase/tree/main/plugin/skills/ase-meta-compat Command: npx skills add https://github.com/rse/ase --skill ase-meta-compat
This Skill provides a comprehensive test to evaluate the compatibility of a language model with the core interpreter primitives used by ASE, ensuring that the model can reliably execute ASE constructs.
Run the self-test to check the compatibility of your current model with ASE interpreter primitives by executing 'ase-meta-compat'.
High-intent search queries and answers about installing and using this skill.
Testing LLM compatibility with ASE interpreter primitives involves running a self-test that evaluates control flow, XML placeholders, regex matching, and arithmetic operations. The skill executes predefined probes and generates a detailed compatibility report with an overall percentage rating.
ASE compatibility testing evaluates four core interpreter primitives: control flow execution, XML placeholder handling, regex matching, and arithmetic operations. The model must execute predefined probes for each capability to determine if it can reliably process ASE constructs.
No external dependencies are required to run the ASE compatibility self-test. The skill operates independently as both the test definition and the system under test, executing its internal scripts to evaluate the language model without needing additional packages or tools.
The self-testing mechanism works by having the skill act as both the test definition and the system under test. It executes various predefined probes against the language model to evaluate its capabilities and reports the results for an overall compatibility rating.
The compatibility report includes a detailed breakdown of the compatibility rating by individual probes and an overall percentage. It identifies exactly how well the language model executes control flow, XML placeholders, regex matching, and arithmetic operations within the ASE toolset.
Language models fail ASE control flow and regex matching tests when they cannot reliably execute the expected interpreter primitives. Running the self-test identifies exactly which constructs cause failures, providing a detailed breakdown of compatibility issues across the evaluated probes.
jinzhezenggroupConfigure ASE MACE backend parameters for ASE workflows.
benjithompsonAnalyze a BlazeMeter execution to identify and prioritize root causes of failures.
mcclowesDefine and generate parsing expression grammars and parsers with Peggy.
Clay-HHKEvaluates code generation models using standard unit-test benchmarks like HumanEval and MBPP.
