What problem does it solve?
DataFusion-Python shares Rust objects with extension libraries through PyCapsules via a family of __datafusion_*__ dunder getters, and deviating from the established convention causes dangling weak references, invisible UDF registrations, and breaking API changes across extension libraries.
Core Features & Use Cases
- Protocol Conventions: Defines the seven rules governing
__datafusion_*__ capsule getters, including session-passing signatures and the ban on constructing SessionContext inside extension libraries.
- Helper Locations: Points to the capsule extraction helpers in
crates/util/src/lib.rs such as ffi_logical_codec_from_pycapsule and ffi_task_context_provider_from_pycapsule.
- Breaking Change Procedure: Specifies the upgrade-guide entries,
api change labels, and Protocol type-hint updates required when a getter signature changes.
- Use Case: When adding a new FFI table provider getter, run the grep enumeration first, match the existing getter signature shape, and add a helper in
crates/util/src/lib.rs rather than hand-rolling capsule extraction.
Quick Start
Review the FFI capsule protocol rules before I add a new __datafusion_table_provider__ getter to the extension example.