Diagnose Apache Arrow memory leak errors in XTDB tests and nodes.
npx skills add https://github.com/xtdb/xtdb --skill arrow-leaks
Please help me install this Agent Skill. Skill: arrow-leaks Source: https://github.com/xtdb/xtdb/tree/main/.claude/skills/arrow-leaks Command: npx skills add https://github.com/xtdb/xtdb --skill arrow-leaks
Arrow "Memory was leaked" errors report the close-time detection stack rather than the actual allocation site, making leaks hard to trace. This Skill explains how to enable Arrow's debug allocator in XTDB's Gradle build, identify the failure-path cleanup gaps that cause most leaks, and avoid misattribution from property-test shrinking.
-D flags and JAVA_TOOL_OPTIONS fail, and how to flip arrow.memory.debug.allocator in build.gradle.kts defaultJvmArgs to capture allocation stack traces.openSlice results handed off without .use or closeOnCatch on throwing code paths.-Piterations and confirm single-op reproductions with a plain deftest, avoiding the shrinker's misleading :smallest case.leader-log-processor allocator; use this Skill to enable the debug allocator, locate the orphaned buffer in the transaction indexing path, and write a minimal reproducing test.Ask the assistant to help track down the Arrow memory leak reported by your failing XTDB test, starting by enabling the debug allocator in build.gradle.kts.
High-intent search queries and answers about installing and using this skill.
Enable Arrow's debug allocator so the leak error gains an event log section with ALLOCATE stack traces pointing to the allocation site. In XTDB, flip `arrow.memory.debug.allocator` to true in build.gradle.kts defaultJvmArgs, since command-line -D flags and JAVA_TOOL_OPTIONS are overridden.
The error carries the close-time detection stack, not the allocation site where the buffer was created. Only the debug allocator records per-allocation stacks, which is why it must be enabled to trace the leak's origin.
XTDB's build.gradle.kts pins `-Darrow.memory.debug.allocator=false` in defaultJvmArgs, which every Test, JavaExec, and clojureRepl task inherits. That pinned value overrides anything passed via command line or environment variables.
Most leaks are failure-path cleanup gaps: a buffer is opened, then an exception unwinds past its close. Look for openSlice and other open calls whose result is handed off without a .use or closeOnCatch on a path that can throw.
The shrinker assumes determinism that leaks do not have, so its :smallest case can misattribute the leak. Bisect with the property test across commits at high -Piterations, then confirm with a single-op plain deftest under the debug allocator.
No. The debug allocator performs per-allocation bookkeeping that adds overhead, so it should only be enabled while hunting a leak. Flip the build.gradle.kts entry back to false before committing.
ProteoWizardIdentify and triage handle, memory, and GC leaks in Skyline tests.
jiayaoqijiaDebug Kurtosis-based Ethereum devnets by tracing logs and applying fixes until blocks advance.
mohitmishra786Profile heap memory allocations on Linux with heaptrack.
yzavyasProfile JVM applications with async-profiler and JFR to identify bottlenecks and memory leaks.