Skills
.Work. You
Rest

spark-best-practices

Community

Master Spark performance with sane defaults.

Data & Analytics#performance#scala#spark#tuning#pyspark#data-processing#spark-optimizations
Authorbaekenough
Version1.0.0
Installs0

System Documentation

What problem does it solve?

Spark workloads often suffer from inefficient configurations and coding patterns that waste time and compute resources. This Skill provides a consolidated set of best practices for PySpark and Scala to help you write faster, more reliable Spark jobs.

Core Features & Use Cases

  • Performance tuning guidance for common bottlenecks (joins, shuffles, caching)
  • Resource management and partitioning strategies to improve throughput
  • Data format and processing recommendations for analytics pipelines

Quick Start

Apply these Spark best practices to your PySpark or Scala job to optimize joins, caching, and partitioning for faster, more scalable data processing.

Dependency Matrix

Required Modules

None required

Components

Standard package

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: spark-best-practices
Download link: https://github.com/baekenough/second-brain/archive/main.zip#spark-best-practices

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 510,000+ vetted skills library on demand.