Develop and execute Spark code on Google Cloud Dataproc clusters.
npx skills add https://github.com/rose4320/Eldonia-Nex --skill gcp-spark-rose4320
Please help me install this Agent Skill. Skill: gcp-spark Source: https://github.com/rose4320/Eldonia-Nex/tree/main/.cursor/skills/gcp-spark Command: npx skills add https://github.com/rose4320/Eldonia-Nex --skill gcp-spark-rose4320
This Skill addresses the challenges of developing, executing, and managing Spark code on Google Cloud Dataproc Clusters and Serverless environments.
Use the gcp-spark skill to execute a Spark ETL job on Dataproc using the provided PySpark script 'etl_job.py'.
High-intent search queries and answers about installing and using this skill.
You can run PySpark ETL jobs on Google Cloud Dataproc by using this Skill to develop and execute Spark code directly on Dataproc Clusters and Serverless environments. It automates large-scale data processing and deployment for your GCP project.
Yes, Dataproc Spark jobs can access and manage data stored in BigLake Iceberg, BigQuery, and Spanner. This integration allows you to query and process managed data directly within your Spark ETL pipelines and ML workflows.
To deploy Machine Learning models on GCP Dataproc, you need PySpark and the gcp-bigdata dependency to interact with GCP services. This Skill provides the necessary environment for large-scale ML model training and execution on Google Cloud.
You can debug execution failures in Spark jobs on Dataproc using the debugging tools provided by this Skill. It helps identify and resolve errors within your Spark jobs and interactive sessions efficiently to ensure smooth data processing.
Yes, this Skill supports developing and executing Spark code on both Dataproc Clusters and Serverless environments. This allows you to choose the appropriate compute model for your specific ETL and data processing needs on Google Cloud.
AradotsoCreate scalable GCP data engineering pipelines with BigQuery and Dataflow.
henkisdabroExport GA4 event data to BigQuery and query nested fields with SQL.
ironkid90Automates Spark code execution on Google Cloud Dataproc with BigLake, BigQuery, and Spanner.
ironkid90Manage Apache Beam pipeline lifecycles on Google Cloud Dataflow.
