skills.rest
arsen-ask-lx
arsen-ask-lxCOMMUNITY
Explore all skills by this author →
GitHub

etl-pipeline

Automates Bitrix-to-PostgreSQL analytics ETL via Trino staging and dbt transforms.

Updated Aug 27, 2026
Search Tags:#workflow#analytics#dbt#trino#etl#mv#bitrix
One-click install
npx skills add https://github.com/arsen-ask-lx/.claude_file_4all --skill etl-pipeline-arsen-ask-lx
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: etl-pipeline
Source: https://github.com/arsen-ask-lx/.claude_file_4all/tree/main/skills/etl-pipeline
Command: npx skills add https://github.com/arsen-ask-lx/.claude_file_4all --skill etl-pipeline-arsen-ask-lx

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automates end-to-end analytics ETL from Bitrix sources to materialized views in PostgreSQL.

Core Features & Use Cases

  • Full ETL pipeline: Bitrix API → Trino (staging) → dbt transforms → MV in PostgreSQL.
  • Debounced and guarded execution with Redis-based locks to prevent concurrent ETL runs.
  • State exposure via AnalyticsState for API-driven health checks and ready status.

Quick Start

Run the full ETL workflow to load Bitrix data into the MV and update analytics dashboards.

Frequently Asked Questions about etl-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate ETL workflows from Bitrix to PostgreSQL materialized views?▼

Automating Bitrix to PostgreSQL ETL involves extracting data via Bitrix API, staging it in Trino, applying dbt transformations, and loading it into materialized views. This workflow supports periodic syncs and dashboard updates with state exposed via AnalyticsState.

How do I prevent concurrent ETL pipeline runs when syncing Bitrix data?▼

To prevent concurrent ETL pipeline runs during Bitrix data sync, use a distributed Redis advisory lock fallback combined with a debounce mechanism. This guards execution to ensure only one ETL workflow instance processes data at a time.

Can I use Trino and dbt together for staging and transforming Bitrix API data?▼

Yes, Trino and dbt can be used together for Bitrix API data by staging raw data in Trino before applying dbt transformations. This prepares the data for final loading into PostgreSQL materialized views for analytics.

How do I trigger a manual ETL workflow sync using Celery tasks?▼

Manual ETL workflow syncs can be triggered using Celery tasks to initiate the Bitrix to materialized view pipeline. This allows on-demand data extraction, transformation, and loading alongside scheduled periodic syncs.

How do I check the health and ready status of an analytics ETL pipeline?▼

Check ETL pipeline health and ready status by querying the AnalyticsState API, which exposes the current state of the Bitrix to PostgreSQL workflow. This enables API-driven health checks for ongoing data syncs.

Related Skills

Explore More Data & Analytics Skills→
gotempshgotempsh
620

temps-platform-setup

Automate installation and initial configuration of a self-hosted Temps platform.

Official
Advanced
ysyecustysyecust
40

clickhouse-io

Automate ClickHouse table design and query optimization for OLAP analytics.

Community
Basic
CodeAtCodeCodeAtCode
14

celery

Orchestrate asynchronous Python tasks across distributed workers with Celery.

Official
Advanced
treasure-datatreasure-data
21

aps-doc-staging

Document Presto/Trino and Hive staging transformations with SQL examples.

Official
Advanced