skills.rest
siddharthchauhan
siddharthchauhanCOMMUNITY
Explore all skills by this author →
GitHub

data-loading

Load and analyze SDTM data from AWS S3 archives.

Updated Aug 27, 2026
Search Tags:#data-analysis#etl#s3#load#domain-detection#sdtm#edc
One-click install
npx skills add https://github.com/siddharthchauhan/ETL --skill data-loading
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-loading
Source: https://github.com/siddharthchauhan/ETL/tree/main/sdtm_pipeline/deepagents/skills/data-loading
Command: npx skills add https://github.com/siddharthchauhan/ETL --skill data-loading

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill addresses the tedious, error-prone process of loading raw EDC data from AWS S3, downloading and extracting ZIP archives, scanning source files, and analyzing data structures to support SDTM pipeline Phase 1.

Core Features & Use Cases

  • S3 Data Ingestion: Load raw EDC data from S3 buckets and locate domain files for processing.
  • ZIP Extraction & File Discovery: Automatically download, unzip, and enumerate extracted CSV/XML files for domain mapping.
  • Source File Scanning & Analysis: Detect available SDTM domains, assess file structure, and summarize key metadata to guide transformation.
  • Use Case: Prepare MAXIS-08 or similar study data for SDTM mapping by previewing domain files and verifying data completeness before transformation.

Quick Start

  1. Load data from S3: specify study_id, s3_bucket, and s3_prefix to fetch RAW_DATA.
  2. Scan the extracted directory to enumerate files and detected domains.
  3. Analyze a representative source file to verify row/column counts and data quality before mapping.

Frequently Asked Questions about data-loading

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I load SDTM data from S3 for clinical trial data ingestion?▼

To load SDTM data from S3, specify your study_id, s3_bucket, and s3_prefix to fetch raw EDC data. The Skill automates downloading and extracting ZIP archives, enumerating CSV/XML files to prepare for domain mapping.

How does domain detection work when analyzing EDC extraction files?▼

Domain detection works by scanning extracted source files and directory structures to identify available SDTM domains. It assesses file structures and summarizes key metadata to guide the subsequent data transformation process.

What is the best way to prepare raw clinical trial datasets for SDTM mapping?▼

The best way to prepare raw clinical trial datasets for SDTM mapping is to scan source files to verify row and column counts, detect available domains, and summarize key metadata to ensure data completeness before transformation begins.

Can I use Python utilities to unzip and enumerate SDTM archives from AWS S3?▼

Yes, you can use Python-based utilities to automatically download, unzip, and enumerate extracted CSV/XML files from AWS S3. This process locates domain files and structures them for mapping within SDTM pipelines.

Does this data ingestion process support both CSV and XML source files?▼

Yes, the data ingestion process supports both CSV and XML source files. After unzipping the raw EDC archives from S3, the tool scans and enumerates all extracted file formats to detect domains and analyze data structures.

What are the limitations of using Python-based utilities for SDTM Phase 1 data loading?▼

This Skill is limited to Phase 1 data ingestion tasks including S3 loading, ZIP extraction, and source file scanning. It does not perform the actual SDTM transformation or mapping, but rather prepares and verifies the raw data structure.

Related Skills

Explore More Data & Analytics Skills→
david-2814david-2814
3

s3

Automate S3 bucket, object, lifecycle, and access-control management via AWS CLI.

Community
Intermediate
mark64oswaldmark64oswald
1

healthsim-trialsim

Generate regulatory-grade synthetic clinical trial data across SDTM domains.

Community
Advanced
wukongnotnullwukongnotnull

wk-html-slides-amber-classified

Convert PPT or markdown slides into a standalone amber-themed index.html site.

Community
Intermediate
RConsortiumRConsortium
93

sdtm-oak

Convert raw EDC/eCRF clinical data into CDISC SDTM domains.

Official
Advanced