Query Treasure Data via Presto or Hive and load pandas DataFrames.
npx skills add https://github.com/treasure-data/td-skills --skill pytd
Please help me install this Agent Skill. Skill: pytd Source: https://github.com/treasure-data/td-skills/tree/main/sdk-skills/python Command: npx skills add https://github.com/treasure-data/td-skills --skill pytd
This Skill provides expert assistance for using pytd (Python SDK) to query and import data with Treasure Data, streamlining Python-based data analysis, ETL pipelines, and integration with data science workflows. It helps users efficiently move data between pandas DataFrames and TD, reducing manual effort and potential errors.
bulk_import, insert_into, spark).pytd for efficient data transfer, iterative processing, and robust error handling within their Python environment.pip install pytd
import pytd import pandas as pd client = pytd.Client(database='sample_datasets') result = client.query('SELECT * FROM nasdaq LIMIT 100') df = pd.DataFrame(result['data'], columns=result['columns']) print(df.head())
High-intent search queries and answers about installing and using this skill.
Query data from Treasure Data into pandas using pytd's Client to execute SQL against Presto or Hive, then convert results into a DataFrame. Install pytd, configure a Client with your database and API credentials, call client.query() with your SQL, and construct a DataFrame from the returned columns and data.
Yes, pytd supports loading pandas DataFrames to TD tables using multiple writer types: bulk_import for high-volume data, insert_into for smaller batches, and spark for distributed processing. Choose the writer based on your data size and performance requirements.
Use pytd's DB-API for iterative retrieval of large datasets instead of loading everything at once. This prevents memory exhaustion and timeouts when processing massive result sets in Python, allowing you to fetch and process data in manageable chunks.
pytd requires Python 3.9 or later and pandas 2.0 or later. Ensure both are installed before using pytd for data querying and loading operations with Treasure Data.
Yes, pytd supports building complete ETL workflows including incremental loading and multi-database operations. Common patterns guide you through robust and scalable data pipeline design for moving and transforming data between pandas and Treasure Data.
pytd requires a valid Treasure Data API key and server endpoint to establish connections. Configure these credentials when initializing the pytd Client along with your target database to execute queries and load data.
microsoftRetrieve and analyze Dataverse records using the Python SDK and Web API.
sourcegraphRun BigQuery SQL queries and export results using the bq CLI.
treasure-dataDocument Presto/Trino and Hive staging transformations with SQL examples.
AradotsoAutomate ETL workflows from the Harvard Art Museums API into SQL databases.