Automates collection and ETL of Qinghai engineering material price data.
npx skills add https://github.com/pengfit/cjt-skills --skill qinghai-price
Please help me install this Agent Skill. Skill: qinghai-price Source: https://github.com/pengfit/cjt-skills/tree/main/qinghai-price Command: npx skills add https://github.com/pengfit/cjt-skills --skill qinghai-price
This Skill automates the collection of Qinghai engineering material price data, streamlining the process of ETL (Extract, Transform, Load) operations.
Run the sync command to synchronize data for the latest period.
High-intent search queries and answers about installing and using this skill.
You can automate Qinghai engineering material price collection by running the sync command to extract data from government websites, transform it through cleaning and classification, and load it into elasticsearch for analysis.
Required Python libraries include requests and beautifulsoup4 for web scraping, pdfplumber and pdf2image for extracting PDF tables, pyyaml for configuration, and elasticsearch for storing the processed engineering material price data.
Yes, this Skill supports extracting material price data from PDF documents using pdfplumber and pdf2image, specifically targeting Qinghai government engineering material price publications for automated data collection.
Yes, the ETL workflow loads cleaned and transformed Qinghai engineering material price data into elasticsearch, enabling searchable indexing and retrieval of government material price records for analysis.
Run the sync command to automatically synchronize and collect the latest Qinghai engineering material price data from government websites, ensuring your dataset stays current through automated ETL processing.
Limitations include dependency on the Qinghai government website structure for scraping with requests and beautifulsoup4, and potential need to update ETL logic if source site layouts or PDF formats change.
pengfitExtract and synchronize Jilin Province material price data to Elasticsearch.
pengfitCollect Hunan engineering material prices and synchronize them to Elasticsearch.
pengfitExtract and transform Huhehaote government website price data via ETL.
pengfitExtract and synchronize Hainan engineering cost material data to Elasticsearch.
