Collect Hunan engineering material prices and synchronize them to Elasticsearch.
npx skills add https://github.com/pengfit/cjt-skills --skill hunan-price
Please help me install this Agent Skill. Skill: hunan-price Source: https://github.com/pengfit/cjt-skills/tree/main/hunan-price Command: npx skills add https://github.com/pengfit/cjt-skills --skill hunan-price
This Skill automates the collection and synchronization of Hunan engineering material price data, streamlining the process of gathering and updating price information.
Use the hunan-price skill to sync the latest material price data from the Hunan government's website.
High-intent search queries and answers about installing and using this skill.
Automate Hunan engineering material price data collection by running a script that scrapes the government website, performs ETL processing to clean and structure the data, and synchronizes the results directly into an Elasticsearch index for analysis.
Sync web-scraped material price data into Elasticsearch by applying ETL processing to extract and transform the raw prices, then loading the structured data directly into an Elasticsearch index for easy access and trend analysis.
Yes, you need Python with BeautifulSoup4 for web scraping, requests for fetching pages, pyyaml for configuration, and the elasticsearch library to load the processed material price data into your Elasticsearch index.
Yes, you can integrate this material price data collection into a larger ETL pipeline to automate the extraction, transformation, and loading of Hunan engineering prices into Elasticsearch for analyzing price trends and supporting business decisions.
ETL processing for scraped material price data works by extracting raw price information from the Hunan government website, transforming it into a structured format, and loading the cleaned data into an Elasticsearch index for synchronization and analysis.
Limitations include dependency on the structure of the Hunan government website, meaning changes to the site layout can break the web scraping scripts, and the need for a properly configured Elasticsearch environment to store the synchronized material price data.
pengfitExtract and transform Huhehaote government website price data via ETL.
patrykkopycinskiRun and debug Elasticsearch API requests in the Kibana Dev Tools Console.
pengfitExtract and synchronize Hainan engineering cost material data to Elasticsearch.
pengfitCollect and synchronize Sichuan construction material prices to Elasticsearch.
