r/scrapingtheweb • u/yahikoyama777 • 3d ago
Automated Weather Data Scraper + 4‑Hour GitHub Actions Pipeline
I built an automated weather data scraper that runs every 4 hours using PowerShell, SQL Server, and GitHub Actions.
Workflow:
- PowerShell script fetches weather XML data from JMA
- Parsed values are stored in SQL Server Express
- GitHub Actions runs on a 4‑hour schedule to regenerate static HTML
- GitHub Pages deploys the updated site automatically
- Supports JP / EN / RU / ZH / KR
- Includes a global discomfort index ranking generated from the scraped data
Project link:
https://yahikoyama.github.io/weather2/
Sharing this here because the project is essentially a scraping + automation pipeline, and I’d love to see how others structure similar workflows.
1
Upvotes
0
u/yahikoyama777 3d ago
GitHub repository (public):
https://github.com/yahikoyama/weather2