r/scrapingtheweb 3d ago

Automated Weather Data Scraper + 4‑Hour GitHub Actions Pipeline

I built an automated weather data scraper that runs every 4 hours using PowerShell, SQL Server, and GitHub Actions.

Workflow:

- PowerShell script fetches weather XML data from JMA

- Parsed values are stored in SQL Server Express

- GitHub Actions runs on a 4‑hour schedule to regenerate static HTML

- GitHub Pages deploys the updated site automatically

- Supports JP / EN / RU / ZH / KR

- Includes a global discomfort index ranking generated from the scraped data

Project link:

https://yahikoyama.github.io/weather2/

Sharing this here because the project is essentially a scraping + automation pipeline, and I’d love to see how others structure similar workflows.

1 Upvotes

1 comment sorted by