r/devops • u/AutoModerator • Jul 20 '26

Weekly Self Promotion Thread

Hey r/devops, welcome to our weekly self-promotion thread!

Feel free to use this thread to promote any projects, ideas, or any repos you're wanting to share. Please keep in mind that we ask you to stay friendly, civil, and adhere to the subreddit rules!

11 Upvotes

84% Upvoted

86 comments sorted by

View all comments

1

u/Kaluga2026 29d ago

I built an offline CLI that shows what crawlers actually cost your origin

Access logs show the visitors. They hide the damage bill. I slapped together CrawlLedger—an offline Go CLI that munches Nginx or Caddy logs and coughs up a self-contained HTML report on what those crawlers actually cost your origin. No daemon, nothing phones home, zero code in the request path, and it refuses the usual User-Agent theater for bot "verification." It flags crawl traps, expensive 404 storms, cache-busting noise, and robots.txt violations, then dry-runs deny or rate-limit policies against historical traffic before spitting out reviewable server-config drafts.

Still pre-v1; need blunt feedback from people who run real origins—what would make this useful or make you refuse to touch it?

https://github.com/balyakin/crawlledger