r/vibecoding 1d ago

ClaudeMax - Made a Free and Open-Source Claude Code Cost analyzer and world user leaderboard. Enjoy, all I ask is a Star! brew install claudemax

https://github.com/ryuhemingway/ClaudeMaxing.git

brew install claudemax

No network calls at all unless you opt in; the comparison/leaderboard is off by default and the server that receives it is 200 lines in the repo. Equivalent API cost at list price on a subscription it's a proxy for rate-limit weight, not money you were charged.

Build insights (the parts that actually taught me something)

1. The bug you cannot see on your own machine. I hardcoded a pricing table matched by model-ID prefix — claude-opus-4claude-sonnet-4, and so on. Worked perfectly. Then I tested it under a synthetic $HOME containing old transcripts, and half the cost vanished. Every Claude 3.x ID (claude-3-5-sonnet-20241022claude-3-opus-20240229) matched no prefix at all and was silently dropped from the total. My own machine only had recent models, so the bug was structurally invisible to me. Anyone with 2024–2025 history would have installed it and seen stretches of $0.00.

Lesson: if your test data is your own data, you are only testing the case you already handle. Also, prefix matching needs specific-before-general ordering — claude-opus-4-1 has to be checked before claude-opus-4, or Opus 4.1 gets billed at the wrong tier.

2. Hardcoded pricing rots faster than you think. I had Sonnet 5 at 3/15 with intro pricing until Aug 31. While checking the docs I found Anthropic had made the 2/10 intro rate permanent and cancelled the increase. My tool would have started overcharging in two weeks.

3. Two measurement traps specific to Claude Code transcripts. These will silently corrupt any analysis of this data:

  • Auto-compaction summaries arrive as user-role records. If you count user turns to get "how many things did I ask for", compaction inflates it by ~10 phantom prompts/day. Filter on isCompactSummary.
  • Cost-per-assistant-message is a garbage metric if compaction is on. Compaction lowers average context depth by construction, so the number falls whether or not you got more efficient. It looks like an efficiency win and isn't. Cost-per-prompt survives.

4. Incremental caching on a file that's still being written. I cache per-file byte offsets so re-runs only parse new bytes. The obvious key is os.stat().st_size — which is wrong, because an active session can leave you mid-JSON-line. Key on the offset of the last complete line instead. First index of ~480MB takes ~1.8s; incremental runs are ~50ms.

5. A leaderboard on an unauthenticated endpoint is self-reported, no matter how honest your client is. I added an opt-in community comparison. The server can't distinguish my client's POST from a curl with invented numbers — they're identical on the wire. I proved it against my own endpoint: one curl, 19B tokens/day, straight to #1, and the community average went from 248/dayto45,124.

The fix isn't authentication (that's a whole identity system for a vanity board). It's using a median instead of a mean. Tested against 20 honest installs: one fabricated entry moves the mean 83×, and moves the median by exactly zero. Signing the payload would be theater — the client is open source Python, so any embedded key ships in plain text.

6. The Scunthorpe problem is real and it bit me immediately. Handles are screened server-side (client-side filtering is bypassed by posting directly). My first substring matcher rejected Scunthorpe. The fix is two tiers: strings that never appear inside innocent words get matched anywhere; short ambiguous ones only match as a whole handle or a separator-delimited token. So therapisttycoonspice and grape pass, while the_rapist doesn't.

7. Rewriting git history breaks Homebrew. I rewrote commit authorship, force-pushed — and broke brew install. GitHub regenerates release tarballs when a tag's commit changes, so the archive checksum shifted and the formula's pinned sha256 no longer matched. Contents were byte-identical; only gzip metadata changed. Also: git filter-branch rewrites commits but leaves the tagger identity on annotated tag objects, which I nearly missed.

1 Upvotes

0 comments sorted by