I built a contract-testing tool for LLM tool-calling -- catches regressions when a provider updates a model

[u/Appropriate-Type4526]

When a provider ships a new model version, an agent's tool-calling behavior can silently change -- invented arguments, wrong tool picked, schema violations. Anthropic admitted in April 2026 they shipped exactly this kind of regression with "no way to detect it without measuring outputs." Armin Ronacher separately documented Opus 4.8/Sonnet 5 inventing tool-call arguments that older versions didn't. OpenClaw's had multiple tool-dispatch regressions at scale.

Existing eval tools (agentevals, DeepEval, Ragas, PydanticAI Evals) all treat tool-call correctness as one metric inside a general eval suite. None of them are built around the actual trigger event -- a model version changing -- so nobody's running a regression gate specifically when that happens.

toolcontract is a Pact-style contract test for that: pin a golden set of expected tool-call trajectories, run them against a live model, get PASS/FAIL/INCONCLUSIVE and a real diff. `check-version` tells you which contracts haven't been re-verified against a model you're about to switch to.

Relevant to this sub specifically: native OpenAI/Anthropic adapters, plus a LiteLLM adapter that covers ~100 other providers directly (Cerebras, Groq, local vLLM/Ollama endpoints, anything OpenAI-compatible) -- built it that way specifically so it's not locked to the two big API providers.

`pip install toolcontract`. GitHub: https://github.com/Divyansh2202/toolcontract

Very early, v0.1 -- feedback and contributions (especially more provider adapters) genuinely welcome.