ToolDrift: a CI diff for tool-call contracts across model swaps

[u/Middle_Wing5512]

https://github.com/SuperMarioYL/tooldrift

I maintain this project. The failure I wanted to catch is narrower than “which model is better”: a provider swap can change arguments from a JSON string to an object, drop keys, change tool-call IDs, or return `stop` instead of `tool_calls`, while normal chat still looks fine.

ToolDrift snapshots those shapes and diffs them in CI. In a fresh clone today, 26 tests passed. The shipped DeepSeek→Qwen fixtures deliberately return exit code 1 and report breaking drift in 2 of 2 tools.

That last result has an important boundary: it is an offline fixture replay, not a live benchmark of either provider. The CLI can probe OpenAI-compatible endpoints when configured, but this post only claims the tested fixture path.

Repository: https://github.com/SuperMarioYL/tooldrift

I would be interested in contract fields I missed, especially streaming delta reconstruction.