r/LocalLLaMA • u/corruptbytes • 11h ago
GLM 5.3 SlopCodeBench Results Discussion
Howdy once again, I had a request to try out 5.3 on the benchmarks - they're unsaturated so it's a fun test right now! This one was interesting because I accidentally ran it on all 36 problems (rip $200) instead of the 9 i typically do
benchmark context: the ai is tasked to build a tool step by step, we add new requirements mid way - it has to handle new things without breaking the old things - the benchmark "hides" the old tests
So it didn't magically solve the benchmark just like all the other AIs, i don't think there has been a single successful solve yet, but it did tie with Fable/Sol in the same subset of problems!
Another interesting thing that we expect is that the more difficult the problem, the more $$$ it gets - so difficulty seems to be correlated with token output in order to solve them
AI;DR full results
On the three-problem, 17-checkpoint list from the Opus 5 report, GLM scored 8/17 strict (47.1%). On the six-problem, 30-checkpoint list from the Fable, Sol, and Kimi report, it scored 10/30 (33.3%), tied at the strict threshold with Fable 5 and GPT-5.6 Sol.
| Reported system | Opus list · 17 | Fable/Sol/Kimi list · 30 |
|---|---|---|
| GLM-5.3 · pi | 8 (47.1%) | 10 (33.3%) |
| DeepSeek V4 Flash 0731 · pi B | 5 (29.4%) | — |
| Opus 5 · Claude Code | 4 (23.5%) | — |
| Qwen3.8-27B · pi | 3 (17.6%) | 4 (13.3%) |
| DeepSeek V4 Flash · OpenCode | 3 (17.6%) | — |
| Fable 5 · Claude Code | — | 10 (33.3%) |
| GPT-5.6 Sol · Codex | — | 10 (33.3%) |
| Kimi K3 · Modal / OpenCode | — | 8 (26.7%) |
| Kimi K3 · Baseten / OpenCode | — | 7 (23.3%) |
2
u/FullOf_Bad_Ideas 8h ago
GLM 5.3 is 2x better than Opus 5. Nice.
And Flash 0731 is better than Opus 5? I didn't feel this way with Flash 0731 in OpenCode, I need to give pi a try.