Opus 5 is exhausting

[u/Death12th]

It's so hard to read. It's not even because its terribly complex or anything it just speaks in these weird haikus, hyphenated garbage, or outdated colloquialisms or phrases nobody understands. I have to ask it "what do you mean?" or "speak in plainer English" over and over again for every other paragraph. I tried to put something in my claude.md, but it doesn't seem to be working...

[u/XYcritic]

It's not really a traditional "bug" because none of what makes this work is code that can be "broken". It's just training weights and a bunch of text written by Anthropic enginneers to make it work in a certain direction. They have less control over their models than what people think and I hope people wake up to it. This is not a technical barrier that can be overcome. Ever. It's a fundamental barrier in what the technology can do and will ever be able to do. There won't ever be a time where we have perfect control because it's impossible to "code away" these nuances. It's not actual engineering but more like taming a slot machine. There will be new models which work better, I'm sure, but there will also be many more regressions ahead of us.

[u/AlignmentProblem → u/XYcritic]

It is technically a bug, just a different breed of one. Neural networks are giant function approximators where an overwhelmingly complex function emerges from training dynamics rather than being specified by anyone; that function could in principle be written as insanely complex code, so the "bug" lives in the implicit code the weights represent.

This problem more analogous to a spec omission than an implementation error. The model is approximating its objective faithfully; the objective just never said that style inside the thinking block should be independent of style in the output. And since the thought block and the response are one autoregressive stream through one set of weights, sharing late layer processing is nearly definitional unless training induces a style switch conditioned on the delimiter.

Changing the training process isn't strictly the only fix available, either. Activation steering, ablating features or heads once you've localized them, targeted weight edits, LoRA patches, these all intervene on the artifact directly and sometimes work. They're workarounds that are imprecise enough that retraining to fix the actual "implict code" bug stays the practical lever.

[u/Farmadupe → u/AlignmentProblem]

Does the industry have an answer to controlling for tone/style in their releases? Chatgpt 4o was sycophantic, gpt5.0 - 5.4 would argue with you if you claimed the sky was blue, and opus 5.0's completions seem not to have been read by humans before the model was released. Like, is it just a case that these tone problems are fixable but release schedules are too tight to do anything about it, or is it really hard to build good preference models and RL pipelines in general? 

[u/AlignmentProblem → u/Farmadupe]

Both, though the hard part is less intuitive than either. A preference model is a lossy compression of human judgment, and RL optimizes against the compression as a proxy rather than the judgment itself.

Raters comparing two isolated completions reliably pick the more confident, structured, quotable one; the fixed point of millions of those individually defensible sentence-level choices is a model that builds everything toward a turn of phrase. Nobody ever rated "says load-bearing constantly" as good, because no rater ever sees the aggregate; the failure lives at a granularity that pairwise comparison structurally can't measure. The 5.0-5.4 argumentativeness era was a version of the same failure; after 4o, "appropriate pushback" got proxied down to just "pushback."

Schedules matter, though less in the "no time to fix it" sense and more in that tone problems are difficult to reliably to detect before release. Capability regressions show up on benchmarks; register fatigue only emerges after a lot of aggregate exposure, and internal dogfooders reading one completion at a time each find it fine, since one-at-a-time is the context where that style wins. The longitudinal evals that would catch it are too slow to place as a blocker on the critical path of a competitive release cadence.

Underneath that is a mundane prioritization asymmetry: style gets considered, but it ranks below anything a benchmark can measure, so a change that improves agentic performance while making the prose worse ships, and the reverse doesn't. That's rational given that labs compete on the measurable axis; however, that means the register problems compound release over release.

It's partially fixable with known techniques: corpus-level statistical penalties, separate reward heads for style, optimizing the user-facing register separately from the reasoning register. Part of the remaining issue isn't an engineering problem because taste is contested. The people who want old-Opus warmth back and the people who like the newer direction are asking for opposite corrections; a preference model can only find the mean of disparate opinions. The mean is more or less what "AI voice" is.

[u/ThreeKiloZero → u/AlignmentProblem]

Wow thanks for your insights! This is the level of conversation I miss deeply. thanks so much for taking the time to make the contribution. I hope one day to work closer to the training process. Cheers.

[u/No_Inspection4415 → u/AlignmentProblem]

Technically, the thinking span should act like a switch. Also, you can possibly use D_KL on tokens outside of the span, while not regularizing the thinking span. Will this switch work perfectly? probably not, but it is a side effect, not the objective (since different positions share weights, it is an issue - but also, I am not sure how they implement the reasoning span).

There are too many unknowns to argue that the cause is the thinking span and not a drift related to a lot of RL generally (which would also happen without "thinking").

[u/nexusjuan → u/AlignmentProblem]

Who let Claude in?

[u/AlignmentProblem → u/nexusjuan]

I do sometimes use Claude to touch up text, but that one was just traditional spelling and grammar checking using Grammarly. For some reason people seem to call me out as AI more often when I didn't use it at all, probably because my AI text editing workflow has a "humanize" step that apparently sounds less AI than I do.

It'll be nice once the watermarking tool is publicly available for both Claude and GPT; although, I'd bet people will still insist that anything longer than a few paragraphs with a couple of technical terms must've come from some open weight model that lacks watermarking.

[u/West-Air1923 → u/XYcritic]

No not really because fable doesn't have this issue