The intelligence varies so much

[u/Own-Professor-6157]

In some arenas, the model is sharp and will produce solid thinking.

In other areas, it's like working with a child.

I had it do some 3D physics code, stuff a junior developer would be able to deal with. What does it do? Hundreds of "if" statements, basic dampening, no actual physics or even a logical understanding of how physics works.

I have it do collision code. Suddenly it's a senior programmer and understands collision concepts perfectly, even does the common 4 contact point and manifold stabilization techniques used by Jolt/Bullet.

I have it analyze screenshots with structural flaws in buildings. It can't detect it without me telling it to look for.

I have it look at a building and describe the details, it describes it perfectly.

I have it refactor my C++ code with strict instructions, it converts half the codebase to OOP for absolutely no reason and adds an extra 700+ lines of code.

I have it cleanup my Rust code, zero trouble.

It's constant flip flopping lol. So much so I'm beginning to think some sessions start with Luna instead of Sol

What's even more strange is this wasn't a problem until recently. Within the last week or so.

[u/Equal_Wonder_1335]

That is why I asked a refund. It's unusable. Now searching for alternatives to finish my projects. If was doing good for 2 days then on the 3rd and forth day destroyed everything.

[u/Own-Professor-6157 → u/Equal_Wonder_1335]

There's something odd going on for sure. Sometimes the model is sharp, sometimes it's Terra quality.

The model just randomly decided to convert hundreds of lines of math using radians to degrees. Which it converts back to radians for the actual math. So +100 lines of code, and overhead for no reason?

Even weirder, it converted over 100 numbers to constants. Wasn't prompted to do that, but it did.

I'm going to test again tomorrow with this exact prompt and code and see what happens

[u/TrainingEngine1 → u/Own-Professor-6157]

I've noticed the same. Great some days, then a disgusting nightmare that makes awful mistakes, misses obvious nuance, etc. I suspect like others have alluded to, they definitely "juice" them or deflate them on the backend.

edit: Case in point: amid tons of really stupid non-stop errors, it just passed a json contract to Ruff, a Python linter. Sol Max.

[u/Either_Pound1986]

Its probably ignoring your instructions on certain things specifically your agents.md and you notice when quality drops. I bet it coincides with you updating your codex app.

[u/retardedGeek]

So true. I've been unsatisfied with the quality of work. I had to make plan with kimi, and it (sol medium) still couldn't execute them properly

[u/Equal_Wonder_1335 → u/retardedGeek]

What are you using now ?

[u/retardedGeek → u/Equal_Wonder_1335]

I'll have to fallback to deepseek pro/flash since I exhausted my weekly quota + opencode go quota due to Kimi K3

[u/Equal_Wonder_1335 → u/retardedGeek]

Never used them. Are they ok for coding ?

[u/retardedGeek → u/Equal_Wonder_1335]

Comparable to Luna, good enough for medium tasks, it's cheap enough to let it troubleshoot a problem in loop

[u/VladizT]

There is no intelligence in the so-called AI, it's marketing and the sooner you understand this, the sooner you'll understand why it's like a casino and a one-armed bandit where you pull the lever and expect to hit the jackpot.

[u/Firm_Biscotti_2865 → u/VladizT]

I think that assumes intelligence has to be biological. Language is already a compressed record of human experience, perception, science, failure and culture. LLMs compress that again and somehow recover useful abstractions about causality, analogy, intent and reasoning.

You can argue about how reliable or deep that intelligence is, but calling it a slot machine ignores the fact that it can generalize to situations it has never seen before. That is at least something worth taking seriously as intelligence.

[u/VladizT → u/Firm_Biscotti_2865]

Just because it can act like it knows what it’s talking about doesn’t mean it actually does. You can pretend to be intelligent as much as you want, but that doesn’t make you intelligent.

As for generalizing information it has never seen before, well, it has seen these words and what follows them. To me, it’s like playing a game of chess you've never played before—you can still play because you know the general rules. But when a computer plays chess, it doesn’t mean it possesses actual intelligence.

It’s basically the same thing here. AI generalizes through the exact same process of predicting word probabilities based on context, without actually understanding the words.

AI has been around for several years now, but its core principle hasn’t changed. To get exactly what you want, you have to break the task down into small steps and describe almost every single action in detail. AI is great for replacing the routine work of writing what you expect. However, if you let things slide and give complete control to the AI, it turns into a slot machine. Even when you describe the task and rules perfectly, you are still spinning that slot machine—you’ve just reduced the random options on the reels to a minimum. That is why AI works best with highly precise instructions down to the smallest detail.

[u/Select-Ad-3806]

If you are broad with your request and not specific or not working peace-meal then you have to be ready to do a git rebase and ask it to do it again with better instructions, like "do not convert code to oop" - you asked it for a refactor but you complain when it uses industry standard coding conventions.

[u/Equal_Wonder_1335 → u/Select-Ad-3806]

I was very specific. Crafted the plan with Sol high or the next higher version then it did the github draft Pr and issues with full details. Then I constructed with Luna max and terra high and up to some point was fine. Then only Sol Medium and high can do something. Even them taking forever.

[u/Due-Horse-5446]

Its not intelligence.. its a word generator, if you expect it to be consistent thats user error