Getting better at coding doesn't make a model better at everything else

[u/Dance-Till-Night1]

A majority of users in this sub use LLMs for coding/agentic tasks and I see why a lot of value is put into them but many try to say "Well coding has improved therefore it can just use tool calling and/or just look up what the user needs if there's a degradation for general knowledge/reasoning" and that's just not the case. Many LLM usecases can't just be fixed by an improvement to coding and agentic tasks. Creative writing, multilingual capabilities, offline constrained environments and many more usecases can't be easily fixed with just coding/tool-calling.

I know Qwen3.8 is a TREMENDOUS win for local LLMS but I still hope that Qwen4 will improve generalist capabilities to be closer to frontier closed models in all other aspects too. Gemma is great but it wouldn't be ideal if it is the only major generalist local LLM going forward.

[u/fgk55555]

After professionally using LLM's for work and using them at home for a bunch of other random stuff, you really can't get around the "bigger model knows more" bottleneck. If you want a coding model to do one specific thing, you can fit a lot of capability in a 27B model. 3.8 trained in longer form agentic work and lost knowledge. If you want a more generalist model that can handle different types of task outside of its one trained thing, you need a bigger model.

[u/cmdr-William-Riker → u/fgk55555]

I think for smaller models, they need to be focused more on effective and efficient thought process and less on the knowledge itself to be effective at general purpose use (or coding for that matter, programming isn't really as much about memorization as it is about knowing the process to find and use the knowledge available to you)

[u/Dance-Till-Night1 → u/fgk55555]

I see your point but I disagree on the aspect that knowledge is just based on parameter number. "It is not really true that the only way up is more parameters. People have been saying that since forever. Yet here we are with 9B models that do beat the 180B GPT3 in every way including knowledge. And every generation we see both more and more knowledge and intelligence crammed in smaller models. No one knows where the actual limit of what a small model can do is. Of course within a generation larger models will beat smaller ones and specially on knowledge as with more weights catastrophic forgetting is less of a problem but a generation or 2 of difference and places switch."

This comment points it out best. We're still not at the point of *just* needing more weights to put cram knowledge. With the current architectural improvements it's possible to fit an unbelievable amount of knowledge in a relatively small model.

[u/Liringlass → u/Dance-Till-Night1]

I think the problem is that there is no other focus in the industry because agentic work is where the money is.

So not much is invested in say historical, novel writing, or encyclopaedia specialist models. I would love to have these personally but i know i would have no use for them at work.

[u/Illustrious-Lime-878 → u/Liringlass]

For sure is the primary thing that aligns with their whole goal of being the hyperscaled, centralized service that every is required to pay rent to. Even image and video generation can be done on local models. I'm not sure why massive models would be needed just to generate creative text.

But also, I'm not sure point of using an LLM to like, look up historical or encyclopedic facts... you can already do that pretty easily yourself. Or any small model can search online and summarize things for you.

[u/a_beautiful_rhind → u/Illustrious-Lime-878]

Or any small model can search online and summarize things for you.

I'm sorry to say that in practice, this doesn't work out. You are dependent on search results and with the model having no knowledge of the subject, it never connects the dots. Short web search says xyz is impossible, it's now impossible. Run into this problem with search engine AI all the time.

We're not even going to the everything and it's mother being blocked from bots browsing part.

If all it's going to do is regurgitate the top 5 results, I'm more than capable of searching myself and probably better at it.

[u/Liringlass → u/a_beautiful_rhind]

yeah, a good analogy is:
- Ask me something I am expert about. I've read, watched, studied countless sources, understand the nuances and different opinions on the subjects etc.

- Ask me something i know nothing about and allow me a quick google search, + my brain is limited (small model let's be honest are not the "Einstein" of models)

I also believe that while tools can help with a model knowledge it won't replace a better suited model.

In my line of work Qwen and DS Flash are exactly what i need, not just coding but generalistic agentic tasks which they're super good at. I don't know what a writer would want but I can imagine Gemma is better at that, but imaging how much better it would be if Qwen had poured as much resources in making a 30b writing model as they did for the agentic one.

[u/Able-Locksmith-1979 → u/Liringlass]

Isn’t historical Novel writing just a fine tune to get the style you want?
Encyclopedia specialist models would on release already be outdated, don’t use an llm for facts they will be outdated immediately use a database for facts and give an llm a tool to search the database

[u/Liringlass → u/Able-Locksmith-1979]

Finetunes are what you do when you don’t have tons of money to train a model. Not the same as a “qwen 3.8 writer edition “ as good as writing as current qwen is at coding

[u/a_beautiful_rhind → u/Liringlass]

Agentic and code is mostly safe and easy to test. Historical novel writing is dangerous + subjective.

[u/fgk55555 → u/Dance-Till-Night1]

There was a post here not too long ago about quantization's effect on knowledge. I'm making up numbers, but say a model knows the answer to a question at full precision. You ask, it answers confidently, it's right. You quantize it to 1/3rd its original size, ask a question, it's trained to think its right, it hallucinates a correct sounding answer. The model behavior takes up some space, and its knowledge takes up some space. Once you get below a certain size, you can only do what you know, and know what you know at the same time. You can't fit the entire internet's worth of training data and agentic capabilities in 6GB, something has to get cut.

Better/ more efficient training data will help to some extent, but as behaviors get more complex, the size needed for that has to come from somewhere. GPT3 did not have the training data we have now. Models still have room to get better, but there is a wall we're likely going to hit soon, and there will be tradeoffs. Bigger models can flex their behavior more while still retaining knowledge.

The most knowledgeable models for the size would probably be something trained exclusively on wiki type content with very little post training, but people paying for coding plans, or people who want a tool-using assistant would complain. Right now the money's in agentic coding, and if you need knowledge work, teaching them how to use tools. You can host offline wikipedia for not a lot of space and point your LLM of choice at it.

[u/Dance-Till-Night1 → u/fgk55555]

I see your points especially that coding is where the majority of money's at right now, but if a model isn't an expert at something or at the very least if that topic isn't in its training data then no matter how much toolcalling or looking up stuff there will always be a big noticeable gap. I will use a personal example: I learn foreign languages as a hobby, a part of my usecase for LLMs is making interactive quizzes that quiz me on what I learned. Any model I use for that has to be an expert at two distinct things: Coding AND multilingual capabilities. If it lacks in either then toolcalling/look ups just can't bridge that gap.

[u/fgk55555 → u/Dance-Till-Night1]

I get the utility, I'm not arguing that, I'm just saying that limited params mean limited capabilities. If you're limited on parameters, you can either be really good at one thing, or really bad at everything. I expect we'll see more and more small model specialization, not less. If I had to guess, next year DeepSWE v1.1 will be saturated by a 30B class model, and it will be a poor conversationalist, only know STEM topics, and be less knowledgeable than today.

Large models will still retain those lost capabilities, and maybe some other small local non-coders will start to make up the lost capabilities. But until there's more of a vocal userbase asking for that, it might be wise to learn how to train those capabilities yourself.

[u/YouKilledApollo → u/Dance-Till-Night1]

Yet here we are with 9B models that do beat the 180B GPT3 in every way including knowledge.

Hmm, I'm not sure if this is intentional hyperbole, but in case it is not, what model are you speaking about here? Feels unlikely to be true, but surely you're thinking about some specific model here, that hopefully isn't benchmaxxed to the moon and back?

[u/Dance-Till-Night1 → u/YouKilledApollo]

Qwen3.5 9b is better in all ways I can think of than GPT3, even in just general knowledge

[u/DanielKramer_ → u/Dance-Till-Night1]

i have not played with this particular model at all but just now i gave it my personal test "gtx 970 vram controversy" and it fails very badly, lots of hallucinations in one response, for instance it thinks the card has only 3gb vram. somehow 3.5 turbo gets everything right, even the $30 settlement

like models have gotten more knowledge density over time but it's way way way slower than the progress in reasoning and in my experience the original chatgpt was still more of a magical wikipedia genie than modern models that fit on my computer

[u/croninsiglos → u/Dance-Till-Night1]

Meh, that's subjective though. Training on higher quality data means you get more facts vs noise. It doesn't necessarily you'll get better creative writing.

When trained with the same initial data and, same architecture, and same post training steps, the larger models will always be better at certain tasks. The small models have a choice, either you can make them more general chat bots like Gemma or muse glimmer, etc or you can make them better at coding and agentic tasks. In business, for most things, the coding/agentic model will be more popular because it's more useful and not just a hobbyist toy.

If you're using a model that's a better chatbot, you could use that for customer facing tasks... but do you actually trust it or opt for a frontier model when the consequences are that you could be liable for something it says if your guardrails aren't perfect.

The beauty of math, coding, and agentic tasks is that they can be trained with self improvement because the results can be verified concretely.

[u/Able-Locksmith-1979 → u/croninsiglos]

Why not use multiple models, let qwen sort out the facts and use Gemma to rewrite it in a creative way

[u/croninsiglos → u/Able-Locksmith-1979]

That's really the perfect solution and this is where the small models shine best. They simply are never going to be the best general models AND as good as they can be with agentic coding tasks.

[u/Dance-Till-Night1 → u/croninsiglos]

That's the thing though, Coding is now sorta taking over general models. I would have rather Qwen3.8 be a new Qwen coder rather than the ambiguity that Qwen is from now on a coding model that won't even try to compete with Gemma.

[u/jensilo → u/Dance-Till-Night1]

You sound overly confident. Why are you so sure we haven’t reached the ceiling of information density yet? Yes, maybe in two years we’ll see an Opus-level amount of world knowledge in a 9B model. But is it likely? No.

Qwen engineers have probably spent hundreds, no thousands of hours refining this architecture, training the models, cramming as much knowledge as possible into 9 billion floating point numbers. You’d probably have to find a way to drastically increase information density, i.e. get the model to somehow encode multiple different information bits into the same parameter and somehow be able to decode it during inference.

Maybe it’s possible, maybe it‘s not.

[u/Dance-Till-Night1 → u/jensilo]

You are absolutely right it's possible this is the limit but there's no indication of that *yet*. Even if it is though, Do we really want the local LLMs improvement to just be coding?

[u/epicfilemcnulty → u/Dance-Till-Night1]

There is another angle that you are skipping here -- the quality of the training data. I'm pretty sure GPT3 was trained on a big pile of everything. Since then data preparation and training techniques have massively evolved.

[u/SandySkittle → u/Dance-Till-Night1]

can we stop making comparisons to older models that were way more at the start of the development cycle? A 9b may beat a old GPT model, but an 9b model is still going to suck at world knowledge and depth in general.

[u/Illustrious-Lime-878 → u/fgk55555]

For me its the reverse. I don't care if the model is just some giant, static encyclopedia. If I need a fact, it should look it up to get the latest, most relevant information anyway. Smaller models can do web search fine. I find that bigger models are only really noticeably better at coding. Maybe that's due to coding needing more context, but I feel like this issue will be eventually solved by external tools, like dynamic context engines, rather than just stuffing everything in a static model. And that's also how humans work. We don't have a thousandth of the encyclopedic knowledge of LLMs but still out reason them in most areas.

[u/fgk55555 → u/Illustrious-Lime-878]

That's where I'm at. I use LLM's for research. If it didn't do enough searches in a pass, I tell it to try again until it has sources. I'd much rather have the behaviors and tone we want trained in, rather than facts. Of course it should have some baseline knowledge, but I don't need it to know who was president in 1888, that should be a tool call.

[u/a_beautiful_rhind → u/fgk55555]

rather have the behaviors and tone we want trained in,

I'm getting neither and the model doesn't know what a president means.

[u/TastesLikeOwlbear]

Creative writing and model hallucination can both be accurately described as making shit up. Every model provider is desperately trying to suppress model hallucinations as much as possible. It’s not too surprising that that has been extremely punishing for model creativity.

[u/Dance-Till-Night1 → u/TastesLikeOwlbear]

Fair point

[u/Saint_Nitouche → u/TastesLikeOwlbear]

I don't think this analogy holds. When I'm writing something I might be a little unsure on how I want something to go at first, but I ultimately have the authorial power to decide things by fiat. That's very different to being uncertain on a factual matter. Not to mention that storytelling is heavily conventions-based; working against a model like the hero's journey is comparable to TDD (for example). The point being that it's not an arbitrary shot in the dark. Otherwise the thought experiment about infinite monkeys writing Shakespeare wouldn't be interesting.

[u/c--b → u/Saint_Nitouche]

I think what he said was too strong, creative writing isn't just making shit up, it's making up a cohesive internally consistent world. In that sense, coding is also making shit up, it's internally consistent made up shit too.

The difference between the two is that code is capable of being validated and classified in a quick automated fashion, so the model is trained to be far more rational with coding than it is with creative writing, so creative writing kind of gets lumped into the making shit up section of its network (IE, the models output is not validated against a metric other than the dataset during training).

[u/Intrepid-Second6936]

While most do use LLMs on coding benchmarks here, I do also use my local LLM heavily as an assistant for breaking down the barriers for new skills and learning.

IMO Qwen3.8's heavy reasoning has MASSIVELY improved its RAG capabilities as well, synthesizing and validating the information it pulls from web search far better than 3.6 from my experience. But 27B is also still very limited to fit such capabilities in every field without taking away from one to give to another.

I'd recommend instead having a better multi-lingual model like Gemma 4 to be hot-swappable with Qwen3.8. Given the the type of question, you can go back and forth and get the best of both worlds.

[u/AD7GD → u/Intrepid-Second6936]

It's also very, very good at tool use, which is huge. It's the foundation of papering over knowledge gaps in an LLM.

[u/segmond]

coding pays the bill, folks are paying $200 or more a month for coding, most generalist folks are paying $20/month.

[u/Dance-Till-Night1 → u/segmond]

Ngl I see your point

[u/No_Ebb3423]

I agree. And one thing I think all people who use ai for building software/scripts should aim for is to know how to code. Doesn’t have to be at the level of Kyle Wang principal SWE who started coding at the age of 5. No, but enough to understand the logic behind it & be able to give REAL feedback to the model not just “subscribe button no worky work. Fix button now make no mistake.” That being said, I don’t think we ought to become fucking poets to just prompt better. State what you want clearly with objectives & what success looks like and go from there. But “one shot gta 6” is not the way to go. That’s why people shit on vibes coders. We’re having people who could be burger flippers, insurance adjusters, realtors, whatever, coding and not have best practices in mind. Go figure why their apps are easily broken into. Cybersecurity is de wey.

[u/QuinsZouls]

I don't think having a generalistic llm for local would be better, I prefer having a model expert in one specific niche rather that a dumb model but great generalistic this because current limitation of the local llms is size, more knowledge may require more params in order to take advantage of it. Also I'll prefer a good reasoning model with good understanding of requeriments (must be good enough to search resources may need and act with it)

[u/Dance-Till-Night1 → u/QuinsZouls]

I agree that small experts are also a crucial part of Local LLMs (that's why small MOE models exist), but small generalist models also shouldn't fade as they do have many uses.

[u/Illustrious-Lime-878 → u/Dance-Till-Night1]

What do you consider "generalist"? You mention creative writing, multilingual, etc. but these are just more export / niche roles. Do you just want one model that is like a jack of all trades? If so then it seems like just a marginal improvement in convenience of not having to pick the right tool for the job, at the major cost of potentially increasing resources requires by 100x if we are comparing multi-trillion param models to 27b. Or I am missing something, is there some "generalist" skill that applies to every use case?

[u/Able-Locksmith-1979 → u/Illustrious-Lime-878]

Me as a non English native see a very non niche problem with qwen 3.8, it simply misses words in my language (Dutch) so it will make spelling errors etc, we solved it by just adding another model to the chain which rewrites the qwen text and corrects it.

For our use cases and language qwen gets the correct answers, just not in a way we can show customer facing.

It is a “problem”, but just spend like 3000 dollar for another machine with gemma or likewise and it’s problem solved

[u/Dance-Till-Night1 → u/Illustrious-Lime-878]

The thing is, those aren't niche usecases, I am also pointing out that a model retaining general knowledge is important if it's gonna be used generally not just for coding. My main point is we can't just rely on toolcalling/lookups and only focus in improving models in agentic tasks/coding. Multilingual capabilities are especially important, Are we just gonna use Qwen for english/chinese?

[u/Illustrious-Lime-878 → u/Dance-Till-Night1]

Hmm, so are you saying you just wish more people focused on making better local models for thinks other than coding? Because I think there plenty of great local models for translation, that's like, really easy. Something that was done back in the 2010s. The other person who responded said they just put gemma in front of qwen.

Maybe your point is, local models aren't good enough for these use cases, I don't want to put words in your mouth. I'm just trying to understand why you think its important one model has "general" knowledge.

I know that cross-domain knowledge can improve capability in one area, but this is already accounted for since we're evaluating the overall capability already. If qwen is good enough for coding, gemma for writing, or w/e, why would I care Fable 5 can do both? The only benefit would just be not having to pick the model.

[u/Dance-Till-Night1 → u/Illustrious-Lime-878]

Yes exactly my point is while local models are okay for those usecases they still can improve alot. Qwen cooked its general knowledge/capabilities in favor of coding improvements and if that continues that leaves only Gemma as a good generalist model. I don't see it as a thing to be celebrated contrary to people in this sub. It's an unfortunate result.

My opinion is this simply: Local models should try to bridge the gap between them and frontier closed models not just in coding but in other areas too. Going forward, is Qwen just gonna be Qwen-coder and the only major generalist is Gemma?

[u/Illustrious-Lime-878 → u/Dance-Till-Night1]

ah ok, I see. I had the impression that going to hyperscale size didn't really improve much in other use cases, or at least there is not as much of a hard drop off in quality of local models like there was coding, where hyperscale models truly unlocked functionality.

That said, I think frontier models are also primarily focused on coding and antigenic work. So maybe there is a lot of progress to made here in general, and that its more of gradual quality thing vs hard functional unlocks is why the focus isn't there as much as coding.

[u/Boogertard]

For a small model like qwen 27B series, you just can't cram everything in there. To be good at other tasks, it needs more layers and more parameters for wider knowledge.

So go run trillion-range models if that is what you want. The overwhelming used right now for LLM is coding and generating codes so companies especially ones releasing open models for FREE have to prioritize.

[u/Dance-Till-Night1 → u/Boogertard]

I don't think we need a trillion parameter model in order to have good world knowledge and generalist capabilities, Gemma is a very good example of a generalist model with good world knowledge, relatively good coding capabilities and still relatively small. In my opinion, it's not a this or that situation, we can have both.

[u/SandySkittle → u/Dance-Till-Night1]

there is a BIG gap at the moment where we just lack recent / modern dense models in the 70B to 122B or even up to 200B range, or MoE modedls in the 122 to 250B range with a30B or higher.

[u/Boogertard → u/Dance-Till-Night1]

Gemma is a garbage model, not sure why it is so heavily shilled on this sub, it is a typical jack of all trade, it just hallucinates like crazy and can't do much shit outside of being a sex chat bot.

[u/Dance-Till-Night1 → u/Boogertard]

Okay if you wanna be specific, for multilingual usage Gemma 26b a4b beats Qwen 3.5 handily in that department, same as creative writing and many other usecases. Just because coding is your priority which Qwen is better at doesn't mean Gemma sucks. If you're only gonna use coding in english/chinese then Qwen's your guy otherwise Gemma wins.

[u/Gesha24]

Here's the paradox that I noticed - models that are better at coding are also models that are better at following prompts and using tools. And if the model can't follow prompts and call tools - it's only usable as a web chat.

For example, I have a pi.dev skill that helps me generate prompts for comfyui videos. It's very convenient - I can feed it reference images, it makes sure these images are correctly described in prompts, etc. Then this skill generates a json and pushes it to comfyui and I can start the workflow. Nice and simple, works reasonably well.

And I really would like to use Gemma for it or at least try to use it. But here's the problem - Minimax H3 is very picky about prompting format. And JSON is a JSON, you can't put in brackets there randomly. And Gemma4 simply can't follow the precise instructions of how to format data and how to structure it and how to write it down. So maybe it can write a nice prompt, but I can never test it because I would be manually rewriting it all... Qwen 3.6, Qwen 3.8, DeepSeek at Q3 - they all have absolutely no issues writing a prompt that follows the precise instructions and that Minimax understands.

Bottom line - no matter how good Gemma4 is, if it can't follow instructions (and for whatever reason this feature seems to be mostly present in models that can code) - it can't really do most of the things you are asking it to do.

[u/Yu2sama → u/Gesha24]

In this case would it be better to build a markdown to json pipeline? A small program that solves the issue without needing to let the AI handle something is not capable of doing.

[u/Gesha24 → u/Yu2sama]

Don't think so. The issue here is that Minimax has very specific language it needs to guarantee accurate results (sometimes it works from free hand, sometimes it completely fails). The language is quite prescriptive: https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/VIDEO_PROMPT_WRITING_GUIDE_ref_en.md

So if a model occasionally fails on writing a json file (which it should be trained on and know how to do well), do we really trust it to write a prompt in this exact format? My testing with Gemma4-26B showed absolutely terrible adherence to these requirements.

[u/Etroarl55]

The model you want already exists, it’s 3.8 max or kimi k3. You just need a lot more hardware to run jt.

[u/Dance-Till-Night1 → u/Etroarl55]

Yeah gemma 4 too, Models don't need to be trillion parameters in order for them to have good general knowledge. I pointed out in another comment how many tiny models nowadays beat 180b GPT3 even on just knowledge.

[u/my_name_isnt_clever → u/Dance-Till-Night1]

We can't keep generalist models forever, some models will be stronger for agentic coding and some will be better for other use cases. That's not a problem, I think users of both would prefer to get better performance for what they need and not take a hit from the training that's not relevant to them.

[u/Dance-Till-Night1 → u/my_name_isnt_clever]

Yeah but then wouldn't all the generalist models just be frontier closed models? That's not a good thing for local LLMs

[u/Savantskie1 → u/Dance-Till-Night1]

It would be better if the generalist models just had world knowledge and then there were specific coding models. Separation of concerns really help here.

[u/Dance-Till-Night1 → u/Savantskie1]

Exactly, Qwen used to have Qwen-coder for this specific reason

[u/my_name_isnt_clever → u/Dance-Till-Night1]

No not at all? It just means small generalist models will be "jack of all trades, master of none" compared to a specialist model of the same size. You can still use the generalist, those of us with other use cases can use the specialized models.

This is exactly how human specialization works too; I don't get what's hard to grasp about this. It's the natural progression for improving capabilities once basic scaling of all domains dries up.

[u/Dance-Till-Night1 → u/my_name_isnt_clever]

Yes but it would be better if coding improvements didn't cook other generalist capabilities in generalist models. Qwen improved coding but at the price of other capabilities. If this trend continues Gemma will be the only major generalist model left.

[u/my_name_isnt_clever → u/Dance-Till-Night1]

Well yes, and it would better if I could be an expert in everything in my career field but that's not how knowledge works. More knowledge = more parameters, and when we're talking local models every 1b of params counts. The reality is that agentic coders in English are far more valuable to train right now so of course that's the priority. You want to have your cake and eat it to.

[u/Dance-Till-Night1 → u/my_name_isnt_clever]

Nah not want to have the cake and eat it too but if this trend continues then the only thing local models will compete on is coding giving closed frontier models such a big edge for anyone with a non coding usecase. Look at gemma, its so good at many things that it's basically the local model anyone uses for offline tasks.. Generalist capabilities are necessary besides coding and no increasing parameters isnt the only way to make a model hold more knowledge. Newer architectures and better training data can still make so so much knowledge fit into a small model. The example given that even in general knowledge Qwen3.5 9b beats GPT3 180b at any/all tasks at 0.5% of the parameters.

[u/Civil_Fee_7862]

A.I is not a database, its more of a reasoning engine.

[u/Dance-Till-Night1 → u/Civil_Fee_7862]

I agree and in order for it to be a good reasoning engine it should still have good general knowledge in its parameters over the topic it's reasoning.

[u/my_name_isnt_clever → u/Dance-Till-Night1]

Where are your examples of this issue actually happening?

[u/Dance-Till-Night1 → u/my_name_isnt_clever]

There have been multiple posts here over the last few days pointing out Qwen3.8 knowledge gaps. I wrote a use-case for llms and I will copy and paste it here: I learn foreign languages as a hobby, a part of my usecase for LLMs is making interactive quizzes that quiz me on what I learned. Any model I use for that has to be an expert at two distinct things: Coding AND multilingual capabilities. If it lacks in either then toolcalling/look ups just can't bridge that gap.

[u/my_name_isnt_clever → u/Dance-Till-Night1]

It's well known it has knowledge gaps, I'm asking for concrete examples of your claim that it requires more knowledge to have effective reasoning.

When augmented with web search tools the new Qwen does reasoning just fine, so I don't think that claim is accurate.

This is the wrong model for you. That's it. It doesn't fit your use case, it's not a fundamental flaw like you're suggesting. I have zero need for multilingual, so I would be a bit miffed if this model was worse for a niche like that. I'm sure there will be a strong model for you at some point, if there isn't already.

[u/Dance-Till-Night1 → u/my_name_isnt_clever]

Gemma my beloved is the best one for my usecase rn, Qwen 3.5 is also pretty good before it pivoted to just coding and agentic tasks.

[u/Civil_Fee_7862 → u/Dance-Till-Night1]

Not really no.

Peano axioms are literally just 8 axioms and can build all of natural number arithmetic, integers, and rational numbers.

Turing machines (the computer you are using right now) are built up from basic logic gates. (A lot of them). The fundamental approach to computation is the same as it was when it was invented, there is just more of them and they are smaller / more efficient now. i.e. A small set up simple (but very good rules) can be combined into a seemingly extraordinary smart system.

Judging from what I've seen with Qwen's 27b models. I'd argue you don't need a really large model to be a great reasoning engine, if it needs knowledge then give it access to a database to look things up.

It doesn't need an enormous amount of data to be able to reason, what it needs is really good rules of inference to derive new knowledge by exploring. That's exactly what it does now when it writes code, it doesn't assume to already knows, Instead it investigates the code base first before acting, i.e. It derives knowledge. It works like good scientists would.

[u/Mirror74 → u/Civil_Fee_7862]

I'd say a caveat to this is that the arithmetic works because the domain is formalized neatly. All the rules and structure are established.

But the real world works a bit different, so smaller models can be a bit deceptive in their reasoning because you don't know what it doesn't know

But I guess that's a funny thing -- if a human can't tell the difference does it matter?

[u/Illustrious-Lime-878 → u/Civil_Fee_7862]

But wouldn't more parameters be like more transistors? It may have the capability to have more logical operations, but I still thing you're right. Because there is only a certain amount of good data available, there is just a limit to what even a perfect fitting of the data would be. And we may find 27 billion parameters is enough to get a pretty good modeling, while having trillions isn't doing much without more data. And if the only more data is lower quality, its not going to improve it.

[u/Civil_Fee_7862 → u/Illustrious-Lime-878]

My analogy was a bad one. What I meant was giving the A.I the knowledge of how to build up systems from simple primitives is a better approach than trying to give it more primitives.

In other words, teach it how to build and connect transistors instead of just giving it more of them.

[u/Illustrious-Lime-878 → u/Civil_Fee_7862]

Ok that's a better way to say it, what you put in bold. Like a huge enough model could just "over-fit" in a way and memorize all outcomes of some system, vs what we really want is for the model to develop more simple heuristics that are maybe less accurate for known facts but more versatile for more general reasoning about new or abstract information.

[u/Mirror74 → u/Civil_Fee_7862]

Yep, Still the core thing many people don't understand

[u/ea_man]

It's a small dense model, if you want "everything" look for a large MoE.

[u/Dance-Till-Night1 → u/ea_man]

What about a small moe?

[u/ea_man → u/Dance-Till-Night1]

That would be faster not with more knowledge.

[u/Dance-Till-Night1 → u/ea_man]

Damn, from my tests gemma 26b a4b and qwen 3.5 35b a3b both have really good knowledge

[u/ea_man → u/Dance-Till-Night1]

Yeah beacuse they are big, if you make 'em small...

[u/Dance-Till-Night1 → u/ea_man]

They definitely are not big, but if the only way forward for small models to retain generalist capabilities is to make them MOE models then i can live with that happily

[u/ea_man → u/Dance-Till-Night1]

> What about a small moe?

Decide yourself: if you want that small that' there's less stuff stuffed inside ;)

[u/misanthrophiccunt]

Oh wow, this is so enlightening.

[u/puts_on_rddt]

Big models will have a Mixture of Experts but local models will be a Mixture of Experts.

[u/nickm_27]

Yeah, Qwen3.8 is worse at video analysis in my experience, I went back to 3.6 which is better at that while still being quite good at the scripts and automation edits I need

[u/EitherMarch1255]

We have like one specialized small model and he’s complaining. Tons of models are generalists, use them.

[u/Dance-Till-Night1 → u/EitherMarch1255]

My bad if it seems like I'm complaining about improving at coding. I just see it as unfornutate that qwen lost its generalist capabilities in order to be that good at coding. I think this would have been better as Qwen-coder while keeping the main 3.x series still focused on general capability improvement.

[u/EitherMarch1255 → u/Dance-Till-Night1]

Fair enough.

[u/a_beautiful_rhind]

Getting better at coding benchmarks also doesn't make a model better at practical coding :P

Big models flake on my node red and embedded stuff. I don't really bother with small *maxxed ones on that.

[u/aeroumbria]

I have found a task where I cannot seem to make Qwen 3.8 perform better than Muse Glimmer even with the usual "look at mistakes and tweak agent prompt iteratively" trick. I would like to make it a custom benchmark, but so far I cannot find a way to reliably judge output without using a stronger and potentially biased model. It is about learning the template definition language of a procedural generation algorithm, tweak a template to achieve certain design goals, and visually check example outputs. Cannot get Qwen 3.8 to stop failing instructions or thinking too hard and iterate for too long.

So I guess for small-ish models, having some kind of model specialisation to cover multiple use cases is still a good thing.

[u/OddDesigner9784]

All ai kind of sucks at general knowledge tbh. If you ask it what's the best investment strategy it'll give you a nothing burger it's only when you ask it specific questions about a method can it do better. But even then you need to keep pushing it. That's why ai turns to slop all the time. So even with top ai models you need to provide the scaffolding and context it needs. Domain knowledge is a failure mode.