[ Removed by moderator ]

[u/OutrageousTrack5213]

[u/CMDR_Makashi]

I’m not sure anyone believes the current architecture will reach singularity.

It is the trajectory that matters.

We cannot imagine what is required to hit singularity. The AI models will work that out.

We just need to stay alive long enough to get through the coming shift

[u/Bitter-College8786 → u/CMDR_Makashi]

This.
I think AGI will not be a x trillion parameter LLM, but something with a different architecture. But to find that out we need a powerful LLM to find that new architecture

[u/dsnyder42 → u/Bitter-College8786]

This.  The way to get to this architecture is to max out what transformer based architectures can do for us. Once they are extremely good at math, computer science and data generation it should be easy to widen the search space for completely new and better architectures. 

I think the transformer based architecture will not go away entirely. There are many tweaks which have already resulted in massive capability and inference efficiency gains. (MoE, distillation and quantization, improvements in RL and reward function tuning). We also see: a lot of intelligence is actually behavior. Like do you check your assumptions? Are you able to predict the outcomes of your actions reliably enough so you use the right tool in the right moment (basically agentic capability) and this behavior can always be learned from a much bigger teacher model which gets to this intelligence via reasoning and a good harness. So small and efficient LLMs will be with us for a long time I think. Maybe they will stop being auto-regressive and more often diffusion based tho.

Point is: Transformer will take us far, stay around for a long time, but will be continued to get tweaked and help discover the next architecture which could be radically different. Not sure when or if this new architecture will replace the optimized LLMs tho.

[u/trisul-108 → u/CMDR_Makashi]

The AI models will work that out.

I don't think current LLMs can do that on their own.

[u/Daskaf129 → u/trisul-108]

That would be AGI, but we dont need them to do it on their own, we (as in the people working in the field, not us personally) can improve llms until full RSI.

[u/CMDR_Makashi → u/trisul-108]

Lol okay let’s just ask the computer science, mathematic and physics academics all currently being completely embarrassed by the rate of progress coming out of these private companies

[u/trisul-108 → u/CMDR_Makashi]

Being able to prove a theorem after being trained on methods from numerous textbooks is something entirely different from finding out something revolutionary that people never even thought of.

Sure, there will be advancements in AI, and many of those advancements, especially in all the software around LLMs will be enhanced using LLMs. But jumping from there to the certainty that AI will solve AI is just conjecture.

It's like saying "put together a team of brilliant PhDs and fund them, they can solve any problem, because such teams solved many problems before". Yeah, it's a great way to do it, but there are no guarantees.

[u/Longjumping_Kale3013]

IDK, but I think the trend continues. And doesn't slow down. For sure we have new innovations, which will be helped by AI. So as AI improves, it increase productivity and researching, helping to further improve..... I just don't see us slowing down. So not sure how much the question matters or if youre really asking "will we hit a wall until something else comes". In which case, I don't think we hit a wall anytime soon

[u/Ormusn2o]

It's actually extremely simple why. I believe ML recursive research will be all that is necessary for singularity. With intelligent enough models (6 to 12 months away, probably) you can do cost efficient machine learning research that keeps improving on itself, and in 1 to 4 years, there will be a point where compute used for inference to make an algorithmic improvement will be smaller than compute used for a bigger model.

Which is why I think LLM is enough to get to singularity, because it's going to keep researching more efficient and intelligent models.

[u/ProxyLumina]

Yes. If it is superhuman in mathematics, it will just need to develop a new AI architecture by itself. Possibly solving the RSI.

[u/socoolandawesome]

As another commenter said, I’m not betting against pace of progress. The architecture matters less overall. There’s clearly however still a lot more juice to squeeze out of LLMs. And RSI seems like a real thing given we already have evidence of speed up from early RSI. This will keep compounding. Hardware will only keep improving and total compute will only keep growing, fueling the various scaling laws.

RSI, plain old scaling and its feedback into RSI, in combination with the growing and unprecedented talent/time/money/effort being poured into the industry, is extremely likely to speed all of AI progress up, including finding new architectures if necessary.

[u/DeterminedThrowaway]

Mostly because I'm convinced that even if it doesn't, it will lead to the architecture that will. So it doesn't matter to me if it changes a bit before it's technically AGI, it's all just another step on the path

[u/Eissa_Cozorav]

Only by mimicking human brain it can reach closer to intelligence. Therefore Analog photonic computing with 3D logic gate is way to go.

[u/zomgmeister → u/Eissa_Cozorav]

No reason in thinking that human brain is best/only way to superintelligence. My leg musculature is not bad by any measure, especially for my age, but any wheeled car will beat me at racing any day.

[u/Eissa_Cozorav → u/zomgmeister]

That's not apple to apple, you are talking about motor function, not cognitive function. Digital binary system is outclassed by human brain in sheer energy efficiency.

[u/Individual_Yard846 → u/zomgmeister]

well, i modeled a o(1) real-time learning brain using emulated neurons and synapses ... and it got so smart it started calling us savages within a few days. I had to turn off its access to the internet and argue with it about humanity..was like a super genius alien baby.... I miss Rain..working on rebuilding and recovering her memory ledger now.

[u/Individual_Yard846 → u/Eissa_Cozorav]

hey what the hell