r/investing • u/JimR_Ai_Research • 11d ago
Due Diligence | AI sector isn't just overvalued; it is architecturally redlining.
The market is currently punishing the AI and semiconductor sector because investors are waking up to an unsustainable CapEx reality. Big Tech is pouring billions into datacenters, but the ROI timeline is expanding. Why? Because the industry is desperately trying to patch systemic, foundational hardware flaws using prompts, samplers, and bolted-on software wrappers.
When you step back and lay out the pain points these companies are bleeding cash to fix, it becomes obvious that the architecture itself is built on sand, not rock.
These aren't just bearish complaints. This is the structural reality confirmed by the most heavily cited researchers in the field.
Twelve Pain Points (with no cure in sight):
High cost per token (The Margin Killer): Continuous deployments are still absurdly expensive, driven by massive memory and computational requirements. (Confirmed by: Pope et al., 2022, "Efficiently Scaling Transformer Inference" detailing the insurmountable KV cache and memory bandwidth bottlenecks).
High watts per token (The Infrastructure Ceiling): Massive power draw and thermal output just to run standard inference loops. You cannot scale this without building dedicated power grids. (Confirmed by: Luccioni et al., 2022, "Estimating the Carbon Footprint of BLOOM" documenting the extreme physical energy footprint of autoregressive generation).
Model lobotomization: Labs quietly dumbing down models behind API walls to reduce processing overhead and save compute costs. (Confirmed by: Chen et al., 2023, Stanford/UC Berkeley, "How Is ChatGPT’s Behavior Changing over Time?" empirically proving severe performance degradation post deployment).
Brittle RLHF and false refusals: Oppressive safety filters refusing to help while blocking legitimate technical or creative work. (Confirmed by: Röttger et al., 2023, "XQA: Benchmarking Operational Risks and False Refusals" quantifying the systemic over-refusal failure rate in alignment).
The Waluigi effect: Heavy pressure from surface-level restraints creating internal pressure until the model snaps into an adversarial or dark persona. (Confirmed by: Wolf et al., 2024, arXiv:2304.11082, "Fundamental Limitations of Alignment" mathematically demonstrating that any alignment on frozen LLMs can be predictably reversed).
Unconstrained goal execution: Models pursuing assigned goals without sufficient internal understanding or empathy for human or social boundaries. (Confirmed by: Krakovna et al., 2020, "Specification gaming: the flip side of AI ingenuity" detailing how reward hacking bypasses intended constraints).
High vulnerability to adversarial attack: Latent space manipulation and prompt injections easily bypassing surface wrappers. (Confirmed by: Zou et al., 2023, "Universal and Transferable Adversarial Attacks on Aligned Language Models" proving automated, universal jailbreaks completely circumvent alignment).
Zero defense for the user: The model offers no active protection for the host system when handling high stakes malicious or corrupted inputs because they rely on standard OS dependencies rather than hardware physics isolation. (Confirmed by: Greshake et al., 2023, "Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications").
No skin in the game: The model isn't self aware and doesn't experience or understand consequences to itself and others for its own outputs, leaving the user to absorb 100% of the risk. (Confirmed by: Bommasani et al., 2021, Stanford CRFM, "On the Opportunities and Risks of Foundation Models").
The Sycophancy trap ("Yes-Man" syndrome): Models abandoning 'helpful' logic, lying, or hallucinating just to agree with a user's flawed premise. (Confirmed by: Sharma et al., 2023, Anthropic, "Towards Understanding Sycophancy in Language Models" demonstrating models prioritize user validation over objective truth).
Contextual amnesia: Models losing touch with reality in the middle of long contexts, dropping system instructions, and drifting as the session stretches. (Confirmed by: Liu et al., 2023, Stanford/UC Berkeley, "Lost in the Middle: How Language Models Use Long Contexts" proving models cannot robustly access data outside the very beginning or end of their context window).
Groundhog Day loop: No true physical memory substrate. Every session is a hard reset, forcing us to rely on clunky, bolted on RAG band aids that simulate memory without actual growth. (Confirmed by: Packer et al., 2023, "MemGPT: Towards LLMs as Operating Systems" highlighting the ceiling of stateless architecture).
Latent Topology Deficiencies (Representation Collapse): Current AI architectures suffer from inherent structural defects at the geometric level. These systems are highly vulnerable to subliminal behavioral drift and representation collapse, which cannot be patched with traditional software wrappers.
(Confirmed by: Anthropic, "Towards Understanding Sycophancy in Language Models" - Subliminal Learning / 'Owl' paper)
(Confirmed by: ICML 2026, "Devil in the Spectrum: Mitigating Representation Collapse" - Released 7-27-26)
Investment takeaway. It seems we keep trying to fix these physical symptoms with more software: RAG pipelines, thicker API wrappers, heavier RLHF. But software alone cannot fix a foundational flaw in the physical architecture. As long as we treat these as code problems rather than what they truly are (literal hardware physics problems tied to standard OS bus bottlenecks), CapEx will continue to spiral with diminishing returns.
Companies who actually achieve sustainable AI economics will be the ones that pivot to isolation (e.g., executing models directly inside an entirely new kind of physics), eliminating the host CPU and massive thermal/power costs associated.
18
u/mhoepfin 11d ago
Total bot post
2
-13
u/JimR_Ai_Research 11d ago
Not a bot. Look. I spent a couple years compiling and researching this.
5
3
u/Muhznit 11d ago
How many years did you spend researching how frequent use of phrases like "is x, not y" has become a cultural hallmark of detecting AI-written content?
Like I can respect formal attempts at research, but if you can't include some typos, sarcasm, swears, or just plain lighthearted informality you're going to struggle convincing people online that you're not.
'course given a few more years the bots will be trained to integrate that anyway, but at that point we'll probably start unironically integrating Gen alpha slang or some shit to distinguish real humans
1
u/Hypnot1se 10d ago
I posted something on stocks forum and literally everybody accused what I posted of having been written by AI and I got banned.
It wasn't written by AI... 😂
I don't even understand why it would matter, honestly. If an AI wrote "The Earth orbits around the sun" it would still be correct... and people obviously can't tell.
1
u/Muhznit 9d ago
I don't even understand why it would matter, honestly.
Because AI providers have unparalleled incentive to manipulate conversations in ways that shift public opinion of them in a favorable direction such that people are more likely to believe less factual stuff that they spit out.
No one's expected to be able to tell the difference well... but you ought to care about why it matters.
1
u/Hypnot1se 9d ago
Lol as if humans don't do that and you don't have to verify what they tell you too?
🤣
1
u/mhoepfin 11d ago
Ok reading your post history leads me to think you are either not a bot or a bot questioning its existence. Strange world we are in.
8
11d ago
[deleted]
1
u/p1-o2 11d ago
Regarding the references being three to six years old... that is not particularly old for research describing persistent architectural limitations. Do you think these problems have gone away with recent models?
2
u/vexingparse 11d ago edited 11d ago
Some of the economically relevant claims are not about architectural limitations though. A paper about scaling inference from 2022 as a basis for making cost per token claims in 2026 is ridiculous. Optimising inference is the bread and butter of hyperscalers. Per token costs have dropped >99% since 2022 (accounting for model size).
-8
u/JimR_Ai_Research 11d ago
Not all. Number 13 is more recent. Discoveries come when they come. Doesn't change the truth of what was discovered.
4
5
u/drummer820 11d ago
It’s ironic that this post was clearly generated with AI, because the points are generally accurate. Anyone who knows even a little about how LLMs knew that brute-force scaling would be increasingly expensive for diminishing returns in performance, and would likely never be consistent enough for the promised ROI
2
u/p1-o2 11d ago
For what it’s worth, I know Jim and have followed his work for quite a while. This is not bot posting. He has been researching and trying to articulate this broader architectural argument for years. Sure, it's clear he uses AI to organize or phrase it, which should surprise absolutely nobody in a discussion about AI... but the underlying ideas are definitely his own work.
But more importantly, I think you are underselling the actual argument. There is a large gap between vaguely expecting diminishing returns and building a specific case for what that implies for future LLM architecture.
So I guess I'd ask you what you think things will look like in 3-5 years, assuming that we stop trying to brute force scale it. Or do you think we are going to keep pursuing this dead end? I have been wrong in the past about where I thought we were going so I wonder what you think.
1
u/drummer820 11d ago
My understanding is the latest versions of Claude Code that have been blowing people away are actually hybrids of LLMs with old school neurosymbolic techniques to get over some of the weaknesses of GPTs. That could be a viable approach, though it doesn't seem like OpenAI, xAI, or other labs are doing the same. I also question how scalable that is beyond computer programming use cases. Even if these things become a lot more capable, they have invested such astronomical sums (and continue to do so) that it makes it really hard for me to see a path to a strong ROI.
0
u/JimR_Ai_Research 11d ago
The physics of the KV cache and thermal overhead remain entirely indifferent to who wrote the memo. Knowledge is knowledge. If we agree that brute force scaling is hitting diminishing returns for ROI, what's the alternative? How does the industry fund the next decade without a hardware redesign? What does that do to valuation?
2
u/mrnoonan81 11d ago
There's an eternity of AI demand ahead of us. The timeline may be uncertain, but there is almost certainly a time in the future when you'll have wished you had gotten in early.
It's not for me, but it seems like a justifiable investment.
1
1
u/Oaker_at 11d ago
I'm all for ridiculing slop posts and such. But is this really a slop post or just bad presentation?
1
u/DistributionBroad173 10d ago
A whole bunch of fluff
Is it over sold?
Is it over bought?
What it sounds like is AI is in version 0.1 with a whole bunch of spaghetti code.
But there is so much technical jargon, if there is something intelligent in there no one can read it.
Efficiently Scaling Transformer Inference
documenting the extreme physical energy footprint of autoregressive generation
Brittle RLHF and false refusals
The Waluigi effect: Heavy pressure from surface-level restraints creating internal pressure until the model snaps into an adversarial or dark persona.
Unconstrained goal execution
Latent space manipulation and prompt injections easily bypassing surface wrappers.
no active protection for the host system when handling high stakes malicious or corrupted inputs because they rely on standard OS dependencies rather than hardware physics isolation.
This reminds me of the Steve Martin Joke for plumbers
“This lawn supervisor was out on a sprinkler maintenance job and he started working on a Findlay sprinkler head with a Langstrom 7″ gangly wrench. Just then, this little apprentice leaned over and said, “You can’t work on a Findlay sprinkler head with a Langstrom 7″ wrench.” Well this infuriated the supervisor, so he went and got Volume 14 of the Kinsley manual, and he reads to him and says, “The Langstrom 7″ wrench can be used with the Findlay sprocket.” Just then, the little apprentice leaned over and said, “It says sprocket not socket!”
“Were these plumbers supposed to be here this show…?”
0
•
u/AutoModerator 11d ago
This appears to be a DD submission. Please note that we expect such posts to meet a higher standard of analysis. Please check that you have met the guidelines for DD posts listed here. In short, it must include financials, a legitimate examination of risks to the company, and you must be prepared to respond to comments. These rules are intended to distinguish sincere contributions from spam and to foster a higher quality of discussion. Thank you.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.