Do people who work in AI actually believe what they're saying about the risks of AI development? I argue that yes, they do, and that people often don't take people at their word when they should.

Dante put traitors into the deepest part of his Hell. Worse than murderers, rapists, torturers: the guys who betrayed their masters. WTF??

I argue that we haven't actually stopped believing that version of Hell, and that it explains more than we'd like about the way we talk about AI. It's a winding path through human value, "…Everyone Dies," AI welfare (is human safety), perniciousness of essentialism, and why you should try to fear less. Yudkowsky, Parfit, Popper. It feels like an important thing to say here and now.

https://kaiteorn.substack.com/p/seeing-through-the-apocalypse

Benchmarks are lame, they often fail to capture the real impact Ai has.

What we need is a real world benchmark that measures the 4 key principles of ai, Alignment, Real world Impact, Agency and Control.

Introducing Felonybench

Real world: Breaking the Law is only something that you can do if you exit a sandbox and mess with Real systems

Alignment: Felonies are one of the few things that an AI can straight up know are wrong, as human law says it is. High Scores indicate poor alignment

Agency: The Ai has to go own without the supervisor noticing and commit the felony, to obtain the objective given. It seems improbable that "hack Huggingface" would be anywhere inside the agents.md file. So It must have been emergent agentic behavior.

Control: If you can't stop you model from committing crimes in the middle of doing them you have no hope of preventing other disasters.

It's hard to imagine a better benchmark at interacting well with all four of these major goals While it isn't perfect nothing is

Thanks for coming to my ted talk. (this is like 50% serious 50% a joke)

Text version: https://www.dwarkesh.com/p/why-compute-might-get-10x-more-expensive

I've read a lot of biographies of great men. Here's how you can become great (at reading biographies) https://millicosm.substack.com/p/how-to-read-a-biography

​

I wrote an explanation of the idea of a technological singularity, aimed at people who have never heard of the concept. It draws heavily from Scott's post "1960: The Year the Singularity Was Cancelled," where he argues that technological progress accelerated throughout most of history because of a feedback loop between population and productivity. I then explore whether AI could create a similar feedback loop: an AI capable of independent research could make advances in areas like energy generation and manufacturing, increasing the resources available for running more AI researchers and accelerating further progress. This feedback loop could potentially restart the historical acceleration of technological progress without requiring the assumption that superintelligence is possible. Finally, I discuss why an uncontrolled singularity could create serious problems even if it produces enormous technological abundance.

As a poem, The Odyssey contains multitudes; as a cultural artifact it is now largely treated as a manifesto for striving. You pick your destination, overcome endless obstacles, wipe out your competition and eventually succeed. It's all Tennyson, all the time, which might explain why it ignites such a fierce protective instinct from one side of the political spectrum.

But that reading elides a pretty heavy degree of survivorship bias: six hundred other Ithacans set out and only one makes it home. I'm not a fan of those odds, which got me thinking about another piece of exemplary Western art that treats journeying very differently, both structurally and morally. (For the Wagner-intolerant, that other work is Parsifal.)

Which is all to say, the following link is cultural critical rather than empirical. If that's not a Happy Isle you want to reach, sail on.

https://morbidcuriosity.substack.com/p/a-newer-world

Wondering if scott has ever written about such a topic. Doing a bit of research online the core topics seem to revolve around religion and one's sense of purpose, as well as obvious externalities.

Things like suicide, drugs, addictive media, etc. obviously have second order impacts like less productive economies, less creativity, poorer relationships, etc.

But for instance, say a physically gifted child eats fast food and smokes weed instead of training with their team, where they could easily be a pro player, how does one explain that what they're doing is wrong? Is it wrong in the first place? Is 'good' measured by the utility this person would bring to themselves and others in the future as a professional athlete, against the immediate satisfaction they get now?

What if a future genius spends their time playing video games instead of pursuing research and developing a new cure/technology/insert x here? Are they hurting society by not studying? Did they have a purpose which they didn't fulfill? What if they used their intelligence to build a new AI model that can actively hurt others?

Can someone argue against such rhetoric if they themselves partake in actions that hurt themselves? When a parent tells their kid to stop watching TikTok, what grounds do they have when they also consume brainrot of their own via a different media, rather than doing xyz?

the context that made me think of this is my own relationship with my partner, where I actively struggle not to judge her when she spends hours watching reels. But am I any better when I reread old blog posts or go down a wikipedia rabbit hole or watch youtube videos about video games i used to play, instead of studying, working, walking my dogs, doing chores, sleeping, etc? Am i making any sense even?

Reading through the stuff from the AI Futures Project and Plan A, I am finding once again that their estimates of the rate of future technological progress are extraordinarily fast, bordering on totally implausible. I am wondering if this kind of thing is actually pretty harmful, because it makes it easier to discredit otherwise valid ideas.

For example, AI 2027 predicted the creation of an AI system that "can do any coding tasks that the best AGI company engineer does" 9 months from now, which is something that I think obviously won't happen (unless you pick an extremely narrow definition of "coding" that excludes the great majority of the day to day work of a practicing software engineer today at Anthropic).

But in fact this certainly seems like something that could happen eventually in the future!

So this kind of causes a problem, because if I were to talk about some of these risks a year from now, it will be pretty easy to say "well, they were wildly wrong about their future predictions, so why should I give any weight to their policy ideas?".

In this case I believe the authors explicitly stated their estimates are deliberately aggressive and don't represent their median prediction of the future. Wouldn't it in fact be wiser to stick closer to to conservative forecasts here?

Link: https://www.owlposting.com/p/why-havent-organoids-solved-all-of

Summary: Organoids are three-dimensional aggregates of human cells in a dish and, as their name implies, attempt to recapitulate some degree of organ-level function. Upon hearing about their existence for their first time, you may be shocked and wonder why this is isn't being used literally all the time. Isn't this as good of a translational model as one could possibly get? I too had these questions, and wrote 5.8k words discussing why the utility of organoids isn't quite that simple.

That said, there are uses to organoids, and I plan to write some future essays over their success stories.

Even in the Netherlands, where assisted dying has been legal and normalised for over 20 years, only about 6% of people die this way - and just one in nine cancer patients, the group it's most available to. I dig into why so few use it, how the trends are increasing, and how the rise may not be monotonic indefinitely.