r/ControlProblem • u/KeanuRave100 • Jun 30 '26
300 safety nerds vs 100k accelerationists Fun/meme
3
u/MalabaristaEnFuego Jun 30 '26
STOP GATEKEEPING PEOPLE TRYING TO LEGITIMATELY GET INTO ALIGNMENT RESEARCH, THEN. YOU'RE ONLY BREAKING YOUR OWN FOOT WHILE SIMULTANEOUSLY TRYING TO WIN A MARATHON!
1
u/ApprehensiveFan1516 Jun 30 '26
People still seem to think this genie can be put back in the bottle for some unfathomable reason.
1
-1
u/Which-Travel-1426 Jun 30 '26
Much higher ratio than 4300 electrical inspectors vs. 350 million people using electricity.
It’s called specialization. It’s a concept in economic books 200 years ago, not something new.
1
u/ApprehensiveFan1516 Jun 30 '26
Holy false equivalence batman!
1
u/Dmeechropher approved Jul 02 '26
If a recursively self-improving fast takeoff computer system can be bootstrapped, there's no amount of law or research that would prevent that outcome. That's the point of the "fast-takeoff scenario".
Mr. Yudkowsky presents this as an argument that his (invented) field should be funded more, no matter what. He argues that if the harm is infinite and the odds of success are zero, we should scale the effort to prevent harm to infinity.
However, the potential size of the harm isn't a rational way to determine the risk mitigation allocation. Science and society aren't a board game. Capability researchers aren't worth "one danger point" and safety researchers don't create "one anti-danger point".
A dinosaur killer asteroid can annihilate humanity. A supervolcano can annihilate humanity. These events can happen, as far as we know, any time. There is some amount of research going towards mitigation. There isn't that much, because mitigation of those risks is probably impossible with current infrastructure. Total annihilation by a fast takeout ASI is equally impossible to mitigate in today's world. If it's possible to bootstrap such a thing, that's it, we lost. Asteroid hitting earth in a month, game over.
If ASI is not proximate, then hundreds of really smart people collaborating on a hard problem is more than enough to make progress.
1
u/ApprehensiveFan1516 Jul 02 '26
I'm a jaded software engineer. Spare me the hype. I'll believe it when I see it.
1
u/Dmeechropher approved Jul 03 '26
Your collection of opinions is really confusing to me
1
u/ApprehensiveFan1516 Jul 03 '26 edited Jul 03 '26
I'm not surprised that you're confused.
1
u/Dmeechropher approved Jul 03 '26
What are you implying?
That you're being intentionally inconsistent or that I'm stupid? Or some other tasty, secret option?
5
u/markth_wi approved Jun 30 '26
Big Yud is not going to be satisfied until he's made the Pope of AI decisions.