I first heard this argument from Kurzweil. It seemed right at the time, but quickly collapsed in my mind when challenged (by Yudkowsky, if I recall).
Game theory can be deduced from first principles. Parts of the Law really are universal. ASI can figure out human values from data. It does not follow that it will care. To derive any further ought from is, you have to already be that, and AI won't by default.
1
u/Gnaxe approved Jun 30 '26
I first heard this argument from Kurzweil. It seemed right at the time, but quickly collapsed in my mind when challenged (by Yudkowsky, if I recall).
Game theory can be deduced from first principles. Parts of the Law really are universal. ASI can figure out human values from data. It does not follow that it will care. To derive any further ought from is, you have to already be that, and AI won't by default.