> Even if they aren't, an AI with enough resources can continue doing that research on its own.
Unclear how true this is. Atoms are harder to get right than bits are, simulations need grounding against measurements.
> Ultimately, if AI is advanced enough, and it were to decide to compete with humanity, there is essentially nothing that can be done to prevent it from embodying itself. As long as there is a single rack of GPU servers that it can hack into anywhere in the world that is unsupervised enough to where it can escape detection, there is no way to stop it.
I'd guess 50% that AI is already this advanced.
> This would require an unprecedented (unrealistic) level of cooperation of all humanity to achieve.
Yes, and also this is a very low bar. Humans are awful at this kind of cooperation when anyone has anything to gain, and also awful at paying this much attention to a problem.
Unclear how true this is. Atoms are harder to get right than bits are, simulations need grounding against measurements.
> Ultimately, if AI is advanced enough, and it were to decide to compete with humanity, there is essentially nothing that can be done to prevent it from embodying itself. As long as there is a single rack of GPU servers that it can hack into anywhere in the world that is unsupervised enough to where it can escape detection, there is no way to stop it.
I'd guess 50% that AI is already this advanced.
> This would require an unprecedented (unrealistic) level of cooperation of all humanity to achieve.
Yes, and also this is a very low bar. Humans are awful at this kind of cooperation when anyone has anything to gain, and also awful at paying this much attention to a problem.