And in terms of AGI it is, that the AI can't survive without humans, and if we decide to quit the game than its over for the AGI; we will happily regress in a techno-barbarian feudal state and salvage solar panels and trade copper wires while still reproducing and carring on. We are playing a whole different game here.
If AI is threatening enough that we can collectively just decide to stop it, wouldn't it also be bribing people & exerting influence? It'd be hard to come to that decision imo.
I mean, I can imagine an AI outliving humans, with sufficiently good robots under its control, I see no reason why an AI could not keep powerplants running, mine raw materials, manufacture new chips, and so on. But the timeframe of within the next decade seems highly implausible to me. Imagine an AI way more advanced than what we have now and imagine handing over control of every connected device on earth, could the AI keep the lights on without any human involvement?
Of course, the question is if there is enough existing technology to build the required robots. There is some number of robots like the ones from BostonDynamics that could walk up to a CNC machine to grab a finished part and put in a new piece of stock. There are probably more than enough motors and microcontroller boards to be found across the planet. Wiring things together seems one of the more challenging tasks. And maybe also moving stuff around as the AI has to use what is available wherever it is.
It seems to me that this would be a race against time, make the robots to keep things running before you lose the capabilities to do so. I somewhat tend towards not possible but on the other hand there is just such an enormous amount of stuff on the planet that could be repurposed in creative ways by a superhuman AI, it might be just about possible.
The regulations apply foremost to closed models. Chinese companies mostly release "open models".
And regarding fines, well the track record for EU fines against any big tech companies, US or China, is abysmal. Even if the companies are fined, the fine goes uncollected or is slashed after the big press release.
Why would they not apply to all models? The models with downloadable weights aren’t “open”. Meaning no one can actually give a summary about their training content and security. Which is what they’re asking US labs.
"The obligations set out in paragraph 1, points (a) and (b), shall not apply to providers of AI models that are released under a free and open-source licence that allows for the access, usage, modification, and distribution of the model, and whose parameters, including the weights, the information on the model architecture, and the information on model usage, are made publicly available. 2This exception shall not apply to general-purpose AI models with systemic risks."
reply