While this article makes some valid points, it basically just ignores the reasons why the law is being passed, that is the potential for open-models to enable bio-attacks, cyberattacks, election manipulation, automated personalised scams, and who knows what else.
One might question why that is. Perhaps it's the case that Jeremy has an excellent response to these points which he has somehow neglected to raise. Or perhaps it's because these threats are very inconvenient for an open source developer.
I'm sure he'd say that open-sourcing models means that all actors have access to defensive systems and that the good guys outnumber the bad guys and it'll all work out well.
And that could be true. Or it could be false. It's not like we really know that everything would work out fine. It's not that we've run the experiment. I mean maybe it works out like that, or maybe one guy creates a virus and then it doesn't really matter how many folk on the other side, but we still get kind of screwed because we can only produce vaccines that fast. It's that's what going to happen? I don't really know, but it's at least plausible. I mean, maybe we'll automate all aspects of vaccine production and be able to respond much faster, but that's dependent on when we develop this technology vs. when AI starts significantly helping with bioweapons with someone then using it for an attack. And at that point it's all so uncertain and up in the air that it's seems rather strange for someone to suggest that it'll all be fine.
As someone who has studied both computer science and molecular biology at postgraduate level I can tell you that the chance of LLMs leading to higher probability of a “bio-attack” compared with a quick Google search is zero.
Do you know how much skill, practice, resourcing and time it takes to develop bio-anything?
You imagine some extremist could somehow use llama version 11 to print viruses from his printer ?
LLMs are not intelligent, they predict text based on what it was trained, if it could somehow build new viruses, weapons then it means the internet has MANY such information so the LLC could predict something useful, so maybe those websites, scientific papers , blog posts need to be deleted because some extremist group or state sponsored group can use them directly plus Natural Intelligence plus good laboratories.
But tell me how can I make my next LLM so it would help on say fighting biologic weapons, creating vaccines but refusing to make evil stuff keeping in mind that jailbreaking is always possible (scientifically proven)
"This paper follows a recent trend of marketing excellent theoretical work as LLMs being capable of secretly plotting behind your back, when the realistic implication is backdoor risk".
Many top computer scientists consider loss of control risks to be a possibility that we need to take seriously.
So the question then becomes, is there a way to apply science to gain greater clarity on the possibility of these claims? And this is very tricky, since we're trying to evaluate claims not about models that currently exist, but about future models.
And I guess what people have realised recently is that, even if we can't directly run an experiment to determine the validity of the core claim of concern, we can run experiments on auxiliary claims in order to better inform discussions. For example, the best way to show that a future model could have a capability is to demonstrate that a current model possesses that capability.
I'm guessing you'd like to see more scientific evidence before you want to take possibilities like deceptive alignment seriously. I think that's reasonable. However, work like this is how we gather that evidence.
Obviously, each individual result doesn't provide much evidence on its own, but the accumulation of results has helped to provide more strategic clarity over time.
OpenAI just released a response to Musk's lawsuit.
One of the emails provided as evidence of their claims is Slate Star Codex's "Should AI be open?"
The emails show Musk forwarding him an email he received to Sam Altman and then Ilya replying that for a hard takeoff scenario, open sourcing could make it easier for a bad actor to reach AGI first and that the right strategy would be to share everything in the short and possibly medium term and that it would make sense to start being less open as they got closer to AGI.
Yeah, at first I read that as it using 26.8% of the original steps, but reducing the number of steps by 26.8% is not that impressive. I wonder whether it actually reduces total search time as there is added overhead of running the neural network.
There's absolutely no way it reduces search time. A* is trivial to run per timestep. The time must be thousands to maybe hundreds of thousands of times slower.
I publish in this area and this is a common thing for reviewers to bring up when authors don't report wall clock time. And then for papers to be rejected. What's the value in making an algorithm that's drastically slower? Not much.
Sounds like half baked research that shouldn't have been published then. Papers are supposed to be an actual advance not a tech preview of something that might be useful maybe if someone else figures it out.
Some people call the flag planting. You publish something that doesn't work with keywords you suspect will be important in the future. Then hope others will cite you.
"Discussing the risks and opportunities in front of us intelligently, e/accs believe, is a sign of a flourishing civil society"
Except e/acc has made a massive contribution to lowering the standard of discourse. Beff talked intelligently on Lex and they are much more reasonable on Twitter Spaces, but on the Twitter timeline itself 90% of their posts are some combination of trash/propaganda/insults.
"Rather, their moral vision is one where more people — including and especially those who consider themselves hands-off today — actively engage with emerging technology and identify concrete plans for its development and stewardship, rather than reflexively backing away from what they don’t understand"
Again, e/acc seems to be all about "build, build, build!" which stands in stark contrast to taking a step back and thinking carefully about the impacts of what you're doing before you do it.
"Discussing the risks and opportunities in front of us intelligently, e/accs believe, is a sign of a flourishing civil society."
Again, this isn't accurate. E/acc is very much not about balance and also very much not about about discussing risks intelligently vs. almost always criticising the people making the claims instead of engaging in discussion on a technical level.
...
Criticism aside, this article paints a picture of e/acc which, while not representative of the movement as it exists, is something which it could choose to grow into.
As someone who majored in philosophy, it's kind of embarrassing that I would struggle to name names as to who might be likely to be seen as the most brilliant by future generations.
I suppose if I'm allowed to name someone who only died recently, I suspect that Derek Parfit will be remembered as someone who was simply brilliant. His teletransportation problem will remain a classic thought experiment and he'll be remembered for his work on population ethics, though it's less clear to me what the future will make of his main project: working out a unified moral theory.
On the other hand, I can name some^ names as to which philosophers are most likely to be seen as influential:
Currently, it's looking like Peter Singer, Toby Ord and Will MacAskill will all have had significant influence through the Effective Altruism movement. Similarly, Nick Bostrom has had significant influence through his book Superintelligence.
Dan Dennet had significant influence by being one of the "Four Horsemen" of New Atheism. His direct influence peaked a long time ago, but it remains to be seen how responsible he will ultimately be seen for the West's decline in religion (or whether this will just be seen as as trend that was happening anyway).
The author is an idiot who is using insults as a crutch to make his case.