This is evidence of example 2. The worst parts of it are infohazardous. Organizations that work at the intersection of biosecurity and AI have strong NDAs and the like. Or so I understand speaking to friends in the space. There was also a paper years ago that showed a tweaked model coming up with 20k novel pathogens each deadly to humans in some number of hours
Either way, the biorisk has orders of magnitude more evidence than wild basilisk speculation. Please do not put those in the same category.
Also your evidence-first approach and no concern without evidence falls you into the turkey problem. Imagine being a turkey and being fed each day. Some other x-risk turkey is saying that the situation is suspicious, that the red barn is where some well-fed turkeys have been taken and none return and that this is happening less and food has increased so maybe all of us will go to barn at once soon. You tell the x-risk turkey they have no evidence a lot of turkeys have ever been sent to the barn which shuts them up. Reasoning by induction would get you "we are happy and thriving 99 days so should be good on the 100th" and then you get slaughtered on the 100th.
Consider the importance of first principles approach where you deduce the possibility of events that will happen only once and never again (like human extinction). Nobody with a "show me the historical evidence" approach would have been able to see the industrial revolution coming.
I think the gradual disempowerment thesis here is well argued enough to be considered the default track for what will happen to us:
https://gradual-disempowerment.ai/
If you disagree with it, where do you disagree with it?
> There was also a paper years ago that showed a tweaked model coming up with 20k novel pathogens each deadly to humans in some number of hours
> If you disagree with it, where do you disagree with it?
How would a malicious actor:
1. Successfully turn these 20k pathogens into actual reproducing viruses / bacteria / etc (in a lab)
2. Successfully turn that lab prototype into something weaponized (i.e. as a bio-terrorism drop in NYC)
And then
3. How is this materially different from today? If we can produce these pathogens in a lab and they can be successfully turned into a weaponizable bioweapon, what is AI accelerating?
1 and 2 have had simple, proof-of-concept answers for more than a decade now:
> 1. Email sets of DNA strings to one or more online laboratories which offer DNA synthesis, peptide sequencing, and FedEx delivery. (Many labs currently offer this service, and some boast of 72-hour turnaround times.)
> 2. Find at least one human connected to the Internet who can be paid, blackmailed, or fooled by the right background story, into receiving FedExed vials and mixing them in a specified environment.
Those routes haven't gotten any more complicated for an AI to use in the intervening time.
And that's just one set of ideas. You can read the comments for dozens more creative ideas, and likely for responses to every objection you can think of. The key is that AI is getting better and better at problem solving, so anything a human can come up with in a few minutes is likely already within its reach.
---
> How is this materially different from today?
Because the should-be-uncontroversial assumption is that AI is going to keep getting better at every step of the process, and be able to do it faster and more at scale. Currently it might struggle, but with a thousand agents? It's already able to find novel math results and cybersecurity flaws. It would be incredibly naive to believe "social engineering" is somehow a unique and unsolvable problem for a sufficiently advanced AI.
For #1, I would seek a laboratory capable of _actually_ creating a _live virus_. That link to lesswrong, while interesting, does not actually indicate how and broadly reads as incredibly speculative.
While this is far out of my wheelhouse, I am not aware of a commercial venture doing actual organic creation or manipulation of material (I.e. capable of creating a modified strain of COVID). This broadly still seems in the nation-state level of lab.
But, just to get this goalpost out of the way, even _if_ such a venture exists, then would it not hold a bunch of doomsday cultists would already try to do this? This is what I’m getting at by my third question. What changes? Why don’t we see modified anthrax attacks in Palestine or Ukraine _today_?
I take no contention with the social engineering argument. I fully accept that LLMs, today and for a while yet, are capable of social engineering their way into anything. I likewise take no contention with the sheer amount of effort LLMs could wield in pursuit of this.
Either way, the biorisk has orders of magnitude more evidence than wild basilisk speculation. Please do not put those in the same category.
Also your evidence-first approach and no concern without evidence falls you into the turkey problem. Imagine being a turkey and being fed each day. Some other x-risk turkey is saying that the situation is suspicious, that the red barn is where some well-fed turkeys have been taken and none return and that this is happening less and food has increased so maybe all of us will go to barn at once soon. You tell the x-risk turkey they have no evidence a lot of turkeys have ever been sent to the barn which shuts them up. Reasoning by induction would get you "we are happy and thriving 99 days so should be good on the 100th" and then you get slaughtered on the 100th.
Consider the importance of first principles approach where you deduce the possibility of events that will happen only once and never again (like human extinction). Nobody with a "show me the historical evidence" approach would have been able to see the industrial revolution coming.
I think the gradual disempowerment thesis here is well argued enough to be considered the default track for what will happen to us: https://gradual-disempowerment.ai/
If you disagree with it, where do you disagree with it?