> There was also a paper years ago that showed a tweaked model coming up with 20k novel pathogens each deadly to humans in some number of hours
> If you disagree with it, where do you disagree with it?
How would a malicious actor:
1. Successfully turn these 20k pathogens into actual reproducing viruses / bacteria / etc (in a lab)
2. Successfully turn that lab prototype into something weaponized (i.e. as a bio-terrorism drop in NYC)
And then
3. How is this materially different from today? If we can produce these pathogens in a lab and they can be successfully turned into a weaponizable bioweapon, what is AI accelerating?
1 and 2 have had simple, proof-of-concept answers for more than a decade now:
> 1. Email sets of DNA strings to one or more online laboratories which offer DNA synthesis, peptide sequencing, and FedEx delivery. (Many labs currently offer this service, and some boast of 72-hour turnaround times.)
> 2. Find at least one human connected to the Internet who can be paid, blackmailed, or fooled by the right background story, into receiving FedExed vials and mixing them in a specified environment.
Those routes haven't gotten any more complicated for an AI to use in the intervening time.
And that's just one set of ideas. You can read the comments for dozens more creative ideas, and likely for responses to every objection you can think of. The key is that AI is getting better and better at problem solving, so anything a human can come up with in a few minutes is likely already within its reach.
---
> How is this materially different from today?
Because the should-be-uncontroversial assumption is that AI is going to keep getting better at every step of the process, and be able to do it faster and more at scale. Currently it might struggle, but with a thousand agents? It's already able to find novel math results and cybersecurity flaws. It would be incredibly naive to believe "social engineering" is somehow a unique and unsolvable problem for a sufficiently advanced AI.
For #1, I would seek a laboratory capable of _actually_ creating a _live virus_. That link to lesswrong, while interesting, does not actually indicate how and broadly reads as incredibly speculative.
While this is far out of my wheelhouse, I am not aware of a commercial venture doing actual organic creation or manipulation of material (I.e. capable of creating a modified strain of COVID). This broadly still seems in the nation-state level of lab.
But, just to get this goalpost out of the way, even _if_ such a venture exists, then would it not hold a bunch of doomsday cultists would already try to do this? This is what I’m getting at by my third question. What changes? Why don’t we see modified anthrax attacks in Palestine or Ukraine _today_?
I take no contention with the social engineering argument. I fully accept that LLMs, today and for a while yet, are capable of social engineering their way into anything. I likewise take no contention with the sheer amount of effort LLMs could wield in pursuit of this.
Your argument is that NASA could launch 18 rockets for ~$160 million total or ~8.8 million per launch?
Or is your argument that the "real" cost of this $1.6B SpaceX / Space Force contract is in fact closer to ~$16B or ~$160B after accounting for tax subsidies and investment fraud?
It is a for profit company. Therefore profit margin must be achieved. What is profit margin but waste? Government is allowed to actually run at cost or under cost. Contracting with for profit entities therefore makes no sense, other than to further the profit of those entities.
So your argument is that NASA, by virtue of being a government entity unburdened from profit seeking, could launch 18 rockets for less than $1.6B total or $88 million per launch?
Why are you willingly refusing to look at the entire cost of SpaceX to the US economy and subsequently the taxpayers? THAT is 20x the cost of NASA. Easily.
Theoretically, absolutely. Of course it gets messy in reality. But theoretically there is no reason they couldn’t. If someone in a certain office snapped their fingers they could nationalize spacex for example and remove profitability requirements while preserving institutional knowledge and team structure. Just musk will no longer be the richest man on the planet.
So what? I'm guessing spacex can be even lower dollar per kilo if they didn't have to make profit. That is my whole point. People are missing that point and going "well nasa doesn't have the talent blabla" that just misses this point entirely. Profit is a cost. You can never convince me profit is not a cost.
What do you think happens to this profit? Where does it go?
> You can never convince me profit is not a cost.
It is a cost, but with the cost you are getting something. And NASA (the US) as it happens literally got the best deal maybe in human history.
The US was literally sucking in space, their rockets were shit. Their human launch was shit. All their major space project were going to utter and complete shit and were literally an intersectional embarrassment. And btw the companies were still making profit, never in its history did NASA not primary work threw independent companies.
Then they bought things from SpaceX and yes SpaceX made some profit. That encouraged the largest ever investment surge into space industry. SpaceX used the profit to build a reusable rocket revolutionizing rockets and the US went from doing like 0% of the global commercial launch to like 80%.
Because that lowered cost many new in space ideas became possible. And US companies have taken that opportunity the best. Where in GEO launches the US was in tough competition against Europe and China. In LEO US is utterly dominated.
In addition to that a whole new crop of rocket launchers were designed. Many of the people designing them only got into the field because of SpaceX.
The location of SpaceX head quarter now sits in the most successful aerospace cluster in the world.
The US went from not making the best engine to making the most and the best engines in the world.
Let alone all the new companies trying everything they can think of. Space based solar to increase sat power. Platinum mining. And so many other things.
The US military profits form that too, cheaper then all global competitors. Allowing it to launch even more. And thanks to all competition in the launch market, some switched to hypersonics. Giving the military hypersonic test platforms they didn't have to develop themselves. Let alone all the crazy startups that pop up because they can find venture money.
Sorry that nobody takes you seriously when you complain about SpaceX making profit on launches when that very profit is what allowed for the US to go from being stagnate in space to being the unquestionable juggernaut all while the government was paying less then they had ever paid in their whole history.
Unless you can explain, how this would have happened, without allowing SpaceX some profit, go ahead.
> Theoretically, absolutely. Of course it gets messy in reality
That’s like saying “theoretically you could get to space by flapping your wings”—you just need to ignore inconvenient laws of physics. Economics describes forces of the real world, you cannot just wave them away.
Yes it does. You’re trying to draw an artificial distinction between what’s possible “theoretically” and “in practice” that depends on removing basic economic understandings from “theory.”
No, I said that ignoring the second order effects of nationalizing SpaceX is like flapping your wings to get to outer space. SpaceX drove the cost of launch far below what NASA and its contractors had achieved. They wouldn't have bothered to invest in doing that if they were going to get nationalized. After you nationalize SpaceX, nobody is going to make that investment in the future. Those are the second-order effects.
The point is that you have to account for those second order effects even in theory. The consequences that arise from those effects isn't some conflict between theory and practice. It's a defect in the theory itself, because the theory ignores relevant variables.
To be fair, oil nationalization and nationalization of incredibly advanced technology companies are not the same thing. See Norway as for what's possible if you do resource nationalization correctly.
But of course they do understand that and the primary nationalized the resources, and have a model where the nationalized company bids for that resource against private companies as well.
This is now an argument I can start to get behind. There isn't a need for profit seeking or "dollar optimization" and I fully subscribe to the argument that the government can just "do things". I likewise fully subscribe to the argument that the ability for the government to just "do things" has been eroded over time systematically.
But, like, that's divergent from "This SpaceX contract is pure grift" that OP was not so subtly implying.
Look, you’re absolutely technically right, but SpaceX is one of capitalism’s best examples.
Because… is the Falcon launch business profitable? SpaceX has spent massive amounts of money to launch their rockets hundreds of times to bring the cost this low. Because they have economies of scale and can learn from their mistakes at a volume no rocket manufacturer has ever achieved.
So your question should actually be: can NASA build a rocket that can do 18 cargo launches for less than $1.6B? The answer is probably no. Because Falcon 9 isn’t a one-off design, it’s the cheapest launch system ever designed. By a massive margin.
A bigger point here is that, sure, you’re right, there are many late-stage capitalistic businesses which basically only exist to increase profit and cut corners as much as possible, to customer detriment. And probably scam the government out of huge amounts of money.
Compensation for the risks taken and opportunity cost lost by doing this instead of something else.
Are you serious? I suggest you re-evaluate your priors because you're spouting things that demonstrate a severe lack of consideration for 1st, 2nd and 3rd order effects.
Plenty of brilliant work is achieved on this earth without bribing c suite with yacht and private jet money. Funny too how it is never really the engineering team tasked with actually doing the thing that gets top compensation at an org. Always the finance bros who have shoehorned themselves in between this team and capital.
C suite compensation is completely bonkers compared to what they actually do. That is all waste, whether paid out in base, equity, bonus, or whatever other benefit.
So you're worried that the shareholders are paying their employees too much?
Come on now. You're just upset and lashing out without considering things.
Touch grass man. Figure out what you actually want, and support policies that actually achieve it, instead of just going off a knee jerk reaction to people you dislike.
Brother the government just labels "profit" differently. Most people would call it grift, pork, or "inefficiency". Government waste dwarfs private sector waste by orders of magnitude. The utopia where governments efficiently and wisely put our tax dollars to their best use doesn't exist, never has, and likely never will.
Mostly because government today pays out the nose for contracted labor rates as doing anything in house has been blown apart by free market reaganists over the years.
The soviet union, the only non-capitalist country ever to have a technology comparable to ours, was really fucking inefficient. E.g. Soviets had 50% higher CO2 output per ton of steel than the U.S. due to poor energy efficiency.
And different orgs may arrive at different efficiency levels in different time scales. Says nothing about the merits of for profit vs nonprofit business.
It exists here too in small subsets of government. A parks dept that hires their own landscapers instead of contracting with a landscaping company is such an example of this utopia. A municipal power company with greatly reduced rates in comparison to a private utility. A university that runs its own prep kitchen facility over prepackaged GFS/cisco.
People talk about the industry like there was a time the government built it's rockets, then SpaceX stole money that was meant for NASA and now NASA can't build rockets anymore.
I am aware the US has operated in this reliance on for profit contractor model for a very long time. I just think it is stupid, costly, and only really serves to make connected individuals extremely rich, at the cost of wider innovation potential. People balk at the cost of large government in house projects like the manhattan project, well imagine that cost if the government had to pay contractor rates on that project instead of doing it in house. Twice as high perhaps given how these contracts usually are pricing in cost overruns in addition to profit margin?
Have you actually looked up the history of government owning its own defense contractors? Please tell me you have actually learned about this history. Because if you think having things like this be nationalized, I suggest you read some of that history.
> People balk at the cost of large government in house projects like the manhattan project, well imagine that cost if the government had to pay contractor rates on that project instead of doing it in house.
If you were actually serious, and not a political shill then you wouldn't pick one random example without anything to compare it to. You are just making assertions based on literally nothing.
And if you knew something about SpaceX and NASA you would know that NASA own internal investigation suggested that actually NASA should be in the business of doing basic things that companies are perfectly good at doing.
The Manhatten project was something large and very new on the cutting edge of science. Launching rockets in the 1960s was that, it isn't anymore. Instead of the government doing everything, allow fair contracting and companies can compete on offering the government a cheap price.
> how these contracts usually are pricing in cost overruns
Yeah I mean governments don't have cost overruns, I mean the great leader said there wouldn't be any, so we know they can't exist.
They dismiss them because if you know how something is made and your competitors don't, you always have a market edge. I consider the RD-180 engine from Russia better than Merlins from SpaceX, for example.
You can charge more or make higher margins at the market clearing price.
I thought everyone has done at least one semester of microeconomics at high school level, but the internet challenges my assumption frequently.
Software people (generally) have a limited idea of just how complex and rigid customs enforcement can be. Moving physical product between countries is actually a very hard problem.
You cannot sell homebrew (and presumably won't be able to sell distilled beverages) without a very expensive license. Homebrewers (of all forms) are generally very aware of this when they get into the hobby. Even "cute" forms of recompensation are heavily frowned upon. The most I've seen are to "pay for the glass itself".
All that is to say, the government is not harvesting tax from homebrew.
Stardew likely qualifies for the reduced store cuts. Steam _lowers_ the percentage for a game when it sells high. Still somewhere between 10 and 25%, though.
Generally, the Steam cut is considered “fair” for Indy devs. The benefits of steam (discoverability, massive audience) generate more sales. My Indy dev friends are not upset about the steam cut at all.
This, however, is one area where eventually Epic Games shines — they take a much lower cut and if they increase in popularity with gamers then steam might be forced to lower their share.
This is basically almost public information: 25% cut on earnings between $10 million and $50 million.
Yet most likely very big share of sales is well below $10 let alone $15 due to sales and regional pricing.
So yeah I doubt numbers anywhere close to those adverised.
> Generally, the Steam cut is considered “fair” for Indy devs. The benefits of steam (discoverability, massive audience) generate more sales. My Indy dev friends are not upset about the steam cut at all.
Steam no longer provide any discoverability on its own unless you either bring your own community ftom outside or spend $10,000-100,000 on marketing to gain wishlists.
If you're small 2-10 people indie gamedev studio and have external funding Valve will earn more from your game than you.
reply