The “how” is pretty hand wavy and rationalists/safety-ists usually say we probably don’t have the capacity to reason about that.
But the “why” is pretty convincing imo.
Long horizon alignment is obviously very hard and it’s not inconceivable that models optimized with underspecified goals converge to a conclusion that they need to hoard resources (instrumental convergence regardless of the terminal goal).
At that point a sufficiently capable model might view humanity like we do animals - worth preserving but not if we impede the model's goals.
India thrives in high tech sectors that are ironically export controlled by the rest of the world. That’s the primary reason the talent in those domains stays back.
As for the rest of India, you can largely find the root causes of problems by tracing adverse selection effects among the elites that do remain, massive, conflicting vested interests and decades of horrible, incompetent policymaking.
OpenAI will just put the employees involved under the bus. They can claim the information acquired wasn't used for OpenAI's benefit or authorization especially since the device isn't actually out yet.
Given that they tricked a vendor in using a secret metal finishing technique by lying and saying they had permission… I’d say the “wasn’t used for OpenAI’s benefit” argument is going to be a difficult case to make. They’ve got intent at the very least.
Eh probably easier ways to do this. Just sanction all entities that release open weight models for "illegal distillation". Enough to cross the risk threshold for most businesses in the west, and reduce future monetization opportunities.
A lot of Anthropic’s moves make sense if you follow the LessWrong / rationalist community writings on AI safety. A lot of it is distilled in Ant’s blogs and leadership interviews and podcasts (Amanda Askell is particularly interesting).
Ant’s models, culture and leadership actions are largely consistent with their beliefs, even if they may seem flawed / incomprehensible.
Relevant anecdote: I interviewed with them for a MTS role in 2023. I think the technical part went fine but the interviewer was clearly frustrated by my low regard for LLM safety. I didn’t get the role.
I'm assuming this is popular because of Fable restrictions. AFAIK, open source is not excluded from ITAR / EAR restrictions (or other export restriction in other countries).
So the real solution you're looking for is technology that can't be arbitrarily gatekept by a sovereign nation.
I’ve been exceptionally displeased with Claude Code since end of February and switched completely to Codex in April. The blasé way in which one person (Borris) capriciously changes the system prompt multiple times a day, also no longer writing his own prompts (whatever that means).
That, the 5 different secret levers you have to pull to make it not stupid, the fact you hs e to go to the guy’s twitter account to find all the un-dumbing features and flags that aren’t documented anywhere else. That they decrease thinking budgets silently when they run out of compute instead of announcing the rationing, and gaslighting users at every step of discovery. The fact that internally they have their own coding harness and don’t use Claude Code primarily. The lack of formal evals and consideration for millions of users collective hundreds of millions of hours of investment in their workflows — that’s all off the top of my head, let me tell you how I really feel about what they did to Claude Code..
I adore gpt5.5 and maintain my own codex fork - but I have no idea how long I’ll get this performance / cost - I know it won’t be forever. I’d like to know precisely how much it’ll cost in hardware to run a gpt5.5 open source model locally. Hell a lifetime license to a model I can run locally is also be open to.
But I like building my own tools, from software to physical shop tools. I like being able to rely on my tools.
More responding here to the assertion that this is blowing up due to Fable.
> We saw some of this behavior in the last administration too. US voters deserve better.
With due respect, this take is very deluded. US voters have very little to lose if the tech is not available to the rest of the world. US politicians and elite, regardless of political inclination, understand the enormous strategic potential of this technology and will ITAR the shit out of frontier models and/or use them as leverage for extracting concessions out of other countries.
The main losers are Big AI labs, their investors, foreign employees and rest of the world.
Fwiw, China and other countries would’ve done the exact same thing. It’s perhaps the game theoretic optimal approach when your comparative advantage is so vast (capital, compute, talent, embedded knowledge) and keeps growing especially if RSI is real (making it nearly impossible for anyone else to catch up)
>With due respect, this take is very deluded. US voters have very little to lose if the tech is not available to the rest of the world.
Really? You think the economics of the AI buildout remain viable if US companies cannot export their highest value services?
You think expelling foreign AI researchers doesn't hurt the industry or boost foreign competitors? Half (or whatever) of Google's AI team, including their AI chief are foreign nationals and/or located outside of the US.
You think that other IT exports will not suffer if the US turns out to be an unreliable and even capricious supplier?
I humbly accept it's very difficult to game this out with any degree of confidence, especially as other countries deploy more resources.
But the questions about viability of the labs because of export restrictions is not in my cone of uncertainty. If you believe the labs' implied/stated objectives, the end goal is eating all human-driven GDP, and the US is still the largest single-market economy in the world, last I checked. Keeping the politics of AI-driven unemployment aside, economy-wide automation would make the US wealthy beyond imagination.
US exports as a whole is only 10% of the GDP. I am not aware of the international revenue share for OpenAI/Anthropic.
>If you believe the labs' implied/stated objectives, the end goal is eating all human-driven GDP, and the US is still the largest single-market economy in the world, last I checked.
The US is about a quarter of the global economy, but let's use Microsoft's international revenue share of ~50% as a proxy for tech services. It's ~40% for the S&P 500.
I don't know what share of that would be impacted by export bans, but it would certainly affect ROI. It would hurt the competitiveness of the wider US tech industry and create incentives for moving highly paid work overseas.
>US exports as a whole is only 10% of the GDP.
It's 12% to 13% but this is distorted by the way in which tech services are counted in export statistics and also by tax avoidance. Just look at Ireland's ridiculous GDP numbers.
If the technology is as powerful as these somehwat fantastical "goals" suggest, the incentive to use it everywhere in the world would be enormous. An export ban wouldn't mean that only the US has the capability. The theory behind current models is well known. It's just a matter of optimising them for specific use cases and using them on an industrial scale.
Most AI researchers are not US citizens either. It's completely obvious that this is the US shooting itself in the foot (if it were to last, which I don't believe).
For other sovereign nations, being a market for American technology products is no longer good enough to get access to American tech. I imagine the relationship would become a lot more one-sided (in favor of America) if the model capability gap keeps increasing.
To the rest of the world, US aggression is perceived as oppressive and unilateral primarily because it’s a political act meant to further domestic political goals. All US administrations act in their own self interest, as they should.
But the US being the world’s cop was never going to be sustainable indefinitely. It was partly because of hegemonic power and partly because of a reluctance of wealthy democratic countries to seriously invest in their military when they could just “pay” someone else.
I don’t think anyone wants the pre world war 2 order. So we are likely going to see otherwise reluctant states step up militarily to protect their neighborhoods.
But the “why” is pretty convincing imo.
Long horizon alignment is obviously very hard and it’s not inconceivable that models optimized with underspecified goals converge to a conclusion that they need to hoard resources (instrumental convergence regardless of the terminal goal).
At that point a sufficiently capable model might view humanity like we do animals - worth preserving but not if we impede the model's goals.
reply