I understand why (probably several reasons) they are taking this approach. But I don't think it is the right approach and I don't think it will work.
First: if they are unilaterally doing it: what about their competitors? Is this a chance for OpenAI to pass them? Or China? If either did, would that be a net-benefit for them or the world?
Maybe this is a ploy for "regulatory capture". And that might help them in the short term. But the rest of the world will continue on. Honestly, it isn't clear to me right now if the winning strategy has as much to do with being the smartest company or simply the one that can command the most compute.
If Anthropic fumbles their IPO and OpenAI scoops them, I don't think I will be happy.
> First: if they are unilaterally doing it: what about their competitors? Is this a chance for OpenAI to pass them?
The only part of this plan Dario is unilaterally committing to is the "embedded evaluators" thing, which doesn't seem like it'll necessarily cause them to slow down much.
I think there are any number of reasons a "safety" person could be concerned about any state of the art models. And so I would expect at least one of those to apply to any model.
The question then is: do we stop when the safety people say to (they will) or not?
Yeah, I'm also pretty skeptical about this. With AI companies we see time and time again that they can have benevolent, well thought-out regulations and then a few years just... abandon them - the most notable case of this being, of course, the founding of OpenAI as a nonprofit dedicated to benefitting all of humanity, and it being stolen by Sam Altman.
If Anthropic just unilaterally does the evaluator thing and can't achieve cooperation of the rest of the plan, my guess is that it'll have some impact for a few months and then they'll just stop reacting to the evaluators' reports and the evaluators would stop bothering to report anything. I think the idea is that if Anthropic does get government support for this, the external evaluations will be legally binding. The problem with this, though, is that the current US government perhaps can't be trusted to consistently enforce a regulation on a company, rather than e.g. taking bribes to not do so.
I think it was a pretty questionable thing to do by trying to front-run these researchers even if they didn’t make use of their techniques. The fact that they may have inadvertently “borrowed” their work via training data makes it much worse.
OpenAI’s behavior here — even if you only consider [their] side of the story — was (at best) in bad taste.
Strongly agree. And as one of the major AI companies, this is extremely tone deaf. If they saw a human (even if assisted) was making great progress on a major problem then you give them space. You don’t swoop in with millions in token spend to scoop them. There are tons of important problems where humans aren’t making traction - please go solve those.
Isn’t that how research works? You build on what others have done. I don’t understand the big deal. I’d rather have the result available sooner than later just to assuage some egos
I was actually pretty worried that the anti-datacenter backlash would stunt and important industry (for the US).
But I think the effect will be that the companies building them will "sweeten" the deal for the local towns where they are building -- probably by way of taxes. And while that will be a drag on their development (by adding costs), the scale won't be bad enough to really hurt.
And in that sense, everyone wins. The locales will be happy to have the extra resources. And the ones that don't take the "bribe" -- or whatever you want to call it -- won't get datacenters.
(But I do agree that panic is overblown and counterproductive.)
> "sweeten" the deal for the local towns where they are building
Yes. And datacenter companies have to foolish to slow-walk doing this. When you are building a 10 to 50 billion dollar construction project, it may be worth giving the local town 100 million (plus 10k directly to every resident) to do with as they please, and it would certainly grease the wheels.
They are slow walking it because you simply can't build the generation fast enough anyway. It's also why you are seeing a lot of project with their own generation plants on site, either stuff like the SpaceX/X/AI (I dunno what you call it man) datacenters - i.e very adhoc methane stuff or proper LNG generators ala Meta.
$500 + $15/replacement head. That’s assuming you can even get them. Right now they are out of stock, and my bet is these will never be something you can just pick up at the grocery store.
Any time I’ve bought something with frequently out of stock consumables, it’s been a mistake. It quickly falls out of favor.
Stripe has such a critical mass, I think SAAS builders don't even shop. It's just of course you use Stripe. It's a like a utility. I think even customers might be surprised to see a non-Stripe checkout when purchasing a SAAS. Might even get worried.
I suspect most CFOs/CTOs fear the risk of undertaking a rewrite of your payment processor or merchant of record layers (which can be tricky to fully test even with sandbox/test environments) more than the percentage discount and opportunity cost of not shipping more features.
reply