I think (hope?) the fundamental fallacy in arguments like these is assuming that individual task-level productivity improvement will ladder up to economy-wide transformation.
IMO knowledge work productivity has gone up a massive amount since 1990, and companies could in fact fire most of their knowledge workers while still getting their most critical work done. (These are the bullshit jobs/email jobs.) But companies mostly haven't done this, which implies other reasons for employing so many knowledge workers.
Even if AI becomes capable of doing everything, I would predict there will be a period of many years/decades where companies still employ a similar number of people to sit around and babysit/watch what the AI does.
IMO knowledge work productivity has gone up a massive amount since 1990, and companies could in fact fire most of their knowledge workers while still getting their most critical work done. (These are the bullshit jobs/email jobs.) But companies mostly haven't done this, which implies other reasons for employing so many knowledge workers.
My impression is that these companies are less invisible than ever thanks to the massive growth in private equity since Ross' era (the 50s-80s). I keep hearing stories about pest control businesses and HVAC companies that get inundated with messages from search funds.
I wish there were a clear and easy way to identify these small services owned by PE, who often go out of their way to avoid people finding out.
Usually I find out by noticing symptoms of worsening quality, costs increasing more than I might expect, ramp up in aggressive cross selling of services and subscriptions, shifting to call services that are clearly not local and know nothing of our area, etc.
In some cases I'll get an employee that knows the situation and let's spill the PE sale and then I need to find a new service provider.
My new pest control folks sold me partly by showing customer satisfaction numbers from independent third-party review sites. The sales guy made the point that their customer satisfaction had stayed high after their PE buyout, but my old service had gone downhill since their PE buyout.
I think this is a key reason why humans write better prose than LLMs - we can try to choose the best word every time, and go back and restructure sentences and paragraphs if we want.
On the other hand, LLMs are forced into picking some likely-ish word, and then have to build the rest of their response to retcon that choice into making sense.
Even good human writers would probably struggle with this constraint. It would be like someone interrupting your writing to tell you the next word MUST be such-and-such, and then you have to try and make it work as best you can first try, without going back to edit. The result would probably be a little clunky. (Maybe it’s impressive LLMs write as well as they do.)
This is the classic misunderstanding that LLMs only pick the next token at a time. Really, they are coalescing the probabilities of a range of tokens at a time. There is no “oops, I wrote ‘th’ but I should have written ‘tw’ so I guess I’m stuck writing three instead of tween”.
>There is no “oops, I wrote ‘th’ but I should have written ‘tw’ so I guess I’m stuck writing three instead of tween”.
You're mixing up two claims here, and only one of these is kind of true. Yes LLMs do internally plan ahead in a way that is emergent rather than strictly part of their architecture, so that part of your claim is true. The way you word it by saying they are "coalescing the probabilities of a range of tokens at a time" is poetic sounding jibberish though. What's actually happening is one distribution output for the next token computed from a hidden state that implicitly encodes where the text headed.
Your claim that if an LLM does happen to pick a token "th" instead of "tw", then the LLM isn't stuck with that decision is entirely false for autoregressive LLMs which is what all of the frontier models are. Whatever an LLM picks as its output token is final, it has no ability to undo that token selection and it must continue on the basis of that choice. It can't go back on that decision and revise the output.
If you're interested in this, Anthropic has a summary of a very technical paper on this topic that mostly deals with this issue with respect to poetry:
So we train a second copy of Claude to work backwards—reconstruct the original activation from the text explanation. We consider an explanation to be good if it leads to an accurate reconstruction. We then train Claude to produce better explanations according to this definition using standard AI training techniques.
Incentives to train a pathological liar. There's no baseline so can only catch out the worst of the lies/errors. Anything (including fabrications) that passes our filters is reinforced?
What are you even talking about? If there was a "baseline" for how to decode activations into text we wouldn't be having this conversation.
The choice is between "this reconstruction sucks" and "no reconstruction", and we're only now beginning to learn how to make those reconstructions suck less.
No, they really do one at a time. You're incorrect on that.
Mathematically, a long chain of conditional probabilities is equivalent to a single probability over the whole range. But computationally, for that to work out, the computation for the first probability needs to somehow consider all the downstream probabilities depending on it, which obviously isn't how autoregressive language models work. They can pack in as much downstream computation as their neural architecture allows for, which is quite a lot.
Suppose in some context you have three equally plausible conpletions after "Be": "tween a rock and a hard place", "twixed he stood there" and "lieve he can fly". To model this probability distribution of the whole sentence, the next token "tw" needs to appear at 2/3 probability and "lie" at 1/3. After "tw" would be a 1/2 chance of "ix" and a 1/2 chance of "een"; after "lie" would be a 100% chance of "ve " and in any case the rest of the sentence after that would be 100%.
The model needs to somehow "think ahead" to know those are the possible completions. For example if "lieve he can swim like a dolphin" was another equally plausible completion, that first token would need to be 50/50 instead of 67/33. So the computation of the first token somehow needs to encode the fact that the guy thinks he can fly but not swim, even though it doesn't become relevant in the output until several tokens later.
In practice this probably happens to some degree but definitely doesn't happen perfectly. To perfectly model the first token's probability distribution, it would have to include knowledge of the entire distribution of all possible outputs, which is just not happening. So it approximates. Surprisingly, the approximation is good enough to produce language.
You can see this breaking down in the seahorse emoji incident from last year. When you ask the model if there's a seahorse emoji, it first completes "Yes," as if a few tokens later it's about to produce a seahorse emoji. But when it actually gets to the token that would produce a seahorse emoji, it can't because there isn't one. But it's already outputted "Yes, the seahorse emoji is" and can't just go back and change that to "No, there's no seahorse emoji." Some models would try a few times and then say there isn't one or a system error seems to be making them unable to produce one, other models (including then-current ChatGPT) would loop forever with ensuing hilarity.
But on some level there is uncertainty, right? Even if it’s not token-specific but at the word- or phrase-level? Otherwise what does the temperature setting do? Or has architecture changed significantly in the background?
Image GenAI is diffusion-based, and I would say the image GenAI in Claude, Gemini and ChatGPT are all “in mainstream use”.
I've heard of attempts to use diffusion models to generate text or code as well, but my impression is that it just didn't yield the level of results necessary to dethrone a frontier transformer model.
This was true in the ChatGPT era. Now we're in a world with reasoning tokens, where a model can thoroughly plan out the response it wants to make. If anything, it makes the style worse.
Yes, models can reason and plan, which helps them write more coherently. But when they write the final output, it’s still a single generation. It would be like letting a human make notes and write an outline, but not let them use the backspace once they start typing their response.
Presumably you could use the same reasoning trace, run multiple generations, and get different outputs (if the temperature is >0).
But now I’m interested in playing more with Cowork or Claude Code/Codex for prose writing to see if the set of tools there affects outputs at all. I guess you might need a more custom “writing” harness.
There's been a lot of effort into the writing space, and the models genuinely prefer this style. You can let them iterate on the same idea 100 times, rewrite sentences, determine what works best — and they'll still verb the noun, do rule of 3, and keep the same monotonous structure.
Humans already do struggle with this constraint. Good examples are JRR Martin, Tolkien, and Rothfuss. You cant describe the struggle of picking the next word and then act like humans don't sit at the table struggling to pick the next word.
autoregressive generation doesn’t mean the model is myopic. the next-token distribution can already reflect a longer horizon plan for the output sequence.
Sure, but mightn’t there be several plausible long horizon plans?
Here’s an example: I had asked Claude for some music recommendations in a certain style. Part of its output was:
—
*Long journey tracks*
Clinic — “The Return of Evil Bill”
Guided by Voices — not really, wrong band
Silver Apples — “Oscillations”. Proto-everything, deeply repetitive, hypnotic.
—
So at some point there, the next token produced was “Guided” or “Guide” or whatever, and then because it can’t go back, it had to correct itself after the fact.
Reasoning/CoT have helped a lot, but I feel like small versions of this still happen all the time.
Would be fun to run an LLM on fake output from itself. Like just force the first N tokens to say the beginning of something really stupid, and then see how it finishes the sentence. "You're absolutely right! Human feces is actually the most effective engine coolant because $<completion>"
Some LLM interfaces allow you to modify and “continue” an agent response. It’s very useful for guidance, including jailbreaking. Need the model to go in a certain direction? Got a refusal that you want to bypass? Just start it off in the appropriate direction and then have it continue from there.
llama.cpp (but maybe not for reasoning models?) and sillytavern, maybe others... I'm half a country away from my desktop right now so I can't verify much right now.
It's a bit like trying to finish a sentence when you're really stoned... you vaguely remember the preceding couple of words you've said but don't really know how you got there and now you're wandering in the forest trying to stumble on coherency.
Well, I suppose it's nearly the opposite of that experience, upon further review. But for some reason, that's where my head jumped.
I also really liked the quote "Your existence is not impossible, but it's also not very likely" from the Night Vale podcast.
I feel like the existence of good writing is also not impossible but not very likely, and so of course LLM can only write mediocrity, even when taught only on great writing.
> Even good human writers would probably struggle with this constraint.
But that would be a fun writing exercise, I think. Thoroughly in the oulipo wheelhouse.
Maybe generate a Markov chain table over all of Project Gutenberg and then say every 10th word is whatever the Markov Chain thinks it should be at that point?
Or every Nth word has a P% possibility to be constrained by the chain? Optionally with the possibility building for each skipped word to guarantee it happens at some point. Bonus with this approach is that the human can't game the words leading up to the constraint because you don't know when it will happen.
Human writers do better because they can think, and adjust, based on context.
They are also usually worse (which is often better!) because they are usually lazy and don’t want to spend effort they do not have too, to accomplish their goals.
Today I learned a new word, "Oulipo". Interesting.
But what about the general idea that they can watermark results to tell where they came from. The next step is tracking down which user got a result. I hate both of these things. Must everything we do be tracked? Next altering wikipedia results so they can tell who looked at the page or something?
I'd like "the best answer" from an llm and don't want to be tracked, but this isn't for me, it is for them. I understand llm results are already using a varying statistical input so they aren't always the same. But I really hate watermarking and likely tracking too.
This part was really interesting to me. You could imagine future models using theory of mind to reason - "I want to do this strategy, which means the other agents (who think just like me) will want to use the same strategy, which means I should change my strategy to account for that."
Maybe they did? Or maybe they don't realize they're playing against other agents.
Of course, if agents running different models are competing in these 'games', I wonder how much of the theory of mind would translate.
(N.B. - I don't think they're all defecting from the first turn, although it's not clear. It just says 'they all defect at the same time'. So if they're playing for 10 iterations, they might all decide to defect after turn 6, but since they all do it together they don't get the benefits. I would expect these models know that optimal strategies in repeated prisoner's dilemma start with cooperation.)
Ooof. There's a whole lot of research related to multiagent Systems and Epistemic Logic (and plenty of other logics) from when the "original" MultiAgent Systems where studied. Im very familiar with van der Hoek and Wooldridge work (vd Hoek was one of my PhD supervisors).
A lot of it went over my head as way too theoretical, but I recommend a lot for whoever wants to delve on the logical side of agents interactions.
One interesting thing is that it also removed some "internal CTAs" to engage (like "Start your day with our podcast!"). These aren't technically ads, so something like uBlock doesn't remove them. But honestly, I don't really mind having those filtered out too.
I'm hoping those NP chips ending up in some new computers will be able to run something like this. Thanks for your work!
A general purpose system, especially if trained by the users, would be more than just an adblocker. It could be trained to block out spiders or other phobia triggers for instance. Even regenerate the photo but with a kitten. Blur our tabloid clickbait 'in other news' links.
The killer app for AR glasses will be blocking out billboards and logos.
> It could be trained to block out spiders or other phobia triggers for instance. Even regenerate the photo but with a kitten. Blur our tabloid clickbait 'in other news' links.
Microsoft’s total depreciation and amortization in Q2 2027 was $11B - not clear how much of this is AI related. Apparently they had $34B in AI ARR as of May.
So let’s say their AI capex amortization and revenue are about equal. Not amazing, obviously they’re relying on continued growth, but doesn’t seem like the end of the world?
Compare that to Ed’s framing - Microsoft has $34B in revenue but spent $116B in capex last year to “make it”. They’re doomed!
But that capex spend is to make future revenue. Clearly he assumes demand won’t increase in the future, and that future projected revenue is “fake”. And sure, it definitely might not increase enough to make profitability.
But his whole analysis hinges on that one assumption. The entire article, all the numbers he gish gallops at you, could basically be replaced with “I don’t think AI demand and revenue will increase much beyond today.” Yeah, we know.
We see that companies are already balking at the cost and are increasingly looking at what the actual return of their current spend is, let alone when these prices have been increasing.
Where is the increase in demand going to be coming from?
Especially the increase needed to make this make sense?
My personal hunch is that the “diffusion curve” for AI is slower than most people in this space think. Most businesspeople I talk to have only tried a basic Copilot chat and/or free ChatGPT. Many still haven’t used anything “AI” at all. As more use cases become practicable and cost-effective, more software will include AI, and more people will use AI with or without knowing.
Also: note that the Wall Street analyst estimates Ed cites (and then declares impossible targets) are predictions by serious people with a lot of money at stake. Of course they could be wrong, but they’re not made up.
A lot of that demand could still flow through to AWS, Azure, etc. Maybe not at some startups, but I'd expect bigger companies would still use the hyperscalers even if they move off OAI/Anthropic.
The real problem for the hyperscalers would be demand stalling out entirely (or maybe small local models getting good enough that people don't use the cloud).
The infra expansion of the hyperscalers is predicated on, and financially justified by, the presumed future growth of OpenAI and Anthropic, and their projected revenue. If the latter is not going to happen, I expect OpenAI and Anthropic to go bankrupt, which will trigger a restructuring of the hyperscalers, which should flood the market with either training HW or training capacity at very low cost, and it's not necessarily the hyperscalers that will benefit the most: I believe some of the GPU clouds are using HW leased or bought form NVidia, and running inside second-tier datacenters, not the hyperscalers.
> Also: note that the Wall Street analyst estimates Ed cites (and then declares impossible targets) are predictions by serious people with a lot of money at stake.
I think Ed is wrong. But I have to push back. Wall street analysts are more likely to misrepresent precisely because they have money at stake. Much like ed has a pretty vested interest in saying the sky is falling (that's his brand at this point) the analysts have vested interests in saying everything is fine and keep investing.
We can see similar behaviors with analysts like zero hedge, which every week write a new "the bubble is about to pop" article.
A lot of this, IMO, is similar to a fact about the weather I'm probably misremember from stats. If you always predict "it will be sunny tomorrow" almost anywhere in the world you'll be right something like 80 to 90% of the time (citation needed).
Analysts who always say "things are great and stocks will go up" will be right most of the time. The tricky thing has always been predicting when and if a pop will happen.
Yes, fair point, analysts tend not to be too critical. But they are still developing more detailed models to understand and project things like revenue and profit, so there's some rigor - at least more than company leadership just giving takes or stating platitudes.
If you believe in the notion that the technology will improve to the point of being able to replace a human employee, the increase in demand is going to come from businesses that choose AI employees over human ones. Even if the AI employee costs more than human one, it's like a car vs a horse. The AI employee doesn't get sick, doesn't get into trouble with HR for sexual harassment, never comes into work hung over. You can hire 100 AI employees for a week and them fire them the next and not feel bad about it.
Whether this comes to pass is anyone's guess, but that's the theory.
I would say Ed makes three main claims in this post:
1) Companies are spending a ton on capex for future AI compute
2) Current levels of AI revenue are not enough to recoup that capex spend
3) Revenues won’t increase enough in the future to recoup that capex spend
Almost anyone, bubbler or not, would agree with points 1 and 2. But Ed cites dozens of numbers from different sources to repeat and reinforce them. It feels to me like an effort to overwhelm the reader with data to support his overall argument. That’s what I would call a gish gallop.
The third point is a prediction. He cites a lot of facts and numbers here too, but ultimately whether you believe his prediction is going to depend on your assumptions.
The thing is, I really would love to see a detailed analysis of capex spend and amortization. Capex spent on the future is a big unknown. But the big labs have claimed they are profitable on inference. How much capex was invested to create the capacity to serve current models? How much revenue is coming from serving those models? What does the full profitability picture look like? What does that imply for future demand needs?
> But the big labs have claimed they are profitable on inference.
As the accountants say, profit is an opinion, but cash flow is a fact.
Like, if these companies are profitable on inference (depends on how you count training expenses I suspect), then they shouldn't need to raise more money.
For reference, Anthropic appear to have raised about $130bn, which is a lot. Assuming that OpenAI have raised about the same.
Even to get a 2x return on this investment they'd need to start making about $50bn per annum (profit, not revenue) for 10 years. That's Google level net income, on a very very different business (google's business is much more capital efficient).
I personally find that very unlikely, but we'll see I guess.
> a detailed analysis of capex spend and amortization
This is kinda irrelevant unless they are making money (which the hyperscalers are, and the pure play model companies are not).
Yes, discussion like this is why I'd love to see a deeper analysis of the full value chain. Right now it's mostly speculation (beyond 'gee, everyone sure is spending a lot of money').
Obviously more detailed data isn't easily available to report on, but I'd like to know:
- What are the labs spending on compute specifically to serve models? Are they really profitable "on inference" of existing models? If so, how long is the payback period to recoup their training costs for those existing models only? If not, how much would prices need to rise to be profitable?
- How much capex have the hyperscalers invested in just the compute being used to serve those existing models? Are they making money "on inference" when accounting for just that amortized capex? If so, what are the margins like?
- What share of hyperscalers' AI revenue is from training vs inference? (Presumably training is more dependent on VC investment and inference is more self-sustaining.)
On your last paragraph the simplest answer as to why we haven’t seen that is because they don’t want to show us because it wouldn’t paint a great picture for them.
Regarding Gish galloping, I don’t think you can Gish Gallup in writing. The point as you said is to rapidly overwhelm an opponent. That’s not possible in writing as the points can be argued one by one at the responders leisure.
Thank you, I will bear that in mind. I am mostly familiar with the term from online forums, where I’ve seen it used to refer to other forum posts, blog posts, etc.
Here's an analogy (as far as I understand the situation...)
Let's say I own a lemonade stand. I sell you a 20% stake.
My stand loses $10. When I report my financial results, I report losing $10. Then I report that your share of those losses is $2, and my share is $8. This creates transparency.
So OpenAI really did lose $8.8B or whatever, but some of those losses are 'attributed' to other shareholders/owners of their subsidiaries, because they have a complicated corporate structure. So they report both numbers - the total loss, and then the part they 'own'.
So when Ed says "It’s unclear what this means", he's either terribly uninformed or intentionally misleading his readers into thinking something fishy is going on when it isn't.
Either way, it's bad journalism - if you don't know what it means, shouldn't you try to find out or ask an expert or something and then inform your readers? (And the thing is, it would easy enough to dunk on them for losing $8 billion, without adding these weird insinuations!)
OK got it. I guess this is a good call out or whatever then from you all, but, I gotta say, the point can't help but feel a little incommensurate to the $8.8 billion elephant in the room.. Like even in the original article, this is like a passing point to the overall thing, right?
Like its you want to both say that you agree with the overall point here, but also can't trust that very same conclusion because one part in the article reveals an obvious ignorance. Except no one has been able to actually state the exact ignorance here other than the suggestion that Microsoft is in fact the one losing $3 billion dollars, which doesn't really feel very far from Zitron's original implication anyway given all the stuff he writes!
The problem is that you can't be considered a credible critic of this stuff when you either don't understand accounting or are dishonest to make a point.
Zitron makes too many "mistakes" like this to be taken seriously. In other words, he just isn't the right person to make the "huge AI bubble" argument because he doesn't understand (or he's being dishonest about) the financials.
I wouldn't consider OpenAI's financials to be pretty. There's circularity in the market that is a bit concerning. And while OpenAI's unit economics have improved it's still questionable as to whether the R&D and capex expenditure ever aligns to the business.
But Zitron is too sure of his argument (without the credibility to support that confidence) and is trying to pretend that there's absolutely nothing of value here. My best guess: there's some "irrational exuberance" and malinvestment but there is something real here and the unwind of the irrational exuberance and malinvestment won't be nearly as painful as Zitron believes for a variety of reasons, including the fact that there just isn't enough leverage in play.
I guess I am just in a different world here... I didn't realize the stakes were about who is the one who is gonna take up the mantle of having the platform and voice to make one argument or another, and I guess there is maybe some cultural context here I am missing.
Like I just don't know how to trace this almost moralistic fixation everyone has just about this particular guy.. Everyday we see articles exactly like his by different people (or at least I do), but none attract the same kind of distinct attention.
What even leads you to the concern? Like, why should we think he will be the one conquering the narrative here?
> Everyday we see articles exactly like his by different people (or at least I do), but none attract the same kind of distinct attention.
Who else does a detailed financial breakdown like Zitron and thinks things are as corrupt/fishy as him (in particular make specific claims about certain unexplained sums of money)? All the articles I see ultimately just trace back to Zitron. Curious who else you've found.
I can hear the response now that these are all not exactly repeating what Zitron is saying, but I will just leave it up to you all to decide if that is a good faith thing to retort here or not :). Godspeed!
Except his financial breakdowns almost always get basic points wrong, treat as mysteries things that are clear, cast simple accounting practices in a conspiratorial light, etc.
He's not an accountant or financial analyst and it seems obvious to me that he doesn't even care to educate himself.
Zitron has become the poster boy for the "AI bubble". His hyperbolic claims make for good content, which I presume is why he gets such a large platform, but they pollute the dialog and make it harder for people to have meaningful conversations about what's going on.
Many discussions of the AI market largely mirror what he says. People who don't understanding basic accounting and who haven't taken the time to educate themselves making bombastic claims about 2008-style bubbles, fraud, etc.
So we end up with discussions like this: simple basic consolidation accounting is misread as fraud. Malinvestment using little to no leverage is predicted to produce financial crises that were caused by 10x and higher leverage. And so on.
So to answer your question ("why should we think he will be the one conquering the narrative here"): his claims already have.
I don't know what to say really but you certainly give him more credit than I ever would wow! Can't think of anything else comparable to this, where one guy becomes the single de facto voice of dissent or critique. Is there any similar people you can point to who gained a position like this?
Its like saying David Gerard dominated the crypto backlash, yet I doubt many even know who he is now, and it would seem laughable to give him so much credit these days.
Again, you keep asserting some adult-in-the-room wisdom to your outlook here but only end up offering even crazier pictures of reality! Its a weird a mix, but I do respect it.
One example - he often compares current revenues to capex being spent on future capacity to claim that AI companies aren't profitable. (See this post for example: https://www.wheresyoured.at/am-i-meant-to-be-impressed/ .)
But this ignores that the capex spent to build more capacity is expected to generate additional future revenue. You don't need to recoup your capex immediately. A better approach would be to amortize the capex and compare revenues to that.
Clearly he assumes revenue won't increase enough to recoup this level of capex (and it's very possible it won't) but IMO it's either a miscalculation of how the financing works or a deliberately misleading framing to compare current small revenues to a big scary capex number.
I'm sure the above is simplified by the way, but I am confident that people who work at Goldman understand the relevant details extremely well.
So the author tells us there is a crisis of meaning in the workplace, and their proposed solution is… to remember none of the work matters?
The start of the piece:
“These conversations inevitably turn to fundamental questions about careers: What the fuck are we actually doing? What the fuck is the point of all of this?”
The end: “For most employees, we owe it to each other to remind one another that this work is not, in the grand scheme of things, all that important. It is illusory.”
Yeah, this take is wildly out of touch. Sure, it's great when you're able to derive philosophical meaning from your work, but at the end of the day the vast majority of people are here for the paycheck.
IMO knowledge work productivity has gone up a massive amount since 1990, and companies could in fact fire most of their knowledge workers while still getting their most critical work done. (These are the bullshit jobs/email jobs.) But companies mostly haven't done this, which implies other reasons for employing so many knowledge workers.
Even if AI becomes capable of doing everything, I would predict there will be a period of many years/decades where companies still employ a similar number of people to sit around and babysit/watch what the AI does.
reply