Yes, discussion like this is why I'd love to see a deeper analysis of the full value chain. Right now it's mostly speculation (beyond 'gee, everyone sure is spending a lot of money').
Obviously more detailed data isn't easily available to report on, but I'd like to know:
- What are the labs spending on compute specifically to serve models? Are they really profitable "on inference" of existing models? If so, how long is the payback period to recoup their training costs for those existing models only? If not, how much would prices need to rise to be profitable?
- How much capex have the hyperscalers invested in just the compute being used to serve those existing models? Are they making money "on inference" when accounting for just that amortized capex? If so, what are the margins like?
- What share of hyperscalers' AI revenue is from training vs inference? (Presumably training is more dependent on VC investment and inference is more self-sustaining.)
Obviously more detailed data isn't easily available to report on, but I'd like to know:
- What are the labs spending on compute specifically to serve models? Are they really profitable "on inference" of existing models? If so, how long is the payback period to recoup their training costs for those existing models only? If not, how much would prices need to rise to be profitable?
- How much capex have the hyperscalers invested in just the compute being used to serve those existing models? Are they making money "on inference" when accounting for just that amortized capex? If so, what are the margins like?
- What share of hyperscalers' AI revenue is from training vs inference? (Presumably training is more dependent on VC investment and inference is more self-sustaining.)