Google splits the TPU, and tells you what it thinks inference is going to cost

Google Cloud Next opened in Las Vegas this morning with the announcement I had pencilled in as the most structurally informative event of the quarter, and I think it more or less lived up to that. The 8th-generation TPU is not one chip. It is two: a training part and an inference part, with different memory architectures, different interconnect topologies, and, this is the bit that matters, different price lists. Google also announced the Gemini Enterprise Agent Platform, Workspace Intelligence, and something called the Virgo Network, which is a data-centre fabric designed, per the release, "for workloads where agents remain resident across sessions of hours or days."¹

I spent the morning reading the keynote transcript, the Cloud blog posts, and the TPU v8 technical brief, and I want to walk through what this bundle is actually saying, because the structural story is more or less in the open and nobody has quite picked it up yet.

The split itself. Until the 7th generation, TPUs were generalist: the same silicon did training and inference, with software scheduling the difference. Splitting the line is an admission that these are now different businesses with different unit economics. The training part, Google is calling it TPU v8p, is the familiar story: high-bandwidth memory, dense interconnect, optimised for the handful of customers doing frontier pre-training runs. The inference part, v8i, strips out a lot of the interconnect, uses a cheaper memory tier, and is priced, per Google's published rate card, at roughly 40% of the per-hour cost of v8p for equivalent FLOPS.²

That 40% number is the one I'd sit with. It is Google's published view of what the inference/training cost ratio should be at the silicon layer, and it is substantially below what NVIDIA's current H-series and B-series lineup implies when used for the same split. If you believe Google's rate card, and Google is the one actually building and operating these at scale, so there is at least a floor of credibility, then the frontier labs running inference on NVIDIA silicon are paying a meaningful premium for hardware that was architected without the training/inference split in mind.

This is the inference economics frame doing exactly what the frame is for. The binding constraint has shifted from training cost to inference cost; the hyperscaler with the most vertically integrated stack is now pricing inference as a distinct, cheaper product; and the implied margin pressure lands on anyone serving frontier-model inference on rented NVIDIA capacity. Which is most of the market.

Virgo. The Virgo Network announcement is where the frame starts to reach beyond silicon. Virgo is a fabric, Google's language, not mine, connecting what the company describes as "persistent agent clusters" across multiple data centres, with the explicit design goal of supporting agents that hold state for hours or days. The technical detail is thin in the public materials, but the economic detail is not. Google is signalling that agentic workloads are a distinct infrastructure SKU, priced and provisioned differently from request-response inference, and that the company has built physical plant specifically for it.

I notice that this is the first time a hyperscaler has built and named infrastructure around the duration of an agent session rather than the volume of tokens. That is a meaningful reframing. A request-response model is priced per token because the unit of work is the token. A persistent agent is priced per, what, exactly? Google did not say, which is conspicuous. The rate card for Virgo was not published today. I would watch for it; whatever unit Google lands on will shape how the rest of the market prices agent workloads for the next two years.

A request-response model is priced per token because the unit of work is the token.

The Gemini Enterprise Agent Platform and Workspace Intelligence. These are the go-to-market layer over the infrastructure, and they tell you who Google thinks is going to buy. Kurian's framing, "the agentic enterprise", is a direct pitch to the enterprise CoE buyer: central IT, not the line of business. Workspace Intelligence is pitched as agents that live inside Docs, Sheets, Gmail, Meet, and do work the human would otherwise have done at a keyboard. Pricing, per the Workspace blog, is "consumption-based, measured in agent-hours."³

Agent-hours. Not seats. This is the SaaS apocalypse frame arriving in the Workspace price list, which is the part of Google's business that has been sold per-seat since 2006. Google is, voluntarily and in public, abandoning per-seat pricing for the agent tier of its flagship productivity suite. I do not think this is because Google wanted to; I think it is because seat pricing for agent work is indefensible when one agent does the work of a team. The interesting question is what the blended ARPU looks like in eighteen months. If agent-hours are priced such that a customer replacing three seats with one agent pays more than they used to, Workspace revenue grows. If not, not. The disclosure to watch is whether Google starts reporting Workspace revenue on a per-customer rather than per-seat basis, which would be the tell that the seat number has become embarrassing.

What this is a case of. This is a hyperscaler using an integrated stack, silicon, fabric, platform, application, to price the entire agentic workload from chip to invoice, and doing so before the market has settled on what an agent workload even is. AWS has not announced an inference-specific chip at this architectural depth. Azure's OpenAI-coupled strategy does not give it the same freedom to split the silicon line. Google is making a bet that if it defines the price structure for agentic compute first, the rest of the market prices relative to it.

The AI performativity frame also applies, lightly. Virgo is capex chasing a product category, persistent agents, that does not yet have a disclosed revenue line at any hyperscaler. Google is building the plant before the demand is booked. This is not unreasonable; it is how the cloud business has always been built. But it is worth naming: a meaningful chunk of today's announcement is spend chasing positioning.

What to watch.

• The Virgo rate card, when it publishes. Specifically, the billing unit.
• Whether NVIDIA responds with a training/inference split of its own in the B-series successor, or holds the generalist line.
• Workspace revenue disclosure in Alphabet's Q2 and Q3 calls. Seat count, or agent-hours, or neither.
• Whether any frontier lab, Anthropic is the one I'd watch, moves material inference workload from NVIDIA to TPU v8i. The 40% number is large enough that somebody will do the maths.

The structural story Google told today is that inference is a different product from training, agents are a different product from inference, and Google is the one pricing all three. Whether the rest of the market agrees is the question the next four quarters answer.

Footnotes

¶