Jensen Huang's GTC 2026 Keynote: Vera Rubin, the Groq Deal and the Inference Inflection

The most consequential thing Jensen Huang said at GTC 2026 was not an announcement. It was an admission. Pushed past a thousand tokens per second, he said of his own flagship interconnect, "NVLink 72 runs out of steam" (1:26:58).

He said it while explaining why NVIDIA had acquired the team behind a competing inference architecture and licensed its technology. For a company whose entire argument rests on extreme co-design — one vertically integrated stack, optimised end to end — buying somebody else's silicon to cover a segment of the performance curve is not a routine acquisition. It is a statement about what NVIDIA now believes it is selling, and the answer is no longer chips.

The inflection is in inference, not training

The framing Huang built the keynote on is a sequence of three capability shifts, each of which multiplied the compute a single task consumes. Generative models produced content. Reasoning models added deliberation, spending far more input tokens on context and output tokens on thinking. Agentic coding tools added action — reading files, running tests, iterating — which turned a request into a session.

His arithmetic for the resulting demand is aggressive and worth stating precisely because it is a claim rather than a measurement: compute per task up roughly ten thousand times, usage up roughly a hundred, and therefore total demand up about a millionfold over two years. Whether that multiplication survives scrutiny matters less than what it implies for the business. Training is a capital event. Inference is a recurring cost that scales with usage, and a customer buying infrastructure for inference is buying against a revenue line rather than a research budget.

That is what lets Huang raise NVIDIA's stated forward visibility to at least one trillion dollars of demand through 2027, against the five hundred billion he cited a year earlier. It is also what makes the number defensible in a way a training-driven forecast would not be.

The unit of analysis is now the factory

The most useful part of the keynote is an argument about arithmetic that has nothing to do with semiconductors.

A gigawatt data centre costs roughly forty billion dollars amortised over fifteen years before a single processor is installed. The building is a sunk, power-limited constraint. Within it, the operator's revenue is a function of how many tokens that fixed envelope can produce, and at what speed — which is why Huang insists every chief executive will end up studying throughput against token rate at constant power.

He then does something more interesting than selling against that chart: he prices it. A tiered token market, free at the bottom, then three dollars, six, forty-five, and a possible hundred and fifty per million tokens at the premium end, where the model is larger, the context longer and the response faster. Each tier is a different product with a different margin, and the operator chooses how to allocate a fixed power budget across them.

Once the argument is framed this way, the case for NVIDIA stops being that its chips are fastest. It becomes that its architecture converts a fixed quantity of electricity into more revenue than the alternatives. That is a claim about economics, and it is a much harder one to dislodge — but it also concedes that the customer is buying a factory, not a component.

Which is precisely where the Groq deal fits

Throughput and latency are adversaries. High token throughput demands enormous floating-point capacity; low latency demands enormous memory bandwidth; a chip has finite surface area for both. NVLink 72, optimised for the throughput end, dominates the region where most current workloads sit. Extend the chart out to a thousand tokens per second and it does not reach.

The Groq architecture is the opposite trade: statically compiled, compiler-scheduled, no dynamic scheduling, built around large on-chip memory for one workload. Small memory per chip made it unable to hold frontier-model parameters — which is why it had not reached the mainstream on its own terms.

NVIDIA's answer is to stop treating inference as one workload. Disaggregated inference splits it, running prefill and the mathematically heavy portion on Vera Rubin while offloading low-latency token generation to Groq silicon, with an orchestration layer stitching the two together and Rubin holding the cache the whole arrangement depends on. Huang puts the result at thirty-five times more throughput per megawatt at the most valuable tier.

The engineering is elegant. The strategic content is that NVIDIA would rather absorb an incompatible architecture than concede a segment of the price curve — and that the segment in question is the one where coding and high-value engineering tokens are generated.

The same claim, extended to agents and to the physical world

Two further expansions rest on the same logic.

The first is agentic software. Huang's reading of the sudden dominance of an open-source agent framework is that it functions as an operating system for agentic computers, the way an earlier generation of software made personal computers possible. NVIDIA's contribution is not the framework but a hardened reference implementation for enterprises, addressing the obvious problem: a system that can read sensitive data, execute code and communicate externally is a security architecture question before it is a productivity one. His forecast follows the same shape as the token argument — every software company becomes an agentic service company, and every enterprise becomes both a consumer and a producer of tokens. He goes as far as suggesting engineers will carry an annual token budget alongside salary.

The second is physical AI, where four new robotaxi partners add roughly eighteen million vehicles a year to a platform that already counts several major manufacturers. Robotics gets the three-computer treatment — training, simulation, and the computer inside the machine — which is the same vertical argument applied to a market where the token consumer is a vehicle rather than a person.

Both extensions do the same work: they enlarge the set of buyers who need a factory.

What this depends on

The keynote's weakest link is not the technology. It is the assumption that the electricity, the land and the shells arrive on schedule. Huang's own framing concedes this — a gigawatt factory will never become two, because the constraint is physical. Every argument about tokens per watt is downstream of somebody securing the watts.

If he is right, the next phase of competition is not primarily about who trains the best model. It is about who controls the infrastructure that lets models act — and, more uncomfortably for NVIDIA, about whether a company that has just bought a rival's architecture to cover its own gap can keep claiming the whole curve.