Nvidia: The Tollgate Every AI Dream Has to Pay to Pass Through

Every argument about artificial intelligence eventually arrives at the same small building in the same industrial park: a data hall full of accelerators, humming, drawing power, and almost all of them made by one company. That is the plain fact behind the phrase Nvidia monopoly. Whether you are a startup in Bengaluru fine-tuning a model, a research lab in Beijing, or a government announcing a sovereign AI programme, your ambition has to pass through a tollgate that one firm operates. It is the oldest story in every gold rush — the person selling shovels does better than almost anyone digging — except this time nearly every shovel on the field comes from the same workshop, and the workshop decides who gets one.

I want to be careful about what this article is and is not. It is not a general essay about big tech concentration in AI; I have written about that separately, and if you want the wider map of who controls the models, the data and the distribution, read the AI monopoly. This piece is narrower and, I think, more important, because it sits underneath the rest: the compute chokepoint. The chips. How one company ended up holding the narrowest part of the funnel, and what that does to prices, to who gets served first, and to the shape of the whole industry.

How the position was actually built

There is a lazy version of this story where Nvidia simply got lucky — the AI wave arrived and washed a graphics-card company onto a beach of gold. That version is wrong, and believing it makes you misjudge how hard this position will be to dislodge.

The real story starts around two decades ago with a bet that looked eccentric at the time. Graphics processors were built to do one narrow thing extremely well: perform the same simple arithmetic on thousands of pixels simultaneously. That is parallel computing. Nvidia's leadership decided that this parallel machinery was not just for rendering games — that it was a general-purpose engine for any problem shaped like “do this small calculation a million times at once.” So they built a software layer to expose the hardware to ordinary programmers, and they gave it away, and they kept investing in it through years when nobody could say what it was for.

Then deep learning arrived, and it turned out that training a neural network is almost entirely matrix multiplication — exactly the shape of problem those chips were built for. That was luck. What was not luck was that when researchers went looking for a way to run their experiments faster, the tooling was already there, mature, documented, and free.

CUDA is the moat, not the silicon

This is the part most commentary gets wrong. People describe the advantage as a hardware lead — faster chips, better interconnects, more memory bandwidth. Those things are real, and they are also the most perishable part of the position. Competitors close hardware gaps. They have closed them before, in every part of the semiconductor industry, and reporting through 2026 suggests rival accelerators from AMD and from the cloud firms' own design teams are now genuinely competitive on raw rack-level performance for at least some workloads.

The durable asset is CUDA: the programming model and the enormous stack of libraries built on top of it. For well over a decade, if you wrote code that trained a model, you wrote it against this layer, or against a framework that was itself written against it. Every optimisation, every kernel, every clever trick for squeezing more throughput out of a training run got contributed back into that ecosystem. Doctoral students learned it. Framework maintainers targeted it first and everything else later. Companies hired engineers whose expertise was, specifically, making things go fast on these particular chips.

The moat is not silicon. It is fifteen years of libraries, and a generation of engineers whose hard-won skill is worth less the moment they switch vendors.

That is a switching cost measured in careers, not in purchase orders. A rival can match the chip. Matching the chip does not give you the twelve years of accumulated tuning that makes the incumbent's chip perform at its benchmark numbers on your actual workload rather than a demo. It does not give you the graduate who already knows the tooling. It does not give you the library that some researcher wrote at 2am in 2019 that your training run quietly depends on. This is why the effort to build open alternatives — vendor-neutral compilers, abstraction layers inside the major frameworks — matters far more than any individual competing chip. The competition that counts is a competition to make the software layer irrelevant, and that work is slow.

Allocation: the power that never appears on a price tag

Here is what I find most striking, and least discussed. When demand for something vastly exceeds supply, the seller stops being a seller and becomes an allocator. Price is only one of the levers. The other lever — quieter, more consequential — is who gets served, and in what order.

For several years now, the queue for high-end AI accelerators has been long. Being at the front of it has been the difference between shipping a frontier model and watching someone else ship it. That makes the allocation decision one of the most powerful acts of industrial planning happening anywhere in the world economy, and it is made inside a private company, according to criteria it is not obliged to publish.

Regulators have noticed. The US Department of Justice opened an inquiry into the company's practices; French and European authorities have run their own investigations; US senators have publicly questioned the structure of its recent deals and investments. The recurring questions are always about the same thing: whether customers who buy the full bundle get better supply than customers who mix vendors, whether switching is penalised, whether the equity investments the company makes in its own biggest customers create a loop that forecloses rivals. The company has consistently denied preferential or coercive allocation and says it allocates fairly. I am not in a position to adjudicate that, and nothing here should be read as a finding of fact. But note that the mere plausibility of the question tells you how much power sits in the allocation decision. Nobody investigates the allocation practices of a company that has competitors its customers can freely walk to.

Is it a monopoly, though?

Strictly, no — and it is worth being precise, because sloppy use of the word lets the defenders win an argument that was never the real one.

A monopoly in the legal sense means a single seller, or something close enough that rivals cannot meaningfully constrain your behaviour. Nvidia is not a single seller. AMD sells competing accelerators and has won significant commitments. Google designs and runs its own tensor processors at enormous scale for its internal workloads. Amazon has its own training and inference silicon and has reportedly explored selling it beyond its own cloud. Microsoft and Meta have their own programmes. Reported estimates for Nvidia's share of AI accelerator revenue in 2026 cluster somewhere in the region of three-quarters to four-fifths, down from a peak that was reportedly higher still — but these numbers are estimates, they vary by who is counting and whether internal silicon is included, and they move every quarter. Treat any specific figure you read, including those, as approximate and already ageing.

So: rivals exist. Share is drifting down. On a courtroom definition, this is not a monopoly.

And yet. What you have is a firm that sets the reference price for the most contested input in the world economy, that decides the order of the queue for it, that owns the software layer its rivals must emulate, and whose gross margins have been reported at levels that in any normal industry would have attracted entry long ago and been competed away. That is monopoly-like power over price and allocation, held without a legal monopoly. The useful question is never “does this meet the statutory definition” but “can the customer walk away, and what does it cost them.” For most buyers right now, walking away means rewriting years of code and retraining their engineers.

The cloud firms are the only real counterweight

The firms building their own accelerators are the most serious competitive pressure, and it is worth understanding why they can do it when a startup cannot. They have the one thing that makes custom silicon rational: enormous, predictable, internal demand. If you are running inference for a billion users, you can justify designing a chip that does only that, badly at everything else, and you can afford to fund the software work to make your own frameworks run on it.

But notice what this actually is. It is not competition arriving to free the market. It is the largest firms vertically integrating away from the chokepoint — solving the problem for themselves, at a scale that nobody else can reach. The startup still rents. The startup rents from those same firms, on their infrastructure, at their prices, which is the subject of a related piece on why the cloud is physical. Partial competition at the top of the market does not become cheaper compute at the bottom of it; it becomes a slightly better negotiating position for five companies.

When the essential input has one supplier, value concentrates at the chokepoint long before anyone downstream notices it is happening.

What the chokepoint teaches

I keep returning to a pattern that this case illustrates with unusual clarity. A technology arrives promising to democratise something. For a while it genuinely does. And then the layer everything else depends on quietly becomes property — owned, priced, gated — by whoever positioned themselves there first. That is how technology gets captured, and compute is the purest example currently running.

The thing to take from it is a habit of looking. When you are handed a story about an industry, do not look at the companies with the famous names and the consumer products. Look for the input that everyone needs and nobody can substitute, and ask how many firms supply it. In AI that question does not stop at Nvidia, either: Nvidia designs its chips but does not manufacture them, and the advanced fabrication and packaging capacity they depend on is itself concentrated in a handful of facilities. The chokepoint has a chokepoint. Follow it far enough and you find that the question of who owns AI is answered less by who owns the models than by who controls the narrow physical passage every model must squeeze through.

Gold rushes end. The diggings get worked out, the boom towns empty, the claims turn out to be worth less than the stories told about them. The shovel-seller, meanwhile, has been paid in cash the entire time, and by the end owns the road into the valley. That is not a scandal. It is the predictable result of a market where one input is essential and one supplier holds it. The scandal, if there is one, is that we keep being surprised.