Two vendors quote you the same dollar-per-GPU-hour, within a few cents of each other. Looks like a clean apples-to-apples comparison, right? It almost never is.
A dollar-per-GPU-hour figure is a bundle, not a fact about the chip, and every vendor bundles differently. Different networking, different storage tier, different commitment length, a different utilization assumption in the denominator, and a support tier that may or may not be in the number. Unbundle the rate first. The comparison usually looks nothing like the two numbers on the page.
Compute only, or compute plus everything around it
A quote can mean “the GPU and nothing else” or “the GPU and everything needed to run it,” and the same headline rate can describe either.
Networking. The fabric is a real capex line, not a rounding error. In the worked reference build in what it actually costs to run a GPU cluster, a 1,152-GPU cluster’s scale-out InfiniBand fabric alone runs about $4M of a roughly $93M build. A separate north-south Ethernet layer adds about $1.2M in the same table. That’s close to $5.2M of a $93M cluster sitting in networking choices that a quote can name or leave vague. If the quote doesn’t say which scale-out fabric is included, and which generation, that capex shows up somewhere: already in the rate, or as a change order later.
Storage capacity and throughput tier. Storage in a GPU quote isn’t one product either. Amazon’s published FSx for Lustre pricing, checked July 2026, lists persistent SSD storage at $0.145 per GB-month, against $0.0230 per GB-month for the frequent-access tier of its Intelligent-Tiering storage class on the same page.
On a round 500 TB of active training storage, that’s 500,000 GB x $0.145 = $72,500/month on the high-throughput tier versus 500,000 GB x $0.0230 = $11,500/month on the lower tier. About $61,000/month apart, more than 6x. And that’s before the separate throughput-capacity provisioning and per-GB monitoring charges Amazon lists on the same page for the Intelligent-Tiering class. “500 TB of storage” tells you nothing about which tier, or which add-ons, you’re actually getting.
Egress. Amazon’s published data transfer pricing, checked July 2026, lists data transfer out to the internet at $0.09 per GB in the first tier, after the first 100 GB/month, which is free. Export a round 10 TB of checkpoints and logs off the cluster in a month and that’s (10,000 - 100) GB x $0.09 = about $891. Most buyers never think to ask whether the headline rate covers it.
Support tier. Amazon’s published support plan pricing, checked July 2026, prices Business Support+ at whichever is greater of $29/month or a tiered percentage of monthly charges: 9 percent up to $10K, 7 percent from $10K to $80K, 5 percent from $80K to $250K, 3 percent above that.
On a round $200,000/month of committed spend, that’s (9% x $10,000) + (7% x $70,000) + (5% x $120,000) = $900 + $4,900 + $6,000 = $11,800/month. An effective 5.9 percent on top of whatever the base rate already implies. Enterprise Support on the same page starts at a $5,000/month floor with its own tiered percentages. None of that shows up in a dollar-per-GPU-hour headline unless you ask.
The utilization assumption hiding in the denominator
A per-hour price is a fraction: total cost over some number of hours. Change the assumed hours and the rate changes, even though nothing about the hardware or the deal did.
The reference build in the cluster economics post puts annual opex for a 1,152-GPU cluster at about $9.1M/yr and assumes 85 percent utilization: 1,152 x 8,760 x 0.85 = 8,577,792 GPU-hours/yr. That’s how it gets to $9.1M / 8.58M GPU-hr = about $1.06/hr in opex alone.
Now run the identical $9.1M/yr against 95 percent utilization, a number a vendor could quote against without telling you: 1,152 x 8,760 x 0.95 = 9,586,944 GPU-hours/yr, and $9.1M / 9.59M GPU-hr = about $0.95/hr.
That’s 11 cents on the exact same cluster, the same opex, the same year. The only thing that moved is the denominator. A rate built on utilization you won’t actually hit is not what you’ll pay per useful hour.
On-demand, reserved, and spot are different products
The most common apples-to-oranges move is putting a pay-by-the-second on-demand rate next to a multi-year reserved rate as if they were the same purchase.
Modal’s public pricing page, checked July 2026, lists on-demand serverless rates of $7.10/hr for B300 and $3.95/hr for H100 SXM5. Lambda’s public pricing page, also checked July 2026, lists on-demand H100 SXM at $3.99 to $4.29/hr depending on cluster size. Those are no-commitment, pay-as-you-go numbers.
Compute Exchange’s public reserved-GPU pricing page, published April 10, 2026, quotes an on-demand H100 SXM rate of about $2.99/hr on the same page where it lists a one-year reserved H100 SXM contract near $1.89/hr and a three-year reserved H100 PCIe contract near $1.84/hr. Put Modal’s on-demand number next to those reserved numbers and it’s $3.95/hr against roughly $1.84 to $1.89/hr. About 2x, and all of it is commitment length, not the chip.
Spot sits at the other end. CoreWeave’s public pricing page, checked July 2026, lists its HGX B200 instance at $68.80/hr on-demand against $34.11/hr as a spot instance. Close to a 50 percent discount, in exchange for the risk that the instance gets reclaimed.
So “what’s the rate?” has three very different correct answers. Say on-demand, reserved or spot, and for how long, before you ask.
How to line up two quotes before you choose
You don’t need to be an engineer. You need to ask what’s inside the number before comparing it to another number.
- Confirm the fabric: InfiniBand or Ethernet, and whether that choice is already in the rate or would be a change order.
- Confirm the storage tier, high-throughput or lower, and ask what bills separately on top. The same capacity number can carry a multiple-x difference in the bill.
- Ask what happens to your bill above the included egress allowance, and whether there is one.
- Ask whether support is in the headline rate or billed on top as a percentage of usage, and at what tier.
- Ask what utilization the quoted rate is built on, and whether that’s realistic for your workload.
- Confirm whether the number is on-demand, reserved or spot, and for what term, before comparing it to anyone else’s.
The arithmetic behind depreciation life, residual value and cost of capital, the levers that move a cluster’s breakeven price, is in what it actually costs to run a GPU cluster. This post is the step before that: make sure the two numbers you’re comparing were ever supposed to be compared.