What does a dollar-per-GPU-hour actually include?
Two vendors quoting the same headline dollar-per-GPU-hour number can mean completely different things, because the rate is a bundle and every vendor bundles differently. Here is what is usually inside it, what usually bills separately, and how to normalize two quotes before you compare them.
Two vendors quote you the same headline number, say a dollar-per-GPU-hour rate within a few cents of each other, and it feels like you have a clean apples-to-apples comparison. You almost never do. A dollar-per-GPU-hour figure is a bundle, not a fact about the chip, and vendors bundle differently: different networking, different storage tier, different commitment length, a different utilization assumption baked into the denominator, and a different support tier that may or may not be included in the number you were quoted. Unbundle the rate first, and the comparison usually looks nothing like the two numbers on the page.
Compute only, or compute plus everything around it
A quote can mean “the GPU and nothing else” or “the GPU and everything needed to run it,” and the same headline rate can describe either one.
Networking. Fabric choice is a real capex line, not a rounding error. In the worked reference build in what it actually costs to run a GPU cluster, a 1,152-GPU cluster’s scale-out InfiniBand fabric alone runs about $4M of a roughly $93M build, on top of a separate north-south Ethernet layer priced at about $1.2M in the same table. That is close to $5.2M of a $93M cluster sitting in networking choices that a quote can name or leave vague. If a quote does not say which scale-out fabric is included and at what generation, that capex has to show up somewhere, either already priced into the rate or as a change order later.
Storage capacity and throughput tier. Storage bundled into a GPU quote is not one product either. Amazon’s published FSx for Lustre pricing, checked September 2, 2026, lists persistent SSD storage at $0.145 per GB-month, against $0.0230 per GB-month for the frequent-access tier of its Intelligent-Tiering storage class on the same page. On a round 500 TB of active training storage, that is 500,000 GB x $0.145 = $72,500/month on the high-throughput tier versus 500,000 GB x $0.0230 = $11,500/month on the lower tier, a headline difference of about $61,000/month, more than 6x, before the separate throughput-capacity provisioning and per-GB monitoring charges Amazon lists on the same page for the Intelligent-Tiering class. “500 TB of storage” tells you nothing about which tier, or which of those add-on charges, you are actually going to get.
Egress. Amazon’s published data transfer pricing, checked September 2, 2026, lists data transfer out to the internet at $0.09 per GB in the first tier, after the first 100 GB/month, which is free. A workload exporting a round 10 TB of checkpoints and logs off the cluster in a month is (10,000 - 100) GB x $0.09 = about $891 that month, and it is a cost most buyers do not think to ask whether the headline rate includes.
Support tier. Amazon’s published support plan pricing, checked September 2, 2026, prices its Business Support+ tier at whichever is greater of $29/month or a tiered percentage of monthly charges: 9 percent up to $10K, 7 percent from $10K to $80K, 5 percent from $80K to $250K, 3 percent above that. On a round $200,000/month of committed spend, that works out to (9% x $10,000) + (7% x $70,000) + (5% x $120,000) = $900 + $4,900 + $6,000 = $11,800/month, an effective 5.9 percent layered on top of whatever the base rate already implies. Enterprise Support on the same page starts at a $5,000/month floor with its own tiered percentages. None of that shows up in a dollar-per-GPU-hour headline unless someone asks.
The utilization assumption hiding in the denominator
A per-hour price is a fraction: total cost over some number of hours. Change the assumed hours and the rate changes with it, without anything about the hardware or the deal changing at all.
The reference build in the cluster economics post puts annual opex for a 1,152-GPU cluster at about $9.1M/yr and assumes 85 percent utilization to get billable hours: 1,152 x 8,760 x 0.85 = 8,577,792 GPU-hours/yr, which is how that post arrives at $9.1M / 8.58M GPU-hr = about $1.06/hr in opex alone. Run the identical $9.1M/yr against a 95 percent utilization assumption instead, a number a vendor could choose to quote against without saying so: 1,152 x 8,760 x 0.95 = 9,586,944 GPU-hours/yr, and $9.1M / 9.59M GPU-hr = about $0.95/hr. That is an 11 cent difference on the exact same cluster, the exact same opex, and the exact same year, purely from which utilization assumption sits in the denominator. A rate quoted against a utilization level you will not actually hit is not the number you will actually pay per useful hour.
On-demand, reserved, and spot are different products
The most common apples-to-oranges comparison is stacking a pay-by-the-second on-demand rate next to a multi-year reserved rate as if they were the same purchase.
Modal’s public pricing page, checked September 2, 2026, lists on-demand serverless rates of $7.10/hr for B300 and $3.95/hr for H100 SXM5. Lambda’s public pricing page, also checked September 2, 2026, lists on-demand H100 SXM at $3.99 to $4.29/hr depending on cluster size. Those are no-commitment, pay-as-you-go numbers.
Compute Exchange’s public reserved-GPU pricing page, published April 10, 2026, quotes an on-demand H100 SXM rate of about $2.99/hr on the same page where it lists a one-year reserved H100 SXM contract near $1.89/hr and a three-year reserved H100 PCIe contract near $1.84/hr. Put the on-demand number from Modal next to the reserved numbers from Compute Exchange and the spread is $3.95/hr against roughly $1.84 to $1.89/hr, a gap of about 2x, entirely attributable to commitment length rather than anything about the chip.
Spot sits at the other end. CoreWeave’s public pricing page, checked September 2, 2026, lists its HGX B200 instance at $68.80/hr on-demand against $34.11/hr as a spot instance, close to a 50 percent discount in exchange for the risk that the instance can be reclaimed. Asking “what’s the rate” without specifying on-demand, reserved, or spot, and for how long, is asking a question with three very different correct answers.
How to normalize two quotes before you choose
None of this requires being an engineer. It requires asking what is inside the number before comparing it to another number.
- Confirm the fabric: InfiniBand or Ethernet, and whether that choice is already reflected in the rate or would be a change order.
- Confirm the storage tier and whether it is priced at a high-throughput rate or a lower one, and ask what is billed separately on top, since the same capacity number can carry a multiple-x difference in the bill.
- Ask what happens to your bill above the included egress allowance, and whether there is one.
- Ask whether support is bundled into the headline rate or billed on top as a percentage of usage, and at what tier.
- Ask what utilization assumption the quoted rate is built on, and whether that is realistic for your workload.
- Confirm whether the number is on-demand, reserved, or spot, and for what term, before comparing it to any other vendor’s number.
The arithmetic behind depreciation life, residual value, and cost of capital, the levers that move a cluster’s underlying breakeven price, is worked in what it actually costs to run a GPU cluster. This post is the companion question: even before you get to breakeven economics, make sure the two numbers you are comparing were ever supposed to be compared.