If your staff use AI at work and the data should not leave the building, there are two ways to pay for it. You rent seats from a vendor, or you buy a few GPUs and run an open model yourself. The second one looks expensive until you divide it by the number of people using it.
All figures are Canadian dollars unless marked US, and they are assumptions as of November 2026. The hardware prices are my estimates, not quotes. The throughput figure is an assumption you should benchmark before you buy anything. I have tried to say which numbers are solid and which are guesses.
The two setups
Both use RTX PRO 6000 class cards. NVIDIA lists 96 GB of memory per card and up to 600 W (checked November 11, 2026), enough to hold a 30B-class open model on one card.
| 1 GPU | 4-GPU server | |
|---|---|---|
| All-in hardware price (assumption) | $33,000 | $115,000 |
| Hardware over 36 months | $917 per month | $3,194 per month |
| Rent for the space (assumption) | $500 (quarter rack) | $1,000 (half rack) |
| Power and cooling | $93 per month | $289 per month |
| Total per month | about $1,510 | about $4,480 |
How I got power and cooling: an all-in electricity rate of C$0.15 per kWh, a cooling overhead of 30 percent (a PUE of 1.3), and an average draw of about 0.65 kW of IT load for the one-GPU box and about 2.0 kW for the four-GPU server. Average draw sits below the maximum because GPUs idle between requests. That works out to 0.85 kW and 2.64 kW at the building’s meter, and 0.85 x 730 hours x $0.15 = $93, 2.64 x 730 x $0.15 = $289.
Two things to notice. Hardware is about 61 percent of the one-GPU monthly cost and about 71 percent of the four-GPU cost. Power and cooling is 6 percent of each. At this scale, electricity is a rounding error next to the hardware and the rent. If you want the power side of that argument in more detail, see the readiness checklist for a building that will host GPU servers.
What the table leaves out: the person who runs it, software, a spare for when something fails, internet connectivity, and any residual value for the hardware after 36 months. I come back to those below.
What a million tokens costs on your own hardware
Tokens are the unit that language models are priced in. Roughly, a token is three-quarters of a word.
Assume a small mixture-of-experts model of the 30B class, with about 3B parameters active per token. I assume one GPU generates about 3,000 tokens per second in aggregate when requests are batched together. This is the number I am least sure of. It depends on the model, the serving software, the context length and how many people are asking at once. Benchmark it on the exact hardware before you rely on it.
Now assume the GPUs are busy 30 percent of the time, which is plausible for an office that works days:
- One GPU: 3,000 tokens/s x 0.30 x 2,628,000 seconds in a 730-hour month = about 2.37 billion tokens per month. Cost: $1,510 / 2,365 million = about $0.64 per million tokens.
- Four GPUs: 12,000 tokens/s x 0.30 x 2,628,000 = about 9.46 billion tokens per month. Cost: $4,480 / 9,461 million = about $0.47 per million tokens.
So the range is about $0.47 to $0.64 per million tokens.
Is that cheap compared with renting the same model?
About the same, which is the point. On OpenRouter, the Qwen3-30B-A3B model was listed at US$0.12 per million input tokens and US$0.50 per million output tokens (checked November 11, 2026, one provider listed). The newer Qwen3-30B-A3B-Instruct-2507 variant was listed at US$0.30 per million output tokens on OpenRouter’s model list the same day. At an exchange rate of C$1.40 per US dollar (the Bank of Canada’s published rate was about 1.40 in late October 2026), that is C$0.42 to C$0.70 per million output tokens.
My C$0.47 to C$0.64 is inside that range. Owning the hardware is not a trick for beating the market price per token. It is a way to get the market price while the data stays in your building, and to know the cost in advance.
What that is per staff member
Divide the four-GPU server’s $4,480 per month across the people using it:
- 1,000 staff: $4.48 per person per month.
- 1,500 staff: $2.99 per person per month.
That is about C$3 to C$5 a month. It also means about 6 to 9 million tokens per person per month are available at 30 percent busy (9.46 billion / 1,500 = 6.3 million; / 1,000 = 9.5 million). A typical staff member asking a few dozen questions a day does not use anywhere near that. Peaks matter more than averages, since a hundred people asking at 9:05 a.m. is not the same as a hundred people spread across a day.
The smaller setup works for a smaller organization. At the same C$3 per head, the one-GPU box at $1,510 serves about 500 people.
Now the other side. Microsoft lists Microsoft 365 Copilot at US$30 per user per month, paid yearly (checked November 11, 2026; Microsoft’s page notes that actual prices vary by country and currency, and it requires a qualifying Microsoft 365 plan). At C$1.40 per US dollar, that is about C$42 per user per month.
- 1,000 users: about C$42,000 per month.
- 1,500 users: about C$63,000 per month.
Against $4,480, that is roughly 9 to 14 times as much. On hardware cost alone, the break-even is around 107 seats ($4,480 / $42). The real break-even is higher once you add the cost of looking after it.
Where the comparison is unfair
I would not make a purchase decision off the headline ratio. Four reasons.
A 30B-class open model is weaker than the frontier. On hard reasoning, long documents, tricky code and difficult writing, the models behind commercial products are better. For everyday staff work, such as summarizing a meeting, drafting a routine email, searching internal documents, rewording a paragraph or answering a policy question, the gap is small. The only way to know is to run your own staff’s real tasks through both and compare. Do that before you buy.
Copilot is more than a model. It sits inside Word, Excel, Outlook and Teams and can reach your Microsoft 365 data. A model on a server in your building has none of that. You would need a chat interface, sign-in, document search and someone to connect them to your systems. That is real work, and it is not in my $4,480.
Someone has to run it. A server needs updates, monitoring, a failure plan and a person who answers when it breaks at 8 a.m. A single server is also a single point of failure, so many organizations would want a second one, which doubles the cost. I have not priced staff time because it varies so much, but it can easily be larger than the hardware line.
Prices change. Hardware depreciates faster or slower than 36 months. Model prices from hosted providers keep falling. The cost per million tokens from a rented API could drop below your hardware cost, and then the case rests entirely on keeping data in-house.
Who this fits
Private AI is worth pricing when the data should not leave the building. That tends to mean:
- municipalities and other public bodies handling resident records;
- clinics and health organizations with patient data;
- law firms and legal departments with privileged files;
- research groups with unpublished or restricted data.
If your data is not sensitive and you already pay for Microsoft 365, renting the seats is usually the lower-risk answer. If your staff mostly use AI for the everyday tasks above and the data is sensitive, the arithmetic says it is worth a serious look.
Common questions
How much does private AI cost per employee? On the hardware alone, about C$3 to C$5 per employee per month in the four-GPU example for 1,000 to 1,500 staff: $4,480 a month divided by the headcount. That excludes the person who runs it and any software around the model.
How much does it cost to run your own AI server? About $1,510 per month for a one-GPU setup and $4,480 per month for a four-GPU server, in the examples above: hardware over 36 months, rent, power and cooling. Power and cooling is only about 6 percent of that.
Is it cheaper than Microsoft 365 Copilot? On hardware cost per seat, yes, by a wide margin: about C$3 to C$5 versus about C$42. The two are not the same product, and Copilot’s integration with Microsoft 365 is part of what you pay for.
What does a million tokens cost on your own hardware? About C$0.47 to C$0.64 at 30 percent utilization, assuming 3,000 tokens per second per GPU, which you should benchmark. That is close to the hosted price of a small open model.
Is a small open model good enough? For routine staff tasks, often yes. For hard reasoning it is weaker than frontier models. Test it on your own work first.
Where should the hardware go? In a room with enough power and cooling. See the building readiness checklist.
If you want a second opinion
If you are weighing this for your organization, the first step is not buying a GPU. It is finding out whether the room can host one, and what the numbers look like with your electricity rate and your headcount. I do that as a fixed-fee compute site readiness review.