← All articles

Two B300 servers: the hard part was power, not the GPUs

This autumn a client asked us to get their first two NVIDIA HGX B300 servers running for AI inference: 8 GPUs each, air-cooled, bare metal. The plan after that was steady growth, up to about twenty servers within a year.

Everyone expected the GPUs to be the bottleneck. They were not. The slow and expensive parts were power, data center space, and the rules around buying the hardware.

The client and the figures are anonymised: provider names are left out and prices are shown as ratios. The engineering numbers are real.

Two B300 servers in numbers: 14-18 kW per server, at peak, A/B fed; ~20 kW per cabinet, air-cooled ceiling; 1.8x per-kW gap, 2 sites, 1 provider; ~35% price spread, same spec, 4 vendors; 1-7 mo lead time, same server; -1/3 3-year colo cost, circuits sized to loadTWO B300 SERVERS IN NUMBERS14-18 kWper serverat peak, A/B fed~20 kWper cabinetair-cooled ceiling1.8xper-kW gap2 sites, 1 provider~35%price spreadsame spec, 4 vendors1-7 molead timesame server-1/33-year colo costcircuits sized to load
Key numbers from the project. Prices are shown as ratios.

Start from watts, not from the GPU

An 8-GPU B300 server draws 14-18 kW at peak, depending on the platform and configuration. It has six power supplies in a 3+3 layout, so it should be fed from two independent power feeds (A and B), and either side must carry the full load.

That changes the conversation with a data center:

  • One server needs a cabinet that can deliver about 17 kW usable on each feed.
  • Two servers are 30-36 kW together, more than most existing colocation cabinets were ever built for.
  • The network switches in the same cabinet add a little more. Count them.

The first question to a colocation provider is not "do you have space", it is "how many kW can you deliver to one air-cooled cabinet, on A and B, and when".

Most colocation is not ready for it

What we heard from the market, in short:

  • Existing halls: typical cabinets were built for 5-15 kW. Our client's own cage could not be upgraded to 40 kW per cabinet.
  • Air-cooled limit: around 20 kW per cabinet was the practical ceiling. That means one B300 server per cabinet, not two.
  • Liquid cooling: rear-door or direct-to-chip cooling for colocation customers was years away at the sites we looked at.
  • New AI halls: built for single large tenants and fully contracted.
  • What was available often came as a whole suite: around 180 kW with a five-year minimum term. Far too big for a first step of two servers.

It took several weeks and three different campuses before we had a quote for exactly what we needed: two cabinets, one server in each.

Compare per kW, all-in

Colocation quotes are hard to compare at first sight. One site charges a fixed price per cabinet with power included. Another charges per cabinet plus metered power with a cooling (PUE) multiplier on top. Bring everything to one number: cost per usable kW per month, all-in.

When we did that for two sites of the same provider, one came out 1.8 times more expensive per kW than the other.

Then size the circuits to the real load. Standard 208 V three-phase circuits, with the usual 80% continuous-load rule:

Usable power per cabinet at 208 V three-phase with A/B feeds: one 50 A pair 14.4 kW, one 60 A pair 17.3 kW (chosen), two 50 A pairs 28.8 kW, two 60 A pairs 34.6 kW. One B300 server peaks at 14 to 18 kW; air-cooled cabinets top out near 20 kW.USABLE kW PER CABINET, 208 V 3-PHASE, A/B0102030401 x 50 A pair14.41 x 60 A pair17.32 x 50 A pairs28.82 x 60 A pairs34.6One B300 at peak: 14-18 kWAir-cooled ceiling ~20 kWChosen: one 60 A pair per cabinet, one server per cabinet
Usable kW with the usual 80% continuous-load rule. One 60 A pair covers a server's peak with full A/B redundancy and costs about a third less over three years than two pairs.

The first proposal had two 60 A pairs per cabinet, sized for a "20 kW per server" request. The real peak was about 15 kW. Dropping to one 60 A pair per cabinet cut the three-year colocation cost by about a third, with no loss of redundancy.

Normalise the spec before comparing quotes

We asked four server vendors for two servers each. The first round of quotes could not be compared: different memory sizes, InfiniBand adapters in some and not in others, different disks, deployment services bundled or not.

So we wrote one reference spec and sent it to everyone:

  • HGX B300 8-GPU, air-cooled
  • 2x 64-core server CPUs
  • 3 TB DDR5
  • 2x 960 GB M.2 in RAID 1 for the OS, 2x 3.84 TB NVMe for data
  • 2x dual-port 100 GbE, no InfiniBand: this is inference on independent servers, not a training cluster
  • 3, 4 and 5 years of onsite support

For the same configuration, the most expensive quote was about 35% higher than the cheapest. Lead times ranged from one to seven months. For a team that wanted to start this year, the lead time decided almost as much as the price.

Rules you will not find in a price list

  • Prepayment. Several vendors wanted 100% payment before shipping.
  • End-user review. For the newest GPUs, some vendors run a compliance review: who uses the hardware, where it will be installed, sometimes a visit to the data center and a look at the company's finances. Plan days to weeks for it.
  • Allocation. At one point a vendor advised against B300 because their lead time was unknown, and offered the previous generation instead.
  • Leasing. The credit line depends on the legal entity that signs, not on the project. A small entity got a limit far below the value of two servers. Choose the entity and prepare its financial statements before you ask for a lease.

A checklist for your first GPU servers

  1. Get the real peak power of the exact server model, and plan for A/B feeds.
  2. Ask colocation providers for usable kW per air-cooled cabinet, not for "space".
  3. Compare sites per usable kW per month, all-in, over the full term.
  4. Size circuits to the measured load, not to a round number.
  5. Write one reference spec and get every vendor to quote it.
  6. Ask for lead time, payment terms and compliance steps together with the price.
  7. Sort out leasing and the signing entity in parallel, not after the quote.

If you are planning GPU capacity and want a second opinion on power, space or quotes, write to us.

Have an infrastructure decision coming up?

Tell us what you are planning and we will come back with how we would approach it.

Email us[email protected]