Size power and cooling for AI and GPU racks

How far the numbers have moved

GPU racks used to be demanding but recognizable. A Hopper-generation rack — H100 or H200 based — draws roughly 30 to 45 kW. Heavy, but a well-designed hall with containment and rear-door heat exchangers can handle it.

The current generation is a different category. NVIDIA's GB200 NVL72 packs 72 GPUs into a single rack at roughly 120 kW. The GB300 version lands between 132 and 142. Announced Rubin-class systems are specified higher still, with the Rubin Ultra generation quoted near 600 kW per rack.

The individual chips tell the same story. An A100 drew 400 W. An H100 draws 700. A B200 reaches 1,000. Put eight 1,000 W accelerators in a 4U chassis and you have an 8 kW thermal problem inside one box, before you count the CPUs, memory, or fans around them.

If your mental model of a dense rack is 15 kW, the thing to internalize is that you are now planning for something roughly ten times that in the same footprint.

Liquid cooling is not a choice at this point

Air cooling is optimized for cabinets in the 8 to 12 kW range and can be stretched to 40 or 50 kW with rear-door heat exchangers — coils bolted to the back of the rack that catch heat on its way out. Beyond that, the physics stop cooperating. Moving enough air through a standard rack footprint to carry 120 kW is not achievable at any fan speed or aisle configuration.

This is not a preference or a cost optimization. Rack-scale AI systems in this class ship liquid-cooled with no air-only variant available. If your facility cannot deliver liquid, you cannot deploy the hardware.

Direct-to-chip is the common approach: coolant is piped to cold plates that sit directly on the GPU and CPU heat spreaders, carrying heat away at the source. Immersion — submerging the whole system in dielectric fluid — is used where density goes higher still.

The facility implications are the part that gets underestimated. You need coolant distribution units, which are the pumps and heat exchangers that sit between your facility water and the rack loops. You need facility water plumbed to the row. And you need a structural review, because a fully populated AI rack with a CDU can exceed the floor loading a raised floor was rated for. Standard cabinets are typically rated in the 2,000 to 3,000 lb range; these deployments can go well past that.

The electrical side changes too

At 120 kW per cabinet, conventional 208 V single-phase whips stop making sense. You would need an impractical number of separate feeds into one rack, and the physical congestion of that many cables becomes its own problem. Three-phase distribution is the minimum, and in practice these deployments use busway — an overhead power track that racks tap into — rather than individual circuits run to each cabinet.

Further out, the industry is moving to 800 volts DC. NVIDIA and its partners have been developing 800 VDC architectures aimed at racks approaching 1 MW, with ecosystem readiness targeted around 2027.

The reason is almost comically physical: at 1 MW per rack, a legacy 54 VDC distribution system would need roughly 200 kg of copper busbar per rack to carry the current. Higher voltage means proportionally less current for the same power, which means less copper. At some point the conductor stops being an electrical decision and becomes a structural and cost one.