TechnologyUnited States

The 250-Kilowatt Rack Becomes the Open Compute Blueprint

Credited to Rackline · Rachel Okonkwo · blogs.nvidia.com

Useful
Photograph · Rackline · Rachel Okonkwo

At this week's Open Compute Project Global Summit in San Jose, much of the news was about the server rack. Read together, the announcements from Nvidia, Meta, Super Micro and AMD point to one shift: the rack, rated at as much as 250 kilowatts and plumbed for liquid, is turning into a published specification that the rest of the supply chain can build to.

The anchor is Nvidia's Oct. 12 contribution of its Vera Rubin rack and compute-tray designs to the Open Compute Project through MGX 1.2, the third generation of its modular reference architecture. Nvidia said the updated specification supports rack deployments up to 250kW and adds rack-scale blind-mate liquid cooling, interoperable rack-scale interconnect and a liquid-cooled 5,000-amp busbar designed for higher-power systems.

The tray inside the rack changes as well. MGX 1.2 specifies fanless, liquid-cooled compute trays, configurable input-output bays and standardized power and cooling interfaces, and adds native support for 19-inch equipment. Nvidia said the design is meant to cut assembly, test and service times.

Nvidia also contributed a Vera CPU rack specification built on the same MGX frame. The company positions it as a companion to the Vera Rubin NVL72 system, handling the tool calling, orchestration and sandboxed execution that AI agents need, while the GPU racks carry training and inference.

The significance lies in the format rather than the hardware. A design published through OCP gives cooling vendors, power-shelf makers, rack fabricators and system integrators a fixed target. Nvidia said the effort helps turn proven AI-factory designs into standards that others can adopt, manufacture and scale, and it paired the rack work with open liquid-cooling connector technologies it calls MQD and MQDB.

The specification also reaches beyond the rack. Nvidia said it joined AMD, Google and Meta in publishing an Open Data Center Specification, version 0.7, covering facility power, cooling and layout. Separately, Google, Meta, Microsoft and Nvidia published a grid-resilience specification describing how large AI sites should stay connected through utility disturbances and recover to near-full load in under two seconds.

Super Micro said on Oct. 13 that it had expanded its portfolio of OCP-accepted systems, including what it called its OCP-contributed MGX 1.2 rack series, engineered exclusively for Vera Rubin NVL72 and Vera CPU platforms and supported by its DLC-2 direct liquid-cooling stack. The same release listed ORv3 racks of 44 and 48 OU with integrated busbars, including MGX 1.2 racks, rated for a static load of 5,000 pounds.

The rest of Super Micro's list shows how much of the rack's surrounding equipment has to change at 250kW. The company introduced a 4U in-rack coolant distribution unit with up to 250kW of cooling capacity, running on 45-degree Celsius warm water. That cooling figure matches the 250kW rack ceiling in Nvidia's MGX 1.2 specification. It also listed liquid-cooled 2 OU Nvidia HGX B300 nodes built to the ORv3 specification, with up to 18 nodes and 144 GPUs per 21-inch rack, plus rear-door heat exchangers rated from 10kW to 120kW to remove residual heat before it reaches the data hall.

Super Micro is also building outside the Nvidia frame. Its release included an AMD Helios platform built on the ORW rack architecture for inference, frontier-model training, fine-tuning and sovereign AI work.

Meta's contribution was in the network, which has to keep up as each rack draws more power. In a joint announcement with Nvidia, Meta said it is adopting Nvidia's Spectrum-6 102.4-terabit-per-second Ethernet switches to build larger Vera Rubin AI factories on open networking. Nvidia said the combination supports agentic AI factories of more than 100,000 GPUs.

The switches land in hardware Meta built for the purpose. Meta's engineering team described IcePack, which it calls its first liquid-cooled network rack, based on OCP's ORV3-HPRv1 rack standard. Inside sit IceCube switch blades, each delivering 128 ports of 800 gigabits per second, in one version built on Broadcom's Tomahawk6C chip and one on Nvidia's Spectrum-6. Nvidia identified the Spectrum-6 version as N-IceCube.

Both versions run FBOSS, Meta's open-source network operating system, on OCP's Switch Abstraction Interface. Meta said the generation doubles bandwidth from its 51.2-terabit platforms while meeting the thermal density next-generation AI data centers require. Liquid cooling, in other words, is no longer confined to the compute tray; Meta has brought it into the switch rack as well.

Meta also said it is launching an Open Switch Platform Architecture initiative within OCP, co-led with Microsoft and Alibaba, to standardize the layer beneath the switch software, with a version 1.0 specification submitted to OCP. Meta argued that this layer has remained proprietary and fragmented, so that each new platform needs months of custom engineering to bring up.

AMD's announcements show the rack format spreading beyond GPU systems. On Oct. 12 the company introduced Helion, an open blueprint for agentic AI racks based on an OCP specification design. One configuration is a CPU-only rack of 6th Gen EPYC processors with up to 32,768 cores, aimed at hyperscalers that want dedicated capacity for running agents. Another combines EPYC nodes, Instinct MI350P GPU nodes and general-purpose compute in one rack for enterprises.

AMD said Helion builds on the open-platform approach of its Helios rack-scale system, comes in liquid- and air-cooled versions, and will be contributed to OCP. The company said Helion configurations are under development with partners and are planned for November 2026. Cisco, Dell, Hon Hai, HPE, Lenovo, QCT, Super Micro and Wiwynn supplied statements of support in AMD's release; HPE's referred to a recent customer agreement to deploy Helios.

Taken together, the CPU racks from Nvidia and AMD point to a quieter consequence of the standard. Agentic workloads need racks of general-purpose processors next to the accelerators, and both companies are now specifying those racks inside the same open frameworks they use for GPU systems, so that power, cooling and service procedures carry across the data hall.

The component makers that sell into the rack were building to the same assumptions. Astera Labs introduced Orion, a rack-scale reference architecture that lets customers choose their own accelerators and CPUs, with a PCIe switch fabric designed to scale from eight to 160 GPUs per pod; the company said designs will be available through OEM and ODM partners beginning in the first quarter of 2027. Celestica introduced the DS6005, a fully liquid-cooled ORv3 co-packaged-optics switch built on Broadcom's Tomahawk 6 Davisson chip for 1.6-terabit fabrics. Credo detailed a 1.6-terabit ZeroFlap active light cable designed to draw 12 watts per cable end, reach 50 meters and ship in 2027.

What remains open is how fast operators can supply the facilities. A 250kW rack needs warm-water loops, coolant distribution and power delivery that many existing halls were not built for, and the facility and grid specifications published at the same summit address that side of the problem. None of the companies disclosed pricing or customer volumes for the new racks.

For the supply chain, though, the question of what to build has been answered in public. The rack ceiling, the busbar, the cooling interface and the tray are now written into open specifications that manufacturers can build to.

Source · blogs.nvidia.com

FrontMethod