Power & SitesUnited States

National Compute Grid Launches to Pool Idle AI Capacity Across Providers

Credited to HPCwire · hpcwire.com7h ago

Useful

For all the talk about a shortage of AI computing power, and the need for more powerful chips and bigger data centers, there’s another side to the story. A significant amount of existing capacity isn’t being used. While tech companies race to build massive new data centers, expensive GPUs in other facilities can sit idle between workloads.

What if all that fragmented computing capacity could be somehow pooled together to become something much bigger. That is the goal of an ambitious new initiative called the National Compute Grid. It is a nationwide network of companies that are capable of pooling GPU resources and making them available wherever they’re needed. At least that is the concept behind it.

The idea is that instead of always building more data centers or buying more powerful chips, why not make better use of the computing resources that are already out there?

Here’s how it would work. Data centers and cloud providers can make their unused computing capacity available through a software platform called Grid Exchange. The network is available to AI companies, startups, researchers, and other users who can find the GPUs they need. The Grid will match them to available computing resources. It takes into account the type of hardware required, available capacity and what customers are willing to pay.

(Shutterstock/PHOTOCREO-Michal-Bednarek)

The grid is designed to support different chip architectures, including GPUs from NVIDIA and AMD and TPUs from Google. That gives customers more options. However, moving workloads between different types of hardware isn’t always straightforward because of software compatibility issues.

The scale of the project is already substantial. At least on paper. According to its organizers, the grid has around 760 megawatts of computing capacity either connected or in the pipeline. The goal is to reach 2 gigawatts by 2030.

According to the organizers, single-tenant data centers use less than 15% of their computing capacity on average. That doesn’t mean 85% of all GPUs are sitting idle, but it does show how much computing power may be going unused at some facilities.

This could be particularly useful for smaller AI companies. Unlike OpenAI and Anthropic, which can afford to sign big, long-term computing contracts, smaller companies don’t always have that luxury. Sam Sinha, head of AI at robotics company 1X, spoke about this problem to Axios. For a smaller developer that needs extra GPUs for a few weeks or a particular project, having somewhere else to turn could be a big help.

Government agencies, academic institutions, and national labs can use it too. Some research projects require more computing power than an institution has available, and getting time on existing systems isn’t always easy. This may provide the much needed computing capacity required by researchers.

The grid could also be useful for President Trump’s Genesis Mission, which needs a lot of computing power for AI research. National Compute plans to contribute $100 million in computing credits to the mission, so researchers would have another way to get access to the GPUs they need.

Participating researchers can tap into computing resources available across the grid. This helps reduce how much the users depend on government supercomputers or dedicated cloud contracts. It would also give National Compute an opportunity to show that its model can work for scientific workloads – not just commercial AI applications.

The timing also matters. The announcement for the initiative comes as the White House prepares to announce more than $1 billion in private-sector commitments to the Genesis Mission. Companies including AMD, OpenAI and Anthropic are expected to take part.

The initiative is being spearheaded by Anjney Midha, who leads the AI holding company Amp. Cloud infrastructure provider Vultr is a confirmed founding member of the consortium and is helping bring computing capacity to the network. National Compute Labs is developing the grid and its exchange platform. Other AI companies, infrastructure operators and investors are involved, although a complete list of members has not been made public.

The consortium has taken a different approach to pricing. According to Vultr, the grid will support a model called “goodput,” where customers are charged for completed, uninterrupted computing work. This is different to just pricing it based on the number of GPU-hours used. National Compute has reportedly attracted more than $5 billion in reservations already. However, that should not be confused with revenue already earned.

There will be challenges ahead, and that is expected. After all, bringing together GPUs from different providers is one thing. Making sure they can reliably handle demanding AI workloads is another challenge entirely.

Training large AI models often requires thousands of GPUs connected through extremely fast networks. Having spare GPUs spread across multiple data centers doesn’t necessarily mean they can be used together for those jobs. Unfortunately, it’s not that simple. Moving large datasets between facilities can also be expensive and time-consuming.

The grid may be more useful initially for smaller training runs, inference and research workloads that don’t require thousands of GPUs working together. Whether it can reliably support larger jobs remains to be seen.

Original · HPCwire

Front Method

Chips & Supply · United States

Software could be the easiest fix for hyperscalers' AI power squeeze, researchers say

Credited to Tom's Hardware · tomshardware.com · 7h ago

Photograph · Tom's Hardware

Data center energy demand is shooting constantly upwards. By 2030, some 945TWh of electricity is expected to be used to meet AI’s demand; the International Energy Agency believes — that's as much electricity as Japan uses today. The energy crunch is inescapable, and the industry’s answer has largely been to focus on the physical: build more efficient GPUs and secure more power supply from an already tight grid. But is that myopic focus on hardware actually worth it? Instead, could most of the work running on those machines be cut back to save power?

Average power usage effectiveness (PUE) has barely changed for six successive years, according to the Uptime Institute’s 2025 survey. PUE can show whether cooling and power systems are wasteful, but doesn’t account for whether the software running on the servers is doing useful work. And given servers account for around 60% of electricity demand in a modern data center, while cooling ranges from about 7% in an efficient hyperscale site to more than 30% in a less-efficient enterprise facility, focusing on eking out more efficiency from the software side of the equation seems sensible.

“The answer to what seems to be the core question – ‘Can software or algorithms meaningfully help with power and energy?’ – is yes, 100%,” said Jae-Won Chung, a PhD candidate in computer science and engineering at the University of Michigan and researcher with the ML.Energy initiative, in an interview with Tom’s Hardware Premium.

Chung suggests computing systems should be seen as a stack with hardware at the bottom, backed up by systems software, algorithms and applications. The hardware end of that equation is the hardest bit to tackle: chips are slow and costly to replace. But the other three layers are easier to change, and any gains in efficiency made there can compound.

ML.Energy’s tests of the Alibaba Qwen 3 235B A22B Thinking model suggest that running inference in FP8 – a slightly lower-precision format – consumed a third less energy than bfloat16 versions on problem-solving tasks. Chung’s ow n Perseus training optimiser identifies parts of a large-model training job that have less work to do, then slows them so they finish alongside busier parts. The system cut training energy by up to 30% without reducing throughput or changing the hardware.

This isn’t a tweak around the edges being made by tinkerers: some of the biggest GPU firms are doing something similar. Nvidia’s Blackwell power profiles fine-tune everything from GPU compute and memory frequencies, power limits, NVLink states and cache settings to fit the workload. Nvidia believes that can save up to 15% of energy while retaining 97% or more of overall performance, allowing power-constrained facilities to run more GPUs, raising throughput by as much as 13%.

‘The loudest rack in the room’

Chetan Visrolia, business development manager for data center infrastructure at SHI, said in an interview with Tom’s Hardware Premium that the most obvious waste he encounters is often legacy equipment supporting old code. “It is always the lowest-hanging fruit for quick savings,” he said. “It can easily be found in any data center, as it is the loudest rack on the floor.”

Cloud instances can also be made a smarter size, and AI prompts that are repeatedly used can be stored in cache rather than recomputed on the fly. There are also efficiencies to be made by getting small models to handle routine requests while holding back the larger ones for harder tasks. At the same time, better batching and compilers can raise accelerator utilisation, and crunching down prompts while throwing limits on outputs can also tamp down the number of tokens processed.

Software can also change when and where electricity is drawn. Sophie Hall, a doctoral student at ETH Zurich’s Automatic Control Laboratory who has studied workload shifting across Google’s global data-center fleet, argues in an interview with Tom’s Hardware Premium that electricity consumption isn’t necessarily the main problem.

“It’s more like: when do they use it, where do they use it, and how is it interacting with the grid?” she said. If batch jobs aren’t time-sensitive, they could be delayed until local demand falls or routed to a region with spare capacity and lower-carbon electricity. Hall’s research uses day-ahead planning and real-time scheduling to respond to grid signals while preserving performance guarantees.

Taking that kind of action was less necessary when hyperscalers built way more capacity than they actually needed and the grid wasn’t a bottleneck, Hall said. But the massive demand for AI has changed the equation.

There are limits, though: individual countries’ data sovereignty rules can prevent a workload from crossing borders, while moving large datasets across networks is something that inference providers may be wary of in case things go wrong. “The data needs to literally, physically, geographically move to a different location,” Hall said. “That’s not that easy.”

The scale of the data center demand powered by AI means that software isn’t a silver bullet or a substitute for better chips and new electricity infrastructure. But it can act as a control layer that squeezes out every last drop from those increasingly expensive assets, while also helping shift demand to places where there is more capacity.

Jevons ’ paradox is also at play, and needs to be considered when thinking about making systems run better. Efficiency doesn’t reduce a facility’s electricity use if each watt saved gets used up generating even more tokens. “Power is the core bottleneck in AI data centers,” said Chung. “We really want to make the best use of every watt we consume.”