Power & Sites · United States

Software could be the easiest fix for hyperscalers' AI power squeeze, researchers say

Credited to Tom's Hardware · tomshardware.com · 7h ago

Photograph · Tom's Hardware

Data center energy demand is shooting constantly upwards. By 2030, some 945TWh of electricity is expected to be used to meet AI’s demand; the International Energy Agency believes — that's as much electricity as Japan uses today. The energy crunch is inescapable, and the industry’s answer has largely been to focus on the physical: build more efficient GPUs and secure more power supply from an already tight grid. But is that myopic focus on hardware actually worth it? Instead, could most of the work running on those machines be cut back to save power?

Average power usage effectiveness (PUE) has barely changed for six successive years, according to the Uptime Institute’s 2025 survey. PUE can show whether cooling and power systems are wasteful, but doesn’t account for whether the software running on the servers is doing useful work. And given servers account for around 60% of electricity demand in a modern data center, while cooling ranges from about 7% in an efficient hyperscale site to more than 30% in a less-efficient enterprise facility, focusing on eking out more efficiency from the software side of the equation seems sensible.

“The answer to what seems to be the core question – ‘Can software or algorithms meaningfully help with power and energy?’ – is yes, 100%,” said Jae-Won Chung, a PhD candidate in computer science and engineering at the University of Michigan and researcher with the ML.Energy initiative, in an interview with Tom’s Hardware Premium.

Chung suggests computing systems should be seen as a stack with hardware at the bottom, backed up by systems software, algorithms and applications. The hardware end of that equation is the hardest bit to tackle: chips are slow and costly to replace. But the other three layers are easier to change, and any gains in efficiency made there can compound.

ML.Energy’s tests of the Alibaba Qwen 3 235B A22B Thinking model suggest that running inference in FP8 – a slightly lower-precision format – consumed a third less energy than bfloat16 versions on problem-solving tasks. Chung’s ow n Perseus training optimiser identifies parts of a large-model training job that have less work to do, then slows them so they finish alongside busier parts. The system cut training energy by up to 30% without reducing throughput or changing the hardware.

This isn’t a tweak around the edges being made by tinkerers: some of the biggest GPU firms are doing something similar. Nvidia’s Blackwell power profiles fine-tune everything from GPU compute and memory frequencies, power limits, NVLink states and cache settings to fit the workload. Nvidia believes that can save up to 15% of energy while retaining 97% or more of overall performance, allowing power-constrained facilities to run more GPUs, raising throughput by as much as 13%.

‘The loudest rack in the room’

Chetan Visrolia, business development manager for data center infrastructure at SHI, said in an interview with Tom’s Hardware Premium that the most obvious waste he encounters is often legacy equipment supporting old code. “It is always the lowest-hanging fruit for quick savings,” he said. “It can easily be found in any data center, as it is the loudest rack on the floor.”

Cloud instances can also be made a smarter size, and AI prompts that are repeatedly used can be stored in cache rather than recomputed on the fly. There are also efficiencies to be made by getting small models to handle routine requests while holding back the larger ones for harder tasks. At the same time, better batching and compilers can raise accelerator utilisation, and crunching down prompts while throwing limits on outputs can also tamp down the number of tokens processed.

Software can also change when and where electricity is drawn. Sophie Hall, a doctoral student at ETH Zurich’s Automatic Control Laboratory who has studied workload shifting across Google’s global data-center fleet, argues in an interview with Tom’s Hardware Premium that electricity consumption isn’t necessarily the main problem.

“It’s more like: when do they use it, where do they use it, and how is it interacting with the grid?” she said. If batch jobs aren’t time-sensitive, they could be delayed until local demand falls or routed to a region with spare capacity and lower-carbon electricity. Hall’s research uses day-ahead planning and real-time scheduling to respond to grid signals while preserving performance guarantees.

Taking that kind of action was less necessary when hyperscalers built way more capacity than they actually needed and the grid wasn’t a bottleneck, Hall said. But the massive demand for AI has changed the equation.

There are limits, though: individual countries’ data sovereignty rules can prevent a workload from crossing borders, while moving large datasets across networks is something that inference providers may be wary of in case things go wrong. “The data needs to literally, physically, geographically move to a different location,” Hall said. “That’s not that easy.”

The scale of the data center demand powered by AI means that software isn’t a silver bullet or a substitute for better chips and new electricity infrastructure. But it can act as a control layer that squeezes out every last drop from those increasingly expensive assets, while also helping shift demand to places where there is more capacity.

Jevons ’ paradox is also at play, and needs to be considered when thinking about making systems run better. Efficiency doesn’t reduce a facility’s electricity use if each watt saved gets used up generating even more tokens. “Power is the core bottleneck in AI data centers,” said Chung. “We really want to make the best use of every watt we consume.”

Power & SitesUnited States

Empowering data centers for sustainability & cost savings

Credited to Data Center Dynamics · datacenterdynamics.com8h ago

Useful

Ablecom AbleCool helps data centers achieve sustainability and cost savings

The rapid growth of AI and HPC is driving a fundamental shift in data center thermal requirements. At the same time, growing energy efficiency requirements are putting greater pressure on operators to improve cooling performance without taking on the cost, complexity and disruption of major infrastructure upgrades.

In this whitepaper, Ablecom explores how rear door heat exchanger (RDHx) systems can provide a practical path to liquid cooling, supporting high-density workloads while minimizing infrastructure disruption. The paper examines the limitations of air cooling, the advantages and trade-offs of different liquid cooling approaches, and how Ablecom’s AbleCool RDHx series can support demanding AI and HPC deployments.

Download the whitepaper now to discover how to:

A version of this story appeared on Empowering data centers for sustainability & cost savings on October 8, 2026

Original · Data Center Dynamics

Front Method

Power & Sites · United States

The truth about data center water usage

Credited to Latitude Media · latitudemedia.com · 8h ago

Of all the concerns surrounding the AI boom, data center water usage tops the list. Opponents claim that massive hyperscaler facilities threaten local drinking supplies and drain aquifers. Proponents say data centers use a negligible amount of water, at least compared to agriculture or conventional power generation sources.

In this episode, Shayle sits down with Tim Shedd, founder of Ebullient Insights and an expert in data center thermal management. They unpack the history and evolution of data center cooling and consider how high-density chip racks have developers ditching air cooling in favor of single-phase direct-to-chip liquid cooling. The two also discuss:

Credits: Hosted by Shayle Kann. Produced and edited by Max Savage Levenson. Original music and engineering by Sean Marquand. Stephen Lacey is our executive editor.

Catalyst is brought to you by GridBeyond. GridBeyond is the world’s leading technology platform for managing distributed and flexible energy resources. Learn more by visiting GridBeyond.com.

Catalyst is brought to you by Antenna Group, the strategic communications and marketing partner behind the biggest names in energy, climate, and infrastructure. For three decades, Antenna has helped breakout companies amplify their stories, build reputation, differentiate from the competition, define new categories and accelerate growth. Learn more at antennagroup.com.

Catalyst is brought to you by EnergyHub. Peak season puts every grid to the test — and the utilities that pass are the ones that built flexible capacity before they needed it. EnergyHub works with more than 230 utilities to coordinate over 2.6 million DERs and more than 3.6 gigawatts of dispatchable flexibility through a single platform designed to perform when it counts most. See what that looks like at EnergyHub.com.

Shayle Kann: I’m Shayle Kann. I lead early stage investments at Energy Impact Partners. Welcome to Catalyst. So, of all the fights over data centers, I think water might be the one where the two sides sort of talk past each other the most. Opponents say data centers are draining aquifers and competing with towns for drinking water. Proponents basically say it’s a nothingburger. It’s a rounding error next to agriculture and other major water consumers, and actually new data centers barely use any water at all.

I think there are two things going on here. The first is about context. So, say a 300 megawatt data center with evaporative cooling uses like 150 million gallons of water per year. Is that a big number or a small number? Just for context, a combined cycle power plant of the same capacity would use roughly three times as much water. Or actually maybe even better, 150 million gallons per year is about what you would use on 550 acres of irrigated soybeans. So, there’s some context.

The second, though, is about history versus future. I think even the data center industry would admit that water consumption was significantly higher in the past. But they’ll also tell you that modern designs, which might be closed loop, consume very little water at all. Perhaps down to essentially just restrooms and kitchens.

So, rather than like trying to argue the numbers, I wanted to understand where the water actually goes. What inside a data center needs cooling? What is the challenge there? How did we do it historically? And what’s changed now that a single rack can draw what used to power a whole row?

It turns out that the answer to how much water the data center consumes is actually mostly a design choice. But it’s a design choice being made right now at gigawatt scale, and the old numbers don’t actually tell you much about the way it’s going.

So, to walk through it, I talked to Tim Shedd, the founder of Ebullient Insights and an expert in data center cooling. That’s coming up next.

Tim Shedd: Thank you. It’s great to be here.

Shayle Kann: Let’s start by having you at the high level walk me through within a data center, what needs cooling, and relatedly, what needs water, and why?

Tim Shedd: Okay, so that’s a big question, but overall, everything needs cooling. Even the little half-watt devices sitting on the motherboard that you never think about, if any one of those fail, the servers fail. The power supply fails, whatever. So, everything needs cooling, and as a thermal engineer, this is something we’re focused on. Making sure that that cooling gets to everything, including the power supplies, including the network switches, every possible thing.

When it comes to water, these days, it’s really kind of important to keep in mind that this push to cool everything with water is really only about three years old. So, everything we’re talking about is evolving. It’s moving very fast, and that’s gonna be part of this discussion, part of many discussions that I have, is understanding, helping people understand that we just don’t know the answers to everything.

But, back to your point, we’re going to focus on those high-load devices, like the processors, but other devices that are having a hard time getting cool, like memory and network ch- chips and so on, are now also requiring water. And we’ll get into maybe talking about the density, but it’s— it’s— it’s driven by the need to put everything really close together in the racks for networking.

Shayle Kann: Right, is the— is a simplified way to think about it just that what is happening over time is that we are densifying everything in the racks, and every individual thing is also becoming more powerful? Like every sub- subsequent Nvidia GPU is more powerful than the previous, and also we are packing as many of them together as we can. And that generates more heat per cubic meter, or whatever unit you want to use, and so there’s more heat to reject and more cooling to be required as time goes on. Is that generally true?

Tim Shedd: That is generally true. I think it’s important to keep in mind that we’re not doing this for fun. This isn’t like trying to make a, you know, Guinness Book of World Records dense rack. We’re doing this because we need to get all of the GPUs to talk to every other GPU in the data center. And that requires network cabling, that requires network switches, and we— those cost a lot of money, actually. And we want to keep them as close together as possible.

Also, by having as many GPUs as possible on the same motherboard, we can use high-speed networking within the server, the NVLink networking, or within the rack in the NVL72 case, or in AMD’s Helios. And there’s just a huge advantage to being able to have that high-speed networking.

So, that’s why we’re trying to keep them all in a rack together. But then, just as you said, every generation is getting higher powered. Again, that’s not for fun. That’s because, you know, Nvidia, AMD, Intel, they’re all learning how to do more with the silicon they’ve got, and run them harder at higher speeds to the point where, in some cases, they’re running them as fast and as hard as they can, limited only by the cooling. Like we can move faster electrically, we can move faster, from a power delivery point of view in many cases, not every case. And we’re limited just by how fast we can get the heat out.

And so that gets back to, where do we need water? Water is a very effective way to cool these high-power devices in a limited space, and those two things together have driven us to direct liquid cooling.

Shayle Kann: Yeah, I think there’s an interesting— from the outside, I’ve seen people getting confused about two things that are happening simultaneously, and I think we can tease them out a little bit. One is, as you said, this pretty recent shift toward liquid cooling, which I want to understand a little bit better. And then simultaneously, the push to have the data center overall consume less water. Right, and those two things on the surface would seem to counteract each other. All of a sudden we’re using water for cooling where we weren’t, and yet how are we trying to use less water in the data center entirely? So, I think we should— we should separate out those threads for a second. Let’s start with the, how are we doing all of this cooling? Can you just talk, before we get into what’s happening today, and you said the last three years we’ve been focused on liquid cooling, like what’s the— how do we— how do we cool chips and racks in a data center historically?

Tim Shedd: Historically, the average rack density was like 9 kilowatts. And with 9 kilowatts, the cheapest, fastest, most convenient way to deploy racks was in an air-cooled data center where I can just put kind of pre-install cooling. I forecast out the next two or three years, I expect to have this load, I can then install the air conditioners around the sides, the computer room air handlers, and then run plumbing to those devices far away from my compute so I don’t have water near my compute.

And then I can— I can use some basic principles to make that efficient by separating the hot and cold air with hot— hot aisles or cold aisles. But otherwise, I can put my compute in, whatever from HPE, from Dell, from Supermicro, it doesn’t matter, they— they can— I can use air, right? Air— air doesn’t care. And, and in fact, it’s— it’s not a bad way to go at all.

The problem is that pushing the air from the sides of the room into the center requires a fair amount of energy from the fans, and there is going to be mixing of hot air and cold air, which makes it less efficient. And I need to get a certain amount of air to every server for every watt that I’m trying to cool, and at some point, that kind of breaks down, at least from that model.

So what we did to start to try to make things more efficient as we went from 9 kilowatts to some high-performance computing going to 30 kilowatts and 40 kilowatts a rack is put an in-row cooler next to it. So, we bring the cooling closer to the heat source. That saves me a lot of energy, actually. A rear door heat exchanger or an in-row cooler, they have kind of the same basic concept of just bringing that heat exchanger as close as possible to the heat source. That way my fans are working very efficiently because I’m only moving heat really closely to my compute, and I’m only using fan power that I need. And, if the racks next to or attached to the rear door heat exchanger don’t need to be cooled, the fans dial back, and I’m saving a lot of energy, actually.

But that requires plumbing now onto the data center floor. It made some people uncomfortable, but overall, you know, it’s a good idea. But even that gets limited at about 80 kilowatts. We’ve, you know, was working for Dell, and we shipped racks regularly at up to 80 kilowatts a rack with rear door heat exchangers, but beyond that, it— it starts to get difficult to pull enough air through to cool. So, that’s what brings us to today. It’s not— you know, rear door heat exchangers are actually really efficient, and, in many cases, almost as efficient as direct liquid cooling up to that power amount, but beyond that, it starts to get really difficult.

Shayle Kann: Okay, so to bring it forward to today then, you said, you know, you were shipping 80-kilowatt racks at Dell. Where are we today? Like what are the in— in a new state of the art frontier inference or training data center, like what’s the rack density?

Tim Shedd: Everybody probably listening to this podcast is aware of the NVL72 or NVL72 racks. Those are rated at 220 kilowatts a rack, including the power supplies and everything else. So, that’s a lot of power in a rack, but that’s actually not the densest racks that are shipping today.

As an example, Dell is installing the Perlmutter supercomputing cluster at Lawrence Berkeley National Labs, and— and that power level kept creeping up. I want to say it’s about 280 kilowatts in a rack, but I don’t remember that exactly, but my point is that that 220 kilowatts is actually not the limit today. There— there are ways to keep packing more and more GPUs into a rack, whether they’re 100% liquid cooled like the NVL72 or they’re a mix of air cooling and liquid cooling like the Perlmutter supercomputer, we can still manage that level. In fact, we can manage— we’ve mapped out up to 400 plus kilowatts, 500 kilowatts in that rack. So, that is what’s possible today, from a cooling perspective and from a power delivery perspective using ORV3, racks and so on. So, yeah, we should expect to see that type of density, continuing on.

Shayle Kann: Right, presumption would be that density continues to increase, that’s the direction of travel. And regardless, it’s a, you know, to your point, like we could do 80 kilowatts air-cooled, maybe we could squeak out a little bit more than that, regardless, we’re a multiple of that now. So, clearly we are forced into doing something other than air cooling, which is liquid cooling. So, walk me through the mechanism. What is— what does liquid cooling actually look like in a rack, and then— and then we should talk about the water side of it.

Tim Shedd: Ultimately, what we have to do is bring water to every device that needs to be cooled with liquid. So, the reason I qualify that is there are different architectures for servers today. Some are like the— the 8-way, DGX, architectures. Today, those are still typically a hybrid-cooled solution, so the GPUs and, and, NVLink and other devices are cooled with water, but storage might still be air-cooled, network cards might still be air-cooled, and so on. So, I’m going to have some mix of airflow through the server as well as— as liquid.

And in other cases, like the NVL72, it’s 100% liquid cooled. So, I’m bringing water into the server and having to route it to every device, somehow touching every device greater than about a half a watt. So, it’s a really intense process once we decide that everything’s going to be liquid cooled.

So, imagine just— it’s a plumbing problem. I’ve got two ports, an in— a supply and return on the server, and I’ve got to route that liquid to every cold plate, to every piece of copper, basically, with enhanced, you know, with fins on it that I’m going to run the water through to pull the heat off of— of all the hot devices.

In addition, we typically want to make this serviceable. So, in other words, we want to make it so that we can remove one GPU out of eight or four that are in there. That means that all those liquid connections have to have some sort of connector that allows us to remove the GPU without spilling liquid all over the place.

So, in addition to the plumbing to just connect all the copper together that we’re using to cool the processors, we also have manifolds, and we have quick connects, and we have hoses and other things. So, the liquid cooling infrastructure is actually quite significant and takes up a significant volume within the server today.

So, as far as the cold plates themselves, they are essentially, just think of it as a basically a piece of 2-millimeter thick copper, maybe it’s 1, 1 and a half millimeters depending, and then we’ve, generally created, people call them different things, but skived fins or fins that are made by shaving a bit of copper off the surface and flipping it up, and, the typical spacing is about 100 microns thick for the copper and about 100 micron spacing between. Historically, we call those microchannels. Microchannel cooling today has a different connotation, we can dive into that if we want, but, they’re typically about 3 millimeters tall, and we’re going to try to arrange them such that we’re flowing water through every one of those microchannels and pulling heat off.

And, that’s non-trivial, but if we do so, they’re very high performance. We can now cool a, you know, 1400-watt, 1800-watt processor using 45-degree C water. That’s phenomenal. That’s actually something we couldn’t do three years ago. So, a lot of progress has been made, and then a lot of additional technology and figuring out how to seal those things so that they don’t, you know, they don’t leak easily, they can be shipped across the oceans and not burst or have any other problems. There’s a lot of engineering has gone into the cold plates in a pretty short time to make them reliable and high performance.

Shayle Kann: Can you orient me a little bit on— there’s different architectures within liquid cooling and you hear these different terms. Just give me like the— the sense of direct-to-chip cooling, immersion cooling people talk about, two-phase — how should I think about categorizing the different ways to do liquid cooling?

Tim Shedd: Direct-to-chip means bringing the coolant to the chip. That’s all what is direct-to-chip, and that can be either single phase or two phase. So, the dominant form today is single-phase direct-to-chip cooling using water with 25% propylene glycol mixed in. So, it’s basically, crudely speaking, it’s basically the antifreeze that you pour into your car diluted to 25%. When you go buy it at the auto parts store, it’s often 50% mixed, pre-mixed.

The really important detail that you don’t even think about when you go pick up a bottle of antifreeze is that that gallon jug that you have has a very small percentage, about 1% to 2% maybe, of a proprietary chemical mix, as a, call it an inhibitor or an inhibitor package that prevents corrosion. So, your car has aluminum and brass and all sorts of other materials in it and lasts for years without corroding and falling apart. That’s actually phenomenal. That’s amazing. So, our partners at, you know, like Valvoline and Dow and Recochem, they’ve done an amazing job developing these— these coolants that are very long lasting and allow us to use many different materials in their systems. That’s the same thing we have to pay attention to in the single-phase liquid cooling system.

So, that’s single-phase liquid cooling. Two-phase liquid cooling is not yet really available at scale. There are companies working on it. I myself had a company that worked on this, so I’m a little, you know, have some history with this, but there are definitely products that are very close to commercialization in this space. And it’s where you take a refrigerant, you pump it to the chip, and you literally boil the refrigerant off of the chips, off— you know, off of the fins. It looks really cool, bubbles are great. It turns out that the performance is very similar to single phase. It’s not really superior in performance, but there are other benefits. I can use much smaller hoses and tubes and so on. So, remember I was saying currently the DLC takes up a lot of real estate inside of the servers? Two phase promises, again, not mass adopted yet, but promises to use a lot less space inside of the servers, leaving more space for innovation on the actual compute.

Immersion can take two forms in single phase and two phase. In single phase, we use an immersion fluid, a coolant, that is often derived from something like a mineral oil, but it’s generally much more sophisticated than this, and— and has additives and other— other inhibitors as well. The problem with oil is, if, you know, if you can imagine it’s a hot day and you’re getting excited, you jump in the pool, and that pool is filled with oil, that would feel pretty not great. It wouldn’t cool you off, right? Because oil is actually a pretty poor heat transfer fluid.

And so, it turns out that it’s good for picking up low-density heat from a large area; it’s not great for picking up heat from the high-performance processors. And so to do tha the emerging community is proposing putting cold plates on the chips themselves and then picking up all the rest of the heat, you know, from those small devices with this— this immersion fluid. And there are pros and cons to that, but that’s the— that’s kind of the trajectory that they’re on.

Two-phase immersion is coming and going and coming again, maybe. The challenge with that is I need to be able to have a fluid that boils and doesn’t allow the combination of vapor and liquid together in a container to get too high of a pressure. Gets a little complicated here, but the boiling point of that fluid really matters. Because if the pressure rises, even, you know, to a third of an atmosphere, and I’m in a great big tank, that can create a lot of force on the lid and create a negative, you know, very bad situation. So I have to manage that vapor, I have to manage the expansion, and so a little complicated there.

There are other things, we actually published— me and some colleagues published a paper on this about four years ago, three and a half years ago now, that went into some of the details of the limitations of two-phase immersion, and it’s, again, going to be kind of stuck at lower power. Still pretty high power, you know, like 1,000-watt processors, but it’s kind of stuck there, in that region for, you know, thermodynamic reasons.

So, really the emphasis in the industry is on the single-phase direct-to-chip cooling because it’s ready to scale. We have a supply chain that’s very robust worldwide. We have liquid coolant suppliers that are now providing— able to provide high-quality coolant worldwide. This is what we need, because we’re at the state where we’re producing and consuming hundreds of thousands of cold plates a month. And by next year, that could climb to maybe close to a million cold plates a month. I mean, this is crazy. You know, in 2022, we might have been talking about 100,000 cold plates a year, right, in the whole industry? So, something on that order, right? And so we’re just scaling super fast, and right now, the technology that can scale and keep up with it is single-phase direct-to-chip.

Shayle Kann: Okay, so now explain the relationship between the cooling architecture and water consumption at the data center level.

Tim Shedd: They’re actually kind of disconnected, and let me explain that. What we want to do is break down the journey of the heat from the chip to the outside into basically three parts, right? We’re going to have to collect the heat from the servers, and then, move that heat through some fluid, air, liquid, whatever, to an exchange point between what we call the TCS, the technology cooling system, which is the part that collects the heat, to the FWS, the facility water system, which is the part that moves the heat to the outside heat rejection system.

And so, the heat rejection system is going to be whatever devices that the data center designer has chosen for that climate, for that region to be, chillers, cooling towers, dry coolers. Whatever’s appropriate for that particular region and— and power level.

And ultimately that interface between the TCS, the cooling architecture that’s collecting the heat, and the FWS is just a heat exchanger. Regardless of whether I’ve got two-phase DLC, single-phase DLC, or immersion, there is a, typically a plate heat exchanger that interfaces between the two systems, and the FWS doesn’t care what was on the other side.

What we care about is having the highest possible temperature on the TCS rejecting the heat to the FWS. Because I want to move the heat from the processor, which is the hottest point in the system, to the outside, I want to try to move that heat downhill the whole way. Then I can just have thermodynamics carry the heat out without requiring an energy input other than pumps. And that’s the most efficient way to move the heat out.

The challenge is that in many climates, if the outside temperature rises above, you know, 100 degrees Fahrenheit, you know, 43 degrees Celsius, that’s pretty warm, and that’s actually warmer than our FWS often wants to be. And so that means we have to push heat uphill, then I need a chiller.

So what we’re doing is we’re looking at the climate, and we’re looking at the heat load, and we’re looking at how much time are we spending in a case where I’m having to push heat uphill? And if it’s just, you know, a relatively small number of hours, I might use something called an adiabatic cooler where it’s basically a dry cooler with some pads, and I just spray the pads with water. This is oversimplifying. My colleagues in the heat rejection industry will grit at that simplified solution, but basically, it’s actually an elegant solution that allows me on those days when I need to dump heat out on a hot afternoon, I can do so basically with just a little bit of water and, do that very efficiently and get that done.

Other climates, I’m going to spend half the year requiring, you know, pushing heat uphill, and then I’m just going to need a refrigerated chiller, and that uses a lot of energy. So, I kind of went a long ways from your original question. I want to kind of disconnect — what our goal is to get the best technology on the TCS that generates the highest temperature to feed the FWS, to feed that the highest temperature to the heat rejection system so that I can use, hopefully, the least amount of energy to push that heat out into the environment.

Shayle Kann: So, that— that all makes sense, but then what does dictate the water consumption, the total water consumption of a data center?

Tim Shedd: Most data centers don’t use much water at all, and they historically haven’t used much water. And it’s kind of frustrating that this whole discussion is even going on. But here’s the deal.

When we were air cooling everything, hyperscalers who are building out thousands of the same rack over and over and over again, their biggest cost is energy input to the data center. So, they were looking at this and saying, well, the most energy efficient way for us to cool this data center is evaporating water.

And they were elegant systems. They are elegant systems that used kind of like I was describing earlier, basically spraying a material, a filler material, and blowing air through it, using evaporation to cool that air down. That’s actually a smart idea. It uses a relatively small amount of water for the amount of cooling power generated or the amount of cooling that’s being done.

But for those select data centers, it did consume, relatively speaking, a lot of water because I’m cooling megawatts, right? And so, I’m going to use a kilogram of water for every 2 megawatts or something like that. So, over time in some communities, they paid attention to this. And to be frank, the hyperscalers were very transparent about this: Google, Meta, Microsoft, very transparent. They’re honest about the amount of water they’re using.

I don’t want to be over negative. People should be paying attention to these things, and I don’t want to be negative on that point, but the problem is the data centers we’re building today aren’t like that. We’re not cooling everything with water, like that.

Even when we’re using a cooling tower, we’re frequently pairing the cooling towers with dry coolers. So, we’re only using the cooling towers to trim the heat rejection on those hottest afternoons. So, as far as how much water consumption is there, I don’t know that number precisely because every data center is going to be different.

Most data centers are built completely closed loop with no cooling towers even, no evaporation. Some are built with, like I said, some evaporation on the hottest afternoons, just for a few hours a year, or maybe tens of hours a year. Others are more dependent on cooling towers, but they’re using it more for trim cooling or just the watts that need to be cooled on those hottest afternoons.

And so what we do know is that the hotter the water— so using a cold plate and direct liquid cooling at the TCS gives me hotter water to the FWS, which gives me more options on the heat rejection, which allows me to use less water if I have to use water at all. So, the lesson here is, do the best I can collecting heat so that I can be as energy efficient rejecting the heat as possible.

Shayle Kann: I guess it’d be interesting maybe to separate out then, you know, clearly what has happened over time is that data centers have— more recent data centers are consuming less water on net. You could make the case that they were never consuming that much water, but it’s like all a matter of perspective. Either way, on a relative basis, they’re consuming less now. And it seems like you’ve given sort of two reasons for that to be true.

One is a different optimization, basically, where like early on the hyperscalers were optimizing predominantly for energy consumption. If you’re optimizing for energy consumption, then you do adiabatic cooling or whatever and— and do use some water. And now if you’re optimizing a little bit more for low water consumption, then you just choose a different method of cooling, and it— you use less. So, that’s one reason.

The other reason being the sort of technical thing that you described of like getting higher temperatures in the TCS so that you have less heat to reject. I don’t know if it’s possible to separate those two from each other, but how much of the change in overall water consumption in modern data centers versus previous generation data centers do you think is from this re-optimization versus the technical advantage of a higher temperature TCS?

Tim Shedd: I would say that a large percentage is using direct liquid cooling or more efficient TCSs, technology. Because if I’m air cooling and I’m not consuming water to do that air cooling in the data center, I’m requiring cold water there to generate the cold air. I’ve got to have cold water going through the heat exchangers to generate the cold air. And that’s going to mean that I’m going to need chillers.

And the most efficient way to run the chillers is often, depending again on the climate and many other things, is often with a cooling tower providing cool water to the chillers so the chillers are consuming less energy, but I am consuming some water. It’s all connected to this balance, as you noted, between water usage and energy.

But if I start by picking up heat with 45-degree C water at the chip, and then my return is 55 degrees C, I mean that’s 135 or something Fahrenheit. Don’t remember the exact conversion. I can reject that heat in most places, except the worst day in, you know, the middle of India in the summer, I can reject that outside relatively easily, right?

And so that has made it, wi- without the use of consuming water or a whole lot of energy even. So, that’s just made a huge difference. As we moved into this AI deployment, and it’s one of the reasons why Jensen makes a pretty big deal about this in his, keynotes, is it’s enabled this buildout to occur with, you know, optimizing the whole supply chain, whether it’s chillers, whether it’s dry coolers, or whether it’s water consumption.

I do want to come back to the fact that these are really large data centers, and some of them have really huge piping infrastructure, and there’s a certain amount of water required to fill all those pipes. And during the installation and startup, we have to flush and clean those pipes, and that does use water. Now, it’s not all consumed. Initially, I can say we were probably dumping a lot of that water.

Again, remember, this is happening over a relatively short time period of a few years. Now you’ll find that most of the companies are pretty sophisticated about that. They have pumping stations that come out and they’re reusing the water. They’re refiltering it, on site often, so that there’s very little wasted water. But it’s still likely that there’s water used from the local water supply to do the filling and operating a data center and actually where it’s being drawn from.

And so, again, it’s an issue with a whole lot of variables to it, and I don’t want to say that a community that’s concerned about water and has seen their water table drop, that they’re not seeing an issue. I don’t want to claim that, you know,they must be wrong. On the flip side, I think if you dig deeper, there’s probably many, many reasons for that, right? And it’s possibly not because of the data center solely, or it’s possibly not because of the cooling system. And maybe that that water is used to build the roads to the data center, so maybe the data center is still responsible, but, you know, it’s not a continuing use.

All that to say is this is a complicated question, but on the whole, water use in data centers is pretty low compared to many other, sources of consumption, and we are continually innovating, and I believe, making more efficient power and cooling systems such that we’re consuming less water for the power that we’re using.

Shayle Kann: I guess to ask you a final question that maybe steps one degree outside of your area of expertise, because it gets more into this question of community acceptance. You know, I think this conversation has, has clarified why this is challenging, because on one hand, you can look at the raw numbers of like how much water have data centers consumed historically and then currently, and again, those numbers are changing, like it’s— it’s falling, the numbers are falling, but you can look at the raw numbers, but this concern that has arisen and the fact that water consumption, perception of water consumption, is one of the two or three big issues right now in the sort of big data center backlash, I think it’s difficult to combat given the nuance associated with it. It would be easy if it weren’t nuanced, and if every new data center could just say, “I promise zero net water consumption, right?” “And here’s how I’m going to do it.” But in the absence of that, it’s complicated, and it’s like one of these things that like political messaging doesn’t do a good job of describing.

So, I wonder if you have a view on, either from a technical and architectural perspective, like should modern hyperscale data centers all do X, Y, or Z, or from a messaging perspective, what is the right way to communicate the relationship between data centers and water consumption moving forward?

Tim Shedd: A tough question, for sure, but in the past year, I would say, you know, Microsoft has been a great example of messaging. They got out front of this question, they were very aggressive in messaging to the community in Wisconsin where they were building a— a large data center. I felt that they were messaging very clearly how they were going to cool and how much water they intended to use and how they were, you know, achieving this. So, kind of doing what you suggested there.

Unfortunately, I still see a backlash to hat data center, and I still see water being brought up as a point, and my feeling is that that, number one, I think it’s table stakes to take the approach that Microsoft did in that case, and I think as an industry, I think we’re getting better about that, maybe a little too slowly, but I think we’re getting better about being really transparent in— in the communications.

I think we could do better literally just inviting the community in to see how this works, if that’s possible. There’s a whole lot of safety and other things, and some privacy concerns, but, you know, I think that would help people see, okay, I get this.

On the other hand, I think being honest about, hey, building a great big building that’s going to consume 500 megawatts of power just uses water. Whether it’s just keeping the dust on the construction site down, or preparing the roads or pouring the concrete, whatever, right? It’s going to consume water, and so there’s going to be a period where water is consumed building a huge data center campus, and communicating that might help.

But this is an opinion or a belief, this isn’t my area of expertise. I’m in this the waters like you are, and seeing that, I wonder if a lot of this just isn’t, a general feeling of discomfort of this new technology that’s being built out all around us, and we’re all uncertain about how it’s going to play out, and it’s pretty easy to take it out on these big non-descript buildings being built and consuming resources, right?

And so I think what we can do as an industry is continually perform outreach, try to explain what we’re doing, be as transparent as possible, try to get away from all the NDAs. I understand sometimes why they’re there, but that just breeds distrust, and we all know that. Again, I understand the reason for that sometimes, but just being transparent.

And then, there’s not much we can do about communication about AI and its application these days. That depends on a few loud voices that you and I don’t have any control over, but, you know, but really trying to work as an industry to be transparent and, and hopefully bring focus on the applications that people do like and do use. I think, you know, autonomous vehicles and autonomous taxis are being used in a huge number of communities, and a lot of people really like them, you know, Waymo and Cyber Taxis now, and other options. Those wouldn’t be possible without these data centers, and of course, all the functions on our phones and whatnot. So, I don’t need to go down that path.

I think this ends up being more of a human issue than a technical issue, and even than a communications issue, and it’s going to take a little while to resolve, but transparency I think is the first step.

Shayle Kann: All right, Tim, this was really interesting. Very much appreciate your time.

Shayle Kann: Tim Shedd is the founder of Ebullient Insights.

This show is a production of Latitude Media. You can head over to latitudemedia.com for links to today’s topics. This episode was produced by Max Savage Levenson. Mixing and theme song by Sean Marquand. Ann Bailey edits the video version of the show. Stephen Lacey is our executive editor. All of our episodes are on YouTube, subscribe to Latitude Media for episodes of this show and Open Circuit. You can also find the audio version of the show, and not have to look at my face, anywhere you get your audio podcasts.