As artificial intelligence (AI), machine learning (ML), and other advanced applications continue to rise in popularity and adoption, there has been a massive influx of these workloads into data centers. To keep pace and meet the cooling requirements of these power-hungry applications, data center owners and operators are looking to design and build new data centers with liquid cooling systems.

In fact, the incredible demand for AI is even driving some owners and operators to retrofit existing data centers with liquid cooling systems.

At the heart of modern liquid cooling systems is a device known as the coolant distribution unit (CDU). But what is the CDU? What purpose does it serve? And what should data center owners and operators consider when purchasing and installing CDUs as part of their data center liquid cooling systems?

To find out, we invited Ben Graham, the Mechanical Project Manager and Estimator at Compu Dynamics, to join us on the first-ever episode of The Modern Data Center Journal Podcast. During our discussion, Ben educated us about the CDU, which he refers to as “The beating heart of the liquid cooling system.”

Click “PLAY” to listen to the episode in its entirety, or scroll down to read a transcript of Ryan and Ben’s conversation.

Transcript: The Modern Data Center Journal Podcast (Season 1, Episode 1)

Ryan Schradin: Welcome to the first-ever episode of The Modern Data Center Journal Podcast. My name is Ryan Schradin, and I’m The Modern Data Center Journal‘s executive editor. I’ll be your host for this and future podcast episodes.

This new podcast program was conceived and created as a forum to talk about some of the largest data center trends that we’re seeing in the industry today. On this podcast, we’ll talk about the technological advancements being driven by modern data centers, we’ll explore the ways new technologies and workloads are shifting and changing the traditional data center as we know it, we’re going to discuss exciting new advancements in the equipment and the devices that are inside and outside the envelope of data centers, and will feature deep data center discussions with industry leaders who know the space better than anyone.

And speaking of people who know spaces better than anyone. Our first guest for this inaugural episode is a previous contributor to the Modern Data Center Journal, and a true genius when it comes to designing, installing and maintaining liquid cooling systems in data centers. He’s a mechanical project manager and estimator at Compu Dynamics with more than two decades of experience working with precision cooling systems. I’d like to welcome Ben Graham to the show.

Ben, thank you so much for joining us today.

Ben Graham: I appreciate that gracious introduction, Ryan. You’re too kind. Before we get going, I wanted to thank you for putting this together. This is an excellent platform, I think our listeners will get a lot out of it.

Ryan Schradin: I don’t just call anyone a genius, man. But hey, it’s great to have you. I’m excited to talk about this stuff. But before we jump into the meat and potatoes of this, before we really get started, why don’t you tell your listeners or our listeners a bit about yourself and what you do at Compu Dynamics.

Ben Graham: My focus at Compu Dynamics is really the liquid cooling aspect of a mechanical project. I look at an engineer drawing that comes through, and [customers] request, “Hey, let’s get this priced up. Let’s get it to a point where we want to get it put together.”

So, I’ll go through that drawing. I’ll look at it. I’ll assess it to make sure that it’s feasible when it comes to, you know, the liquid cooling aspect. There’s a little bit of detail involved in that, obviously. I also get proposals together for smaller clients that have liquid cooling requirements, we basically do a design/build for them.

So, we’ll take what they require at the server rack level, and basically work back from that point back. We’ll look at all the racks. We’ll look at what’s required within the rack and throughout the white space, then all the way back to the chiller. We’ll [then] create a full working system for them.

Once those proposals are out there and approved or awarded, we’ll go through and order and put together all the components and pieces. We’ll do the full assembly and installation at the facility. I’ll oversee all of those aspects to make sure that what was designed is being delivered and that it’s fully operational and optimized for their particular cooling needs.

Ryan Schradin: It sounds like you spend a lot of your day designing, pricing, and building liquid cooling systems for data centers, which makes the topic of today’s podcast very meaningful.

Now, at the risk of pulling the curtain back too far or showing too much of how the sausage is made…podcast hosts DO talk to their guests before the podcast. You and I had a chat about what you wanted to talk about today, and you said – quite poetically, I might add – that you wanted to talk about the CDU as the heart of the liquid cooling system. And my response to that was, “What’s a CDU?”

So, let’s start with the basics. For our listeners who might have as much knowledge as I do. What’s the CDU? And what does the CDU do?

Ben Graham: Yes, of course. The CDU – or coolant distribution unit – supplies the lifeblood of the system; or the glycol, if you will. The CDU maintains specific loop temperatures and automatically adjusts for fluid flow requirements.

When there are liquid cooling requirements within the servers themselves, some of them have internal valves and things [that will] modulate. But, if the operator is putting in new compute or new nodes, the pressures will change. So, a CDU will automatically adjust to those pressures, and will maintain fluid flow and temperature based on the requirements.

All of those things aside, they also act as a buffer between the main building loop – the chilled water – that typically is not filtered well enough for on-chip reasons and purposes. Also, that main building loop – the chilled water – is typically too cold for on-chip applications. That fluid, if it were put directly onto those chips, would condensate, causing droplets to form and moisture to come in contact with the compute node. So, the CDU will maintain those loops above dewpoint, preventing that from taking place.

Ryan Schradin: Well, that makes sense. Now, say the CDU is kind of this foundational part of a liquid cooling system. These liquid cooling systems, the ones that leverage the CDUs, do we need those in all data centers? Why would a data center owner and operator want one of these liquid cooling systems with the CDU in it versus, say, a traditional air cooling system?

Ben Graham: That’s a great question, Ryan. The biggest thing I consider is the transition of the wattage of the processors or the nodes. When that transition is somewhere around 300 to 400 Watts, it becomes much more advantageous to cool using fluid. And those processors and nodes in that range of 300 Watts and above.

Ryan Schradin: So, as these power rack densities are going up, liquid cooling is the answer. Do you have any kind of particular workload or application that’s kind of driving those higher rack densities?

Ben Graham: So, what we’re seeing is [that there are] higher compute requirements for machine learning (ML) and artificial intelligence [AI] software. To physically do that work takes a lot more energy. [It takes] more wattage for those processors to perform. And those higher levels of performance, obviously, come with a lot more heat generation.

So, with that higher heat generation, we’re trying to pull [more heat] out of the nodes. So liquid cooling is really the most efficient method to do that.

Ryan Schradin: So, it sounds like AI, ML, and a lot of these advanced workloads got a lot more power density in the rack, a lot more heat. Liquid Cooling is really the only way to go now.

In some conversations that I’ve had – even with some of your associates at Compu Dynamics – I’ve kind of learned that there are a lot of different flavors of liquid cooling, right? Like, it’s not just one thing. There’s liquid to the rack. There’s liquid to the chip. Immersion cooling is a form of liquid cooling.

Do all of those different kinds of liquid cooling flavors or approaches require a CDU? Or do you only need the CDU for one or two of them?

Ben Graham: The short answer is, “No.” But with that said, there are aspects that need to be engineered or prerequisites that need to be met before you can really design the CDU out of the equation. A couple that come to mind are loop temperatures, [which] as I mentioned before need to be maintained above dew point. So, you’d almost have to have a dedicated cooling system in place to engineer the CDU out.

Also, the fluid itself typically – on a normal building loop – is a fluid that isn’t exactly clean enough to utilize for on-chip or liquid cooling, meaning that it needs to be filtered to 500 microns or less for the chilled door aspects. If you go into DLC or direct liquid cooling, that filtering needs to be much better – around 200 microns of filtering for that fluid. The fluid itself needs to be a specific fluid design that will not cause corrosion to take place.

As long as you can come to the table with your plant cooling that addresses those things, you could possibly design the CDU out of it. But, for a lot of current buildings and infrastructures, the CDU is the easy solution that prevents all kinds of other infrastructure changes [from needing to] take place.

Ryan Schradin: Yeah, that sounds like it’s pretty necessary. And it certainly seems like it saves you a lot of work and heartache.

Now, in previous discussions you and I have had, it sounds like – at least to me – that there are different kinds of CDUs. And I’m not quite sure exactly what those different kinds are. What exactly differentiates them? Can you take a minute to talk through, you know, what the different kinds of CDUs are and what is different between those different kinds?

Ben Graham: Of course! I guess we’ll start out with the in-rack version of a CDU. Now, this CDU is designed to slide onto a shelf bracket inside the rack. This particular CDU is capable of right around 100 kilowatts of heat rejection, plus or minus a certain percentage, depending on your primary loop temperatures and things like that. But they are typically about 100 kilowatts. They can pipe directly into a chilled door or rear door heat exchanger. And then that piping can be connected to one or two other racks next to it to provide cooling for those racks. Or that CDU can be piped into the rack manifold, providing cooling for just that rack alone.

The next step up would be a floor mount version. These floor mount versions range in capacities from 150 kilowatts to 2.5 megawatts of cooling capacity. The piping also transitions as the capacities increase. So, you can start out with an inch and a half piping for your lower capacity unit, all the way up to six-inch piping for your high-end capacity units. And, as you might imagine, the pumps within change in size, as well. So, you’ve got your in-rack units, [which] are small little pumps [that] may be consuming around 300 or 400 watts of energy, all the way up to your largest floor mount unit that might have a 30 horsepower pump, and the options that go into these units can vary.

You could order a unit with just a single pump or a simplex unit, where you might have a couple of the simplex units piped in series or in parallel. Or, you can order units with a built in strainer for the filtering aspect of DLC. They also have primary flow meter options and coolant feeder options – which are basically reservoirs and small little pumps that’ll maintain the loop pressure and fluid capacity. There [are also] a few other electrical options depending on your geolocation. Obviously, in the States, we see a lot of 3-phase, but they also manufacture for other voltages and frequencies that are prevalent in the other geo markets.

Ryan Schradin: Let’s say I was a data center owner or operator, and I’m making a CDU purchasing decision. We’ve got the floor mount and some of the other different [CDU options] that you talked about there. Why would I choose one over the other? What considerations what I have to keep in mind? What considerations go into choosing which type of CDU I’m going to use?

Ben Graham: Well, there’s a few considerations that you would want to keep in mind. The primary consideration is, “What is connected to what am I cooling with this unit? Am I just connecting to chilled doors or rear door heat exchangers? Or am I connecting it to direct liquid cooling (DLC)?”

Those two aspects would be almost a completely different ordering package. The chilled doors don’t necessarily require a lot of the bells and whistles as a DLC unit would. The chilled doors wouldn’t necessarily include a primary flow meter, the secondary strainer, or the glycol feeder options. But, when it comes to the DLC version, I would wholeheartedly shift and utilize those options to the fullest. It just makes life a lot easier when you’re dealing with the DLC to have those baked into the unit.

And then, lastly, [I would consider], “Do I want a higher level of redundancy? Should I use two units and put those next to each other, piping them in series so that if one has an issue, the other unit can take up the slack? Or do I just want to rely on a single unit to accomplish the job?” So, those are varying aspects [that I would] try to focus on – and that I think the customer should focus in on – when making any of these choices.

Ryan Schradin: Let’s stick with that for a minute. Let’s [keep] our data center owner and operator hat on and talk about buying decisions. We’ve all had to buy stuff that’s new technology, or we’re not too knowledgeable about, and there’s nothing worse than that, right? You don’t know what you don’t know. And there’s always a lot of questions about [whether] you’re making the right choice.

So, I’m a data center owner or data center operator and I’m designing and building a new data center that’s going to have a liquid cooling system in it. Or, slightly worse, I’m retrofitting an old data center to have a liquid cooling system in it. I have to buy a CDU. What do I need to know? What do I have to keep in mind? What do I have to come prepared with to make an informed CDU buying decision?

Ben Graham: So, the biggest thing, when you’re purchasing a CD, you want to make sure that the CDU can do the job – meaning transfer the heat and provide the fluid flow rate that’s required. And then, the next step is – once you have identified the flow rate and capacity – [is to identify] what are the other options for that unit that might not necessarily break the bank, but also provide good value and ease of operation down the line? If I wanted a fully redundant unit, take that in mind as well as how they are going to be piped into the system.

So, I want to understand those things before pulling the trigger on an order. [I also want to understand] what manufacturer options I want and ensure that the units [I’m purchasing have those options]. Because there’s nothing worse than getting a system up and running and it’s a nightmare to service. So, you also want to look down the road on the serviceability side of things.

Ryan Schradin: Excellent. So, I guess they can keep all those considerations in mind and think about all those things and make a buying decision or just call you and have you do it for them.

Ben Graham: That would be the easy button – if you will. I pride myself on assessing those scenarios and coming up with a really good solution.

It benefits both the data center owner and the personnel who will be maintaining the system down the road. It provides a really good bit of insight, as well, because then we really can assess the fundamental [requirements] of what [the system is] being used for and if the design is adequate or over-engineered. It’s a really good [process to follow]. I can get in there and assess what is needed, [which] tends to make everyone happy because they get a solution that works.

Ryan Schradin: That’s excellent. Well, I bet this has been an incredibly educational primer into the world of liquid cooling and CDUs. I think everybody who’s listening is coming away with a better understanding of what the CDU is, why it’s the heart of the liquid cooling system, and what they have to keep in mind when they’re choosing a CDU or making a CDU buying decision.

This has been great! And I want to thank you so much for taking the time to talk to us and educate us a little bit today.

Ben Graham: Of course! You’re very welcome! I wholeheartedly appreciate your time, as well as putting this together. I think that anyone that’s [making] this decision would definitely benefit from understanding a little bit more when it comes to the CDUs and how they are the lifeblood of any cooling solution.

Ryan Schradin: I think you’ve done our listeners a service today, and I do want to speak to our listeners for a second. I want to thank you guys so much for joining us for this, our very first episode of The Modern Data Center Journal Podcast.