The increased adoption of advanced artificial intelligence (AI) applications and solutions has significantly impacted modern data centers. Data center owners and operators that build or retrofit data centers for AI workloads have to design and construct their data centers with the unique requirements of AI applications in mind.
With modern AI applications often running on power-dense racks that generate tremendous amounts of heat, how data centers are cooled and powered also needs to be considered. Traditional air-cooled data centers are no longer sufficient for AI applications.
So, how can data center owners and operators keep their data centers cool? And how can they free up power that was used to run cooling systems to run more compute resources?
According to Bret Davis, a Senior Project Engineer at CoolIT Systems (CoolIT), the answer lies in liquid cooling. We recently sat down with Bret to discuss the evolution of data center cooling systems, the need for liquid cooling in the age of AI, and why direct-to-chip cooling is gaining traction for AI data centers.
The Modern Data Center Journal (MDCJ): Can you tell our readers a bit about CoolIT’s products and solutions?
Bret Davis: CoolIT specializes in scalable liquid cooling solutions for the world’s most demanding computing environments. In the enterprise data center, AI, and high-performance computing markets, we partner with global processor and server design leaders to develop the most efficient and reliable liquid cooling solutions for their products.
Our Direct Liquid Cooling technology enables dramatically increased rack densities, component performance, and power efficiencies. This is incredibly important right now, as AI and machine learning (ML) workloads are driving up rack densities across the data center industry. Today, we are the global leader in liquid cooling, and our direct liquid cooling system is the most trusted in the market.
MDCJ: What about AI has made direct liquid cooling essential?
Bret Davis: With the rise of AI, CPUs and GPUs are generating unprecedented heat while their operating temperature requirements for peak performance are decreasing.
In the next few years, processor roadmaps for server CPUs are expected to reach 600W, and GPUs are expected to exceed 1,200W. This raises the need for a more efficient cooling solution than what traditional air cooling can provide.
Server and rack designs are also densifying. Today, rack power densities can exceed 70kW per rack in normal operation, making it practically impossible to cool with traditional air-cooling methods that need space between servers to accommodate sufficient airflow with large heat sinks and server-level fans. That’s where direct liquid cooling comes in. It can cool these high-heat components in a very dense space, making it much more efficient than air cooling.
MDCJ: There are numerous different kinds of liquid cooling. Why does CoolIT believe that direct liquid cooling is the best method? Are there certain use cases or environments where other kinds of liquid cooling – immersion, rear door exchangers, etc. – might be a better alternative?
Bret Davis: Compared to immersion cooling, direct liquid cooling supports higher TDP processors, reduces cooling energy consumption by up to 50 percent, and virtually eliminates fluid costs. We feel this makes it the superior choice for AI data centers and high-performance computing environments. Unlike immersion, direct liquid cooling fits within the current rack and row data center layout, making it an easier transition for data centers to introduce liquid cooling into their racks.
“Developing a long-term strategy that includes potential upgrades, evolving technologies, and changing data center needs ensures that the chosen liquid cooling solution remains effective and relevant as technologies and requirements evolve.” – Bret Davis
Our single-phase direct liquid cooling technology is the most mature technology in the market now, with more and more data centers implementing and leveraging DLC. DLC offers significant benefits over other liquid cooling methods. For example, CoolIT’s patented coldplates, utilizing Split-Flow technology, ensure the hottest components are targeted with the coolest liquid first. This allows our users to manage more than 1,500 W processor TDP and more than 300 kW per rack.
Our customers have reported significant savings, including at least a 10 percent reduction in energy bills (OPEX) and more than a 50 percent decrease in capital expenditure (CAPEX). Additionally, we’ve seen impressive improvements in Power Usage Effectiveness (PUE), with examples lowered from 1.30 to 1.02.
MDCJ: At the recent Thought Leadership Summit sponsored by Compu Dynamics and InterGlobix, many of the speakers and experts agreed that power is a limiting factor in the continued growth and expansion of the data center industry and AI. How can liquid cooling help data center owners and operators get the most out of the available power in the data center?
Bret Davis: In most data centers, power consumption is split up, with 40 percent going towards compute, 40 percent going to the cooling system, and 20 percent being used by other IT equipment, including power supply systems, communication equipment, and storage equipment.
Cooling represents the largest opportunity for data centers to reduce power consumption. Implementing direct liquid cooling can improve energy efficiency, operating costs, and sustainability practices.
MDCJ: I understand sustainability is becoming a larger issue for data center owners and operators. How can liquid cooling make data centers more sustainable?
Bret Davis: Direct liquid cooling can significantly enhance the sustainability of data centers in multiple ways.
“Compared to immersion cooling, direct liquid cooling supports higher TDP processors, reduces cooling energy consumption by up to 50 percent, and virtually eliminates fluid costs. We feel this makes it the superior choice for AI data centers and high-performance computing environments.” – Bret Davis
First, it offers significantly lower PUE ratings, indicating more efficient energy use and reduced operational costs. It also enables the more effective capture and reuse of heat, allowing data centers to recycle heat for other purposes such as heating swimming pools, providing district heating, and supporting agricultural operations like greenhouse heating, further enhancing overall energy efficiency and sustainability.
Adopting technology – such as direct liquid cooling – that increases energy efficiency while enabling heat reuse programs would help a data center meet its sustainability goals as well as lower operating expenses (OPEX) in the AI-era.
MDCJ: What considerations should data center owners and operators keep in mind when choosing a liquid cooling solution?
Bret Davis: First and foremost, they should plan for the future! Developing a long-term strategy that includes potential upgrades, evolving technologies, and changing data center needs ensures that the chosen liquid cooling solution remains effective and relevant as technologies and requirements evolve.
In addition to planning, they should look for three key characteristics in their liquid cooling systems. The first is scalability. This includes opting for components and pipe dimensions that can handle increased flow rates for future expansion.
The next factor is reliability. To ensure maximum uptime, data center owners or operators should incorporate dependable components and redundancy into the design. Ultimately, uptime is everything.
“In the next few years, processor roadmaps for server CPUs are expected to reach 600W, and GPUs are expected to exceed 1,200W. This raises the need for a more efficient cooling solution than what traditional air cooling can provide.” – Bret Davis
Finally, they should consider the environmental Impact. This includes assessing the environmental advantages, such as lower energy usage and efficient heat reuse. Eco-friendly systems boost sustainability and help meet regulatory and corporate environmental goals. They’re also more economical in the long run.
MDCJ: What services are available for data center owners and operators who might be concerned about implementing or maintaining a liquid cooling system?
Bret Davis: CoolIT offers a team of industry experts to assist in the design, implementation, and maintenance of your liquid cooling system. Our specialists address all concerns during the conceptual phase, ensuring full confidence in the liquid-cooled architecture.
We also have an extensive network of Authorized Service Providers who can assist with deploying and maintaining liquid cooling solutions worldwide, ensuring that they operate at peak efficiency throughout their lifespan.
Click HERE to learn more about AI’s impact on data center design, construction, power, and cooling.
