Immersion cooling: a key technology for the energy challenges of AI

Immersion cooling is seen as a potentially efficient answer to the new energy-related challenges associated with artificial intelligence.  In the current field of innovation, AI has been transformed from an emerging technology into an essential component of global digital infrastructure.  According to a Gallup poll, 99% of Americans interact with AI based products on a weekly basis, reflecting how it has become an integral part of everyday life.

Behind the ubiquity of AI lies an extensive network of high performance data centres, designed to withstand intense computational loads, as well as complex machine learning and cloud computing applications.  Nevertheless, this exponential increase in calculation power brings with it a corresponding increase in heat production, making traditional cooling technologies less and less sustainable.

What is immersion cooling?

The term immersion cooling refers to a thermal management technology in which electronic components (typically servers and calculation units), are immersed in a non-conducting dielectric fluid.  This fluid, which can consist of specialist mineral oils or engineered silicons, directly absorbs the heat generated by the tecnichal components and transfers it via heat exchangers.

Unlike air-based cooling systems, which are subject to thermodynamic limits related to te low thermal capacity and conductivity of air, the fluids used in immersion cooling have a far greater capacity to absorb and dissipate heat.  This permits more efficient management of operational temperatures and reduces the need for mechanical components such as fans or powerful pumps.

There are two main approaches to immersion cooling:

• single phase immersion cooling: in this configuration, the dielectric fluid remains in a liquid state throughout the process. The fluid is heated in contact with the IT components, then pumped towards a heat exchanger where it cools and is subsequently recirculated around the system.  This technology is easier to implement and the fluid’s lifecycle is relatively stable;
• double phase immersion cooling: this more advanced variant uses fluids such as fluorocarbons, which evaporate at low temperatures. The heat emitted by the servers makes the fluid boil, transforming it into steam.  This steam is then condensed by cooling coils placed above or around the immersion environment and the condensed fluid returns to the tank.  This process creates a continuous, highly efficient thermal cycle, reducing the need for mechanical pumping.

Advantages of immersion cooling

The introduction of immersion cooling brings numerous benefits, in terms of performance as well as financial and environmental sustainability.  According to McKinsey & Co., cooling accounts for about 40% of a data centre’s total energy consumption.  Thanks to the dialectric fluids’ superior thermal capacity, immersion cooling can drastically reduce this consumption, while also lowering the PUE (Power Usage Effectiveness), a key indicator of data centre energy efficiency.

Immersion systems also dissipate heat more efficiently, allowing for a higher density of components per rack.  This is particularly useful for AI and High Performance Computing (HPC), where high performance GPU units release elevated quantities of heat into limited spaces.  As the immersed servers do not need internal fans for thermal control, there are fewer mechanical components which could potentially fail.  This reduces the need for maintenance and extends the hardware’s lifecycle.

The absence of high power air cooling systems also considerably reduces the level of noise generated.  In addition, thermal fluctuations are more efficiently managed, reducing the risk of component damage caused by peaks in temperature.  Finally, with appropriate planning and the use of sustainable fluids, immersion cooling can reduce CO₂ emissions and improve hydrological efficiency, especially when compared to traditional, evaporation-based cooling systems.

Challenges to be overcome and future prospects

Despite the advantages outlined above, immersion cooling must still overcome several obstacles if it is to become established as an industrial standard.  One of the main challenges regards its compatibility with existing infrastructure.  The majority of data centres currently in operation are designed for air based cooling and converting such centres to make them compatible with immersion based systems can be a complex, costly process.

Another issue involves hardware compatibility.  Servers currently on the market were mainly designed for use in dry environments and some of their components and materials are not necessarily suitable for immersion in dielectric fluids.  Without common standards or close collaboration with the original equipment manufacturers (OEMs), it is difficult to create fully optimised hardware for use in immersion environments.

The managemement of the fluids’ lifecycle is also a critical factor.  Although the dielectric fluids used are stable, they still need continuous monitoring and regular maintenance and must be disposed of according to specific environmental regulations.  Furthermore, their production and distribution could create bottlenecks in the supply chain, especially if there is a surge in demand.

Staff training represents a barrier which must not be overlooked.  Considering that data centre operators, (who are usually trained to manage air cooled systems), can find that they lack the necessary skills to operate complex immersion systems, investment in specialised training programmes is crucial to ensure a smooth, efficient transition.

The global data centre market, currently valued at around 383 billion dollars, is undergoing constant growth.  The rise in demand for energy and regulatory pressure regarding efficiency and sustainability make it the widespread implementation of immersion refrigeration technologies increasingly likely.

Prospects for future development focus on three key factors:

• advances in the chemistry of the fluids;
• standardisation and collaboration between the big cloud operators;
• hybrid and retrofit solutions.

These strategies aim to combine immersion refrigeration with traditional technologies, allowing existing infrastructures to evolve gradually with less need for structural intervention.  Such approaches provide an essential link between current operational requirements and the technological ambitions of the future.

Translated by Joanne Beckwith