FlexCoolDC: Datacenter Cooling Flexibility for Harmonizing Water, Energy, Carbon, and Cost Trade-offs

In an era of growing water demand and amplifying climatic disruptions, water is an increasingly limited and precious resource. As datacenters continue to proliferate due in particular to growing artificial intelligence demand, datacenter reliance on water for cooling is drawing increasing criticism. In this paper, we investigate the water and energy consumption, carbon emissions, and cost associated with the energy demand (including its cooling) from a datacenter, in four diverse locations (California, Texas, Germany, and France). We consider water use both in evaporative cooling and power generation, employing a hybrid datacenter cooling model that is operable with and without evaporative cooling. Subsequently, we examine solutions that expand the equipment’s operational range and actively exploit the cooling design redundancy. Moreover, we investigate the benefits of making datacenters dynamic, that is changing operating balance hourly. The results show that using dry cooling in California reduces the water average consumption by 4.34 million of gallon per month with minimal increase in TCO (0.7%). However in France, water saving increases TCO by up to 3.9x more than in California. The story for carbon emissions is more complicated, dry cooling increases total carbon emissions in some geographies (California, Texas), but has minimal impact in others (France). Operating cooling equipment more aggressively enables wider datacenter operation ranges and the reduction of carbon emissions by 1.36%. This approach also improves the PUE of dry cooling and can produce lower TCO in the future.


INTRODUCTION
Datacenters have grown in size, while the density of energy housed within their racks has also increased considerably.The demand for computing power, driven by generative artificial intelligence (AI), along with the advent of new-generation hardware architectures, is announced to serve as a catalyst for the accelerated growth of datacenters.This expansion could further accentuate the already significant environmental challenges posed by these facilities.As the environmental impact of datacenters is often solely associated with their energy consumption and the resulting carbon footprint, a numerous research work have been dedicated to increasing the energy efficiency of the datacenters, through the reduction of the cooling systems' energy consumption.However, both metrics are not exhaustive to characterize a sustainable datacenter.For instance, water, used both in the power generation and the heat removal processes, is not captured by these metrics.Yet, although water is often taken for granted, it is a finite resource that becomes more scarce as the effects of global warming intensify.In fact, "only 2.5% is fresh and only 1% of that is accessible for direct human use; furthermore the water system is closed, i.e. there is a finite quantity in the ecosphere.Population growth and related land occupation and use, agriculture and irrigation and industrialization mean that demand is continually increasing " [25].The US Environmental Protection Agency (EPA) predicts that at least 40 US states will experience water shortages by 2024 [2].In front of people's vital needs, using water to provide Internet services is not a priority, thus it urges to reduce water consumption of current datacenters.
The use of water for cooling large-scale datacenters tends to provoke conflicts with local residents, and often leads to their temporary closure.In North Holland, the Microsoft's datacenters initially designed to consume between 12 and 20 million liters of water per year, exceeded expectations by consuming 84 million liters in 2021 [33].That occurred during a period of severe drought in the region, resulting in a nine month ban on hyperscale datacenters.Public opposition has also halted construction projects undertaken by Google and Facebook [33].Moreover, social tensions over water resources are poised to intensify and/or extend to other regions where datacenters are (or planned to be) built, as expectation for datacenters growth calls for more cooling capacity and water use.
Understanding the environmental impact of datacenter operations is a complex task.Indeed, optimizing a datacenter's energy efficiency (commonly the Power Usage Effectiveness metric) may result in an unintended shift of the impact onto other equally (or even more) critical indicators, such as water consumption, cost, etc.The objective of this paper is threefold.We aim to 1) estimate, for various geographic regions, the water savings, energy consumption, potential financial overhead, and carbon impact of using a water lossless mechanical cooling design for datacenters, 2) investigate if employing the redundancy and operating the datacenters out of the cooling system specifications limits can compensate these financial and carbon offsets, 3) make a trade-off analysis between saving water and energy/carbon, based on the electricity mix and water scarcity in locations.Our contributions are to • build a hybrid datacenter cooling system model, • characterize the value of water in datacenter operations in varied geographies and seasons, • study the water-power and then the water-carbon trade-offs in varied geographies and seasons, • explore operating the datacenter's cooling equipment "out of specification" frontiers and exploiting the redundancy in order to reduce the energy and cost overhead, and • examine if the benefits of dynamic DC approach can compensate for using less water, and thus more energy.
The rest of this paper is organized as follows.Section 2 presents the problematic of the article.Section 3 describes the cooling system design and the models used in the study.Section 4 presents the methodology.Section 5 presents the results.Section 6 presents recent work on datacenters water and energy consumption.

PROBLEM
Before the use of water-based cooling systems, datacenters and other industries relied on dry cooling.Dry cooling uses ambient air to lower the temperature of a coolant (water) without losing any of it, through a heat exchanger.However, it suffers from low efficiency, which is emphasized during hot summer periods [63].In other words, it uses a non negligible amount of energy, thus causing significant carbon emissions.Therefore, this cooling mode has been progressively replaced by evaporative cooling in datacenters.This happened to be a good option when electricity grids were mostly powered by fossil fuels, which are carbon-emission intensive.Today, the question of water availability is a critical topic, all the more so as the needs of populations are constantly increasing and datacenters are more water-hungry than ever.Conversely, electricity grids have evolved to massively integrate renewables and nuclear power plants.Such a grid mix helps to mitigate the carbon impact of consumers and consequently, dry cooling could become an effective solution.However, the trade-off needs to be assessed for each region, considering the region's energy mix and weather conditions.Another aspect comes into play when choosing the design of cooling system: the cost of energy.Although the use of dry cooling may not induce high pollution when a datacenter consumes energy from low-carbon grids, the electricity bills may remain a concern.The key question for a datacenter operator is "how much does saving water cost?".If it results in a significant loss of revenue, "which flexibility can be exploited in the cooling system to reduce or compensate these costs?".In particular, the mechanical equipment composing the cooling system has operating ranges (temperature and flow) recommended by the American Society of Heating, Refrigerating and Air-Conditioning Engineers [6] (ASHRAE) through its standard 9.9.These operating conditions are a set of rules of conduct evolving over time and are meant to ensure high system reliability.Operating equipment beyond these recommended limits (which may actually be more conservative than physical limits) could be a solution to offset costs.Indeed, such a solution increases the cooling capacity of the equipment, enabling it to evacuate more heat using (little) extra power consumption, so making it more profitable for the datacenter operator.In addition, exploiting the redundancy of the cooling system could reduce investment costs and thus offset the overhead generated by the use of dry cooling.
Finally, designing an hybrid cooling system capable of dynamically alternating and/or optimally sharing the heat load between both cooling modes according to the location, the season and the grid patterns could make a datacenter benefit from the advantages of each mode.We name such techniques dynamic datacenter operations.In this study, we evaluate for datacenters located in different zones, the impact of saving water (using dry cooling compared to evaporative cooling) on the power consumption, carbon emission and cost, then we investigate ways to mitigate those impacts and finally propose dynamic operation strategies to optimize water, power, carbon and costs.

DATACENTER DESIGN AND MODELING 3.1 Cooling system description
We consider an hyper-scale datacenter whose cooling system is shown in Figure 1.The cooling system employs a multi-level design consisting of interconnected loop systems, each introducing a cold medium that absorbs heat and undergoes subsequent cooling cycles.It is a three-loops system made of a datacenter floor, a process loop and a condenser loop.We have incorporated a 20% heat capacity buffer (safety margin) into the equipment design of each loop and ensure N+1 redundancy.In the datacenter floor, the in-rack hotcold aisle design [9] is adopted.Typically, an in-rack cooler adds an air-to-water heat exchanger referred to as Computer Room Air Conditioner (CRAC) at the back of a rack so the hot air exiting the servers immediately flows over coils cooled by water.The resulting fresh air is recirculated and injected at the front-side of the rack.In the datacenter floor, each rack is provided with such a heat exchanger.
The water used to chill the air within the CRACs originates from the process loop.The interaction in the CRACs causes the water to be heated and thereafter to flow towards some centrifugal chillers.Typically, a centrifugal or water-cooled chiller cools water by submerging evaporator and condenser coils in two separate compartments connected by a refrigeration system.The refrigeration consists of a compressor, expansion valve, and piping.The incoming water from the datacenter floor is pumped over the evaporator coils where it is cooled down and is then suitable to be reused in the CRACs.Meanwhile, the heat is propagated to the chillers condensers side, heating water in the condenser loop.
In the last step, the condensers' hot water flows to cooling towers where the heat will be dissipated in the atmosphere.Pumps are also used in the condenser loop to ensure the water dynamics.In this study, we consider mechanical draft dry-wet hybrid cooling towers [71,73].They are governed by two operation principles taking place in different units opened to the surrounding environment: the dry and the wet sections.The hot water originating from the chillers arrives first to the dry section located at the top of the towers, which typically consists of air-cooled heat exchangers.Hence, there is an indirect contact between the ambient air and the water, resulting in heat transfer by convection.Since heat is carried out without loss of water from the loop, the process is called dry cooling.The resulting pre-cooled water then drains into the wet cooling section located at the base of the cooling towers, where a direct contact is established with the ambient air.This causes heat and mass transfer, leading to the humidification of air which is in turn released in the atmosphere by giant fans, and the water temperature drops.The cold water at the exit of the cooling towers flows back to the chillers condensers for the next heat removal iteration.In Figure 1, blue lines represent the cold state of a fluid (water or air) and the red color designates its hot state.Green is meant for the inputs and purple for the decision parameters.

Physics modeling
We make the following assumptions as in [36,71]: 1) there is no energy loss to the environment, i.e the loops are perfectly closed, 2) the transient phase of the heat exchangers is neglected (only the steady states are considered), 3) the physical properties of air and water are temperature independent (constant thermal density).Table 2 of Appendix A presents the symbols used in the models.

3.2.1
Heat exchange in the closed loops.The heat removal in the CRACs and the chillers is modeled based on similar mechanisms: mass and energy conservation.Let's call  the amount of heat generated in the datacenter at a given time.This heat load circulates in the successive loops.Equation 1a presents the heat transfer in the datacenter floor, the evaporators and condensers of the chillers as in [49,65].Equation 1c and Equation 1b represent respectively the recommended/standard fluids flow and temperature limits in the equipment.These limits are specified in the equipment datasheets.In this study, we consider a balanced distribution of the heat load in the equipment of the same type.For instance, the mass and temperature of air refrigerated in the CRACs are identical; and that is generalized to the other stages of the system.
Where    , and    , are respectively the minimum and maximum fluid  (water or air) flow rate in the equipment  (CRAC, chiller or cooling tower).   , and    , are respectively the minimum and maximum fluid  temperature rate in .

3.2.2
Heat rejection in the cooling towers dry sections.The dry section heat exchanger considered in this study is a gas-liquid device containing tubes in the water side arranged in several rows and gas side fins.We adopted the -NTU modeling [69,71,73].We consider a constant water flow at the entrance and the exit of dry section (as well as the wet unit presented in Section 3. UA is the total heat transfer coefficient and is expressed as in Equation 3a.In this representation, ℎ   is the air-side heat transfer coefficient (Equation 3b).It is a function of a characteristic heat parameter (  ), the effective windward area of the heat exchanger (   ), thermal conductivity of air (), the characteristic flow parameter detailed in Equation 3d and the air Prandtl number ( ).
In the water-side, ℎ  model is based on the Gnielinski [26] (Equation 3f) and the Nusselt numbers (Equation 3h).In these equations,   stands for the friction factor [21] and is presented in Equation 3g where   and   are respectively the Reynolds and Prandtl numbers in water side,   is the surface roughness of the heat exchanger.The water-side area (  ) is presented in Equation 3e.
3.2.3Heat rejection in the wet section of the cooling towers.In the wet section, ambient air humidification consists in transferring the latent heat from the water to the ambient air, which increases the specific humidity of the ambient air.Thus, a part of water evaporates with heat, lowering the temperature of the recirculating water.Let    be the amount of heat dissipated in the wet sections.The wet section model is based on mass conservation, energy conservation as in [36,71].Water-side and air-side heat exchange are respectively presented in Equation 5a and Equation 5b. , is the specific heat of vapor considered constant,  is the specific heat of vaporization i.e the amount of energy needed to evaporate a kg of water,  , , is the humidity of air leaving the wet section.The humidity is calculated as in Equation 5c [36,52].We consider the efficiency of the wet section to be fixed, corresponding to the optimal operation conditions.Thus, the temperature of water at the exit of the wet section can be derived from the efficiency (Equation 5d).The wet bulb temperature (  ) is the minimum possible temperature of the outlet water when the tower operates at 100% efficiency.Equation 5e evaluates the   given specific weather conditions, as does this online calculator [24].Finally, the temperature of air leaving the wet section can be derived from Equation 5a and Equation 5b.
The heat removed in the cooling towers is  =     +    .We assume that the air steaming from the wet and dry sections is blended evenly.As a result, the humidity of the air released into nature is as in Equation 6a and its temperature is presented in Equation 6b.   , and   , represent the enthalpy of air at the outlets of the dry and wet sections respectively.Enthalpy in cooling towers is deeply developed in [36].

Power consumption modeling
The CRACs and the cooling towers power consumption is mainly due to the fans used to control and circulate air [70,72].Pump and fan have similar consumption curves, which are a cubic function of the mass flow of the fluids they convey, as shown in Equation 7.
Where  0, and  0, are the nominal flow rate of fluid  and the nominal power consumption of equipment .  is the power consumption of equipment  when it operates to deliver the mass flow rate   . is either a pump or a fan and  is water or air.
The power consumption of a chiller linearly depends on the amount of heat it removes (Equation 8a).The coefficient of linearity is called coefficient of performance (COP).The COP is dynamic according to the chillers operation state.In this work, we used the Lee's simplified (LS) model [37] which is a mutation of the Gordon-Ng universal model, to predict the COP (Equation 8b).
where  1 ,  2 and  3 are coefficients that can be approximated by linear regression on the COP curves provided by the manufacturer and  ℎ max is the cooling capacity of the chiller.

Water usage modeling
We consider both water consumed by the cooling system (on-site water consumption) and at the electricity generation (indirect water consumption).On-site, water is lost only in the wet section of the cooling towers due to three phenomena [52,64] : 1) water evaporation, 2) wastewater treatment and 3) drift (or windage) water loss.Let's call  % the percentage of water evaporated in the cooling towers.This evaporation loss is influenced by the specific climate as shown in Equation 9b, using the model developed by Qureshi and Zubair in [53].This model predicts the percentage evaporation loss with a maximum error of 6.6% compared to the ASHRAE's rule of thumb [5].The amount of water evaporated can be derived by multiplying  % by the duration and the water flow rate in the cooling tower (converted in gallon/s using water density).As water evaporates from a cooling tower, the dissolved solids in the water remain in the tank, increasing its concentration.Thus, some water need to be replaced.This process is called blowdown.We assume the condenser loop cycles potable water until the concentration of dissolved solids is roughly   (specified by the manufacturer) times the supplied water.Drift is the carryover of small water droplets (non evaporated) from the cooling tower to the ambient air by the exhaust air.Usually, it is specified in the cooling towers datasheets, as a percentage (%) of the vaporized water.Equation 9c shows the blowdown and windage water usage.Indirect water consumption is due to the electricity generation.It depends linearly on the energy consumption with the coefficient of linearity called energy water intensity factor (EWIF).A particular grid's EWIF is estimated using Equation 9d where   is the percentage of energy generated from the source  (solar system, coal, natural gas, nuclear... ) and     is the volume of water consumed per kWh of energy generated by that source.The coefficients     are location-dependent.Specific water usage values for different regions around the world are detailed in [55].Where   is the power consumption of the cooling system.The datacenter power consumed is assumed totally converted into heat [19,52], so its value is .  ,   ,   and   (gallon) are respectively water lost through evaporation, the blowdown water, the volume of drift water and the indirect water consumption.

Total
Cost of Ownership modeling.Two components contribute to the cooling system cost: capital expenditure (CAPEX) and operational expenditure (OPEX).The CAPEX (Equation 10b) includes 1) the investment and installation costs and 2) seasonal (annual) expenses (like insurance, land lease, salaries, planned maintenance, etc.) induced for the ownership of the components and known as fixed operation and maintenance costs.In order to convert the initial capital cost to an annual capital cost, we used the capital recovery factor (CRF) [34] shown in Equation 10d.
The OPEX (Equation 10c) contains expenses like electricity bills, unplanned maintenance which depend on the actual operation of the equipment.In this study, the OPEX is assumed to be dominated by the electricity bills.
Where  ∈ {, ℎ,  , } is a component of the cooling system,   the number of  units,  max, (kW) is the nominal capacity of component ,  , ($/ ),  & , ($/ . ) are respectively the investment and fixed operation and maintenance costs,   () ($/ ℎ) is the instantaneous electricity tariff,  and   are respectively the interest rate and the lifespan of equipment , 8760 is the number of hours in a year.

STUDY 4.1 Objectives
This study examines the water and electricity consumption, as well as the carbon footprint, of the cooling system in both evaporative and dry modes, in four regions of the world (California, Texas, France, and Germany) presenting different weather conditions and energy mixes.In fact, using solely evaporative cooling is expected to cause significant water loss.One can make considerable water savings by using dry cooling which is unfortunately less energy-efficient and may lead to a high carbon footprint, exorbitant electricity bills, and high indirect water consumption (that would offset the on-site water savings).The first step of our study is to characterize the value of water in datacenter operations and the water-power trade-off in varied geographies and seasons.
Secondly, we examine the potential of operating cooling equipment (CRACs and chillers) slightly outside of their specified operation ranges to reduce power consumption in order to address the environmental challenges of dry cooling.Moreover, in regions with a low-carbon grid and low indirect water consumption, the economic implications of using dry cooling may be the only factor deterring datacenter operators from using it.As the demand for Internet services grows, datacenter demand and heat generation increase every year.Following the demand requires to increase the datacenters capacity, which may also command to invest in additional cooling equipment.Thus we also explore the long-term potential of exploiting the redundant equipment to save on investment and to offset the bills.
Finally, we investigate the benefits of adopting a dynamic datacenter and the associated requirement of trade-offs.More precisely, we propose two optimization problems that aim to: 1) share the heat load between the dry and wet sections and find the operation points of these sections to harmonize water, power consumption, carbon emission and costs 2) alternate between the evaporative and dry modes with regard to the local precipitations and grid status.

Methodology
The simulations are based on a 1h discrete time-step Δ.In this study, power demand designates the instantaneous power (MW) imported from the grid, and power consumption and energy consumption (MWh) are used interchangeably.

4.2.1
Datacenter and AI growth.We modeled the datacenter load growth forecast using the Compound Annual Growth Rate (CAGR).The CAGR of datacenter load in the Cloud is estimated in several regional and national reports and summarized in Table 1.Cloud + AI demand includes the additional AI demand.The AI CAGR methodology was proposed by Lin et al. [39].For the USA, we estimated the AI growth by extrapolating NVIDIA's most recent revenue released in August 2023 [45].We first calculate the number of GPUs sold by NVIDIA based on the price of A100 hardware ($10,000 per GPU [38], consuming in average 300W [44]).Then, we use the methodology proposed by Chien et al. [13] to forecast the corresponding datacenter capacity supported by these GPUs.By normalizing this increase to a median estimate of overall datacenter capacity, we arrive at a 4.12% increase in average US CAGR.However, some regions experience higher growth rates for datacenters due to favorable conditions (good communication networks, low operating costs, etc.).We adjust the CAGR as in Equation 11. (11) Where  +, is the CAGR of Cloud+AI in region  (California , Texas),  , (whose value is 7.21% as estimated from IEA's 2015-2022 data [1]) and  , are respectively the average US cloud and the average AI estimated growth (4.12%).
, is the CAGR of cloud datacenters in the region  .
In the French and German case studies, the AI annual revenue growth is detailed respectively in [62] and [61].We assume that this growth is also due to the installation of NVIDIA A100 GPUs and used the aforementioned process to estimate the AI CAGR.In this study, we consider a datacenter of 20MW capacity in 2022.Thus, following the region/country of interest, it grows according to the local CAGR.This may correspond to adding new racks to the datacenter floor, each with a CRAC.

Dynamic Datacenter
Operation.We consider datacenters whose heat rejection is provided by a hybrid cooling tower.We investigate two approaches making the cooling system dynamic: 1) sharing the heat load between the cooling towers' dry and wet sections, 2) alternating between dry and evaporative cooling modes according to seasonal water availability (rainfall), the real-time energy mix in the grid (carbon intensity and pricing), and the specific demands of the system.Let's define  0 = 1 −   /  where   and   are respectively the instantaneous and annual average precipitations (in mm), and  0 = ( / ).(/)where  and  are the instantaneous and annual average carbon intensity of a given grid,  and  are the instantaneous and annual average TCO.In this study, we do not consider predicting future events.Thus, we consider the median parameters (  ,  and ) of the previous year (2021).
The first approach considers intermediate operating points involving the simultaneous functioning of the dry and wet sections of the cooling towers.Thus, the aim is to find the optimum   (heat dissipated in the wet sections) which maximize both the percentage of water saved and minimize that of power overhead (Equation 12a).Moreover, we assign some water budget to each datacenter, with regard to the local rainfall (Equation 12b).This approach is meant to find the best trade-off between water consumption, power consumption, carbon emissions and costs.
Where  and  are weights respecting  +  = 1.We define .  ,max and  ,min are respectively the water and power consumption when the datacenter operates in full evaporative mode. ,min is its water consumption when operating in the exclusive dry mode.
The second approach chooses the evaporative mode when precipitations can balance the carbon and costs (Equation 13).In the reverse, the dry mode is chosen.
4.2.3Input data.In this study, we consider the Vertiv Liebert DCD50 [66] heat exchanger in the racks.It is associated to the YD dual centrifugal chiller of compressor reference K7, evaporator reference K4 and condenser reference K4 [18].The wet section of the cooling tower is based on the Evapco series 3000 Model S3E-1424-14W-2 [23] and the dry section heat exchangers from [71,73].
In the process and condenser loops, we consider the 12-LNS-32 Horizontal Single-Stage Centrifugal Pump [68].The equipment specifications, weather, grid and other inputs used to conduct this study are summarized in Table 3 of Appendix B. We consider a hand-made diurnal workload which model is described in [14].The workload is generated for one day and we assume a daily periodicity in the computation demand to build an annual workload.Moreover, to ensure meaningfully conclusions, we use the same workload in the different locations considered in this study.Let's call  () the datacenters' computation demand at time  and  the daily average computation demand.The datacenter's electrical power demand (which is equivalent to the amount of heat ) is considered logarithmically dependent on computing power demand, as shown in Equation 14.
Where  0 is a scaling constant and  is a constant that ensures a positive power demand supported by the datacenters (0 ≤  ≤ 20 + 4 of safety margin).

EVALUATION 5.1 Water, power and carbon trade-off analysis
Let's consider a 20MW datacenter located in California (connected to the CAISO grid), Texas (connected to the Ercot grid), Germany or France.First, we operate the datacenter both in evaporative and dry modes for the year 2022, in order to quantify the amount of water that can be saved by (in dry mode compared to the evaporative mode), the energy consumption overhead and the economical efforts that it would require from the datacenter operator.We define the energy overhead as the difference between the energy consumption in dry cooling and that in evaporative cooling mode.We also define the water monetary value as the ratio between the cost overhead due to dry cooling and the volume of water it allows to economize, as in Equation 15.
water value ($/gallon) = TCO in dry cooling -TCO in evaporative cooling  in evaporative cooling - in dry cooling (15) The results are shown in Figure 2. All our target locations show seasonal variation in energy consumption overhead and in the water value.In summer, rising temperatures in the four locations further reduce the energy-efficiency of dry cooling, leading to much higher energy consumption compared to the evaporative cooling (Figure 2a).As a result, the water value increases (Figure 2b).Moreover, the power consumption in California and Texas is higher than in the chosen European locations.In fact, the temperatures in these states were higher in 2022, reaching extreme peaks in summer.The peak of energy overhead in California (the highest) reaches 204% that of Germany (the lowest peak).On average, the worst energy consumption surplus is recorded in Texas, and represents 219% of the smallest recorded in France.Thus, France and Germany are better suited to cooling the datacenter in terms of energy consumption.However, energy consumption does not reflect the cost of switching from evaporative to dry cooling.Figure 2b illustrates that: operating a datacenter with dry cooling in France is on average 3.9 times more expensive than in California, despite lower energy increase in France.The higher electricity tariffs in Europe are the main reason for this observation.It can also be noted that the 2022 French day-ahead prices being slightly greater on average than the German ones, this cost is higher in France.Figure 11 of Appendix D presents the percentage of cost increase compared to the evaporative cooling.We observe that dry cooling increases the costs by less than 2% on average, in all the locations.
To understand how efficiently water is used under both operating modes, we analyze two key metrics defined by the Green Grid [8]: 1) Water Usage Effectiveness (WUE=[  +   +   ]/) shows how much water the datacenters use on-site, per kilowatthour (kWh) of energy consumed by the IT equipment -excluding energy required for cooling-, 2) Water Usage Effectiveness Source (WUE  = [  +   +   +   ]/) takes a broader view, considering both the on-site and indirect water used per kWh of IT load.As illustrated in Figure 3, the dry cooling scheme results in WUE of zero as it eliminates water loss on-site.In the evaporative cooling mode, the WUE is season-dependent.California shows the highest WUE average at 2.36L/kWh (standard deviation of 0.46), followed by Texas at 2.42L/kWh (0.28 deviation).Germany and France show lower averages of 2.03L/kWh (0.11 deviation) and 2.04 L/kWh (0.15 deviation), respectively.In dry cooling mode, the WUE  shows minimal variation throughout the year.In contrast, evaporative cooling shows seasonal fluctuations in WUE  , which represents on average 3x, 1.46x, 1.73x and 2.8x the WUE respectively in California, Texas, Germany and France.In fact, water footprint of the datacenters is significantly influenced by the water intensity of the electricity source.This indirect water demand is particularly noticeable in California and France, where hydropower and nuclear power dominate the electrical grids.These energy sources, while considered clean, require important quantities of water [55].
Figure 4 provides a detailed breakdown of water consumption in the four locations, separating the water used on-site from the water used to generate the electricity for powering the cooling system (CS) and operating the IT load.In evaporative cooling, IT operation is the dominant consumer of water in California and France.On average, IT load accounts for 7.75 million gallons monthly (or 57.6% of the total) in California and 6.02 million gallons (or 54.73%) in France.Direct on-site water usage is the second-highest contributor at 4.47 million gallons (or 33.13%) in California and 3.87 million gallons (or 35.21%) in France.Conversely, Germany and Texas grids have less water-demanding electrical grids.This flips the water consumption pattern and on-site water usage becomes the dominant factor.For example, in Texas, on-site consumption accounts for a significant portion (4.58 million gallons on average monthly or 68.49%) compared to the indirect IT water usage (1.77 million gallons or 26.46%).Similarly, in Germany, on-site consumption is 3.86 million gallons (57.92%) and the indirect IT water consumption is 2.38 million gallons (33.72%).Interestingly, the indirect water used by the cooling system is the lowest in all the locations, which represents an attractive flexibility to adopt dry cooling.Figure 4 shows an increase in indirect water consumption of dry cooling system compared to that of evaporative cooling.However, this increase remains significantly lower than the overall water savings achieved by eliminating water evaporation in the cooling system.In fact, on average, dry cooling would need to use 3.5x, 3.52x, 9.1x, and 13.5x more water for cooling, respectively in France, California, Germany, and Texas, to outweigh the on-site water consumption.
Figure 5a (top) shows the power consumption of the cooling system in the four locations.Dry cooling in California can potentially save monthly 4.34 million gallon of water on average, with an average increase of 111.3 kWh of power consumption.Germany, France and Texas allow on average to make respective water savings of 3.83, 3.82 and 4.54 millions gallons per month while they lead respectively to an increased power consumption of 70.9kWh, 68.6kWh and 150.3kWh compared to the evaporative cooling.As stated in Section 4.2.3, we consider the same workload in the four locations, with a daily periodicity.Hence, the IT power consumption does not impact the conclusions and will be disregarded in the rest of our analysis.
Figure 5b (top) shows the carbon footprint of the cooling system depending on the data center location.The evaporative cooling system generates on average 80.5tCO2eq/month, 257.4tCO2eq/month, 465.5tCO2eq/month, and 511.1tCO2eq/month respectively for France, California, Texas and Germany locations, a carbon footprint attributable to their power demand.On average, the dry mode increases their carbon emissions by respectively 4.3tCO2eq/month (or 5.11%), 23.4tCO2eq/month (or 9.1%), 50.5tCO2eq/month (or 10.85%) and 25.6tCO2eq/month (or 5%) compared to the evaporative cooling.As the French energy mix is less carbon intensive (nuclear energy source mainly), the rise in power demand is mitigated.The cooling system in California has a 3.3x higher carbon footprint than the one in France and increases the overall water consumption by 31.64%.Conversely, the cooling system in Texas and Germany generates  respectively 6.1x and 6.3x the carbon footprint recorded in France, to consume respectively 39.1% and 39.2% less water.The bottom portion of the figure illustrates the carbon footprint associated with running the data centers in each location.Similar carbon impacts as those of the cooling system emerge, though at different scales.Therefore, considering the IT's carbon footprint would not affect our conclusions.To isolate the impact of each cooling configuration on carbon emissions, we will exclude IT-related carbon footprint from the remaining analysis.

Extending the operation ranges improves dry cooling performances
The ASHRAE standard 9.9 [6] (last updated on March 2021) makes recommendations on the temperature ranges under which cooling equipment (such as CRACs and chillers) ought to operate for guaranteed reliability.Yet, improvements are anticipated to yield enhanced efficiency and resilience, enabling these systems to exceed the recommended limits.Thus, we propose to operate the cooling equipment (in dry mode) beyond the specifications within reasonable ranges, in order to increase their capacity and reduce the carbon emissions and costs.This could permit the circulation of colder or hotter fluids, and/or at higher flow rates.Let's consider three scenarios in which: 1) the datacenters in the four locations operate in evaporative mode within the specifications, 2) they operate in dry mode within the specifications and 3) they operate in dry mode beyond the recommended specifications with an exceeding margin limited to 30%.The latest scenario implies an increase in inlet air temperature limits of the CRACs by 4 to 6°C and an increase in inlet water temperature limits of the chillers by 0 to 4.5°C.
Alternatively, the circulating fluids may flow up to 30% faster.
Figure 6 shows the monthly power usage effectiveness (PUE) for the three scenarios at each location.The PUE is a metric used to evaluate the energy efficiency of a datacenter.We observe that in all the locations, operating the datacenter with higher ranges improves its efficiency.Utilizing this approach is particularly beneficial during winter months (January, November, December) as it achieves an efficiency equivalent to or slightly surpassing that of evaporative cooling.However, increasing the operating ranges by 30% is insufficient to compensate the power consumption overhead induced by dry cooling.In fact, it reduces the PUE by only 0.0025 to 0.0027 (or 0.21% to 0.22%) in the datacenter.A broad spectrum of operation ranges is needed, but that could impact the equipment reliability.A comprehensive investigation into the permissible limits and the failure risks with overheating is left for future research.
Figure 7 shows the carbon emissions in the four locations, under the three scenarios.The datacenter operating with broad operation  ranges in France, California, Texas and Germany achieves respectively an average carbon reduction by 1.16 (1.37%), 3.9 (1.39%), 6.33 (1.23%) and 7.32 (1.36%) tCO2eq/month.Some observations made for the PUE are also applicable for the carbon emissions: operating datacenter equipment beyond its specified limits can lower carbon emissions but is far from offsetting the increased carbon impact associated with operating in dry mode.

Exploiting the redundancy may reduce the water value
The past few decades have witnessed an extraordinary growth in datacenter demands, driven by the explosion of Internet-connected devices.As a result, the datacenters, whose duty it is to provide services over the Internet, have enlarged their power capacity in order to respond to or to anticipate this demand.This growth is accommodated through the construction of new datacenters, the expansion of existing datacenters with sufficient space, or the replacement of existing computing equipment with higher-density alternatives.
Consequently, the cooling system may need to be adapted.Dry cooling mode presents an economic drawback for datacenter operators as they pay more for electricity.In this section, we make an extrapolation of water usage and evaluate the cooling system costs in two scenarios: 1) evaporative cooling is utilized across the datacenters, and system resizing is considered annually when its capacity cannot guarantee the N+1 redundancy; 2) dry cooling is considered, actively utilizing the redundant equipment without resizing the cooling system.While the latest scenario supports the datacenters heat load, it does not guarantee N+1 redundancy.Figure 8 shows the results for the four locations, from 2022 to 2028.Purple lines and bars stand respectively for the TCO and water consumption in the first scenario.Similarly, silver lines and bars represent the TCO and water usage in the second scenario.In this figure, asterisks (*) appear within a bar when redundancy is guaranteed in all the loop, and they are absent when at least one stage of the cooling system lacks redundancy.The result shows that in France, the datacenter growth is lower and does not require a sacrifice of redundancy.As a result, evaporative cooling remains the cost-effective option compared to dry cooling.Exploiting redundancy in Texas, Germany, and California results in a lower dry cooling cost compared to evaporative cooling, in the long term.This is attributed to savings on new equipment investments, which offset and even surpass the additional expenses incurred by dry cooling usage.However, the datacenters in these regions concede their redundancy in 2025, 2026, and 2027, respectively.Moreover, evaporative cooling exhibits a faster rate of water usage compared to dry cooling, exacerbating the water scarcity in the coming years.
The proliferation of artificial intelligence (AI) applications is fueling the rapid growth of datacenters.Let's examine the impact of AI load demand on water consumption and total cost of ownership (TCO) in datacenters across the four locations.The datacenter in France will retain its redundancy through 2028, rendering dry cooling cost reductions ineffective.However, in the other locations, the active use of the redundant equipment has a clear economical impact but it accelerates the loss of redundancy.For instance, Texas and Germany datacenters will not show redundancy from 2024 on (or respectively one and two years earlier than when no additional service is supported) and the Californian sacrifices its redundancy from 2026 (one year earlier than the datacenter with no AI).
This section highlights redundancy as a flexibility to mitigate the financial costs associated with the use of dry cooling.This strategy is only beneficial when the datacenter annual growth is sufficiently high (at least 8.5% in this study).The integration of AI services in datacenters further enhances the potential of redundancy.Nevertheless, this approach introduces an element of risk in the event of equipment failure.Without guaranteed redundancy, some compute nodes may need to be halted, increasing the likelihood of overheating in the datacenter and potential damage to facilities.

Operating a Dynamic Datacenter
We examine the appropriateness of adopting dynamic datacenter operation in two scenarios (detailed in Section 4.2.2).In the first scenario, named simultaneous operation, the heat load is shared between the dry and wet sections of the cooling towers in order to maximize the water savings while containing the power consumption increase.In the second one, named alternating operation, the cooling system switches hourly between the dry and the evaporative cooling modes according to the local water availability and grid characteristics.Both scenarios are compared to the fully evaporative cooling, considered as baseline.Figure 12 of Appendix E presents the monthly rainfalls, the average carbon intensity and tariffs of the local grids considered.
Figure 10 shows the percentages of water savings made by the simultaneous and alternating operation modes, and the counterbalance in power, carbon and cost overheads.The absolute values are shown in Figure 13 of Appendix F. The simultaneous solution reduces the average monthly water consumption by 32.14%, 60.2%, 53.22% and 32.2% respectively in France, Texas, Germany and California.Conversely, it increases the monthly average power consumption by respectively 4.2%, 9.2%, 4.5% and 8.5% .Moreover, it increases the monthly average carbon emissions by respectively 4.2%, 9.1%, 4.1% and 8.2%.Concerning the costs (TCO), the approach increases them respectively by 1.4%, 0.8%, 1% and 0.7%.Optimally sharing the load between dry and evaporative cooling achieves high water savings with relatively low impact on the other indicators.
Compared to the dry operation mode (Figure 14 in Appendix G), the simultaneous mode saves energy and reduces carbon emissions on average by 0.3% to 1.5% across the four locations.As for the TCO, it is reduced by less than 0.3%.However, this operation mode consumes more water: 7.59%, 4.48%, 2.57% and 0.88% in respectively Texas (where precipitations were the highest), Germany, France and California (where water scarcity was high in 2022).Nevertheless, the relaxation in water savings could be seen as an attractive feature as water is consumed when it is locally abundant.
The alternating operation scheme further reduces the total monthly average water consumption by 34.47% in France, 63.1% in Texas, 63.1% in Germany and 55.82% in California.It represents a gain of 0.7% to 3%, compared to the simultaneous solution.Conversely, the alternating solution increases the power consumption, carbon emissions and TCO (<1% compared to the simultaneous scheme).
We also compare the alternating mode to the dry operation mode (see Figure 14 in Appendix G).It reduces the power consumption and carbon emissions by less than 0.7% across all the locations.Especially in California that showed severe water stress in 2022, the alternating operation reduces the power consumption, carbon emission and cost by (only) respectively 0.06% 0.03%, and 0.02%.It means that most of the time, the cooling system works in dry mode.In an other hand, the alternating operation requires more water than the dry cooling mode.The additional demand is between 0.2% (in California) and 4.68% (in Texas) compared to dry cooling.In summary, the best compromise for dynamic datacenters is the simultaneous mode, which better capitalizes on the opportunities offered by one indicator to reduce the impact of the other indicators.

RELATED WORK
Thermal management is essential for datacenters to maintain the optimal temperature, humidity, and airflow levels for their equipment to operate reliably and efficiently.There are many datacenter cooling system designs available, differing by technology, energy and water consumption profiles.Air free-cooling is a simple design that does not require water to operate.In fact, it allows to use naturally cool air to lower the temperature in the datacenter, without the need for mechanical refrigeration.This technique is broadly used in research projects [7,59].However, it is only applicable in specific meteorological conditions: locations with colder temperature (< 20°C) and higher humidity (> 60%) [59].Therefore, many locations are not suitable for this design.Even regions with average satisfactory weather conditions may have unsuitable periods during the year.Nevertheless, free-cooling is used in association with mechanical refrigeration in some real-world data-center like Amazon Web Service (AWS) [4], in order to take advantage of cheap and water-friendly cooling when the weather is favorable.Unfortunately, free-cooling will become less applicable as the global climate warms.Moreover, an exclusive use of free-cooling in large datacenters could be non efficient and challenging to deploy, as air has a low thermal density.Alternatively, a closed loop cooling system [7] uses water to cool down air in the server room.This water is not wasted as the system uses air-cooled chillers to remove the heat from the water, which is then recirculated in the loop.Yet, this design consumes a significant amount of energy to ensure a safe operation in a given datacenter.
In large scale datacenters [9,72] (at the scale of MegaWatts) and supercomputers like the U.S. Department of Energy's ARM-based supercomputer Astra (1.2MW) [27] and the RIKEN research institute's supercomputer Fugaku (up to 40MW) [43], the generated heat is conveyed by water through mechanical refrigeration processes and dissipated into the atmosphere via giant cooling towers.Modern cooling towers dissipate the heat load by means of water evaporation that allows to save about 10% of energy and as much carbon, according to Google estimations [28].Mytton [42] examined the global water consumption of datacenters and pointed out the lack of transparency of some data center operators on this figure.Chen et al. [12] proposed a quantification of the on-site and indirect DC water consumption (water used when generating the power consumed by the DC).The authors also proposed the Water Scarcity Usage Effectiveness (WSUE) as a sustainability metric that captures the impacts of water consumption on regional water availability.Whitehead et al. [25] developed a life cycle assessment tool.This tool helps datacenter designers to understand the environmental impacts of various cooling systems, considering both energy use and water consumption.By analyzing these factors, designers can identify trade-offs and opt for greener solutions for their facilities.A more extensive selection of green metrics for analyzing individual key performance indicators of data centers is detailed in [54].Karimi et al. [35] also conducted a study in which a dry, an evaporative and an hybrid cooling designs are compared, in the greater Phoenix area.However, this study does not consider variability in the locations, the carbon impact and the costs related to each configuration.In [32], the authors proposed a batch job scheduling algorithm, called WACE which dynamically adjusts geographic load balancing and resource provisioning to minimize the water consumption along with carbon emission and electricity cost, in geographically distributed datacenters.In contrast, our study focuses on comparing different cooling setups for datacenters that handle the same workload.This can help identify optimal locations and configurations for single datacenters.
Major cloud providers are developing the next generation of cooling technologies to reduce and/or compensate their water and energy consumption.For instance, Microsoft [60] and Google [11] plan to be water-positive by 2030, i.e to collect more water than their datacenters average consumption per year.The water consumption is tackled in two ways: reducing water use intensity (water consumed per MegaWatt) and replenishing water in the scarce regions where datacenters are implanted.By harvesting rainwater in excess, these firms also seek to support water security for the riverside communities.As for the energy consumption, it is tackled by the progressive experimentation and adoption of energy-efficient cooling technologies like the immersion cooling [41,58].

CONCLUSIONS
In this study, we explore the utilization of dry cooling to alleviate water scarcity and minimize datacenters' reliance on water resources.However, dry cooling exhibits lower energy efficiency, thus increasing the datacenters' power load, their overall operational costs, and their carbon footprint.In regions with a high nuclear and/or hydro power penetration, a non-negligible amount of water is also consumed to power the datacenters.We compared the water consumption, the carbon emissions and energy consumption in four locations namely California, Texas, France and Germany.
To address the limitations of dry cooling, we have proposed expanding the operational ranges of cooling equipment.The results indicate that broadening the operational margins by 30% can lead to a carbon footprint reduction of 1.36%, but cannot compensate the associated drawbacks, particularly the financial ones.Therefore, we have proposed a proactive approach that utilizes redundant equipment for datacenter cooling.This solution offers long-term economies on the system, compared to the evaporative cooling system design that maintains N+1 redundancy.The study shows that with regards to the location, the season and the grid status, dynamic datacenters may benefit from both evaporative and dry cooling.By testing two approaches (alternating and simultaneous), we found that both decrease substantially water consumption (up to 63% compared to fully evaporative mode) in the four locations, for relatively low power, carbon and cost increases.
In the coming work, we aim to conduct an examination of the permissible limits of the cooling equipment and the potential risks associated with extending the operation ranges.Additionally, we will simulate scenarios of failure during the exploitation of redundancy and analyze the responsiveness of datacenters.

C ESTIMATION OF THE INDIRECT WATER CONSUMPTION.
In this study, we use the water consumption factors estimated by [55] in its Appendix 4.However, this estimation does not provide some water consumption factor for the waste source in Germany.Hence, we extrapolate this value based on the water usage in France.We assume the same scale between average water factor (including waste) and waste source in France and Germany.The extrapolation is made as follows.
Waste water factor (Germany) Waste water factor (France)

=
Average water factor (Germany) Average water factor (France) (16) We use the hourly power generation (detailed by generation source) provided in [50] (France and Germany) and [40] (California and Texas).For some locations, like Germany and California, some portions of power generation are of unknown or non specified source.Hence, we apply a proportional average water consumption factor (sum of the known factors by power source divided by the number of generating sources in the location).
Moreover, in California, the local grid imports electricity from the interconnected grids.This imported electricity is from unknown source.On another hand, the local operator (CAISO for California) released a detailed breakdown of the annual energy generation by source type in [16], including the imported power.Hence, we assumed that, at each time step (of one hour), the imported energy has the same mix as the annual value.Thus, we estimate the imported power water factor as follows.

E PRECIPITATIONS AND GRID CHARACTERISTICS
The electricity prices considered in this study as being paid by the datacenter operator are based on hourly day-ahead market prices.Theses prices may not reflect the actual prices paid by consumers, e.g. as they do not include tax and network charges, while presenting a different (higher) average value than wholesale/retail prices as representing only the share of electricity that is sold on the dayahead market.However, they were considered in this study as a good illustration of fluctuating prices envisaged to be more and more proposed to consumers as part of dynamic tariffs.These tariffs are intended to make the consumers more aware of the (closer to) real-time state of the power systems, thereby incentivizing them France-Germany pricing [50] Germany Weather [67] Others' weather [15] precipitations (2021 and 2022) [48] CAISO energy mix [16,40] ERCOT energy mix [22,40] German energy mix [47,50] French energy mix [50,56] Carbon intensity [40] Energy water factors [55] to adapt their consumption in order to help the electrical grids.Figure 12 presents the monthly precipitation recorded in the four locations, the carbon intensity and the tariffs of the relevant grids.

Figure 2 :
Figure 2: Extra energy and cost of using dry cooling.

Figure 3 :
Figure 3: Water effectiveness of the cooling system.
Figure 5a (bottom) shows the energy consumption in the four locations.

Figure 4 :
Figure 4: Water consumption (million of gallons) in the four locations.

Figure 5 :
Figure 5: Power consumption and carbon emissions.

Figure 6 :
Figure 6: Power usage effectiveness in the four locations

Figure 7 :
Figure 7: Carbon footprint of the cooling systems in the four locations (note different scales)

Figure 8 :
Figure 8: Cost of evaporative cooling with N+1 redundancy guarantee and that of dry cooling exploiting the redundancy, in a Cloud datacenter with no AI (note different scales).

Figure 9 :
Figure 9: Cost of evaporative cooling with N+1 redundancy guarantee and that of dry cooling exploiting the redundancy, in a Cloud datacenter with AI (note different scales).

Figure 10 :
Figure 10: Dynamic datacenters' performance relative to the full evaporative cooling scheme (in percentage).
Figure 11  presents the percentage of cost increase when using dry cooling, compared to the evaporative cooling.Dry cooling increases the costs by less than 2% on average, in all locations.

Figure 11 :
Figure 11: Percentage of cost increase when using dry cooling, compared to the evaporative cooling

Figure 13
Figure13shows water saving made by adopting the alternating and simultaneous operation modes, and the counterbalance in power, carbon and cost overheads.

Figure 12 :
Figure 12: Monthly precipitations and average grid characteristics in 2022

Table 1 :
Datacenter Capacity and Estimated Growth Rate

Table 2
presents the list of symbols used in this study.

Table 2 :
Symbols used in the models Outer and inner diameters of the tubes in the cooling tower dry section (m) n Number of tube rows in the cooling tower dry section n Number of tubes per row in the cooling tower dry section A Fluid (air or water)-side surface in the cooling tower dry section (m 2 ) A   Windward area of the heat exchanger in the cooling tower dry section( 2 ) Pr and Re Prandtl (no dimension) and Reynold ( 2 /) numbers respectively k Thermal conductivity ( /.)Ny and Ry Characteristic heat transfer and flow parameters respectively (m −1 )  and  Dynamic viscosity (Pa.s) and density (kg/m3) respectively NTU Number of heat transfer units f  Friction factor in the dry section tubes  Surface roughness of the dry section of cooling tower (m) Q Heat removed by the cooling system (kW) H Enthalpy (KJ/Kg) COP Coefficient of performance P Power consumption (W) EWIF Energy Water Intensity Factor (L/KWh)

Table 3
summarizes the data used to conduct this study.