Air vs Immersion Cooling for AI Servers
For AI servers, immersion cooling is generally better than air cooling for high-density GPU clusters, but air cooling remains the more practical and cost-effective choice for most deployments. The best option depends on your server density, power budget, and facility infrastructure.
Background: Why AI Servers Run Hot
AI training and inference workloads are dominated by GPUs and specialized accelerators (like TPUs) that draw far more power than traditional CPUs. A single high-end GPU can consume 300–700 watts under load, and a fully populated server with 8 GPUs can draw 5–10 kW or more. Air cooling, which relies on heat sinks and fans, struggles to remove that much heat from a compact chassis without excessive airflow noise and ducting. Immersion cooling submerges servers in a dielectric fluid that has much higher thermal conductivity and heat capacity than air, allowing it to absorb and transfer heat far more efficiently.
How Each Cooling Method Works
Air cooling is the standard approach: heat sinks attached to chips transfer heat to air, which is pushed through the chassis by fans. In a data center, hot air is exhausted into the room and removed by computer room air handlers (CRAHs) or in-row coolers. Direct-to-chip liquid cooling is a hybrid where cold plates sit on the chips and carry heat to a coolant loop, but the rest of the server remains air-cooled.
Immersion cooling comes in two main forms:
- Single-phase immersion: Servers are fully submerged in a tank of dielectric fluid (often a synthetic hydrocarbon or ester oil). The fluid absorbs heat and is pumped to a heat exchanger, where it transfers heat to a secondary water loop.
- Two-phase immersion: The fluid boils at a low temperature (e.g., a fluorocarbon like 3M Novec), and the vapor rises to a condenser coil at the top of the tank, where it condenses and drips back down. This phase change absorbs a large amount of latent heat.
Key Comparisons: Performance, Cost, and Practicality
Cooling capacity: Air cooling can handle typical rack densities of 10–20 kW per rack with proper design. Beyond that, you need liquid cooling. Immersion can handle 100 kW per rack or more, making it the only option for the densest AI clusters.
Energy efficiency: Immersion eliminates the need for server fans and reduces or eliminates compressor-based cooling. Data center PUE (Power Usage Effectiveness) can drop from ~1.3–1.5 (air) to ~1.05–1.1 (immersion). That translates to lower electricity bills and a smaller carbon footprint.
Capital cost: Air cooling is cheap to deploy—you already have the infrastructure. Immersion requires specialized tanks, dielectric fluid (which is expensive, often $10–$20 per liter), and significant facility modifications. The fluid alone for a large tank can cost hundreds of thousands of dollars.
Operating cost: Immersion saves on energy but adds maintenance complexity. The fluid must be monitored for contamination, and pumps and heat exchangers need upkeep. Air cooling is simpler and cheaper to maintain.
Reliability and serviceability: With air, you can swap a failed component in minutes. With immersion, you must lift the server out of the tank, drain the fluid, and dry it before servicing—a much slower process. Some operators keep spare servers ready to swap in while the failed one dries.
Common Problems and Misconceptions
Misconception: Immersion cooling is maintenance-free. While the fluid does not evaporate (in single-phase), it can absorb moisture, particulates, and dissolved metals over time, which can affect its dielectric properties. Filtration and periodic testing are required.
Misconception: Air cooling is obsolete. For most enterprise AI workloads—especially inference with a few GPUs per server—air is still the standard. The industry is moving toward liquid cooling for new high-density deployments, but the installed base of air-cooled servers will remain for years.
Problem: Fluid compatibility. Not all server components are compatible with immersion fluids. Some plastics, adhesives, and rubber seals can degrade. Server manufacturers now offer immersion-ready models, but retrofitting older hardware is risky.
Problem: Weight and structural load. A tank filled with dielectric fluid is extremely heavy. A typical 10-server immersion tank can weigh over 1,500 kg, requiring reinforced flooring.
How to Choose: Factors to Consider
When deciding between air and immersion cooling for your AI servers, evaluate these factors:
- Server density: If you plan to exceed ~20 kW per rack, air cooling will not suffice. Immersion (or direct-to-chip liquid cooling) becomes necessary.
- Power and space constraints: Immersion allows you to pack more compute into a smaller footprint, which is valuable if floor space is limited or expensive.
- Budget: Calculate total cost of ownership over 3–5 years, including fluid replacement, maintenance, and energy savings. Immersion has higher upfront costs but lower operating costs at scale.
- Service model: If your team needs rapid hardware swaps, air is easier. If you can afford spare servers and longer downtime for maintenance, immersion is viable.
- Facility infrastructure: Immersion requires a source of cool water (for the heat exchanger) and adequate floor load. Air cooling requires sufficient airflow and hot/cold aisle containment.
Cost Breakdown and ROI
Air cooling costs are mostly in the facility (CRAHs, ducting, containment) and the server fans. Immersion adds:
- Tank hardware: $50,000–$150,000 per tank depending on size and features.
- Dielectric fluid: $10–$20 per liter; a large tank holds 1,000–2,000 liters.
- Facility modifications: reinforced flooring, water loops, pumps, and controls.
- Ongoing maintenance: fluid filtration, testing, and replacement (typically every 5–10 years).
Energy savings can be substantial: a 1 MW IT load with PUE 1.4 (air) uses 1.4 MW total; with PUE 1.1 (immersion), it uses 1.1 MW. Over a year, that saves about 2.6 million kWh, which at $0.10/kWh is $260,000—enough to pay for the immersion system in a few years.
Safety and Environmental Considerations
Dielectric fluids used in immersion cooling are generally non-flammable (especially fluorocarbons) or have high flash points (hydrocarbons). However, they can be hazardous if inhaled or if they contact skin. Proper PPE and ventilation are required during maintenance. Two-phase fluids like Novec have high global warming potential (GWP) and are being phased down in some regions. Single-phase fluids (e.g., synthetic esters) are more environmentally benign but still need careful disposal. Air cooling has no such chemical risks but can create hot spots and requires significant electricity for fans and compressors.
Alternatives to Full Immersion
If immersion seems too drastic, consider these middle-ground options:
- Direct-to-chip liquid cooling: Cold plates on the CPU/GPU carry heat to a coolant loop. This handles the highest heat loads without submerging the whole server. It is easier to retrofit than immersion and is becoming the industry standard for new AI data centers.
- Rear-door heat exchangers: These attach to the back of a rack and use water to cool the exhaust air, allowing higher rack densities without full liquid cooling inside the server.
- Hybrid cooling: Use air for lower-density racks and liquid for high-density GPU racks, optimizing cost and performance.
Frequently Asked Questions
Is immersion cooling safe for hard drives and other components? Most immersion fluids are non-conductive, so they are safe for electronics. However, spinning hard drives are not recommended because the fluid adds drag and can cause mechanical stress. Solid-state drives (SSDs) are fine. Always check component compatibility with the specific fluid.
Can I retrofit my existing air-cooled servers for immersion? It is not recommended. Server components may have plastic or adhesive materials that degrade in the fluid, and the warranty is usually void. Immersion-ready servers are designed with sealed connectors and compatible materials. Retrofitting is possible for some models but risky and often not cost-effective.
How long does it take to service a server in an immersion tank? The process involves lifting the server out, letting it drain, and then drying it. Depending on the fluid and tank design, this can take 15–30 minutes for the physical removal, plus several hours of drying before the server can be opened. Many operators keep a spare server to swap in immediately, then service the wet one later.
Does immersion cooling reduce performance or overclocking headroom? No—in fact, it often improves performance because lower temperatures allow GPUs to maintain boost clocks longer. Some operators have successfully overclocked submerged GPUs beyond what air cooling would allow, though this increases power draw and heat output.
What is the typical PUE for immersion-cooled data centers? Reported PUE values for immersion installations are often between 1.02 and 1.10, compared to 1.3–1.5 for traditional air-cooled facilities. This is because immersion eliminates most mechanical cooling and reduces fan power. Actual values depend on climate and facility design.
Is immersion cooling worth it for a small business with a few AI servers? Probably not. The high upfront cost of tanks and fluid is only justified when you have enough compute density to benefit from the energy savings and space efficiency. For a single rack or less, air cooling or direct-to-chip liquid cooling is more cost-effective.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.