Coolant Flow Rates & Temps for Liquid-Cooled AI Servers
This guide walks you through selecting coolant flow rates and temperatures for liquid-cooled AI servers, covering the key calculations, component limits, and practical tuning steps. You’ll learn how to balance pump power, cooling capacity, and thermal margins to keep GPUs and CPUs within spec while maximizing energy efficiency.
What You’ll Need
- Server hardware specs: GPU/CPU thermal design power (TDP) in watts, maximum allowable junction temperature (Tj,max), and coolant inlet temperature range from the manufacturer.
- Cooling loop components: pump curve (flow vs. pressure), radiator or CDU (coolant distribution unit) capacity in kW, and tubing/quick-connect pressure drop ratings.
- A way to measure flow: flow meter (e.g., paddle wheel, ultrasonic) or pump speed/RPM with a known curve.
- Temperature sensors: inlet and outlet coolant temperature probes, plus server inlet air temp if using hybrid cooling.
- Optional: thermal simulation software or a spreadsheet for heat balance calculations.
Step-by-Step
- Determine total heat load (Q_total). Sum the TDP of all components in the loop. For example, if you have 8 GPUs at 350W each and 2 CPUs at 280W, Q_total = 8*350 + 2*280 = 3360W. Add 10-20% for power supplies and other losses, giving ~3700-4000W.
- Pick a target coolant temperature rise (ΔT). For AI servers, a common ΔT across the cold plate is 5-10°C (9-18°F). Smaller ΔT means higher flow but better thermal uniformity. Start with ΔT = 5°C for high-density GPU clusters.
- Calculate required flow rate. Use the heat balance formula: Flow (L/min) = Q (kW) / (ΔT (°C) * 4.18 kJ/(kg·K) * density (kg/L)). For water, density ≈ 1 kg/L. Example: Q=4 kW, ΔT=5°C → Flow = 4 / (5 * 4.18) = 0.191 L/s = 11.5 L/min. For 25% propylene glycol, adjust specific heat to ~3.8 and density to ~1.02, giving ~12.6 L/min.
- Check flow rate per device. Divide total flow by number of parallel branches. If you have 8 GPUs in parallel, each gets ~1.4 L/min. Most cold plates require 0.5-2 L/min per device; verify your GPU’s spec. If below, increase total flow or reduce parallel branches.
- Set coolant inlet temperature. Typical range is 20-35°C (68-95°F) for AI servers. Lower inlet temps improve thermal margin but increase condensation risk and chiller energy. Start at 25°C (77°F) and adjust based on Tj margin.
- Verify junction temperature margin. Measure GPU/CPU Tj under full load. If Tj is below Tj,max by more than 15°C, raise inlet temp or reduce flow to save energy. If within 5°C of Tj,max, lower inlet temp or increase flow.
- Adjust pump speed. Use the pump curve to find the flow rate at your system’s pressure drop. If flow is too low, increase pump speed or reduce restrictions (e.g., larger tubing, fewer bends). If flow is too high, reduce speed to save power—your ΔT will rise.
- Monitor and fine-tune. Log inlet/outlet temps, flow, and Tj over a 24-hour period under varying load. Adjust inlet temp setpoint and pump speed to keep Tj within 10-15°C of Tj,max for best efficiency without risking thermal throttling.
Common Mistakes
- Using too high a ΔT. A 15°C rise may reduce flow requirements but leads to hot spots on cold plates and uneven cooling. Stick to 5-10°C.
- Ignoring pressure drop. High flow through small tubing can cause cavitation or pump overload. Always check the pump curve against system resistance.
- Forgetting coolant properties. Glycol mixtures have lower specific heat and higher viscosity, requiring higher flow rates for the same heat removal.
- Setting inlet temp too low. Below the dew point causes condensation on cold plates, risking short circuits. Use dew point monitoring.
- Not accounting for altitude. At high altitudes, coolant boiling point drops, so you may need higher pressure or lower temperatures.
Tips
- Use the rule of thumb: 1 L/min per kW of heat load for a 15°C ΔT, or 2 L/min per kW for a 7.5°C ΔT. Adjust for coolant type.
- For AI servers with variable loads, use variable-speed pumps that respond to temperature sensors—this can cut pump energy by 30-50%.
- Consider using a CDU that provides a secondary loop. This isolates the server loop from facility water, allowing higher inlet temps in the facility loop (e.g., 32°C) while maintaining a cooler server loop.
- Always install a filter (50-100 micron) to protect cold plates from debris.
- Test with a dummy load or stress test (e.g., FurMark, Prime95) to simulate worst-case heat.
Troubleshooting
- Flow rate is lower than calculated. Check for air locks, clogged filters, or closed valves. Bleed air from the loop. Measure pressure drop across each component to find the restriction.
- Junction temps are too high despite adequate flow. Check cold plate mounting torque and thermal paste application. Ensure the cold plate is making full contact with the chip.
- Condensation on cold plates. Raise inlet temp above dew point, or insulate cold plates and tubing with closed-cell foam.
- Pump is noisy or cavitating. Increase suction pressure (e.g., raise reservoir height) or reduce pump speed. Check for dissolved air in coolant.
- Coolant temperature rises over time. The radiator or CDU may be undersized. Clean radiator fins, check fan operation, or increase facility water flow.
Frequently Asked Questions
What is the ideal coolant flow rate for a single GPU?
Most GPU cold plates work well with 0.5-2 L/min. For a 350W GPU, 1 L/min with a 5°C ΔT is a safe starting point. Check your specific model’s spec—some require higher flow for optimal performance.
What coolant temperature should I use?
For AI servers, 25-35°C inlet is typical. Lower temps (20°C) improve thermal margin but increase energy and condensation risk. Higher temps (40°C+) reduce chiller energy but may require higher flow rates to keep Tj in check.
Can I use regular water?
Yes, but only if you add corrosion inhibitors and biocides. Use deionized water with a corrosion inhibitor specifically designed for cooling loops. Plain tap water will cause corrosion and mineral deposits.
How do I calculate the required flow rate?
Use the formula: Flow (L/min) = Q (kW) / (ΔT (°C) * 4.18 * specific_gravity). For water, specific gravity is 1. For 25% glycol, it’s about 1.02 and specific heat is about 3.8 kJ/kg·K.
What is the difference between direct-to-chip and immersion cooling?
Direct-to-chip uses cold plates attached to the chips, with coolant flowing through them. Immersion cooling submerges the entire server in a dielectric fluid. Direct-to-chip is more common for AI servers because it allows higher power densities and easier maintenance.
How do I know if my pump is big enough?
Calculate the total pressure drop of your loop (tubing, fittings, cold plates, radiator). Then check the pump curve: it should provide your required flow rate at that pressure drop. If the pump operates near its maximum head, it may be undersized.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.