AI Data Center Network Capacity Planning

Updated 5 November 2026 · Standard formula, shown below

This guide walks through planning network capacity for AI and high-density data centers, covering fabric design, oversubscription, cabling, and power/cooling constraints. You’ll learn how to size leaf-spine architectures, choose speeds, and validate your plan before deployment.

What You’ll Need

Expected AI workload characteristics (training vs. inference, GPU count, model size)
Server and GPU specifications (NIC speeds, PCIe generations, GPU-to-GPU traffic patterns)
Existing data center floor plan, rack layout, and power/cooling capacity
Bandwidth requirements per application (e.g., storage replication, east-west traffic)
Network equipment specs (switch port densities, supported speeds, buffer sizes)
Budget and vendor lead times
Tools: spreadsheet or network simulation software (optional)

Step-by-Step

Define the AI workload profile. Determine if the data center will run training, inference, or both. Training typically requires low-latency, high-bandwidth east-west traffic (GPU-to-GPU), while inference may have more north-south traffic. Note the number of GPUs, model size, and whether you’re using distributed training (e.g., data parallel, model parallel).
Calculate per-server bandwidth needs. For each AI server, identify the NIC speed (e.g., 100GbE, 200GbE, 400GbE) and the number of NICs per server. Multiply NIC speed by the number of NICs to get total server bandwidth. For example, a server with 8x 100GbE NICs needs 800 Gbps.
Design the leaf-spine fabric. Choose a two-tier leaf-spine topology for most AI clusters. Each leaf switch connects to a set of servers (typically 32-48 ports for 100GbE). Each spine switch connects to every leaf switch. The number of spine switches determines the oversubscription ratio.
Determine oversubscription ratio. For AI training, aim for a low oversubscription ratio (1:1 or 2:1) to avoid congestion. Calculate the total leaf uplink bandwidth (leaf ports * spine count * port speed) and compare to the total downlink bandwidth (server ports * port speed). For example, a leaf with 48x 100GbE downlinks (4.8 Tbps) and 8x 400GbE uplinks (3.2 Tbps) gives a ratio of 1.5:1.
Choose switch speeds and port densities. Select leaf switches with enough 100GbE or 400GbE ports for servers and uplinks. Spine switches should have high-density 400GbE ports (e.g., 64 ports) to connect to many leaves. Ensure the switch ASIC buffer size is adequate for AI burst traffic (at least 16-32 MB per port).
Plan cabling and optics. For short distances (within a rack), use DAC cables. For inter-rack or inter-row, use active optical cables (AOC) or transceivers with fiber. For distances over 100m, use single-mode fiber with 400G-DR4 or 400G-FR4 optics. Consider breakout cables (e.g., 1x400G to 4x100G) for flexibility.
Account for power and cooling. High-density AI racks can consume 20-40 kW or more. Verify your data center’s power per rack and cooling capacity. If insufficient, plan for liquid cooling or reduce density. Network switches also add heat; ensure proper airflow.
Simulate or prototype the fabric. Use network simulation tools (e.g., ns-3, OMNeT++) or build a small-scale testbed with representative traffic to validate the design. Measure latency, throughput, and packet loss under load.
Document the design and create a bill of materials. List all switches, optics, cables, and patch panels. Include part numbers and quantities. Plan for spares (e.g., 10% extra optics).
Plan for scalability. Design the fabric so you can add more spine or leaf switches without disrupting existing traffic. For example, use a Clos topology that allows adding spines one at a time. Ensure the leaf switches have spare uplink ports.

Common Mistakes

Ignoring GPU-to-GPU traffic patterns. AI training often requires all-to-all communication; if the network is oversubscribed, training performance degrades significantly.
Underestimating buffer needs. AI traffic is bursty; switches with insufficient buffers drop packets, causing TCP timeouts and reduced throughput.
Forgetting about storage traffic. Checkpointing and reading training data can consume significant bandwidth. Ensure the network handles both compute and storage traffic.
Not accounting for cable lengths. Using DAC beyond its specified length (e.g., 5m) causes errors. Measure actual distances and choose appropriate media.
Oversubscribing the spine. A 3:1 or higher ratio may be fine for general cloud, but AI training needs near 1:1.

Tips

Use a leaf-spine design with at least 4 spine switches to provide redundancy and load balancing.
Consider using RDMA over Converged Ethernet (RoCEv2) for low-latency GPU communication; ensure lossless fabric with PFC and ECN.
Plan for future speed upgrades by using optics that can be reused (e.g., 400G optics can break out to 4x100G).
Monitor network utilization during early deployment to validate assumptions and adjust capacity.
Coordinate with server and storage teams to ensure NIC settings (MTU, flow control) are aligned.

Troubleshooting

High latency or packet loss: Check for oversubscription, buffer exhaustion, or faulty optics. Use `ethtool -S` on servers to see drop counters. Verify PFC/ECN settings.
Training throughput lower than expected: Run a network benchmark (e.g., iperf) between two servers. If throughput is lower than NIC speed, check for duplex mismatches, cable issues, or switch congestion.
Spine switch CPU spikes: This may indicate a broadcast storm or routing loop. Check for misconfigured VLANs or STP issues.
Cabling errors: Use optical time-domain reflectometer (OTDR) or built-in diagnostics to verify fiber quality and connector cleanliness.
Power/cooling overload: If racks trip breakers or overheat, redistribute servers or add cooling. Consider using higher-efficiency switches.

Frequently Asked Questions

What oversubscription ratio is best for AI training?

For large-scale training, aim for 1:1 (no oversubscription) or at most 2:1. Anything higher can cause significant performance degradation due to congestion.

Should I use 100GbE or 400GbE?

It depends on GPU density. If you have 8 GPUs per server with 100GbE each, 400GbE uplinks are needed to avoid oversubscription. For smaller clusters, 100GbE may suffice.

How many spine switches do I need?

At least 4 for redundancy. For a 1:1 oversubscription, the number of spines equals the total leaf downlink bandwidth divided by the spine port speed. For example, 48x100GbE downlinks (4.8 Tbps) requires 12 spines if each spine has 100GbE ports, or 3 spines if they have 400GbE ports.

What is the difference between DAC and AOC?

DAC (direct attach copper) is cheaper and lower power but limited to ~5m. AOC (active optical cable) can run up to 100m and is more flexible, but costs more. For rack-to-rack, AOC is common.

Can I use standard Ethernet switches for AI?

Yes, but they must support RoCEv2 (lossless Ethernet) with PFC and ECN. Additionally, they need large buffers and low latency. Some vendors offer AI-specific switches, but standard high-end switches often work.

How do I plan for future expansion?

Design the fabric with spare spine ports and leaf uplink ports. Use a modular approach where you can add spines without changing leaves. Also, choose optics that can be reused for higher speeds.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.