Air-Gapped AI Deployment: GPUs, Storage, Network, Power
An air-gapped AI deployment needs enough GPU memory and compute for its models and workloads, fast and durable storage for models and data, an isolated network sized for traffic between systems, and power and cooling sized for the full equipment load. The right configuration depends on what the AI must do, how many people use it, and whether it must train models or only run inference.
What “air-gapped” means for AI
An air gap means the deployment has no direct network connection to the public internet or another less-trusted network. It does not mean the equipment has no network: GPUs, storage, servers, and approved user terminals still need to communicate inside the isolated environment. Some organizations use a strict physical separation; others enforce isolation with controlled gateways and procedures. The latter may be described as disconnected or restricted rather than fully air-gapped.
Isolation changes operations as much as hardware selection. Model files, software updates, security patches, documentation, and sometimes license information must reach the environment through a controlled process. Plan how to inspect and transfer those materials before deployment. An air gap by itself does not make a system secure: compromised removable media, excessive access, unpatched software, and exposed internal services remain risks.
Choose GPUs for the workload
Start with the model and task, not a GPU count. Inference runs a trained model to produce answers; training changes model parameters and typically requires substantially more compute, memory, and time. Fine-tuning sits between them, but its resource requirements vary greatly with the method, model size, sequence length, and batch size.
GPU memory is often the first practical constraint. The model’s weights must fit in memory, along with runtime overhead, input and output context, and any temporary values needed during computation. Quantization can reduce memory needs, but may affect quality or supported operations. Splitting a model across GPUs can make larger models usable, at the cost of more complex configuration and communication between devices.
- Small inference service: A single GPU may be sufficient for a compact model and limited concurrency, provided its memory fits the model and working context.
- Higher throughput: More GPU capacity can serve additional requests in parallel, but performance also depends on batching, software, CPU, and memory bandwidth.
- Training or fine-tuning: Estimate the specific model, training method, data volume, and target completion time. Do not assume that a configuration suitable for inference can train the same model.
Check accelerator compatibility with the server, drivers, libraries, and inference or training software. Confirm physical dimensions, slot spacing, supported interconnects, and whether the chosen configuration can be serviced locally. Benchmark a representative workload: advertised peak compute alone does not predict real application performance.
Plan storage for data and operations
Storage must hold more than the final model. Include source and prepared datasets, model versions, indexes or other derived artifacts, logs, temporary files, operating-system images, backups, and update packages. A prototype may fit on one server’s local drives; a multi-user deployment may need shared storage so that several compute nodes can access the same approved data.
Capacity and speed are separate questions. Large sequential reads matter when loading datasets or model files; low-latency access and high throughput can matter for active workloads and shared services. Choose storage based on measured access patterns, not a generic claim that one drive type is always best. Decide which data needs fast access and which can reside in slower archival or backup storage.
- Keep a verified, recoverable copy of models, configuration, and important data.
- Use checksums or equivalent integrity checks when transferring files into the isolated environment.
- Set retention rules for datasets, logs, checkpoints, and temporary outputs so capacity does not quietly run out.
- Test restoring backups; a backup that has never been restored is not a demonstrated recovery plan.
Build an isolated network that still works
Air-gapped systems need internal connectivity. Network design should cover user access, GPU-to-GPU communication where applicable, storage traffic, management, monitoring, and backup. These flows may have different performance and access requirements. A small inference installation might use ordinary server networking, while distributed training or heavily shared storage can require faster links and more careful topology.
Separate services by function where practical, restrict who can reach management interfaces, and document permitted paths. Provide internal name resolution, time synchronization, authentication, logging, and software or package repositories as needed; services that are normally obtained online must have an offline substitute or a documented manual process. Keep a clear inventory of network ports, addresses, dependencies, and approved data-transfer points.
Measure traffic using the intended workload. A fast GPU can wait on a congested storage path or network, and adding accelerators without checking communication overhead may yield little improvement. Test concurrent users, large file transfers, and recovery operations—not just a single benchmark run.
Size power, cooling, and the room
Power planning must account for the complete installed system: accelerators, CPUs, memory, storage, fans, switches, and other equipment. Use the manufacturers’ specifications and a qualified facilities or electrical professional to determine circuit, rack, connector, and distribution needs. Do not estimate capacity from GPU count alone. Consider peak demand, startup behavior, redundancy, and what happens if a power supply or circuit fails.
Nearly all electrical energy consumed by equipment becomes heat in the room. Cooling must therefore handle the actual equipment load and maintain the environmental conditions specified for the hardware. Dense GPU servers can concentrate substantial heat in a small rack footprint; verify airflow direction, rack layout, room cooling capacity, and monitoring before installation. A UPS may provide ride-through or an orderly shutdown, but it does not replace adequate circuits, cooling, or a tested backup-power plan.
Common planning failures and trade-offs
A frequent mistake is buying GPUs first and discovering that memory, power, cooling, or physical space is insufficient. Another is sizing only for a model’s weights while overlooking context length, concurrency, storage, and software overhead. Avoid both by creating a workload profile and validating a representative end-to-end system before scaling out.
Air-gapped environments also have ongoing support costs in staff time and operational complexity, even when no external service fees apply. Updates and vulnerability fixes require a controlled intake, review, transfer, and installation process. Decide who owns that process, how urgently critical fixes can be applied, and how the deployment will be recovered after hardware failure.
For a small, fixed workload, one server with local storage may be simpler than a distributed cluster. A cluster can add capacity and resilience, but brings more networking, orchestration, and maintenance requirements. The best choice is the smallest design that meets measured performance, availability, and recovery objectives—not the largest hardware budget.
Frequently Asked Questions
Can an air-gapped AI system use a network?
Yes. It can use an isolated internal network for users, servers, storage, and management. The defining constraint is the absence of an uncontrolled connection to external networks, not the absence of all connectivity.
How many GPUs does an air-gapped AI deployment need?
There is no universal number. It depends on model size, GPU memory, inference concurrency or training goals, and required response time. Test the intended model and workload on candidate hardware.
Does air-gapped AI need internet access for updates?
No direct internet access is required, but software and security updates still need a controlled route into the environment. Establish a process for reviewing, verifying, transferring, and installing them.
Is one server enough?
It can be for a small inference service or a limited proof of concept. Shared users, larger models, training, availability requirements, or separated storage and management needs may call for multiple systems.
What is the biggest overlooked requirement?
Power and cooling are common surprises, especially with dense accelerators. Storage capacity, offline maintenance, and recovery testing are also easy to underestimate.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.