Troubleshoot Slow Cloud GPU Instances

Updated 7 November 2026 · Standard formula, shown below

This guide helps you diagnose and fix slow or unstable performance on cloud GPU instances. You’ll learn systematic checks for network, storage, driver, and configuration issues, plus practical steps to restore expected speed and reliability.

What You’ll Need

Access to your cloud provider’s console (AWS, GCP, Azure, etc.)
SSH access to the instance
Basic command-line familiarity
Your workload details (model size, dataset location, expected performance)

Step-by-Step

Check GPU utilization and temperature. Run nvidia-smi to see GPU usage, memory, and temperature. If utilization is near 100% and temperature is high (above 80°C), the GPU may be throttling. Compare with your expected workload.
Verify the correct GPU is being used. In PyTorch, run torch.cuda.is_available() and torch.cuda.get_device_name(). For TensorFlow, check tf.config.list_physical_devices('GPU'). Ensure your code isn’t accidentally using CPU.
Monitor CPU and RAM usage. Use htop or top. If CPU is pegged at 100% while GPU is idle, you have a data-loading bottleneck. Check for memory swapping (high si/so in vmstat).
Test network bandwidth. Run iperf3 between your instance and a test server, or use curl to download a large file. Compare with your provider’s expected network performance. If slow, consider a different instance type or region.
Check disk I/O. Run iostat -x 1 or fio to measure read/write speeds. If your dataset is on a standard HDD, move it to SSD or use a managed filesystem like EFS or GCS FUSE.
Inspect driver and CUDA versions. Run nvidia-smi to see driver version and CUDA version. Compare with your framework’s requirements. Mismatched versions can cause instability and poor performance.
Review instance type and quotas. Check if you’re using a burstable instance (like t-series) that throttles CPU after credits run out. Also ensure you haven’t hit your vCPU or GPU quota limits.
Look for noisy neighbors. On shared hardware, other tenants can affect performance. If you notice inconsistent performance, try a dedicated instance or a different availability zone.
Check for thermal throttling. If temperature is consistently high, consider reducing workload intensity, improving cooling (if physical), or selecting a different instance type with better cooling.
Restart the instance. A simple reboot can clear transient issues like memory leaks or stuck processes. Before rebooting, save your work and terminate any running jobs.

Common Mistakes

Ignoring CPU bottlenecks: Many users focus on GPU utilization but overlook data loading. Use --num_workers in PyTorch or tf.data pipelines to parallelize data loading.
Using the wrong instance type: Selecting a GPU instance with insufficient CPU or network bandwidth for your workload.
Not updating drivers: Outdated drivers can cause crashes and suboptimal performance. Always use the latest stable driver from your cloud provider’s marketplace.
Overlooking storage performance: Using standard storage for large datasets that require high IOPS. Provision SSD or NVMe storage.
Misconfiguring CUDA: Setting CUDA_VISIBLE_DEVICES incorrectly can cause the GPU to be underutilized or cause errors.

Tips

Use monitoring tools: Set up CloudWatch, Stackdriver, or Azure Monitor to track GPU metrics over time and identify patterns.
Profile your workload: Use nvidia-smi in a loop (watch -n 1 nvidia-smi) to see real-time usage. For deeper analysis, use profiling tools like Nsight Systems or TensorBoard.
Optimize your code: Use mixed precision training (e.g., AMP) to speed up training and reduce memory usage.
Use spot/preemptible instances wisely: They are cheaper but can be terminated. For unstable performance, avoid spot instances if your workload is time-sensitive.
Consider distributed training: If you need more power, scale out to multiple GPUs using Horovod or PyTorch DDP, but ensure your network bandwidth supports it.

Troubleshooting

If the steps above don’t resolve the issue, try these additional checks:

Check system logs: Run dmesg or journalctl -xe to look for hardware errors, OOM kills, or driver crashes.
Test with a simple workload: Run a minimal GPU benchmark (e.g., matrix multiplication) to isolate whether the issue is with your code or the instance.
Compare with a fresh instance: Launch a new instance with the same configuration and run your workload. If the new instance performs better, the original may have degraded hardware.
Contact support: If you’ve exhausted all options, open a support ticket with your cloud provider, providing the metrics you’ve collected.

Frequently Asked Questions

Why is my GPU utilization low even though I’m running a heavy model?

This often indicates a data-loading bottleneck. Check CPU usage and disk I/O. Increase the number of data loader workers and ensure your dataset is on fast storage.

Can I fix performance issues by upgrading to a more expensive instance?

Sometimes, but not always. If the bottleneck is network or storage, a more powerful GPU won’t help. Identify the bottleneck first using the steps above.

How do I know if my instance is being throttled?

Look for a drop in performance over time, especially after sustained high usage. Use nvidia-smi to check temperature and power limits. If the clock speed is lower than the base clock, throttling is likely.

What should I do if my instance becomes unresponsive?

Try to SSH in and check system load. If you can’t, use the cloud console to reboot the instance. After reboot, check logs for errors that may have caused the instability.

Is it normal for GPU performance to vary between sessions?

Some variance is expected, especially on shared hardware. However, large and consistent drops indicate a problem. Monitor over time to see if it’s correlated with certain times of day or other factors.

Do I need to update CUDA versions regularly?

Only if your framework or libraries require it. Check compatibility before updating. Sometimes newer CUDA versions can improve performance, but they can also introduce instability if not matched with the right driver.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.