How to Measure Energy Use per AI Inference and Training Run

Updated 5 November 2026 · Standard formula, shown below

This guide explains how to measure the energy consumption of a single AI inference or a full training run. Youâ€™ll learn what hardware and software tools to use, how to record power draw over time, and how to convert that into kilowatt-hours (kWh) and cost. This is useful for budgeting, comparing model efficiency, and reducing your carbon footprint.

What You’ll Need

A computer with the AI workload (GPU, CPU, or both) â€“ local or cloud instance.
Access to the command line (terminal) on that machine.
Python installed (for most tools).
Admin/root privileges to install software and read hardware sensors.
For cloud: access to the cloud providerâ€™s billing or monitoring dashboard.
Optional: a wall power meter (like a Kill A Watt) if measuring the whole system from the outlet.

Step-by-Step

Identify the hardware youâ€™re measuring. Energy use is typically dominated by the GPU(s). Use nvidia-smi (NVIDIA) or rocm-smi (AMD) to list your GPUs. For CPU-only, note the CPU model and TDP (thermal design power) as a rough upper bound.
Choose a measurement tool.
- For GPU power: NVIDIA GPUs expose power draw in watts via nvidia-smi (e.g., nvidia-smi --query-gpu=power.draw --format=csv). AMD GPUs have similar via rocm-smi --showpower.
- For whole system: Use a tool like powertop (Linux) or energy_usage (macOS) to estimate CPU and other components. For the most accurate whole-system number, use a physical wall meter.
- For cloud instances: Most providers (AWS, GCP, Azure) offer per-second or per-minute power/energy metrics in their monitoring consoles (e.g., CloudWatch, Cloud Monitoring). Look for â€œpowerâ€� or â€œenergyâ€� metrics.

Create a logging script. Youâ€™ll need to sample power draw at regular intervals (e.g., every 0.1 to 1 second) during the run. Hereâ€™s a simple Python loop that logs GPU power to a CSV file:

import subprocess, time, csv, datetime

with open('power_log.csv', 'w', newline='') as f:
    writer = csv.writer(f)
    writer.writerow(['timestamp', 'power_watts'])
    while True:
        # For NVIDIA GPUs; adjust for AMD or CPU
        result = subprocess.run(['nvidia-smi', '--query-gpu=power.draw', '--format=csv,noheader'], capture_output=True, text=True)
        watts = float(result.stdout.strip().split()[0])  # first GPU
        writer.writerow([datetime.datetime.now().isoformat(), watts])
        time.sleep(0.1)  # sample every 100 ms

Run this script in the background before starting your inference or training.

Start the logging script. In a terminal, run the script (e.g., python3 power_logger.py). Leave it running.
Run your AI workload. Execute your inference or training script exactly as you normally would. Make sure the workload runs to completion.
Stop the logger. After the workload finishes, stop the logging script (Ctrl+C). You now have a CSV with timestamps and power in watts.
Calculate total energy. Energy (in watt-hours) = average power (watts) Ã— time (hours). To get this from the log:
1. Load the CSV into a spreadsheet or Python.
2. Compute the mean of the power column.
3. Compute the total duration: (last timestamp – first timestamp) in seconds, then divide by 3600 to get hours.
4. Multiply mean power Ã— duration in hours to get watt-hours (Wh). Divide by 1000 to get kWh.
For example, if the average power was 250 W and the run took 2 hours, energy = 250 Ã— 2 = 500 Wh = 0.5 kWh.
Convert to cost (optional). Multiply the kWh by your electricity rate (e.g., $0.15 per kWh). For the above example, cost = 0.5 Ã— 0.15 = $0.075.
For cloud, use built-in metrics. If youâ€™re on a cloud instance, skip the logging script and instead note the start and end times, then query the providerâ€™s monitoring for the average power during that window. Many providers also give you the energy directly in kWh.

Common Mistakes

Measuring only the GPU, not the whole system. The GPU is the biggest consumer, but CPU, memory, and fans add up. If you need total system energy, use a wall meter or include CPU power.
Sampling too infrequently. Power draw fluctuates rapidly during training. Sampling every 5 seconds can miss spikes and underestimate energy. Use at least 1 Hz (once per second), and ideally 10 Hz for accurate averages.
Forgetting to include idle power. If the system is doing other tasks during the run, those watts get counted. Try to close other applications.
Confusing watts and watt-hours. Watts are instantaneous power; watt-hours are energy. You need to integrate over time.
Not accounting for multiple GPUs. If you use 8 GPUs, sum the power from all of them. The script above only logs the first GPU.

Tips

Use a dedicated measurement tool. For NVIDIA GPUs, nvidia-smi is built-in and reliable. For CPU, tools like turbostat (Linux) or Intel Power Gadget (Windows) give per-package power.
Run a warm-up. Let the hardware reach a steady temperature before logging to avoid including initial warm-up spikes.
Measure multiple runs. Energy can vary due to thermal throttling or other processes. Run 3 times and take the median.
For accurate whole-system, use a wall meter. Plug the machine into a power strip with a meter and record the reading before and after the run. The difference is the energy used.
For cloud, check if the provider already gives energy in kWh. Many do, which saves you the integration step.

Troubleshooting

nvidia-smi shows â€œN/Aâ€� for power.draw. This happens on some older GPUs or virtualized environments. Try using nvidia-smi -q -d POWER to see if any power info exists. If not, youâ€™ll need a wall meter or cloud metrics.
The logging script crashes with a permission error. Some sensors require root. Run the script with sudo or add your user to the appropriate group (e.g., video on Linux).
Power readings fluctuate wildly. Thatâ€™s normal. Increase the sampling rate (e.g., 0.05s) and use a longer run to get a stable average.
Cloud monitoring doesnâ€™t show power. Some providers only offer CPU utilization, not power. In that case, use a local script on the instance (if you have access) or estimate based on instance typeâ€™s typical power draw (check provider documentation).
The workload finishes before you stop the logger. Thatâ€™s fineâ€”the log will include a bit of post-run idle time. You can trim the log to the exact run duration if you know the start/end times.

Frequently Asked Questions

How do I measure energy for a single inference?

Run the inference multiple times (e.g., 100) to get a measurable duration. Log power during that batch, then divide total energy by the number of inferences to get per-inference energy.

Do I need to measure CPU power too?

For a typical GPU workload, the GPU dominates (often 70-90% of system power). If you need a precise total, include CPU and other components. For a quick estimate, GPU-only is often sufficient.

What is the difference between power and energy?

Power is the rate of energy use (watts). Energy is the total amount used (watt-hours or kWh). You need both to calculate cost.

Can I use the same method for training and inference?

Yes, the method is identical. The difference is that training runs are longer and often use more power due to higher utilization.

How do I measure energy on a cloud GPU instance?

Check your providerâ€™s monitoring dashboard for power or energy metrics. If not available, run a logging script on the instance (if you have SSH access) or estimate based on the instance typeâ€™s typical power draw.

Is it better to measure from the wall or from software?

Wall meters measure the entire system, including power supply losses, which is the most accurate for total energy. Software tools are easier and give per-component data, but may not include all losses.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.