Local AI Model RAM and VRAM Requirements Explained

Updated 5 November 2026 · Standard formula, shown below

For local AI, a practical starting point is 16 GB of system RAM and 8 GB of VRAM for smaller, quantized language models; 32 GB of RAM and 12–16 GB of VRAM gives more room for capable models and longer prompts. The exact requirement depends on model size, numerical precision, context length, and whether the model runs on a GPU, CPU, or both.

RAM and VRAM: What Each One Does

RAM is the computer’s general-purpose working memory. It holds the operating system, applications, and model data when inference runs on a CPU, and it may also support a model that does not fit entirely in graphics memory. VRAM is memory on a discrete graphics card; it holds GPU model weights and other data needed for calculations. Integrated graphics typically share system RAM rather than having a separate pool of VRAM.

These capacities are not interchangeable in every setup. A GPU with 8 GB of VRAM cannot normally use all 32 GB of system RAM as if it were its own fast memory. Some software can offload part of a model to system RAM, but moving data between memory pools can slow generation considerably. Leave memory for the operating system and other programs instead of assuming every listed gigabyte is available to the model.

How Model Size Turns Into Memory Use

A model’s parameters are its learned numerical weights. A rough first estimate for weight storage is parameter count multiplied by bytes per parameter. At 16-bit precision, each parameter takes about 2 bytes; at 8-bit, about 1 byte; and at 4-bit, about half a byte before format-specific overhead. These are estimates, not complete hardware requirements.

A 7-billion-parameter model at 16-bit precision needs about 14 GB just for weights; at 4-bit, the raw estimate is about 3.5 GB.
A 13-billion-parameter model at 4-bit has a raw weight estimate around 6.5 GB.
A 70-billion-parameter model at 4-bit has a raw estimate around 35 GB, before additional memory needs.

Quantization reduces the precision used to store weights, often making a model fit on smaller hardware. Actual files and runtime use vary because quantization formats have metadata and some layers may use different precision. Lower precision can also affect output quality, though the impact depends on the model and task.

Why the Minimum Is Not the Whole Requirement

Weight storage is only part of inference memory. The runtime needs space for temporary calculations, and text-generation models commonly use a key-value cache (KV cache) to retain information about earlier tokens in a conversation. That cache grows with context length—the amount of prompt and generated text the model can consider—and its size depends on model architecture, batch size, and cache precision.

This is why a model that appears to fit based on its file size may still fail to load, run out of memory with a long prompt, or leave too little room for other applications. Image, audio, and video models have their own memory demands; image resolution, batch size, and the particular pipeline can matter as much as model parameter count. There is no single RAM or VRAM figure that applies to every local AI task.

Practical Hardware Tiers

Use these ranges as planning guidance for local text-generation models, not guarantees. Check the chosen model’s documentation and the requirements of the application that loads it.

Basic experimentation: 8–16 GB system RAM and roughly 4–8 GB VRAM can run some small, aggressively quantized models. Expect limited model choice, shorter contexts, or slower performance.
Comfortable entry point: 16–32 GB system RAM and 8–12 GB VRAM suit many smaller quantized models and everyday testing, subject to context length and software overhead.
More flexibility: 32 GB system RAM and 16–24 GB VRAM can accommodate a broader range of quantized models, larger contexts, and some multimodal workloads, though not every model or setting.
Large models: Models in the tens of billions of parameters may require multiple GPUs, substantial unified memory, or CPU offloading. A 70-billion-parameter model at 4-bit needs roughly 35 GB for raw weights alone, so a 24 GB graphics card cannot hold all of those weights.

These are not speed rankings. Generation speed also depends on GPU architecture, memory bandwidth, CPU, software, and the model itself. A machine can technically run a model while producing tokens too slowly for comfortable use.

How to Choose for Your Workload

Start with a particular model and application rather than buying hardware around a vague promise that it “runs AI.” Find the model’s parameter count, available quantization formats, recommended context length, and any stated memory guidance. Then account for the operating system, the application, and other programs you plan to keep open.

Choose the task: chat, coding, document search, image generation, or another workload can use different models and memory patterns.
Choose model size and quantization: smaller or more heavily quantized weights generally need less memory, with possible trade-offs in capability or output quality.
Set a realistic context: longer conversations and larger documents can increase KV-cache use. Start with the intended prompt length, not just the model’s maximum advertised context.
Check the runtime’s placement options: determine whether it supports GPU execution, CPU execution, or partial offloading, and how it reports memory use.
Leave headroom: avoid planning for the model to consume every gigabyte. A small margin helps prevent loading failures when context or application use changes.

Common Misconceptions and Troubleshooting

“The model file fits, so it will run.” Not necessarily: weights are only one component, and loading can require extra temporary memory. Try a smaller quantization, shorter context, or a smaller model if loading fails.

“More system RAM makes any GPU model fit.” System RAM can help with CPU inference or supported offloading, but it is not equivalent to VRAM. Offloading can make a model usable at the cost of speed.

“The advertised context length costs nothing.” A long context can increase memory use, and the amount varies by architecture and runtime. If memory use rises during a long conversation, reduce the context or use a model and cache configuration that fits your hardware.

Also check whether the system is using the intended GPU and whether another application is consuming its memory. Reported memory can differ between tools because some show reserved capacity and others show active use.

Frequently Asked Questions

Is 8 GB of RAM enough to run an AI model locally?
It can be enough for some small, quantized models, especially with CPU inference, but it leaves little room for the operating system and applications. Many users will find 16 GB more practical.

Can I run a local model without a GPU?
Yes. CPU inference can run models using system RAM, but it is often slower than a suitable GPU. The required RAM still depends on the model, quantization, context, and runtime.

How much VRAM does a 7B model need?
There is no fixed figure. A 7-billion-parameter model’s raw weights are about 14 GB at 16-bit or 3.5 GB at 4-bit; runtime overhead and context-cache memory mean the usable VRAM requirement is higher than the raw 4-bit estimate.

Does a longer context need more VRAM?
Usually, yes for transformer language models, because the KV cache stores information for tokens in the context. The increase depends on model architecture, batch size, and cache settings.

Can system RAM and VRAM be added together?
Not as one uniform pool of fast GPU memory. Some runtimes can place parts of a model in system RAM, but transfers and CPU work can reduce performance.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.