How Much RAM and VRAM Do Local LLMs Need?

Updated 9 November 2026 · Standard formula, shown below

For local language models, 16 GB of system RAM is a practical starting point, while 32 GB gives more room for larger models and multitasking. For a graphics card, plan on roughly 8 GB of VRAM for small quantized models and 12–16 GB or more for many 7B–14B models; the exact requirement depends on model size, quantization, context length, and software overhead.

What RAM and VRAM Do

RAM is the computer’s general-purpose working memory. It holds model data when a model runs on the processor, as well as the operating system, applications, and other active tasks. VRAM is memory on a graphics card, used by the GPU for model weights and the computations needed to generate text.

A model’s parameter count—such as 7B, meaning about seven billion parameters—does not directly state its memory requirement. Parameters are stored using numerical formats of different sizes. A model using 16-bit weights takes about two bytes per parameter, so 7 billion parameters require roughly 14 GB just for weights. That is a lower-bound estimate, not a complete system requirement.

Quantization stores weights with fewer bits, reducing their memory footprint, often with some quality trade-off. A 4-bit version of a 7B model may use around 4 GB for weights, but real use needs additional memory for metadata, temporary computation, and the context cache. File size can provide a useful clue, but it does not equal the full amount of RAM or VRAM needed while running.

Approximate Memory by Model Size

The following are broad planning ranges for quantized models, commonly around 4-bit, with enough extra memory beyond the weights for a usable setup. They are not guarantees: quantization format, context length, inference software, and operating-system use all affect the result.

3B–4B models: often workable with 8 GB of system RAM; 4–6 GB of VRAM can be sufficient for a GPU-focused setup.
7B–8B models: 16 GB of system RAM is a sensible baseline; allow roughly 6–8 GB of VRAM for a fully GPU-resident, quantized model.
13B–14B models: 24–32 GB of system RAM is more comfortable; around 10–16 GB of VRAM may be needed, depending on settings.
30B–34B models: expect substantially more memory: often 24 GB or more of VRAM for a quantized GPU setup, or 48 GB or more of system RAM when running mainly on a CPU.
70B models: a quantized version can still require around 40 GB or more for weights alone. A practical setup generally needs multiple GPUs with sufficient combined usable memory, or a large amount of system RAM and patience.

These ranges describe approximate capacity, not speed. A model that fits in memory may generate slowly, especially on a CPU or when some layers must be moved between system RAM and VRAM.

Why Context Length Changes the Answer

Memory use does not stop at model weights. During generation, the system also stores a key-value cache (KV cache), which preserves information about the conversation so the model does not need to recompute the entire context for every new token. The cache grows with context length and is larger for models with more layers or wider architectures. Its size also depends on the cache’s numerical format and inference implementation.

This is why a model that runs comfortably with a short prompt may run out of memory with a long document or conversation. If a tool lets you set a context limit, increasing it can increase memory use even when the model itself is unchanged. Other applications using the GPU also reduce the VRAM available to the model.

Choosing Between System RAM and VRAM

If you want to run a model on a graphics card, check that the model weights, cache, and software overhead fit within available VRAM. Leave a margin rather than planning to use every advertised gigabyte: the display, driver, and other applications may reserve some memory. A card with 8 GB of VRAM is therefore not a reliable home for every model whose downloaded file is under 8 GB.

CPU inference uses system RAM instead. It can run models too large for a graphics card, but generation is often slower. Some software can split a model between VRAM and RAM; this can make a model fit, but transferring data between memory types may reduce speed. Unified-memory computers share a pool between CPU and GPU tasks, so the total memory is not all freely available to the model.

For occasional small-model use: 16 GB system RAM and a GPU with 8 GB VRAM can be a reasonable entry-level combination.
For more flexibility: 32 GB system RAM and 12–16 GB VRAM provide room for larger quantized models and longer contexts.
For larger models: prioritize available memory capacity first, then consider GPU speed, memory bandwidth, and the software’s support for your hardware.

How to Estimate Your Requirements

Use the specific model’s published file size and format as a starting point, then account for its runtime needs. A useful checklist is:

Identify the exact model variant and quantization. A 4-bit file and a 16-bit version of the same model can have very different memory requirements.
Decide whether it will run on the CPU, GPU, or a split of both. Compare the relevant memory pool with the model’s expected runtime use.
Choose a realistic context length. Long prompts and large context settings increase cache memory.
Reserve memory for the operating system, inference software, and other applications. Do not treat total installed memory as entirely available.
Check actual usage after loading the model. If it fails or slows sharply, lower the context limit, close other GPU applications, or choose a smaller or more heavily quantized model.

Common Misconceptions and Limits

“A 7B model needs exactly 7 GB.” No: 7B counts parameters, not bytes. The storage format determines weight size, and runtime overhead adds more. “The model file fits, so the model will run.” Not necessarily; loading, cache allocation, and computation need extra room.

“More RAM makes generation faster.” Extra capacity can prevent out-of-memory errors, but speed depends strongly on the processor or GPU, memory bandwidth, software, and how much of the model fits in fast memory. Quantization also is not a universal quality or speed guarantee: results depend on the model and quantization method.

Frequently Asked Questions

Is 16 GB of RAM enough to run a local LLM?
Yes, for many small or quantized 7B–8B models, especially with a short context. It may not be enough for larger models, long contexts, or heavy multitasking.

Can I run an LLM without a GPU?
Yes. Many inference tools can use a CPU and system RAM. Expect lower generation speed than a suitable GPU in many cases, and choose a model that fits comfortably in RAM.

How much VRAM does a 7B model need?
A quantized 7B model may fit in about 6–8 GB of VRAM in some setups, but context cache and overhead can push the requirement higher. A 16-bit version needs much more.

Can system RAM make up for insufficient VRAM?
Some software can place part of a model in system RAM, but this is not equivalent to having enough VRAM. It can work at reduced speed, and performance depends on the hardware and software.

Does a longer context require more memory?
Usually, yes. A longer context generally increases KV-cache use, so a model that fits with short prompts may exceed available memory at a much larger context setting.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.