How Much RAM Does a Local LLM Need?

Updated 7 November 2026 · Standard formula, shown below

This guide answers the most common question about running large language models on your own hardware: how much RAM (and VRAM) you actually need. The short answer: for a 7B-parameter model quantized to 4-bit, you need about 6–8 GB of free VRAM (or system RAM if running on CPU), while a 13B model needs roughly 10–12 GB, and a 70B model needs 40–48 GB. The exact amount depends on the model size, the quantization level, and the context length you plan to use.

Why RAM Matters for Local LLMs

When you run a local LLM, the model’s weights (the numbers that define its behavior) must be loaded into memory before inference can start. The memory can be either VRAM (on your graphics card) or system RAM (if you run on CPU). The model’s size is measured in parameters—billions of weights—and each weight occupies a certain number of bytes depending on the precision. A 4-bit quantized 7B model, for example, uses about 3.5 GB for the weights alone, plus overhead for the KV cache, activations, and the inference engine. If you don’t have enough free memory, the model will either fail to load or run extremely slowly due to swapping.

Quantization: The Key to Fitting Big Models

Quantization reduces the precision of the weights, trading a small drop in quality for a large reduction in memory. The most common formats are:

FP16/BF16 – 2 bytes per weight, full quality, high memory use.
8-bit (INT8) – 1 byte per weight, slight quality loss, moderate memory.
4-bit (INT4/NF4) – 0.5 bytes per weight, noticeable but acceptable quality for most tasks, low memory.
2-bit – 0.25 bytes per weight, significant quality loss, rarely recommended.

For example, a 7B model in FP16 needs 14 GB of memory just for weights, but in 4-bit it needs only 3.5 GB. Most local LLM users choose 4-bit or 8-bit to fit larger models on consumer hardware.

How to Estimate Memory for Any Model

The basic formula is: Memory (GB) ≈ (Parameters × Bytes per Weight) + Overhead. Overhead includes the KV cache (which grows with context length), the inference engine’s buffers, and the operating system’s own usage. A practical rule of thumb is to add 1–2 GB for a 7B model, 2–4 GB for a 13B model, and 6–10 GB for a 70B model. For a more precise estimate, use the following steps:

Find the model’s parameter count (e.g., 7B, 13B, 70B).
Multiply by the bytes per weight for your quantization (FP16: 2, INT8: 1, INT4: 0.5).
Add the KV cache: roughly 0.5–1 GB per 1,000 tokens of context for a 7B model, scaling linearly with model size.
Add 1–2 GB for the inference engine and system overhead.

Practical Memory Requirements by Model Size

Here are typical free memory requirements for running popular model sizes with 4-bit quantization and a 2,000-token context (adjust for your context length):

1–3B models (e.g., TinyLlama, Phi-3-mini): 2–4 GB – run on almost any laptop.
7B models (e.g., Llama 3, Mistral): 6–8 GB – run on mid-range GPUs or 16 GB RAM systems.
13B models (e.g., Llama 2 13B): 10–12 GB – need a GPU with 12+ GB VRAM or 32 GB RAM.
30B models (e.g., Mixtral 8x7B): 20–24 GB – high-end consumer GPU (RTX 3090/4090) or 64 GB RAM.
70B models (e.g., Llama 3 70B): 40–48 GB – require dual GPUs or a server with 128 GB RAM.

If you run on CPU with system RAM, add about 25% more because the inference engine uses additional buffers. Also, the operating system and other programs need memory, so always subtract that from your total.

Common Mistakes and Misconceptions

Confusing VRAM with system RAM – A GPU’s VRAM is separate; you cannot use system RAM for GPU inference unless you use CPU-only mode, which is much slower.
Ignoring context length – The KV cache grows linearly with the number of tokens you input. A 32K context can double or triple memory usage.
Assuming “7B” means 7 GB – The actual memory depends on quantization, not the parameter count alone.
Forgetting about the GPU’s other usage – Your display and other applications use VRAM, so the “free” VRAM is what matters, not the total.

How to Check Your Available Memory

Before downloading a model, check how much free memory you have. On Windows, open Task Manager (Ctrl+Shift+Esc) and look at the Performance tab for GPU memory and RAM. On Linux, use free -h for RAM and nvidia-smi for GPU VRAM. On macOS, use Activity Monitor. Subtract the memory used by other programs to get the free amount. Then compare with the requirements above.

What If You Don’t Have Enough RAM?

If you are short on memory, you have a few options:

Use a smaller model – Drop from 13B to 7B, or from 7B to 3B.
Increase quantization – Switch from 8-bit to 4-bit (quality drops slightly).
Reduce context length – Use a shorter prompt or limit the response length.
Use CPU offloading – Some tools can offload part of the model to system RAM, but this slows inference dramatically.
Close other applications – Free up VRAM by closing browsers and games.

Frequently Asked Questions

Can I run a 7B model on 8 GB RAM?

Yes, if you use 4-bit quantization and have no other significant memory usage. A 7B 4-bit model needs about 4 GB for weights plus 1–2 GB overhead, so 8 GB total is enough, though it will be tight. On a GPU with 8 GB VRAM, it works well.

Is more RAM always better?

For local LLMs, yes—more memory lets you run larger models or longer contexts. But beyond a certain point (e.g., 128 GB for most users), the bottleneck becomes the CPU or GPU speed, not memory.

Does the model’s quality drop with 4-bit quantization?

Yes, but the drop is small for most tasks. In benchmarks, 4-bit models often score within 1–2% of the full-precision version. For casual use, the difference is usually unnoticeable.

Can I use system RAM as VRAM?

No, system RAM cannot be used as VRAM on most consumer systems. However, you can run the model entirely on CPU using system RAM, which is slower but works. Some tools like llama.cpp support CPU inference efficiently.

How much RAM do I need for a 70B model?

For a 70B model in 4-bit, you need about 40 GB of memory. This means either a GPU with 48 GB VRAM (like an RTX A6000) or a system with 128 GB RAM running on CPU. Dual consumer GPUs (e.g., two RTX 3090s) can also work with proper software support.

Does the operating system affect memory requirements?

Yes. Windows typically uses 2–4 GB RAM at idle, while a minimal Linux server might use under 1 GB. On a GPU, the display driver and desktop environment consume VRAM (often 0.5–2 GB). Always account for this when checking free memory.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.