Do MoE models need less VRAM?

No. Mixture-of-experts models activate only some parameters per token, which improves speed, but all experts must sit in memory, so VRAM follows total parameter count.

LLM VRAM Calculator

Q: How much VRAM does a 70B model need?

At Q4_K_M quantization (~4.8 bits per weight), a 70B model needs roughly 42 GB plus overhead, which is more than any single consumer GPU offers but fits comfortably on two 24 GB cards or a 64 GB Mac. At FP16 it needs ~168 GB.

Q: What does quantization do?

Quantization stores each weight with fewer bits, commonly 4 or 5 instead of 16, which shrinks memory to roughly a third with a small quality loss. Q4_K_M is the popular sweet spot.

Find out whether a model fits on your hardware before downloading 80 GB of weights. Pick an open model (or enter a parameter count), choose a quantization level, and see the estimated memory it needs against popular GPUs and Apple Silicon Macs. Estimates cover the weights plus typical inference overhead at moderate context lengths.

Model

Params (B)

Quantization

Estimate = parameters × bits-per-weight ÷ 8, plus ~20% overhead for the KV cache, activations and runtime buffers at a moderate context of 4K to 8K tokens. Long contexts need more. "Tight" means within 1 GB of the limit: possible with a short context and nothing else using the GPU. MoE models (Llama 4, DeepSeek) still need all experts in memory, so the total parameter count is what matters; only compute scales with the active parameters. On Apple Silicon, weights load into unified memory; leave a few GB for macOS. Prefer paying per token instead? Compare hosted prices in the API cost calculator.

Questions

How much VRAM does a 70B model need?

At Q4_K_M (~4.8 bits per weight), a 70B model needs roughly 42 GB plus overhead, which is more than any single consumer GPU offers but fits comfortably on two 24 GB cards or a 64 GB Mac. At FP16 it needs ~168 GB and moves into multi-A100/H100 territory.

What does quantization actually do?

Quantization stores each weight with fewer bits, usually 4 or 5 instead of 16, which shrinks memory to roughly a third with a small quality loss. Q4_K_M is the popular sweet spot; below ~3 bits, quality degrades noticeably.

Do MoE models like DeepSeek R1 need less memory?

No. Mixture-of-experts models activate only some parameters per token (37B of DeepSeek R1's 671B), which makes them faster, but every expert must still sit in memory, so VRAM requirements follow the total parameter count.