Memory & Efficiency
Quantization & Precision
IntermediateQuantization is the single highest-leverage lever for fitting big models on real hardware. By storing each number in fewer bits, you shrink memory and bandwidth - often with near-zero quality loss. This page builds from how a float is laid out in bits, to the methods that do it, to the hardware that makes it fast.
Fewer bits, same model
A model's weights are just billions of numbers. Store each one in 2 bytes instead of 4, and the model halves in size. Drop to 1 byte (FP8) or half a byte (INT4) and it halves again - and again. The catch is precision: too few bits and the numbers can no longer represent what the model learned. Modern quantization is the science of cutting bits right up to that line, and Blackwell-class hardware now makes the low-precision math faster, not just smaller.
Anatomy of a number - where the bits go
Every format splits its bits into a sign, an exponent (range), and a mantissa (precision). Quantization is the art of throwing bits away while keeping the number useful.
Total bits
16
Bytes / value
2
Max magnitude
±3.4e38
vs FP32 size
0.50×
BF16 - Brain float. The training & inference default. Keeps FP32's exponent range, trades mantissa - no loss scaling needed.
The format ladder
Each rung halves the bytes. BF16 is the lossless default; FP8 is the current serving sweet spot; FP4/INT4 push the frontier with per-block scaling to stay accurate.
| Format | Bytes / value | Where it's used |
|---|---|---|
| FP32 | 4 B | Master weights, optimizer states |
| FP16 | 2 B | Legacy mixed precision (needs loss scaling) |
| BF16 | 2 B | Default for weights & training |
| FP8 (E4M3/E5M2) | 1 B | Forward/grad on Hopper & Blackwell |
| INT4 / NF4 | 0.5 B | Weight-only serving, QLoRA fine-tune |
| FP4 (NVFP4/MXFP4) | 0.5 B | Blackwell-native, per-block scaled |
Size it: precision vs. what fits
The same model swings wildly in footprint depending on precision. A 70B model needs 4 GPUs at FP32 but fits a single H100 at INT4. Pick a model and watch the bars - and the quality cost - move.
Precision explorer - shrink the model, keep the quality
Halving the bytes per parameter halves the memory footprint. Pick a model and see how each precision changes what fits on a single GPU - and what it costs in quality.
Reference. Rarely used for serving - pure overhead.
Effectively lossless. The default for weights.
~1% degradation on most tasks. Native on Hopper/Blackwell.
GPTQ/AWQ 4-bit. Small quality hit, huge memory win.
Weights only - KV cache and activations add more. Quality figures are illustrative; real degradation is task- and method-dependent. Reference GPUs: H100 = 80 GB, H200 = 141 GB HBM. Below INT4, per-block scaling (NVFP4/MXFP4) becomes essential to hold quality.
What, exactly, gets quantized
Quantization isn't one knob. You can quantize the weights, the activations, or the KV cache - independently - and each targets a different bottleneck.
Weight-only
Quantize weights, keep activations in BF16. Dequantize on the fly.
Cuts the dominant memory term; minimal quality loss.
Weight + Activation (W8A8 / W4A8)
Both weights and activations low-precision - the matmul runs in low precision too.
Faster compute via Tensor Cores, not just smaller memory.
KV-cache quantization
Store the KV cache itself at FP8/INT8 instead of BF16.
Halves cache footprint & bandwidth per decode step.
How it's done
The toolbox splits into post-training (fast, no retraining) and training-aware (best quality, expensive). These are the names you'll meet in every model card.
PTQ
minutes · no retrainingPost-Training Quantization
Trained BF16 model
frozen weights, already converged
↓
Calibrate on a few batches
measure activation ranges → pick scales
↓
Round weights to low-bit
GPTQ / AWQ / SmoothQuant
↓
Quantized model ships
small quality drop, no GPUs spent
QAT
full training run · best qualityQuantization-Aware Training
Train with fake-quant inserted
forward pass rounds, gradients stay full-precision
↓
Model learns to tolerate rounding
weights adapt around the coarse grid
↓
Bake in the real low-bit weights
no calibration gap to recover
↓
Quantized model ships
holds accuracy even at 2–4 bits
The named recipes
PTQ - Post-Training Quantization
Quantize an already-trained model. No retraining, minutes to run. The default path for serving - GPTQ, AWQ, and SmoothQuant all live here.
QAT - Quantization-Aware Training
Simulate quantization during training so the model learns to tolerate it. Best quality at low bit-widths, but costs a full training run.
GPTQ / AWQ
Weight-only PTQ to 4-bit. GPTQ minimises layer-wise error; AWQ protects the most salient weight channels. The backbone of 4-bit serving.
SmoothQuant
Migrates activation outliers into the weights so both can go to INT8/FP8 (W8A8). Enables activation quantization, not just weights.
QLoRA (NF4)
Fine-tune a 4-bit NF4 frozen base with small LoRA adapters. Puts 70B fine-tuning on a single 48 GB GPU.
GGUF (llama.cpp)
K-quant block formats (Q4_K_M, Q5_K…) for CPU/edge inference. How a 70B model runs on a laptop.
Try it: round the weights yourself
Quantization is, at heart, snapping each weight onto a coarse grid. Drop the bit-width to see the grid get coarser and the rounding error grow - then toggle per-block scaling to watch that error shrink back. This is the exact trick behind NVFP4/MXFP4.
Quantization playground - round the weights, watch the error
A weight matrix is just numbers. Quantizing snaps each one onto a coarse grid of allowed values. Fewer bits means a coarser grid and bigger rounding error - per-block scaling fights back by giving each block of 4 its own range.
Levels
16
4-bit grid
Bytes / value
0.50
vs 2 B (BF16)
RMS error
0.0279
lower is better
Max error
0.0600
worst single weight
Per-block scaling gives each block of 4 its own range, so a block of small weights doesn't waste its levels on an outlier elsewhere. Turn it off to see the error jump - that's the whole idea behind NVFP4/MXFP4.
Hardware makes low precision fast
Smaller numbers aren't just cheaper to store - on the right silicon they're cheaper to multiply. Each GPU generation adds native Tensor Core support for a lower format, roughly doubling throughput. Hopper made FP8 a first-class citizen (~2× BF16); Blackwell adds FP4/NVFP4 (~2× FP8 again), with per-block micro-scaling to hold accuracy at 4 bits.
Native FP8 (E4M3/E5M2) Tensor Cores - ~2× BF16 throughput. DeepSeek-V3 was trained in FP8 on this generation.
Adds native FP4 (NVFP4/MXFP4) - ~2× FP8 again. Per-block scaling keeps 4-bit usable for inference at frontier scale.
Quantization composes with everything
Lower precision shrinks weights and the KV cache, and it changes how many GPUs your parallelism plan needs. It's the cheapest lever you have - pull it first, then size the rest.