Modern frontier large language models, whether spanning 8 billion, 70 billion, or 405 billion parameters, are massive collections of decimal numbers. In their native uncompressed format (typically 16-bit brain floating point, or BF16), each parameter consumes 2 bytes of high-bandwidth memory (VRAM). A 70-billion-parameter model thus demands over 140 gigabytes of raw video RAM merely to sit idle in memory, before allocating a single byte to user context or intermediate activation tensors.
The Anatomy of a Number
In standard floating-point representation, a number is partitioned into three distinct binary fields: a sign bit, an exponent, and a mantissa (fraction). This allows computers to represent numbers across astronomical dynamic ranges, from subatomic fractions ($10^{-38}$) to cosmic magnitudes ($10^{38}$):
| Format | Bit Width | Structure (Sign / Exp / Mantissa) | Discrete Values | Memory per 70B Model |
|---|---|---|---|---|
| FP16 / BF16 | 16 bits | 1 sign, 5/8 exp, 10/7 mantissa | 65,536 | ~140 GB |
| INT8 | 8 bits | 1 sign, 7 integer magnitude | 256 | ~70 GB |
| INT4 / FP4 | 4 bits | 1 sign, 3 magnitude (or 2 exp / 1 mantissa) | 16 | ~35 GB |
The Squeeze
Quantization is the algorithmic art of discarding 75% to 87.5% of binary data while preserving the semantic computations learned during billions of GPU training hours. When quantizing a layer to 4 bits, every single floating-point number must be forced into one of exactly 16 discrete bins (from -8 to +7 in signed INT4).
The standard affine quantization formula maps a continuous float $W$ to an integer $q$ via a scale factor $s$ and zero-point $z$:
Why It Works
If you rounded the numbers in your bank account or a space shuttle flight trajectory to 16 buckets, catastrophe would follow instantly. Why do neural networks survive this brutal crushing? Because neural networks are overparameterized, non-linear ensembles. The vast majority of weights in a matrix are distributed normally around zero. Shifting an individual weight by +/- 0.005 barely alters the post-activation output vector when thousands of incoming connections are summed together.
However, as we will explore in the next chapters, this resilience is deceptive. While superficial grammar and vocabulary survive 4-bit compression effortlessly, deep recursive reasoning, mathematical chain-of-thought, and precise coding logic sit on precarious precision cliffs.
Quantization from FP16 to 4-bit reduces memory footprint and memory bus transit by exactly 4x, transforming a model that requires an entire $300,000 server rack into one that fits on a single workstation or smartphone. But compressing millions of continuous weights into 16 discrete levels creates subtle rounding ripples that compound catastrophically across deep transformer layers.
Continue Your Exploration
This series translates the core principles of numerical precision, dynamic range, outlier feature clipping, and hardware Tensor Cores into accessible visual models. For full empirical telemetry, mathematical error bounds, and hardware benchmarking, consult our foundational monograph: Quantization Reasoning Degradation: 5 Empirical Rules.
Foundations of LLM Quantization & Low-Bit Inference
An empirical exploration through INT8/INT4/FP4 quantization, straight-through estimators, outlier features, and hardware tensor cores.