People always ask how big a model really is and what it takes to host it. Forget marketing slides—here’s the napkin math you can use in the field.
At FP8, one parameter equals one byte. A 1 B model needs 1 GB of vRAM. That’s it. The GPUs that used to push pixels are now just expensive memory sticks:
H100 80 GB
H200 140 GB
B200 180 GB
Those numbers are per card and you almost always run eight in a box.
So a 40 B model already pushes you past any desktop card. Add the KV cache and even an 80–96 GB RTX 6000 Pro or single H100 is gasping.
Take Qwen3-235B-A35B. The “A35B” means only 35 B are active at once, a trick to survive narrow pipes. Still, the full 235 B must live somewhere. Round up to 235 GB plus KV. Two H200s (2×140 GB) or four H100s (4×80 GB) will hold it—barely. And the new 256 K context upgrade? That KV cache just doubled. (Nobody hosts this at scale except
@comput3ai—more later.)
Now drop the coding variant, Qwen3-Coder-480B. It's Qwen3 with additional training to be good at coding tasks. Same footprint math, but extra layers mean extra KV. Eight H200s will serve it, but you’ll be red-lining.
DeepSeek-R1 clocks in at 671 B. That’s 671 GB naked. Eight H100s give 640 GB—game over. Eight H200s give 1.14 TB, so it fits. Then DeepSeek drops a new checkpoint with more layers and the KV cache inflates again. You start pricing two 8-GPU H200 nodes.
Kimi K2 is 1 T—one trillion parameters. One terabyte of vRAM minimum. Moonshot’s own doc says “don’t try this on fewer than sixteen H200s,” i.e., two 8-way boxes. The B200 finally looks relaxed: 8×180 GB = 1.44 TB. That leaves enough headroom for the 128 K KV cache and still lets you breathe.
Rule of thumb: if the model won’t fit, you cannot ship. So grab the B200s—we’ve got them warm and waiting at
@comput3ai.