GPU guides

Vendor specs and measured plow vs vLLM results for the NVIDIA and AMD GPUs we run on: H100, MI300X, MI325X, MI350X and MI355X.

• NVIDIA H100 SXM: 80 GB, 3.35 TB/s

  The GPU most of our builds and comparisons run on.

• AMD Instinct MI300X: 192 GB, 5.3 TB/s

  192 GB of HBM3 memory on one card.

• AMD Instinct MI325X: 256 GB, 6 TB/s

  The MI300X compute with 256 GB of faster memory.

• AMD Instinct MI350X: 288 GB, 8 TB/s

  CDNA 4 with FP4 and 288 GB; our long-context and GLM-5.3 results.

• AMD Instinct MI355X: 288 GB, 8 TB/s

  The higher-clocked MI350; where we brought up Kimi-K3 on day zero.

Every number on these pages is on benchmarks, with its settings and source.