Wiki/Demand
HBM content per AI accelerator
The shape of AI memory demand starts with a per-socket number. Each generation of accelerator carries more HBM than the last, and the rate of growth has outpaced anything in commodity DRAM history.
The accelerators
| Accelerator | HBM | Bandwidth | Stack config | Source |
|---|---|---|---|---|
| Nvidia H100 (SXM5) | 80 GB HBM3 | 3.35 TB/s | 5× 16 GB | Nvidia H100 datasheet |
| Nvidia H200 | 141 GB HBM3e | 4.8 TB/s | 6× 24 GB | Nvidia H200 product page; PNY datasheet PDF |
| Nvidia B200 (Blackwell) | 192 GB HBM3e | 8 TB/s | 8 stacks, 8-high | Nvidia Blackwell architecture page |
| Nvidia GB200 superchip | 384 GB HBM3e | — | 2× B200 + Grace | Nvidia Blackwell pages |
| Nvidia B300 / GB300 (Blackwell Ultra) | 288 GB / 576 GB HBM3e | — | 12-high stacks | Nvidia developer blog, "Inside Blackwell Ultra" |
| AMD Instinct MI300X | 192 GB HBM3 | 5.3 TB/s | 8× 24 GB | AMD MI300X datasheet PDF; Hot Chips 2024 |
| AMD Instinct MI325X | 256 GB HBM3e | 6 TB/s | — | AMD MI300-series page |
| AMD Instinct MI355X / MI350 | 288 GB HBM3e | 8 TB/s | — | AMD MI350 product page |
The trajectory
H100 to B300 — two product generations — is a 3.6× jump in HBM per GPU (80 GB → 288 GB). At the superchip level, GB300's 576 GB is a 7.2× jump from a single H100. An NVL72 rack of GB300 carries 14–21 TB of HBM3e per Nvidia's own developer blog.
AMD's roadmap converges at the same number: MI300X (192 GB) → MI325X (256 GB) → MI355X (288 GB). Both vendors arrive at ~288 GB per GPU by 2025–2026, an industry-wide signal that frontier-model serving requires that capacity.
Why each refresh is a memory refresh
H100 → H200 is the cleanest example: same Hopper compute die, 76% more HBM (80 GB → 141 GB) and 43% more bandwidth. Nvidia priced H200 as a distinct SKU. The midcycle product was, in commercial substance, a memory product.
The pattern repeats with MI325X (256 GB on the same MI300 compute) and Blackwell Ultra (288 GB 12-high stacks on the same Blackwell die). The competitive axis in 2024–2026 is HBM capacity, not flops.
The number to remember
288 GB per GPU × the volumes Nvidia and AMD are shipping is the demand-side input to the rest of this wiki. A single GB300 superchip carries enough HBM to populate ~750 high-end gaming PCs. The 3:1 wafer trade ratio between HBM and DDR5 (Mehrotra, Micron Q1 FY26 call) means the same wafer pool that would otherwise serve PCs, phones, servers, and cars is being routed to a small number of accelerator stacks.
See supply/hbm-packaging-bottleneck.md for what that does to the wafer pool, and demand/hyperscaler-buying.md for how that capacity is being booked.