Research SuperCluster (RSC)
Meta, United States (2022) — 16,000 x NVIDIA A100 SXM (80GB)
8 × A100 SXM (80GB) per server
4 servers per rack
= 32 accelerators per rack, drawing 12.8 kW (accelerators only)
Rack layout Estimated
NVIDIA DGX A100 carries 8 A100s (6U, ~6.5 kW); rack density is a typical assumption.
500 racks holding 2,000 servers and 16,000 accelerators.
At roughly 3 m² per rack including aisle space, that's about 1,500 m² of floor — roughly 3.6 basketball courts. Floor area is estimated: real halls also carry cooling, power, and networking equipment this doesn't count.
If fully dedicated, this cluster could train...
| Known training run | Time on this cluster | Originally took |
|---|---|---|
| GPT-3 (175B) | 1.8 days | duration not published |
| Llama 2 (70B) | 4.9 days | 35.0 days on 2,048 x A100 SXM (80GB) |
| Llama 3.1 (405B) | 220.3 days | 54.0 days on 16,384 x H100 SXM (80GB) |
| PaLM (540B) | 14.8 days | duration not published |
| Chinchilla (70B) | 3.3 days | duration not published |
Hypothetical: assumes the whole cluster runs one job at the vendor-default MFU, with no downtime, restarts, or contention — Estimated. "Originally took" is the run's own reported hardware and duration where published, for contrast.
Source: Meta AI blog, 'Introducing the AI Research SuperCluster' (2022): initial buildout of 6,080 A100 GPUs, with a planned full build-out of 16,000 A100-80GB GPUs. 16,000 is Meta's own announced target scale for RSC's full build-out, not necessarily the exact count operating at every point in time.