Most people would be happy just to get their mitts on a single GPU, in the current landscape. The US Department of Energy’s Argonne National Laboratory is much more fortunate, having procured a whopping 2,240 NVIDIA A100 Tensor Core GPUs (based on Ampere) for its Polaris supercomputer to “supercharge researcher and discovery.”

As such,

Source: Hot Hardware – NVIDIA Powers Polaris Supercomputer With 2,240 GPUs For Massive AI Workloads