GPU accelerated computing-from hype to mainstream, the rebirth of vector computing

Acceleration technologies, in particular GPUs and Cell, are receiving considerable attention in modern-day HPC. Compared to classic accelerators and traditional CPUs, these devices not only exhibit higher compute density, but also sport significant memory bandwidth and vector-like capabilities to stream data at bandwidth of 100 GB/s or more. The latter qualifies such accelerators as a rebirth of vector computing. With large-scale deployments of GPUs such as Tokyo Tech's TSUBAME 1.2 supercomputer facilitating 680 GPUs in a 100-Teraflops scale supercomputer, we can demonstrate that, even under a massively parallel setting, GPUs can scale both in dense linear algebra codes as well as vector-oriented CFD codes. In both cases, however, careful algorithmic developments, especially latency hiding, are important to maximize their performance.