Optimizing Hardware Resource Partitioning and Job Allocations on Modern GPUs under Power Caps

Eishi Arima, Minjoon Kang, Issa Saba, Josef Weidendorfer, Carsten Trinitis, Martin Schulz

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

CPU-GPU heterogeneous systems are now commonly used in HPC (High-Performance Computing). However, improving the utilization and energy-efficiency of such systems is still one of the most critical issues. As one single program typically cannot fully utilize all resources within a node/chip, co-scheduling (or co-locating) multiple programs with complementary resource requirements is a promising solution. Meanwhile, as power consumption has become the first-class design constraint for HPC systems, such co-scheduling techniques should be well-Tailored for power-constrained environments. To this end, the industry recently started supporting hardware-level resource partitioning features on modern GPUs for realizing efficient co-scheduling, which can operate with existing power capping features. For example, NVidia's MIG (Multi-Instance GPU) partitions one single GPU into multiple instances at the granularity of a GPC (Graphics Processing Cluster). In this paper, we explicitly target the combination of hardware-level GPU partitioning features and power capping for power-constrained HPC systems. We provide a systematic methodology to optimize the combination of chip partitioning, job allocations, as well as power capping based on our scalability/interference modeling while taking a variety of aspects into account, such as compute/memory intensity and utilization in heterogeneous computational resources (e.g., Tensor Cores). The experimental result indicates that our approach is successful in selecting a near optimal combination across multiple different workloads.

Original languageEnglish
Title of host publication51st International Conference on Parallel Processing, ICPP 2022 - Workshop Proceedings
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450394451
DOIs
StatePublished - 29 Aug 2022
Event51st International Conference on Parallel Processing, ICPP 2022 - Virtual, Online, France
Duration: 29 Aug 20221 Sep 2022

Publication series

NameACM International Conference Proceeding Series

Conference

Conference51st International Conference on Parallel Processing, ICPP 2022
Country/TerritoryFrance
CityVirtual, Online
Period29/08/221/09/22

Keywords

  • Co-Scheduling
  • GPUs
  • MIG
  • Power Capping

Fingerprint

Dive into the research topics of 'Optimizing Hardware Resource Partitioning and Job Allocations on Modern GPUs under Power Caps'. Together they form a unique fingerprint.

Cite this