Hierarchical Resource Partitioning on Modern GPUs: A Reinforcement Learning Approach

Urvij Saroliya, Eishi Arima, Dai Liu, Martin Schulz

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

GPU-based heterogeneous architectures are now commonly used in HPC clusters. Due to their architectural simplicity specialized for data-level parallelism, GPUs can offer much higher computational throughput and memory bandwidth than CPUs in the same generation do. However, as the available resources in GPUs have increased exponentially over the past decades, it has become increasingly difficult for a single program to fully utilize them. As a consequence, the industry has started supporting several resource partitioning features in order to improve the resource utilization by co-scheduling multiple programs on the same GPU die at the same time.Driven by the technological trend, this paper focuses on hierarchical resource partitioning on modern GPUs, and as an example, we utilize a combination of two different features available on recent NVIDIA GPUs in a hierarchical manner: MPS (Multi-Process Service), a finer-grained logical partitioning; and MIG (Multi-Instance GPU), a coarse-grained physical partitioning. We propose a method for comprehensively co-optimizing the setup of hierarchical partitioning and the selection of co-scheduling groups from a given set of jobs, based on reinforcement learning using their profiles. Our thorough experimental results demonstrate that our approach can successfully set up job concurrency, partitioning, and co-scheduling group selections simultaneously. This results in a maximum throughput improvement by a factor of 1.87 compared to the time-sharing scheduling.

Original languageEnglish
Title of host publicationProceedings - 2023 IEEE International Conference on Cluster Computing, CLUSTER 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages185-196
Number of pages12
ISBN (Electronic)9798350307924
DOIs
StatePublished - 2023
Event25th IEEE International Conference on Cluster Computing, CLUSTER 2023 - Santa Fe, United States
Duration: 31 Oct 20233 Nov 2023

Publication series

NameProceedings - IEEE International Conference on Cluster Computing, ICCC
ISSN (Print)1552-5244

Conference

Conference25th IEEE International Conference on Cluster Computing, CLUSTER 2023
Country/TerritoryUnited States
CitySanta Fe
Period31/10/233/11/23

Keywords

  • GPUs
  • Reinforcement Learning
  • Resource Management
  • Scheduling

Fingerprint

Dive into the research topics of 'Hierarchical Resource Partitioning on Modern GPUs: A Reinforcement Learning Approach'. Together they form a unique fingerprint.

Cite this