Probabilistic Job History Conversion and Performance Model Generation for Malleable Scheduling Simulations

Isaías Comprés; Eishi Arima; Martin Schulz; Tiberiu Rotaru; Rui Machado

doi:10.1007/978-3-031-40843-4_7

Probabilistic Job History Conversion and Performance Model Generation for Malleable Scheduling Simulations

Isaías Comprés, Eishi Arima, Martin Schulz, Tiberiu Rotaru, Rui Machado

Informatik 10 - Lehrstuhl für Rechnerarchitektur und Parallele Systeme

Publikation: Beitrag in Buch/Bericht/Konferenzband › Konferenzbeitrag › Begutachtung

Abstract

Malleability support in supercomputing requires several updates to system software stacks. In addition to this, updates to applications, libraries and the runtime systems of distributed memory programming models are also necessary. Because of this, there are relatively few applications that have been extended or developed with malleability support. As a consequence, there are no job histories from production systems that include sufficient malleable job submissions for scheduling research. In this paper, we propose a solution: a probabilistic job history conversion. This conversion allows us to evaluate malleable scheduling heuristics via simulations based on existing job histories. Based on a configurable probability, job arrivals are converted into malleable versions, and assigned a malleable performance model. This model is used by the simulator to evaluate its changes at runtime, as an effect of malleable operations being applied to it.

Originalsprache	Englisch
Titel	High Performance Computing - ISC High Performance 2023 International Workshops, Revised Selected Papers
Redakteure/-innen	Amanda Bienz, Michèle Weiland, Marc Baboulin, Carola Kruse
Herausgeber (Verlag)	Springer Science and Business Media Deutschland GmbH
Seiten	82-94
Seitenumfang	13
ISBN (Print)	9783031408427
DOIs	https://doi.org/10.1007/978-3-031-40843-4_7
Publikationsstatus	Veröffentlicht - 2023
Veranstaltung	38th International Conference on High Performance Computing, ISC High Performance 2023 - Hamburg, Deutschland Dauer: 21 Mai 2023 → 25 Mai 2023

Publikationsreihe

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Band	13999 LNCS
ISSN (Print)	0302-9743
ISSN (elektronisch)	1611-3349

Konferenz

Konferenz	38th International Conference on High Performance Computing, ISC High Performance 2023
Land/Gebiet	Deutschland
Ort	Hamburg
Zeitraum	21/05/23 → 25/05/23

Zugriff auf Dokument

10.1007/978-3-031-40843-4_7

Andere Dateien und Links

Verknüpfung zur Publikation in Scopus

Dieses zitieren

Comprés, I., Arima, E., Schulz, M., Rotaru, T., & Machado, R. (2023). Probabilistic Job History Conversion and Performance Model Generation for Malleable Scheduling Simulations. in A. Bienz, M. Weiland, M. Baboulin, & C. Kruse (Hrsg.), High Performance Computing - ISC High Performance 2023 International Workshops, Revised Selected Papers (S. 82-94). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Band 13999 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-40843-4_7

Comprés, Isaías ; Arima, Eishi ; Schulz, Martin et al. / Probabilistic Job History Conversion and Performance Model Generation for Malleable Scheduling Simulations. High Performance Computing - ISC High Performance 2023 International Workshops, Revised Selected Papers. Hrsg. / Amanda Bienz ; Michèle Weiland ; Marc Baboulin ; Carola Kruse. Springer Science and Business Media Deutschland GmbH, 2023. S. 82-94 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{ac7f7191cef24cbe845310ce478389e3,

title = "Probabilistic Job History Conversion and Performance Model Generation for Malleable Scheduling Simulations",

abstract = "Malleability support in supercomputing requires several updates to system software stacks. In addition to this, updates to applications, libraries and the runtime systems of distributed memory programming models are also necessary. Because of this, there are relatively few applications that have been extended or developed with malleability support. As a consequence, there are no job histories from production systems that include sufficient malleable job submissions for scheduling research. In this paper, we propose a solution: a probabilistic job history conversion. This conversion allows us to evaluate malleable scheduling heuristics via simulations based on existing job histories. Based on a configurable probability, job arrivals are converted into malleable versions, and assigned a malleable performance model. This model is used by the simulator to evaluate its changes at runtime, as an effect of malleable operations being applied to it.",

keywords = "Job History, Malleability, Scheduling, Supercomputing",

author = "Isa{\'i}as Compr{\'e}s and Eishi Arima and Martin Schulz and Tiberiu Rotaru and Rui Machado",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 38th International Conference on High Performance Computing, ISC High Performance 2023 ; Conference date: 21-05-2023 Through 25-05-2023",

year = "2023",

doi = "10.1007/978-3-031-40843-4_7",

language = "English",

isbn = "9783031408427",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "82--94",

editor = "Amanda Bienz and Mich{\`e}le Weiland and Marc Baboulin and Carola Kruse",

booktitle = "High Performance Computing - ISC High Performance 2023 International Workshops, Revised Selected Papers",

}

Comprés, I, Arima, E, Schulz, M, Rotaru, T & Machado, R 2023, Probabilistic Job History Conversion and Performance Model Generation for Malleable Scheduling Simulations. in A Bienz, M Weiland, M Baboulin & C Kruse (Hrsg.), High Performance Computing - ISC High Performance 2023 International Workshops, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Bd. 13999 LNCS, Springer Science and Business Media Deutschland GmbH, S. 82-94, 38th International Conference on High Performance Computing, ISC High Performance 2023, Hamburg, Deutschland, 21/05/23. https://doi.org/10.1007/978-3-031-40843-4_7

Probabilistic Job History Conversion and Performance Model Generation for Malleable Scheduling Simulations. / Comprés, Isaías; Arima, Eishi; Schulz, Martin et al.
High Performance Computing - ISC High Performance 2023 International Workshops, Revised Selected Papers. Hrsg. / Amanda Bienz; Michèle Weiland; Marc Baboulin; Carola Kruse. Springer Science and Business Media Deutschland GmbH, 2023. S. 82-94 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Band 13999 LNCS).

Publikation: Beitrag in Buch/Bericht/Konferenzband › Konferenzbeitrag › Begutachtung

TY - GEN

T1 - Probabilistic Job History Conversion and Performance Model Generation for Malleable Scheduling Simulations

AU - Comprés, Isaías

AU - Arima, Eishi

AU - Schulz, Martin

AU - Rotaru, Tiberiu

AU - Machado, Rui

PY - 2023

Y1 - 2023

N2 - Malleability support in supercomputing requires several updates to system software stacks. In addition to this, updates to applications, libraries and the runtime systems of distributed memory programming models are also necessary. Because of this, there are relatively few applications that have been extended or developed with malleability support. As a consequence, there are no job histories from production systems that include sufficient malleable job submissions for scheduling research. In this paper, we propose a solution: a probabilistic job history conversion. This conversion allows us to evaluate malleable scheduling heuristics via simulations based on existing job histories. Based on a configurable probability, job arrivals are converted into malleable versions, and assigned a malleable performance model. This model is used by the simulator to evaluate its changes at runtime, as an effect of malleable operations being applied to it.

AB - Malleability support in supercomputing requires several updates to system software stacks. In addition to this, updates to applications, libraries and the runtime systems of distributed memory programming models are also necessary. Because of this, there are relatively few applications that have been extended or developed with malleability support. As a consequence, there are no job histories from production systems that include sufficient malleable job submissions for scheduling research. In this paper, we propose a solution: a probabilistic job history conversion. This conversion allows us to evaluate malleable scheduling heuristics via simulations based on existing job histories. Based on a configurable probability, job arrivals are converted into malleable versions, and assigned a malleable performance model. This model is used by the simulator to evaluate its changes at runtime, as an effect of malleable operations being applied to it.

KW - Job History

KW - Malleability

KW - Scheduling

KW - Supercomputing

UR - http://www.scopus.com/inward/record.url?scp=85171379941&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-40843-4_7

DO - 10.1007/978-3-031-40843-4_7

M3 - Conference contribution

AN - SCOPUS:85171379941

SN - 9783031408427

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 82

EP - 94

BT - High Performance Computing - ISC High Performance 2023 International Workshops, Revised Selected Papers

A2 - Bienz, Amanda

A2 - Weiland, Michèle

A2 - Baboulin, Marc

A2 - Kruse, Carola

PB - Springer Science and Business Media Deutschland GmbH

T2 - 38th International Conference on High Performance Computing, ISC High Performance 2023

Y2 - 21 May 2023 through 25 May 2023

ER -

Comprés I, Arima E, Schulz M, Rotaru T, Machado R. Probabilistic Job History Conversion and Performance Model Generation for Malleable Scheduling Simulations. in Bienz A, Weiland M, Baboulin M, Kruse C, Hrsg., High Performance Computing - ISC High Performance 2023 International Workshops, Revised Selected Papers. Springer Science and Business Media Deutschland GmbH. 2023. S. 82-94. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-40843-4_7