Application-Driven Requirements for Node Resource Management in Next-Generation Systems

Edgar A León (1) , Balazs Gerofi (2) , Julien Jaeger (3) , Guillaume Mercier (4) , Rolf Riesen (5) , Masamichi Takagi (2) , Brice Goglin (6)

1 LLNL - Lawrence Livermore National Laboratory
2 RIKEN CCS - RIKEN Center for Computational Science [Kobe]
3 CEA-DAM - Centre d'Études de Limeil-Valenton
4 Bordeaux INP - Institut Polytechnique de Bordeaux
5 Intel Corporation [USA]
6 TADAAM - Topology-Aware System-Scale Data Management for High-Performance Computing

Emerging workloads on supercomputing platforms are pushing the limits of traditional high-performance computing software environments. Multi-physics, coupled simulations, big data processing and machine learning frameworks, and multi-component workloads pose serious challenges to system and application developers. At the heart of the problem is the lack of cross-stack coordination to enable flexible resource management among multiple runtime components. In this work we analyze seven, real-world applications that represent emerging workloads and illustrate the scope and magnitude of the problem. We then extract several themes from these applications that highlight next-generation requirements for node resource managers. Finally, using these requirements, we propose a general, cross-stack coordination framework and outline its components and functionality.

Domaines: Architectures Matérielles [cs.AR]; Modélisation et simulation; Calcul parallèle, distribué et partagé [cs.DC]; Système d'exploitation [cs.OS]

Edgar A León, Balazs Gerofi, Julien Jaeger, Guillaume Mercier, Rolf Riesen, et al.. Application-Driven Requirements for Node Resource Management in Next-Generation Systems. ROSS 2020 : International Workshop on Runtime and Operating Systems for Supercomputers, Nov 2020, Atlanta, GA / Virtual, United States. ⟨hal-02950635v1⟩