Hierarchical DAG scheduling for Hybrid Distributed Systems

TitleHierarchical DAG scheduling for Hybrid Distributed Systems
Publication TypeConference Paper
Year of Publication2015
AuthorsWu, W., A. Bouteiller, G. Bosilca, M. Faverge, and J. Dongarra
Conference Name29th IEEE International Parallel & Distributed Processing Symposium (IPDPS)
Date Published05-2015
Conference LocationHyderabad, India
Keywordsdense linear algebra, gpu, heterogeneous architecture, PaRSEC runtime
AbstractAccelerator-enhanced computing platforms have drawn a lot of attention due to their massive peak com-putational capacity. Despite significant advances in the pro-gramming interfaces to such hybrid architectures, traditional programming paradigms struggle mapping the resulting multi-dimensional heterogeneity and the expression of algorithm parallelism, resulting in sub-optimal effective performance. Task-based programming paradigms have the capability to alleviate some of the programming challenges on distributed hybrid many-core architectures. In this paper we take this concept a step further by showing that the potential of task-based programming paradigms can be greatly increased with minimal modification of the underlying runtime combined with the right algorithmic changes. We propose two novel recursive algorithmic variants for one-sided factorizations and describe the changes to the PaRSEC task-scheduling runtime to build a framework where the task granularity is dynamically adjusted to adapt the degree of available parallelism and kernel effi-ciency according to runtime conditions. Based on an extensive set of results we show that, with one-sided factorizations, i.e. Cholesky and QR, a carefully written algorithm, supported by an adaptive tasks-based runtime, is capable of reaching a degree of performance and scalability never achieved before in distributed hybrid environments.
Project Tags: 
External Publication Flag: