%0 Journal Article
%J Proceedings of the Royal Society A
%D 2020
%T Mixed-Precision Iterative Refinement using Tensor Cores on GPUs to Accelerate Solution of Linear Systems
%A Azzam Haidar
%A Harun Bayraktar
%A Stanimire Tomov
%A Jack Dongarra
%A Nicholas J. Higham
%K GMRESLU factorization
%K GPU computing
%K half precision arithmetic
%K iterative refinement
%K mixed precision solvers
%X Double-precision floating-point arithmetic (FP64) has been the de facto standard for engineering and scientific simulations for several decades. Problem complexity and the sheer volume of data coming from various instruments and sensors motivate researchers to mix and match various approaches to optimize compute resources, including different levels of floating-point precision. In recent years, machine learning has motivated hardware support for half-precision floating-point arithmetic. A primary challenge in high-performance computing is to leverage reduced-precision and mixed-precision hardware. We show how the FP16/FP32 Tensor Cores on NVIDIA GPUs can be exploited to accelerate the solution of linear systems of equations Ax = b without sacrificing numerical stability. The techniques we employ include multiprecision LU factorization, the preconditioned generalized minimal residual algorithm (GMRES), and scaling and auto-adaptive rounding to avoid overflow. We also show how to efficiently handle systems with multiple right-hand sides. On the NVIDIA Quadro GV100 (Volta) GPU, we achieve a 4×−5× performance increase and 5× better energy efficiency versus the standard FP64 implementation while maintaining an FP64 level of numerical stability.
%B Proceedings of the Royal Society A
%V 476
%8 2020-11
%G eng
%N 2243
%R https://doi.org/10.1098/rspa.2020.0110
%0 Generic
%D 2020
%T Mixed-Precision Solution of Linear Systems Using Accelerator-Based Computing
%A Azzam Haidar
%A Harun Bayraktar
%A Stanimire Tomov
%A Jack Dongarra
%A Nicholas J. Higham
%X Double-precision floating-point arithmetic (FP64) has been the de facto standard for engineering and scientific simulations for several decades. Problem complexity and the sheer volume of data coming from various instruments and sensors motivate researchers to mix and match various approaches to optimize compute resources, including different levels of floating-point precision. In recent years, machine learning has motivated hardware support for half-precision floating-point arithmetic. A primary challenge in high-performance computing is to leverage reduced- and mixed-precision hardware. We show how the FP16/FP32 Tensor Cores on NVIDIA GPUs can be exploited to accelerate the solution of linear systems of equations Ax = b without sacrificing numerical stability. We achieve a 4×–5× performance increase and 5× better energy efficiency versus the standard FP64 implementation while maintaining an FP64 level of numerical stability.
%B Innovative Computing Laboratory Technical Report
%I University of Tennessee
%8 2020-05
%G eng