Multi-Level Checkpointing and Silent Error Detection for Linear Workflows

TitleMulti-Level Checkpointing and Silent Error Detection for Linear Workflows
Publication TypeJournal Article
Year of Publication2017
AuthorsBenoit, A., A. Cavelan, Y. Robert, and H. Sun
JournalJournal of Computational Science
Date Published04/2017
KeywordsDynamic programming algorithm, Error detection, fail-stop errors, fault-tolerance, Linear workflows, Multi-level checkpointing, silent errors, Verification mechanism

We focus on High Performance Computing (HPC) workflows whose dependency graph forms a linear chain, and we extend single-level checkpointing in two important directions. Our first contribution targets silent errors, and combines in-memory checkpoints with both partial and guaranteed verifications. Our second contribution deals with multi-level checkpointing for fail-stop errors. We present sophisticated dynamic programming algorithms that return the optimal solution for each problem in polynomial time. We also show how to combine all these techniques and solve the problem with both fail-stop and silent errors. Simulation results demonstrate that these extensions lead to significantly improved performance compared to the standard single-level checkpointing algorithm.