%0 Generic
%D 2019
%T A Collection of White Papers from the BDEC2 Workshop in San Diego, CA
%A Ilkay Altintas
%A Kyle Marcus
%A Volkan Vural
%A Shweta Purawat
%A Daniel Crawl
%A Gabriel Antoniu
%A Alexandru Costan
%A Ovidiu Marcu
%A Prasanna Balaprakash
%A Rongqiang Cao
%A Yangang Wang
%A Franck Cappello
%A Robert Underwood
%A Sheng Di
%A Justin M. Wozniak
%A Jon C. Calhoun
%A Cong Xu
%A Antonio Lain
%A Paolo Faraboschi
%A Nic Dube
%A Dejan Milojicic
%A Balazs Gerofi
%A Maria Girone
%A Viktor Khristenko
%A Tony Hey
%A Erza Kissel
%A Yu Liu
%A Richard Loft
%A Pekka Manninen
%A Sebastian von Alfthan
%A Takemasa Miyoshi
%A Bruno Raffin
%A Olivier Richard
%A Denis Trystram
%A Maryam Rahnemoonfar
%A Robin Murphy
%A Joel Saltz
%A Kentaro Sano
%A Rupak Roy
%A Kento Sato
%A Jian Guo
%A Jen s Domke
%A Weikuan Yu
%A Takaki Hatsui
%A Yasumasa Joti
%A Alex Szalay
%A William M. Tang
%A Michael R. Wyatt II
%A Michela Taufer
%A Todd Gamblin
%A Stephen Herbein
%A Adam Moody
%A Dong H. Ahn
%A Rich Wolski
%A Chandra Krintz
%A Fatih Bakir
%A Wei-tsung Lin
%A Gareth George
%B Innovative Computing Laboratory Technical Report
%I University of Tennessee
%8 2019-10
%G eng
%0 Journal Article
%J The International Journal of High Performance Computing Applications
%D 2018
%T Big Data and Extreme-Scale Computing: Pathways to Convergence - Toward a Shaping Strategy for a Future Software and Data Ecosystem for Scientific Inquiry
%A Mark Asch
%A Terry Moore
%A Rosa M. Badia
%A Micah Beck
%A Pete Beckman
%A Thierry Bidot
%A François Bodin
%A Franck Cappello
%A Alok Choudhary
%A Bronis R. de Supinski
%A Ewa Deelman
%A Jack Dongarra
%A Anshu Dubey
%A Geoffrey Fox
%A Haohuan Fu
%A Sergi Girona
%A Michael Heroux
%A Yutaka Ishikawa
%A Kate Keahey
%A David Keyes
%A William T. Kramer
%A Jean-François Lavignon
%A Yutong Lu
%A Satoshi Matsuoka
%A Bernd Mohr
%A Stéphane Requena
%A Joel Saltz
%A Thomas Schulthess
%A Rick Stevens
%A Martin Swany
%A Alexander Szalay
%A William Tang
%A Gaël Varoquaux
%A Jean-Pierre Vilotte
%A Robert W. Wisniewski
%A Zhiwei Xu
%A Igor Zacharov
%X Over the past four years, the Big Data and Exascale Computing (BDEC) project organized a series of five international workshops that aimed to explore the ways in which the new forms of data-centric discovery introduced by the ongoing revolution in high-end data analysis (HDA) might be integrated with the established, simulation-centric paradigm of the high-performance computing (HPC) community. Based on those meetings, we argue that the rapid proliferation of digital data generators, the unprecedented growth in the volume and diversity of the data they generate, and the intense evolution of the methods for analyzing and using that data are radically reshaping the landscape of scientific computing. The most critical problems involve the logistics of wide-area, multistage workflows that will move back and forth across the computing continuum, between the multitude of distributed sensors, instruments and other devices at the networks edge, and the centralized resources of commercial clouds and HPC centers. We suggest that the prospects for the future integration of technological infrastructures and research ecosystems need to be considered at three different levels. First, we discuss the convergence of research applications and workflows that establish a research paradigm that combines both HPC and HDA, where ongoing progress is already motivating efforts at the other two levels. Second, we offer an account of some of the problems involved with creating a converged infrastructure for peripheral environments, that is, a shared infrastructure that can be deployed throughout the network in a scalable manner to meet the highly diverse requirements for processing, communication, and buffering/storage of massive data workflows of many different scientific domains. Third, we focus on some opportunities for software ecosystem convergence in big, logically centralized facilities that execute large-scale simulations and models and/or perform large-scale data analytics. We close by offering some conclusions and recommendations for future investment and policy review.
%B The International Journal of High Performance Computing Applications
%V 32
%P 435–479
%8 2018-07
%G eng
%N 4
%R https://doi.org/10.1177/1094342018778123
%0 Conference Paper
%B ACM MultiMedia Workshop 2017
%D 2017
%T Efficient Communications in Training Large Scale Neural Networks
%A Yiyang Zhao
%A Linnan Wan
%A Wei Wu
%A George Bosilca
%A Richard Vuduc
%A Jinmian Ye
%A Wenqi Tang
%A Zenglin Xu
%X We consider the problem of how to reduce the cost of communication that is required for the parallel training of a neural network. The state-of-the-art method, Bulk Synchronous Parallel Stochastic Gradient Descent (BSP-SGD), requires many collective communication operations, like broadcasts of parameters or reductions for sub-gradient aggregations, which for large messages quickly dominates overall execution time and limits parallel scalability. To address this problem, we develop a new technique for collective operations, referred to as Linear Pipelining (LP). It is tuned to the message sizes that arise in BSP-SGD, and works effectively on multi-GPU systems. Theoretically, the cost of LP is invariant to P, where P is the number of GPUs, while the cost of more conventional Minimum Spanning Tree (MST) scales like O(logP). LP also demonstrate up to 2x faster bandwidth than Bidirectional Exchange (BE) techniques that are widely adopted by current MPI implementations. We apply these collectives to BSP-SGD, showing that the proposed implementations reduce communication bottlenecks in practice while preserving the attractive convergence properties of BSP-SGD.
%B ACM MultiMedia Workshop 2017
%I ACM
%C Mountain View, CA
%8 2017-10
%G eng
%0 Journal Article
%J International Journal of Parallel Programming
%D 2005
%T New Grid Scheduling and Rescheduling Methods in the GrADS Project
%A Francine Berman
%A Henri Casanova
%A Andrew Chien
%A Keith Cooper
%A Holly Dail
%A Anshuman Dasgupta
%A Wei Deng
%A Jack Dongarra
%A Lennart Johnsson
%A Ken Kennedy
%A Charles Koelbel
%A Bo Liu
%A Xu Liu
%A Anirban Mandal
%A Gabriel Marin
%A Mark Mazina
%A John Mellor-Crummey
%A Celso Mendes
%A A. Olugbile
%A Jignesh M. Patel
%A Dan Reed
%A Zhiao Shi
%A Otto Sievert
%A H. Xia
%A Asim YarKhan
%K grads
%B International Journal of Parallel Programming
%I Springer
%V 33
%P 209-229
%8 2005-06
%G eng
%0 Conference Proceedings
%B Proceedings of 16th IMACS World Congress 2000 on Scientific Computing, Applications Mathematics and Simulation
%D 2000
%T Seamless Access to Adaptive Solver Algorithms
%A Dorian Arnold
%A Susan Blackford
%A Jack Dongarra
%A Victor Eijkhout
%A Tinghua Xu
%K netsolve
%B Proceedings of 16th IMACS World Congress 2000 on Scientific Computing, Applications Mathematics and Simulation
%C Lausanne, Switzerland
%8 2000-08
%G eng