Active I/O Streams for Heterogeneous High Performance Computing
Fabián E. Bustamante and Karsten Schwan
In Proc. of Parallel Computing (ParCo), 1999.
College of Computing
Georgia Institute of Technology
Atlanta, GA 30332, USA
This email address is being protected from spambots. You need JavaScript enabled to view it.
, This email address is being protected from spambots. You need JavaScript enabled to view it.
Abstract
We are concerned with the attainment of high performance in I/O on distributed, heterogeneous hardware. Our approach is to combine a program's data retrieval and storage actions with operations executed on the resulting active I/O streams. Performance improvements are attained by exploitation of information about these operations and by runtime changes to their behavior and placement. In this fashion, active I/O can adjust to static system properties derived from the heterogeneous nature of distributed CPU, storage, and network devices, and it can respond to dynamic changes in system's conditions, thereby reducing the total bandwidth needs and/or the end-to-end latencies of I/O actions. Our prototype of an active I/O system, called Adios, implements I/O as a directed network comprised of streams originating at sources, destined for sinks, and routed through a number of intermediate vertices that act on the data units traversing the stream. Adaptive resource allocation methods based on this model are under development, with the goal of improving I/O performance of complex parallel programs running in shared heterogeneous computing environments.
