June 22–26, 2014
Leipzig, Germany

Presentation Details

Name: The SIOX Architecture – Coupling Automatic Monitoring & Optimization of Parallel I/O
Time: Thursday, June 26, 2014
10:30 am - 11:00 am
Room:   Hall 5
CCL - Congress Center Leipzig
Breaks:10:30 am - 11:00 am Coffee Break
07:30 am - 10:30 am Welcome Coffee
Speaker:   Julian Kunkel, DKRZ
Abstract:   Performance analysis and optimization of high-performance I/O systems is a daunting task. Mainly, this is due to the overwhelmingly complex interplay of the involved hardware and software layers. The Scalable I/O for Extreme Performance (SIOX) project provides a versatile environment for monitoring I/O activities and learning from this information. The goal of SIOX is to automatically suggest and apply performance optimizations, and to assist in locating and diagnosing performance problems.
In this paper, we present the current status of SIOX. Our modular architecture covers instrumentation of POSIX, MPI and other high-level I/O libraries; the monitoring data is recorded asynchronously into a global database, and recorded traces can be visualized. Furthermore, we offer a set of primitive plugins with additional features to demonstrate the flexibility of our architecture: A surveyor plugin to keep track of the oberserved spatial access patterns; an fadvise plugin for injecting hints to achieve read-ahead for strided access patterns; and an optimizer plugin which monitors the performance achieved with different MPI-IO hints, automatically supplying the best known hint-set when no hints were explicitely set. The presentation of the technical status is accompanied by a demonstration of some of these features on our 20 node cluster. In additional experiments, we analyze the overhead for concurrent access, for MPI-IO's 4-levels of access, and for an instrumented climate application. While our prototype is not yet full-featured, it demonstrates the potential and feasability of our approach.

Julian Kunkel, DKRZ; Michaela Zimmer, University of Hamburg; Nathanael Hübbe, University of Hamburg; Alvaro Aguilera, TU Dresden/ZIH; Holger Mickler, TU Dresden/ZIH; Xuan Wang, HLRS; Thomas Bönisch, HLRS; Andriy Chut, HLRS; Jakob Lüttgau, University of Hamburg; Roman Michel, University of Hamburg; Johann Weging, University of Hamburg