June 22–26, 2014
Leipzig, Germany

Presentation Details

Name: RADAR: Runtime Asymmetric Data-Access Driven Scientific Data Replication
Time: Wednesday, June 25, 2014
02:15 pm - 02:45 pm
Room:   Hall 3
CCL - Congress Center Leipzig
Breaks:01:00 pm - 02:15 pm Lunch
Speaker:   John Jenkins, Argonne National Laboratory
Abstract:   Efficient I/O on large-scale spatiotemporal scientific data requires scrutiny of both the logical layout of the data (e.g., row-major vs. column-major) and the physical layout (e.g., distribution on parallel filesystems). For increasingly complex datasets, hand optimization is a difficult matter prone to error and not scalable to the increasing heterogeneity of analysis workloads. Given these factors, we present a partial data replication system called RADAR. We capture datatype- and collective-aware I/O access patterns (indicating logical access) via MPI-IO tracing and use a combination of coarse-grained and fine-grained performance modeling to evaluate and select optimized physical data distributions for the task at hand. Unlike conventional methods, we store all replica data and metadata, along with the original untouched data, under a single file container using the object abstraction in parallel filesystems. Our system results in manyfold improvements in some commonly used subvolume decomposition access patterns. Moreover, the modeling approach can determine whether such optimizations should be undertaken in the first place.

John Jenkins, Argonne National Laboratory; Xiaocheng Zou, North Carolina State University; Houjun Tang, North Carolina State University; Dries Kimpe, Argonne National Laboratory; Robert Ross, Argonne National Laboratory; Nagiza Samatova, North Carolina State University