| Abstract: |
|
As concurrency and complexities continue to increase on high-end machines, in terms of both the number of cores and the level of storage hierarchy, managing I/O efficiently becomes more and more challenging. As we are moving forward, one of the major roadblocks to exascale is how to manipulate, write, read big datasets quickly and efficiently on high-end machines. In this tutorial we will demonstrate I/O practices and techniques that are crucial to achieve high performance. Part I of this tutorial will introduce parallel I/O in general, including file systems, I/O, and middle-ware libraries. We will summarize the lessons and the key techniques that the team gained through years of collaboration with domain scientists at U.S. Department of Energy leadership computing facilities. This experience and knowledge resulted in the creation of ADIOS – the library that makes scaling I/O much easier and flexible. Part II will discuss the concept of ADIOS I/O abstraction, the binary-packed file format, and I/O methods (including streaming) along with the benefits to applications. It will be followed by a hands-on session on how to write/read data, and how to use different I/O componentizations. Part III will show users how to run in-situ visualization using VisIt/Paraview, data transformation plugins and data streaming.
Content Level
- 40% Beginner, teaching attendees the basic of file systems, I/O libraries
- 40% Intermediate, training attendees how to use the basic APIs in these systems
- 20% Advanced, training attendees how to gain high levels of performance on HPC using the systems.
Attendee Requirements
The attendees are required to bring a laptop with at least 20GB free disk space and are required to have basic knowledge of either C or FORTRAN, along with MPI programming. We will provide USB Thumb Drives which has VirtualBox and a complete Linux virtual machine, so that students can have a sample system which they can run on their own laptops. Access to the network is required. |
|
;){kind=link}