June 22–26, 2014
Leipzig, Germany

Session Details

Name: New Benchmarks for Ranking HPC Systems
Time: Thursday, June 26, 2014
09:00 am - 10:30 am
Room:   Hall 2
CCL - Congress Center Leipzig
Breaks:07:30 am - 10:30 am Welcome Coffee
Chair:   Erich Strohmaier, LBNL
Abstract:   The TOP500 is a unique, surprisingly popular, and successful project to define and rank the largest supercomputers worldwide. It provides an invaluable resource for research about the adoption of new technologies in HPC. The ranking is based on performance numbers obtained with the HPL (High Performance Linpack) benchmark. Since its inception in 1993, the biggest criticism and suggestion for improvement was to consider a "more realistic" benchmark for its ranking. This, however, has proven a difficult task. Early attempts by the TOP500 authors to use some of the NAS Parallel Benchmarks failed due to the lack of measurements on a similar large variety of systems as HPL enjoys. This held true for any other popular benchmark since such as the HPCC suite. Attempts by other groups such as the GRAPH500 are facing similar difficulties. In this session we will highlight three new attempts for defining complementary performance measures useful in or next to the TOP500.

  • Jack Dongarra – the father of HPL itself – and Mike Heroux will present first experiences and new improvements for his newest benchmark called HPCG;
  • Tilmann Rabl will talk about a project to create an alternative listing for "Big Data Systems";
  • Mark Adams will present an new benchmark called HPGMG based on algebraic multigrid solvers, which are widely used in HPC and provide more balanced loads for all parts of HPC systems.
Presentations: HPCG: One Year Later
09:00 am - 09:30 am
  Jack Dongarra, University of Tennessee & ORNL
  Mike Heroux, Sandia National Laboratories
Crafting Benchmarks for Big Data
09:30 am - 10:00 am
  Tilmann Rabl, University of Toronto & bankmark
A Proposal for a New Top500 Metric
10:00 am - 10:30 am
  Mark Adams, LBNL