Slide from the TOP500 BoF session @ SC17 in Denver

1. Highlights of
the 50th
TOP500 List
SC17,
Denver,
November 14,
2017
Erich
Strohmaier

2. 41ST LIST: THE TOP10
# Site Manufacturer Computer Country Cores Rmax
[Pflops]
Power
[MW]
1
National Supercomputing
Center in Wuxi
NRCPC
Sunway TaihuLight
NRCPC Sunway SW26010,
260C 1.45GHz
China 10,649,600 93.0 15.4
2
National University of
Defense Technology
NUDT
Tianhe-2
NUDT TH-IVB-FEP,
Xeon 12C 2.2GHz, IntelXeon Phi
China 3,120,000 33.9 17.8
3
Swiss National Supercomputing
Centre (CSCS)
Cray
Piz Daint
Cray XC50,
Xeon E5 12C 2.6GHz, Aries, NVIDIA Tesla P100
Switzerland 361,760 19.6 2.27
4
Japan Agency for Marine-Earth
Science and Technology
ExaScaler
Gyoukou
ZettaScaler-2.2 HPC System,
Xeon 16C 1.3GHz, IB-EDR, PEZY-SC2 700Mhz
Japan 19,860,000 19.1 1.35
5
Oak Ridge
National Laboratory
Cray
Titan
Cray XK7,
Opteron 16C 2.2GHz, Gemini, NVIDIA K20x
USA 560,640 17.6 8.21
6
Lawrence Livermore
National Laboratory
IBM
Sequoia
BlueGene/Q,
Power BQC 16C 1.6GHz, Custom
USA 1,572,864 17.2 7.89
7
Los Alamos NL /
Sandia NL
Cray
Trinity
Cray XC40,
Intel Xeon Phi 7250 68C 1.4GHz, Aries
USA 979,968 14.1 3.84
8
Lawrence Berkeley
National Laboratory
Cray
Cori
Cray XC40,
Intel Xeons Phi 7250 68C 1.4 GHz, Aries
USA 622,336 14.0 3.94
9
JCAHPC
Joint Center for Advanced HPC
Fujitsu
Oakforest-PACS
PRIMERGY CX1640 M1,
Intel Xeons Phi 7250 68C 1.4 GHz, OmniPath
Japan 556,104 13.6 2.72
10
RIKEN Advanced Institute for
Computational Science
Fujitsu
K Computer
SPARC64 VIIIfx 2.0GHz,
Tofu Interconnect
Japan 795,024 10.5 12.7

3. AVERAGE SYSTEM AGE
0
5
10
15
20
25
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
Age[Months]
7.6 month

4. 0
20
40
60
80
100
1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016
RANK AT WHICH HALF OF
TOTAL PERFORMANCE IS ACCUMULATED

5. PERFORMANCE DEVELOPMENT
1.00E-01
1.00E+00
1.00E+01
1.00E+02
1.00E+03
1.00E+04
1.00E+05
1.00E+06
1.00E+07
1.00E+08
1.00E+09
1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016
59.7 GFlop/s
422 MFlop/s
1.17 TFlop/s
93 PFlop/s
549 TFlop/s
845 PFlop/s
SUM
N=1
N=500
1 Gflop/s
1 Tflop/s
100 Mflop/s
100 Gflop/s
100 Tflop/s
10 Gflop/s
10 Tflop/s
1 Pflop/s
100 Pflop/s
10 Pflop/s
1 Eflop/s

6. PERFORMANCE DEVELOPMENT
1.00E-01
1.00E+01
1.00E+03
1.00E+05
1.00E+07
1.00E+09
1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016
June 2008
June 2013
SUM
N=1
N=500
59.7 GFlop/s
422 MFlop/s
1.17 TFlop/s
93 PFlop/s
549 TFlop/s
845 PFlop/s
1 Gflop/s
1 Tflop/s
100 Mflop/s
100 Gflop/s
100 Tflop/s
10 Gflop/s
10 Tflop/s
1 Pflop/s
100 Pflop/s
10 Pflop/s
1 Eflop/s
10 Eflop/s

7. PROJECTED PERFORMANCE DEVELOPMENT
1.00E-01
1.00E+00
1.00E+01
1.00E+02
1.00E+03
1.00E+04
1.00E+05
1.00E+06
1.00E+07
1.00E+08
1.00E+09
1.00E+10
1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016 2018 2020
SUM
N=1
N=500
1 Gflop/s
1 Tflop/s
100 Mflop/s
100 Gflop/s
100 Tflop/s
10 Gflop/s
10 Tflop/s
1 Pflop/s
100 Pflop/s
10 Pflop/s
1 Eflop/s
10 Eflop/s

8. ANNUAL PERFORMANCE INCREASE
OF THE TOP500
1
1.2
1.4
1.6
1.8
2
2.2
2.4
2.6
1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016
Moore’s Law
TOP500
TOP500: Averages

9. United
States, 29%
China, 40%
Japan, 7%
Germany, 4%
France, 4%
United
Kingdom,
3%
Italy, 1%
Netherlands, 1% Others, 11% United States
China
Japan
Germany
France
United Kingdom
Italy
Netherlands
Others
COUNTRIES / SYSTEM SHARE

10. 0
100
200
300
400
500
1993
1995
1997
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
China
Korea, South
Italy
Canada
France
United Kingdom
Germany
Japan
United States
COUNTRIES

11. 0
1
10
100
1,000
10,000
100,000
2000
2002
2004
2006
2008
2010
2012
2014
2016
TotalPerformance[Tflop/s]
US
EU
Japan
China
PERFORMANCE OF COUNTRIES

12. 0
100
200
300
400
500
1993
1995
1997
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
Russia
China
Europe
Japan
USA
PRODUCERS

13. HPE, 122,
24%
Lenovo, 81,
16%
Inspur, 56, 11%
Cray Inc., 53,
11%
Sugon, 51, 10%
IBM, 19, 4%
Bull, 17, 4%
Huawei, 19, 4%
Dell EMC, 16, 3%
Fujitsu, 12, 2%
Penguin
Computing, 10,
2%
Others, 44, 9% HPE
Lenovo
Inspur
Cray Inc.
Sugon
IBM
Bull
Huawei
Dell EMC
VENDORS / SYSTEM SHARE
# of systems, % of 500

14. HPE, 165,
20%
Lenovo, 128,
15%
Cray Inc., 94,
11%Sugon, 77, 9%IBM, 54, 6%
Inspur, 51, 6%
Huawei, 44, 5%
Bull, 40, 5%
Dell, 39,
5%
Fujitsu, 29, 3%
Penguin C., 24,
3%
NRCPC, 14, 2% others, 86, 10% HPE
Lenovo
Cray Inc.
Sugon
IBM
Inspur
Huawei
Bull
Dell
VENDORS / PERFORMANCE SHARE
Sum of Pflop/s, % of whole list

15. Cray Inc., 18,
36%
HPE, 7,
14%
IBM, 5, 10%
Fujitsu, 4, 8%
Lenovo, 2, 4%
Penguin
Computing, 2,
4%
Bull, 2, 4%
Others, 10,
20%
Cray Inc.
HPE
IBM
Fujitsu
Lenovo
Penguin Computing
Bull
Others
VENDORS (TOP50) / SYSTEM SHARE

16. 0
10
20
30
40
50
60
70
80
90
100
110
120
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
Systems
PEZY-SC
Kepler/Phi
Xeon Phi Main
Intel Xeon Phi
Clearspeed
IBM Cell
ATI Radeon
Nvidia Volta
Nvidia Pascal
Nvidia Kepler
Nvidia Fermi
ACCELERATORS

17. PERFORMANCE SHARE OF ACCELERATORS
0%
5%
10%
15%
20%
25%
30%
35%
40%
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017
FractionofTotalTOP500
Performance
Xeon Phi Main
Accelerators

18. • Both projects worked for several year to unify measurement and
reporting approaches
(EEHPC-WG: Energy-Efficient HPC Working Group ).
• Ultimately this lead us to combine data collection and curation in one
site and system.
• Both lists will continue to be published at the same time (ISC and SC).
• We are working on combining past data-sets and sites.
• Both sites will be hosted and maintained by the ISC Group.
TOP500 - GREEN500

19. Computer
Rmax/
Power
Shoubou system B, ZettaScaler-2.2 Xeon 16C 1.3GHz Infiniband EDR PEZY-SC2 17.0
Suiren2, ZettaScaler-2.2 Xeon 16C 1.3GHz Infiniband EDR PEZY-SC2 16.8
Sakura, ZettaScaler-2.2 Xeon 8C 2.3GHz Infiniband EDR PEZY-SC2 16.7
DGX Saturn V, NVIDIA DGX-1 Volta36 Xeon 20C 2.2GHz Infiniband EDR Tesla V100 15.1*
Gyoukou, ZettaScaler-2 Xeon 16C 1.3GHz Infiniband EDR PEZY-SC2 14.2
Tsubame 3.0, SGI ICE XA Xeon 14C 2.4GHz Intel Omni-Path Tesla P100 SXM2 13.7*
AIST AI Cloud, NEC 4U-8GPU Xeon 10C 1.8GHz Infiniband EDR Tesla P100 SXM2 12.7
RAIDEN GPU subsystem, NVIDIA DGX-1 Xeon 20C 2.2GHz Infiniband EDR Tesla P100 10.6
Wilkes-2, Dell C4130 Xeon 12C 2.2GHz Infiniband EDR Tesla P100 10.4
Piz Daint, Cray XC50 Xeon 12C 2.6GHz Aries interconnect Tesla P100 10.4*
MOST ENERGY EFFICIENT ARCHITECTURES
[Gflops/Watt]* Efficiency based on Power optimized HPL runs of equal size to TOP500 run.

20. POWER EFFICIENCY
0
1,000
2,000
3,000
4,000
5,000
6,000
2008 2009 2010 2011 2012 2013 2014 2015 2016 2017
Linpack/Power[Gflops/kW]
TOP10
TOP50
TOP500

21. ENERGY EFFICIENCY
0
2,000
4,000
6,000
8,000
10,000
12,000
14,000
16,000
18,000
2008 2009 2010 2011 2012 2013 2014 2015 2016 2017
Linpack/Power[Gflops/kW]
TOP500 Average
Max-Efficiency ZettaScaler-2.2
Tsubame 3.0
BlueGene/Q
Cell
Mic
AMD FirePro
Tsubame KFC
NVIDIA K20x – K80
ZettaScaler-1.6 c
DGX SaturnV

22. • Longstanding interest to augment HPL with other
benchmarks.
• Publishing HPCG numbers together with the TOP500.
• Submission still go to Jack and Mike first.
• 61 HPCG entries which made the TOP500
(not necessarily the top61 HPCG measurements!).
– 47 last June
• Ability to resort and filter on our web-lists.
• Top10 … Mike Heroux
TOP500 - HPCG

23. 41ST LIST: THE TOP10
# T Site Manufacturer Computer Country
HPCG
[Pflop/s]
Rmax
[Pflop/s]
HPCG/
Peak
HPCG/
HPL
1 10
RIKEN Advanced Institute for
Computational Science
Fujitsu
K Computer
SPARC64 VIIIfx 2.0GHz,
Tofu Interconnect
Japan 0.6027 10.5 5.3% 5.7%
2 2
National University of
Defense Technology
NUDT
Tianhe-2
NUDT TH-IVB-FEP,
Xeon 12C 2.2GHz, IntelXeon Phi
China 0.5801 33.9 1.1% 1.7%
3 7
Los Alamos NL /
Sandia NL
Cray
Trinity
Cray XC40,
Intel Xeon Phi 7250 68C 1.4GHz, Aries
USA 0.5461 14.1 1.2% 3.9%
4 3
Swiss National Supercomputing
Centre (CSCS)
Cray
Piz Daint
Cray XC50,
Xeon E5 12C 2.6GHz, Aries, NVIDIA Tesla P100
Switzerland 0.4864 19.6 1.9% 2.5%
5 1
National Supercomputing
Center in Wuxi
NRCPC
Sunway TaihuLight
NRCPC Sunway SW26010,
260C 1.45GHz
China 0.4808 93.0 0.4% 0.5%
6 9
JCAHPC
Joint Center for Advanced HPC
Fujitsu
Oakforest-PACS
PRIMERGY CX1640 M1,
Intel Xeons Phi 7250 68C 1.4 GHz, OmniPath
Japan 0.3855 13.6 1.5% 2.8%
7 8
Lawrence Berkeley
National Laboratory
Cray
Cori
Cray XC40,
Intel Xeons Phi 7250 68C 1.4 GHz, Aries
USA 0.3554 14.0 1.3% 2.5%
8 6
Lawrence Livermore
National Laboratory
IBM
Sequoia
BlueGene/Q,
Power BQC 16C 1.6GHz, Custom
USA 0.3304 17.2 1.6% 1.9%
9 5
Oak Ridge
National Laboratory
Cray
Titan
Cray XK7,
Opteron 16C 2.2GHz, Gemini, NVIDIA K20x
USA 0.3223 17.6 1.2% 1.8%
10 13
GSIC Center, Tokyo Institute of
Technology
HPE
Tsubame 3.0
SGI ICE XA,
Xeon E5 14C 2.4GHz, OmniPath, NVIDIA P100
Japan 0.1886 8.1 1.6% 2.3%

24. SC17 HPCG HIGHLIGHTS
• Top 10 machine experience a serious rearrangement.
• US returns to the Top 3 club.
• Trinity gets an upgrade and improves its HPCG score from
180 TF to 550 TF
• Piz Daint passes TaihuLight with improved result.
• TSUBAME 3.0 submits a new result with 4x improvement in
performance.
• Mare Nostrum 4 shows HPCG performance on Intel Skylake cores.
• First Volta results from the recently released DGX-1V system.
• International Space Station computer by HPE submits HPCG result!