Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

HPC Computing Trends

853 views

Published on

Chris Willard from Intersect360 Research gave this talk at the Stanford HPC Conference.

"Intersect360 Research returns with an annual deep dive into the trends, technologies and usage models that will be propelling the HPC community through 2018 and beyond. Emerging areas of focus and opportunities to expand will be explored along with insightful observations needed to support measurably positive decision making within your operations."

Intersect360 Research's deep knowledge of HPC, coupled with strong marketing and consulting expertise, results in actionable intelligence for the HPC industry - insights and advice that allow customers to make decisions that are measurably positive to their business. The company's end-user-focused research is inclusive from both a technology perspective and a usage standpoint, allowing Intersect360 Research to provide its clients with total market models that include both traditional and emerging HPC applications.

Watch the video: https://insidehpc.com/category/hpc-software/cloud-hpc/

Learn more: http://www.intersect360.com/
and
http://hpcadivosorycouncil.com

Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Published in: Technology
  • Be the first to comment

  • Be the first to like this

HPC Computing Trends

  1. 1. HPC Market Update HPC Advisory Council, February 2018 Addison Snell Christopher Willard, Ph.D. Laura Segervall
  2. 2. Intersect360 Research in 2017 10 year history of HPC analyst business Covering “high performance data center” markets, including traditional HPC and supercomputing, high-performance enterprise, cloud, big data, and hyperscale New hyperscale advisory service Weekly podcast, “This Week in HPC” Market research partners of TOP500.org, contributing editors to The Next Platform, media partnerships with HPCwire, insideHPC Sponsors of HPC Advisory Council
  3. 3. HPC Market, 2016 - 2021 • Total worldwide HPC market (servers, storage, software, etc.) reached $35.6 billion in 2016, up 3.5% from 2015. • Servers were the largest component, reaching $11.5 billion (3.3% growth over 2015). • Total market is forecast to grow to $43.9 billion in 2021, 4.3% CAGR. • Growth has been driven significantly by commercial HPC markets; this continues over the next five years.
  4. 4. A Caution About Forecasting “When you create a forecast, it has to be bound by what is realistic, whereas reality itself has no such limitations.” – Chris Willard Chief Research Officer Intersect360 Research
  5. 5. HPC Market Revenue by Product Class ($K)
  6. 6. Total HPC Revenue by Product Category • Servers will lead the market in terms of absolute revenue growth, adding about $2.9 billion to the market by 2021 • Cloud (aka utility computing) growth is based on a low starting point; by 2021 the segment will capture only about 3% of total revenue • Software revenue is challenged by a combination of: growth in open source options, consolidation, bundling with hardware, and using software as a loss leader to promote other products • Storage is the highest-growth segment other than cloud and will add roughly $2 billion in growth in the next five years
  7. 7. Total HPC Market Revenue • 2011-16 actual, 2017-21 forecast • The HPC market will grow at a 4.3% CAGR between 2016 and 2021 • Revenues in HPC will grow by $8.3 billion to about $43.9 billion • Industrial HPC is the driving engine of the market High Performance Computing Market by Economic Sectors -- Total Market Forecast by Economic Sectors ($M) 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 CAGR Government 7,396 8,453 8,425 8,418 8,936 9,217 9,363 9,517 9,702 9,920 10,194 2.0% Academia 4,888 5,768 6,028 5,976 6,470 6,619 6,612 6,609 6,612 6,629 6,667 0.1% Industry 13,385 15,478 16,694 17,142 18,978 19,751 20,949 22,274 23,727 25,322 27,083 6.5% Total 25,669 29,698 31,147 31,536 34,384 35,587 36,923 38,400 40,042 41,871 43,944 4.3%
  8. 8. HPC 2016 Revenue by Vertical Academic, 18.6% Government, 25.9% Commercial, 55.5%
  9. 9. HPC 2016 Revenue by Vertical Academic, 18.6% Government, 25.9% Commercial, 55.5%
  10. 10. Commercial HPC Growth Five-Year Forecast Revenue Growth in HPC, over 2016 Base Intersect360 Research, 2017
  11. 11. Potential Five-Year Growth of HPC “Solve”: U.S. Council on Competitiveness, with Intersect360 Research, 2014 Free Download: http://www.compete.org/reports/all/2695-solve
  12. 12. Best Metric to Justify HPC Investments “Solve”: U.S. Council on Competitiveness, with Intersect360 Research, 2014 Free Download: http://www.compete.org/reports/all/2695-solve
  13. 13. HPC System Market Share
  14. 14. HPC Storage Revenue Share Model • Includes disk (NAS, SAN, scratch, etc.), flash, tape • Revenue assigned to the final seller (may be OEM from other vendor)
  15. 15. The Hyperscale Market • Included since our 2007 HPC methodology as “ultrascale internet” • In 2015, recognized “Hyperscale” as distinct market in proximity to HPC; began process of separating it from HPC • Defined hyperscale methodology with market research and feedback from interested clients • Published definitions and methodology report in April 2016
  16. 16. Hyperscale Market Definition The hyperscale market consists of arbitrarily scalable, web-facing application infrastructure that is distinct from general IT investment
  17. 17. HPC vs. Hyperscale • Primary focus is performance • Large jobs • Is still “HPC” on only a few nodes – there is entry-level HPC • Largest supercomputers are >$100 million • Primary focus is scalability • Many jobs • At lower levels it is just enterprise computing – no entry-level • Largest hyperscale installations are >$1 billion Similarities Application infrastructure that is distinct from general enterprise Applications are arbitrarily scalable Top tier users push boundaries and influence technology throughout industry Differences HPC Hyperscale
  18. 18. Hyperscale Market Size and Tiers • Intersect360 Research has a tiered model for the hyperscale market, as follows: 2016 estimates Intersect360 Research, 2017
  19. 19. AI / Deep Learning • In 2016, Intersect360 Research placed AI in the hyperscale market – Similar to but distinct from HPC – Low precision, intensely parallel, strong affinity to public cloud • DL is not a vertical market. It is an algorithmic or computational approach • We are monitoring spending patterns – Are budgets increasing? – What is the effect on configurations? – On premise versus cloud
  20. 20. Sizing Deep Learning • We have looked at the size of the deep learning market opportunity from several angles: – Supplier-side information on market expansion – Demand-side market analysis on acquisitions – Top-down analysis of the hyperscale market – Analyst opinion • Our independent methods have produced consistent views of the current opportunity in deep learning
  21. 21. Deep Learning Size and Growth • According to our analysis, the deep learning market opportunity is $2.0B to $2.5B for 2016 – Represents about 6.5% of the hyperscale market – Data presented one-on-one to clients of Hyperscale advisory service • We expect growth rate to be very high for the next two years, 2017-18, close to 100% Y/Y • After that, growth will moderate, and possibly even contract by end of five-year period
  22. 22. The Top of All Things in HPC, 2017 Site Census Snapshot Analysis January 2018 Christopher Willard, Ph.D., Addison Snell, Laura Segervall
  23. 23. Introduction • The purpose of this report is to provide the top suppliers, packages, or categories within specific segments. – Shares reported are based on Site Census survey results, not total sales. – Top is defined as companies accounting for 50% market share or the top five, whichever comes first. • Our goals for collecting and maintaining site census data include: – Developing demand-side data for tracking and modeling fundamental characteristics of the HPC market in such areas as computer room contents, vendor positioning, technology adoption, user purchase patterns, etc. – Collecting and comparing demand-side data to supply-side data in order to accurately frame the entire HPC ecosystem. – Tracking trends in these characteristics over time. – Supporting our HPC market-sizing and forecasting models. • Survey was conducted in the third quarter of 2017. • Responses were received from 454 sites which responded to at least one section of our last three site census surveys. Entries reported in two or more surveys were counted once.
  24. 24. What is “Top” • Given the wide range of product categories and technologies combined with the variable number of vendors selling into any category a strict count base definition of “top” (e.g. top 10, top 3) is not meaningful in all cases. • We define “top” as the fewest companies necessary to account for 50% share in a given category or the top five, whichever comes first. • We ignore this rule for some software categories because of either too few category options or too many companies needed to get to 50% • Note: These charts indicate responses to our surveys of the HPC user community. Survey responses are not always indicative of market share by revenue. For inquiries on revenue share, contact Intersect360 Research.
  25. 25. Top System Suppliers • HPE, Dell EMC, and Lenovo remained the top three HPC system suppliers, accounting for 45% share of all mentions. • At commercial sites, HPE accounted for about 36% of the reported installations, compared to Dell, the next closest supplier, with about 13% of reported installations.
  26. 26. Top Accelerator* Supplier • 40% of all systems reported have accelerators installed. • NVidia was installed in more than three-fourths of the systems with accelerators. • Intel Xeon Phi was installed in about 15% of systems yet accounted for 43% of the total number of accelerators reported, due to several high density system installations. • FPGAs are installed in less than 5% of the systems. Top Suppliers by Share of Systems (N=310 systems) Top Suppliers by Share of Total Accelerators (N=84,597 accelerators) * Accelerators includes GPUs, co-processors, and FPGAs
  27. 27. Top Storage System Suppliers • DDN’s share of reported storage systems in our Site Census survey increased by 2% between 2016 and 2017. • IBM’s share of mentions also grew by about 1%, while Dell EMC’s share remained flat. • Amazon Web Services continues to be the top named supplier of cloud storage. Cloud storage accounted for about 4% of the mentions.
  28. 28. Top File Systems reported with storage systems Top Storage Management Software Suppliers Top Storage Management Software Packages • IBM was the leading supplier with its multiple storage management packages (Spectrum Scale, Spectrum Protect, HPSS). Spectrum Scale accounts for about 70% of IBM’s mentions. • Including all derivatives of Lustre and Spectrum Scale, the share for both is almost the same. Storage Management Software Package No. of Storage Systems Share of Systems Lustre and derivatives 76 24% Spectrum Scale and derivatives 75 24% Panfs 16 5% Data OnTap 11 3% NFS 11 3% Other 128 40% Total 317 100% Source: Intersect360 Research, 2017 Storage Management Software Supplier No. of Storage Systems Share of Systems IBM 102 25% DDN 38 9% Dell EMC 28 7% Intel 28 7% Lustre.org 26 6% Other 181 45% Total 403 100% Source: Intersect360 Research, 2017
  29. 29. Top Interconnects/Network Suppliers • In all locales, two vendors account for over 50% of the mentions, with Mellanox being the only supplier in all three locales. Entries not specifying supplier were not included in total. • Mellanox gained share in every segment, with the largest increase (5%) being for storage networks. • Intel’s Omni-Path increased its presence for both interconnects and storage, accounting for roughly 2% of all mentions in both locales. Top System Interconnect Suppliers Top Storage Interconnect Suppliers Top LAN Suppliers
  30. 30. Source of HPC Middleware • ISV and Open Source account for about 96% of all primary middleware reported. • Commercial sites have a preference for ISV supplied middleware. • Top ISVs were Intel with its compilers and programming environment software, MathWorks with Matlab, and IBM with primarily its job and storage management software. Source of Primary Middleware by Sector Source of Primary Middleware Reported
  31. 31. Top Primary Middleware Categories • Programming Environment and Compilers account for 56% of all mentions. Programming Environment includes mathematical applications, parallel tools, and other tools. • Top package and share of mentions for each of the top three usage categories were: • Programming Environments: MathWorks Matlab, 14.8% • Job Management: SchedMD Slurm, 21.2% • Compiler: GNU Compiler Collection, 19.8% • Each top package accounted for double the mentions of the second most mentioned package in these three categories.
  32. 32. • About 76% of applications reported were Open Source or ISV software (generally-available applications). • Top applications are primarily used for traditional HPC application – chemistry, fluid dynamics, and structural analysis -- with the exception of WRF, a weather forecasting application. • ISV and in-house packages are still the majority at commercial sites, representing about 85% of the packages reported. • Open source accounts for more than 50% of the packages reported at academic and government sites. Top Generally-Available Applications
  33. 33. Summary • HPE became the largest HPC vendor reported in our Site Census surveys, accounting for roughly 21% of all mentions. Much of this increase in mentions is due to the acquisition of SGI systems. • NVidia continues to dominate the accelerator market in system installations. Their lead in total processors installed shrank to less than 5% with installation of several large Intel Phi-based systems. • Mellanox gained share of mentions in every locale, with storage showing the largest gain. • Middleware profile remained consistent with the prior year with programming environment, job management, and compilers being the top three subcategories. • Top ten generally-available application packages continue to primarily be in traditional segments – chemistry, fluid dynamics, and structural analysis.
  34. 34. HPC Market Update SC17 – November 2017 Addison Snell Christopher Willard, Ph.D. Laura Segervall

×