In this deck from the 2015 PBS Works User Group, Thomas Leung from the GE Global Research Center presents: HPC Across the Enterprise: How HPC Transforms the Corporate Information Technology Ecosystem. "The commercial world uses significant HPC resources for simulation and product design. An increasing number of HPC systems are deployed in the commercial space and their scale is getting larger and larger. These advanced systems push limits in every aspect of Enterprise IT. Accommodating such systems within the enterprise is a challenge, and there have been many recent changes to enterprise IT infrastructures and architectures resulting from the need to support HPC. Although Commercial HPC is part of the Supercomputing family, Commercial HPC has unique focus areas and challenges. Cloud, Data Lake, Industrial Internet, Internet of Things and Big Data have created a huge impact on HPC systems. Each of these initiatives offers new opportunities to both HPC itself and to the enterprise. HPC systems must change themselves to accept and accommodate these changes. This presentation covers how HPC transforms science in a corporate information technology ecosystem. It also explains the differences between Research HPC and Commercial HPC in terms of architecture, usage, resource planning and challenges." Learn more: http://www.pbsworks.com/pbsug/2015/agenda.aspx Watch the video presentation: https://youtu.be/bpP-HSLvzPg Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

1. Imagination at work
HPC in Enterprise
How HPC transforms the Corporate
Information Technology Ecosystem
Thomas Leung – Sept 10, 2015
make happen
© 2015 General Electric Company - All rights reserved

2. Who am I?
'No one ever changed the world without first being labeled a 'troublemaker‘ ~ Author
Unknown
© 2015 General Electric Company - All rights reserved

3. Agenda
 What is Industrial HPC
 Technology, design and
architecture
 Position of HPC at enterprise
 HPC as a service
 Challenges and future focus
*IDC HPC and ROI Pilot Study 10/2013
© 2015 General Electric Company - All rights reserved
Average of
revenue per
HPC dollar is
$462.4*

4. What is industrial HPC and
difference to Research HPC?
© 2015 General Electric Company - All rights reserved

5. Industrial HPC (aka Commercial / Enterprise HPC)
• Commodity hardware
• Small – large systems
• Co-located data center
• Resource management based on business priorities between business units. (long
run time.)
• Computational process is part of the design workflow.
• High availability and stability with 24 /7 operation. Business critical processes /
projects support
• Multiple years planning with yearly funding / commitment. (in most cases)
• Multiple source of funding with multiple plans/ directions.
• Highly integrated with enterprise IT systems / internal infrastructure.
• Legacy code and latest code support
© 2015 General Electric Company - All rights reserved

6. System is busy all the time….
Industrial HPC systems operates 24 x 7 with over 95% utilization.
© 2015 General Electric Company - All rights reserved

7. HPC expansion & requirements
HPC systems grow all the time with requirements of both types of HPC.
Research HPC
requirements
Enterprise IT
Requirements
Business Requirements
Industrial HPC
requirements
© 2015 General Electric Company - All rights reserved

8. Technology, Design, Architecture

9. Bleeding edge technology focus on scale and speed. HPC system will see more “defects” in future.
Research
HPC
© 2015 General Electric Company - All rights reserved

10. Pull and Push force – on design decision
New technology adoption and wide range of applications support made HPC system design harder.
Vendors: I have the
latest and greatest
technology. Trust me!
It works.
Users: Why aren’t
we using the latest
greatest
technology?
Cloud provider:
Cloud can do
everything now.Finance:
Faster,
More,
Cheaper are
always
better!
Business Leaders:
It is stable now, I
don’t want
anything new and
risky.
Engineer: I have
to support codes
developed 20
years ago.
Managers:
Downtime is not
acceptable.
© 2015 General Electric Company - All rights reserved

11. HPC changes enterprise cyber security
More collaborators, high bandwidth, low latency connections leads to new challenges.
HPC systems
Enterprise
Defense
Global Users
Machines
National Labs
Enterprise
Network
Enterpris
e Network
© 2015 General Electric Company - All rights reserved
HPC systems

12. HPC is moving Enterprise IT
Networking Storage Infrastructure
/Cloud
Application
Mostly 10Gb NAS , Iscsi, FC
storage
Co-located data center
with low power
requirement
Java, .net
10Gb / 40Gb
Ethernet only
Everything is
backup
Virtual machine Max 244GB ram
(EC2)
1 - 10Gb internet
connection
TB – PB storage VM hypervisor controls
resource
No MPI
programming, Java
IaaS only IaaS only PaaS only SaaS only
HPC
40Gb / 100Gb
Ethernet with FDR /
ERR Infiniband
Some workflow
need TB ram
PB storage with fast
IO
Fortran,
Multi thread with
MPI Programming
Resource Mgr
controls
PaaS / SaaS
Enterprise IT
© 2015 General Electric Company - All rights reserved

13. Marriage between Big Data and HPC……
Distributed storage and processing framework in a stable & scalable environment.
Big Data system makes “data” more accessible to HPC. It enhances the simulation efficiency and effectiveness.
Big Data
HPC
© 2015 General Electric Company - All rights reserved

14. Position of HPC in Enterprise
ERP / Enterprise
systems
HPC
inside
Past: ERP / Enterprise systems are
the center of the business workflow /
decisions.
HPC is just one of the engineering
tool kits.
Future: Enterprise Hadoop platform
becomes the “CENTER” of all decisions
and workflow.
© 2015 General Electric Company - All rights reserved

15. Understand availability and redundancy of HPC
Single Cray system outage
Recover 100TB data from Tape
Fixing incorrect job submission
Fixing Job submission errors
HPC issues associate with system down
time
Not a lot of system issues can be fixed in less than 1
min.
This is the area of improvement!
© 2015 General Electric Company - All rights reserved

16. Redundancy improves availability?
© 2015 General Electric Company - All rights reserved

17. Please help me solve these problem.
© 2015 General Electric Company - All rights reserved

18. Our Future……
© 2015 General Electric Company - All rights reserved

19. HPC team evolving
HPCers are hard to get. Talent search for HPC positions are harder.
system administrators, storage administrator, database administrators, software
developers, network administrators, solution architect, technologist, business
analyst, project managers, researcher, computer scientist, leaders.
Modern HPC
team
HPC team is usually a group of
geeks…..
Traditional
HPC team
Future key requirements:
• Big Data knowledge & hands on experience – able to build production hadoop system and support
operation
• Programming – manage systems with provided API
• Application support – Optimization, New platform adoption and migration
• Data science expertise, business expertise
• Experience with large systems – Really large systems
• Industry specific domain knowledge – Enhance partnership with users and business.
© 2015 General Electric Company - All rights reserved

20. Focus areas for industrial HPC
High availability
• Establish automated
failover capability
• Enhance Job level
recovery after system
failure
• Don’t forget all
surrounding IT services.
24 x 7 service
• 24 x 7 system
monitoring
• 24 x 7 user support
• Full debug and
compiling environment
available
Multiple systems strategy vs
single system strategy
• Easy for compute resource
but hard for storage
solutions.
• Multiple location planning
Automation
• HPC service shall be
automated as Web
service
• API driven management
activity / functions will
increase
© 2015 General Electric Company - All rights reserved
Reduce cost
• Software License
cost
• Storage cost
• Platform cost
• $ per CPU cores

21. Call for actions
Resource
Manager
Big
Data
VDI
HPCCloud
Storage
© 2015 General Electric Company - All rights reserved

22. Q&A
© 2015 General Electric Company - All rights reserved

23. Backup
© 2015 General Electric Company - All rights reserved

24. Research HPC
• Commodity hardware
• Huge Systems
• Dedicated Data Center
• Resource management based on fair share between users /
programs. (relatively short run time.)
• Handle computational process / workflow only.
• Extended MTBF with parts fail all the time.
• Multiple years planning with one-time funding / commitment.
• Centralized source of funding – University / Government.
• Independent from IT systems / infrastructure.
• latest code support
© 2015 General Electric Company - All rights reserved

25. Size of HPC systems
Small
~100 cores
Eg:
Workgroup
<$100k
Medium
~500 cores
Eg:
Departmental
$100k - $250k
Large
~1000 cores
Eg:
Divisional
$250k - $500k
X- Large
>2000 cores
Eg:
Supercomputer
>$500k
XX – Large
>10000 cores
Commercial HPC
Research HPC
Research HPC covers National Lab, Government Agency
and universities
Industrial, Finance, Oil &Gas, Manufacturing companies
© 2015 General Electric Company - All rights reserved

26. Image citation / source
CDN Locations | EdgeCast, www.edgecast.com
Gauging Skills | Autism After 16www.autismafter16.com
Service Availability: I'll See Your Five 9s and Raise You One More ...aits.org
MySQL Master/Slave Management with mysql-master-ha | CRGNetwww.cirgan.net
Illustration Depicting Two Triangular Warning Road Signs With A ...www.shutterstock.com
Review: Thomas Pynchon's “Bleeding Edge” | The American Readertheamericanreader.com
Can Crowd Testing replace an Internal Quality Team? | 99testsblog.99tests.com
VIDEO: Find the balance between push and pull can help your ...www.teamsupport.com
FCoE Adoption Curve - Wikibonwikibon.org
Online Customers are Like Toddlers, They Don't Like Getting ...www.contenthook.com
Stkit MOC] Iron Man Hulkbuster statue: A LEGO® creation by St Kit ...www.mocpages.com
LEGO Ideas - Iron Man Hulkbuster armorideas.lego.com
How does an ERP system work? - Quorawww.quora.com
hadoops-role-in-enterprise-architecture-7-638.jpg?cb=1364477105www.slideshare.net
Get Free or Cheap Legal Fees Help in the USwww.grantfundingexpert.org
Migration | QualITasqualitas.co.id
Respect for system administratorsgeekshumor.com
Sincerely Stephanie: The Digit-al Dozen September Spectacular: Geekswww.sincerelystephanienails.com
Huntington's Disease Youth Organization - Young adults - Ask a ...en.hdyo.org
ABigFave | Flickr - Photo Sharing!www.flickr.com
Peak Hour Protests Immobilise Bangalore - The New Indian Expresswww.newindianexpress.com
Competitive Cost Benchmarking Software for Manufacturerswww.mtisystems.com
Zenith Software announces India Expansion PlansDATAQUESTwww.dqindia.com
© 2015 General Electric Company - All rights reserved