How HPC Transforms the Corporate Information Technology Ecosystem

1. Imagination at work HPC in Enterprise How HPC transforms the Corporate Information Technology Ecosystem Thomas Leung – Sept 10, 2015 make happen © 2015 General Electric Company - All rights reserved

3. Agenda  What is Industrial HPC  Technology, design and architecture  Position of HPC at enterprise  HPC as a service  Challenges and future focus *IDC HPC and ROI Pilot Study 10/2013 © 2015 General Electric Company - All rights reserved Average of revenue per HPC dollar is $462.4*

5. Industrial HPC (aka Commercial / Enterprise HPC) • Commodity hardware • Small – large systems • Co-located data center • Resource management based on business priorities between business units. (long run time.) • Computational process is part of the design workflow. • High availability and stability with 24 /7 operation. Business critical processes / projects support • Multiple years planning with yearly funding / commitment. (in most cases) • Multiple source of funding with multiple plans/ directions. • Highly integrated with enterprise IT systems / internal infrastructure. • Legacy code and latest code support © 2015 General Electric Company - All rights reserved

7. HPC expansion & requirements HPC systems grow all the time with requirements of both types of HPC. Research HPC requirements Enterprise IT Requirements Business Requirements Industrial HPC requirements © 2015 General Electric Company - All rights reserved

8. Technology, Design, Architecture

10. Pull and Push force – on design decision New technology adoption and wide range of applications support made HPC system design harder. Vendors: I have the latest and greatest technology. Trust me! It works. Users: Why aren’t we using the latest greatest technology? Cloud provider: Cloud can do everything now.Finance: Faster, More, Cheaper are always better! Business Leaders: It is stable now, I don’t want anything new and risky. Engineer: I have to support codes developed 20 years ago. Managers: Downtime is not acceptable. © 2015 General Electric Company - All rights reserved

11. HPC changes enterprise cyber security More collaborators, high bandwidth, low latency connections leads to new challenges. HPC systems Enterprise Defense Global Users Machines National Labs Enterprise Network Enterpris e Network © 2015 General Electric Company - All rights reserved HPC systems

12. HPC is moving Enterprise IT Networking Storage Infrastructure /Cloud Application Mostly 10Gb NAS , Iscsi, FC storage Co-located data center with low power requirement Java, .net 10Gb / 40Gb Ethernet only Everything is backup Virtual machine Max 244GB ram (EC2) 1 - 10Gb internet connection TB – PB storage VM hypervisor controls resource No MPI programming, Java IaaS only IaaS only PaaS only SaaS only HPC 40Gb / 100Gb Ethernet with FDR / ERR Infiniband Some workflow need TB ram PB storage with fast IO Fortran, Multi thread with MPI Programming Resource Mgr controls PaaS / SaaS Enterprise IT © 2015 General Electric Company - All rights reserved

13. Marriage between Big Data and HPC…… Distributed storage and processing framework in a stable & scalable environment. Big Data system makes “data” more accessible to HPC. It enhances the simulation efficiency and effectiveness. Big Data HPC © 2015 General Electric Company - All rights reserved

14. Position of HPC in Enterprise ERP / Enterprise systems HPC inside Past: ERP / Enterprise systems are the center of the business workflow / decisions. HPC is just one of the engineering tool kits. Future: Enterprise Hadoop platform becomes the “CENTER” of all decisions and workflow. © 2015 General Electric Company - All rights reserved

15. Understand availability and redundancy of HPC Single Cray system outage Recover 100TB data from Tape Fixing incorrect job submission Fixing Job submission errors HPC issues associate with system down time Not a lot of system issues can be fixed in less than 1 min. This is the area of improvement! © 2015 General Electric Company - All rights reserved

19. HPC team evolving HPCers are hard to get. Talent search for HPC positions are harder. system administrators, storage administrator, database administrators, software developers, network administrators, solution architect, technologist, business analyst, project managers, researcher, computer scientist, leaders. Modern HPC team HPC team is usually a group of geeks….. Traditional HPC team Future key requirements: • Big Data knowledge & hands on experience – able to build production hadoop system and support operation • Programming – manage systems with provided API • Application support – Optimization, New platform adoption and migration • Data science expertise, business expertise • Experience with large systems – Really large systems • Industry specific domain knowledge – Enhance partnership with users and business. © 2015 General Electric Company - All rights reserved

20. Focus areas for industrial HPC High availability • Establish automated failover capability • Enhance Job level recovery after system failure • Don’t forget all surrounding IT services. 24 x 7 service • 24 x 7 system monitoring • 24 x 7 user support • Full debug and compiling environment available Multiple systems strategy vs single system strategy • Easy for compute resource but hard for storage solutions. • Multiple location planning Automation • HPC service shall be automated as Web service • API driven management activity / functions will increase © 2015 General Electric Company - All rights reserved Reduce cost • Software License cost • Storage cost • Platform cost • $ per CPU cores

24. Research HPC • Commodity hardware • Huge Systems • Dedicated Data Center • Resource management based on fair share between users / programs. (relatively short run time.) • Handle computational process / workflow only. • Extended MTBF with parts fail all the time. • Multiple years planning with one-time funding / commitment. • Centralized source of funding – University / Government. • Independent from IT systems / infrastructure. • latest code support © 2015 General Electric Company - All rights reserved

25. Size of HPC systems Small ~100 cores Eg: Workgroup <$100k Medium ~500 cores Eg: Departmental $100k - $250k Large ~1000 cores Eg: Divisional $250k - $500k X- Large >2000 cores Eg: Supercomputer >$500k XX – Large >10000 cores Commercial HPC Research HPC Research HPC covers National Lab, Government Agency and universities Industrial, Finance, Oil &Gas, Manufacturing companies © 2015 General Electric Company - All rights reserved

26. Image citation / source CDN Locations | EdgeCast, www.edgecast.com Gauging Skills | Autism After 16www.autismafter16.com Service Availability: I'll See Your Five 9s and Raise You One More ...aits.org MySQL Master/Slave Management with mysql-master-ha | CRGNetwww.cirgan.net Illustration Depicting Two Triangular Warning Road Signs With A ...www.shutterstock.com Review: Thomas Pynchon's “Bleeding Edge” | The American Readertheamericanreader.com Can Crowd Testing replace an Internal Quality Team? | 99testsblog.99tests.com VIDEO: Find the balance between push and pull can help your ...www.teamsupport.com FCoE Adoption Curve - Wikibonwikibon.org Online Customers are Like Toddlers, They Don't Like Getting ...www.contenthook.com Stkit MOC] Iron Man Hulkbuster statue: A LEGO® creation by St Kit ...www.mocpages.com LEGO Ideas - Iron Man Hulkbuster armorideas.lego.com How does an ERP system work? - Quorawww.quora.com hadoops-role-in-enterprise-architecture-7-638.jpg?cb=1364477105www.slideshare.net Get Free or Cheap Legal Fees Help in the USwww.grantfundingexpert.org Migration | QualITasqualitas.co.id Respect for system administratorsgeekshumor.com Sincerely Stephanie: The Digit-al Dozen September Spectacular: Geekswww.sincerelystephanienails.com Huntington's Disease Youth Organization - Young adults - Ask a ...en.hdyo.org ABigFave | Flickr - Photo Sharing!www.flickr.com Peak Hour Protests Immobilise Bangalore - The New Indian Expresswww.newindianexpress.com Competitive Cost Benchmarking Software for Manufacturerswww.mtisystems.com Zenith Software announces India Expansion PlansDATAQUESTwww.dqindia.com © 2015 General Electric Company - All rights reserved

How HPC Transforms the Corporate Information Technology Ecosystem

You are reading a preview.

Check these out next

How HPC Transforms the Corporate Information Technology Ecosystem

More Related Content

Slideshows for you (20)

Similar to How HPC Transforms the Corporate Information Technology Ecosystem (20)

More from inside-BigData.com (20)

Recently uploaded (20)

How HPC Transforms the Corporate Information Technology Ecosystem