Gaining a Competitive Edge with HPC Cloud Computing
•
1 like•469 views
In this deck from the HPC User Forum in Detroit, William Edsall from Dow presents: Gaining a Competitive Edge with HPC Cloud Computing. "At Dow, Information Research provides leading edge information technology capability and information resources to meet strategic R&D initiatives. IR delivers a combination of Information Technology, R&D experience and physical science knowledge to develop capabilities and analysis that help deliver R&D growth and innovation commitments. IR overall is focused on making Dow R&D the best at using information technology and informatics for a competitive advantage." Watch the video: https://insidehpc.com/2018/09/gaining-competitive-edge-hpc-cloud-computing-dow/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Recommended
Recommended
More Related Content
More from inside-BigData.com
More from inside-BigData.com (20)
Recently uploaded
Recently uploaded (20)
Gaining a Competitive Edge with HPC Cloud Computing
- 2. William J. Edsall – The Dow Chemical Company High Performance Computing Architect • BA Applied Mathematics • 11 years HPC experience, 15 years IT • 3 kids ages 0, 2 and 4 • Self-proclaimed “ex cloud critic” 2 About the presenters
- 3. 3 About Dow High Performance Computing On premise HPC capabilities • Two Intel CPU based HPC Clusters • >85% AVG utilization • Nvidia K80 and V100 coprocessors • High end HPC desktops HPC workload at Dow • Computational Chemistry • Computational Fluid Dynamics • Molecular Dynamics • Computer Aided Engineering • Finite Element Analysis • Optimization • Deep Learning / Machine Learning • X-ray Tomography
- 4. GOAL: Develop processes and leverage a hybrid On Premise / Cloud Bursting environment by 2019. 4 Executive Summary 2019 Today Mar May Jul Sep Nov 2019 Mar May Jul Users Trained 6/29/2019 Success: Working On Premise / Cloud Hybrid Environment 7/7/2019 Proof of Concept with Rescale and Microsoft Azure 3/1/2018 - 7/1/2018 7/1/2018 - 3/1/2019 HPC Cloud Data Security Model 7/1/2018 - 3/1/2019 Cost Recovery Model 7/1/2018 - 3/1/2019 Azure Active Directory Integration 7/1/2018 - 3/1/2019 ISV Licensing Model 7/1/2018 - 3/1/2019 Data Transfer (Dow to Azure)
- 5. 5 Cloud Bursting Opportunity CPU Hours Demand Urgency High-End Desktop HPC Cluster Cloud Bursting Opportunity Dow Enterprise Workstation Special Cases Extreme memory, massive GPU farms and massive parallel jobs (we don’t have resources to solve on premise) A few CPU 100s of CPU 1,000s of CPU NoUrgencyTimeSensitive Computational Chemistry Today Computational Chemistry Tomorrow
- 6. • True demand difficult/ impossible to capture • Utilization always follows capacity trend 6 Challenges with on premise HPC Model 2015 2016 2017 2018 HPC Utilization HPC Capacity Dow HPC Capacity / Utilization Trend 2015-2018
- 7. 7 Challenges With HPC Cloud Bursting Model HPC Cloud Data Security Model Cost Recovery Model Azure Active Directory Integration Independent Software Vendor (ISV) Licensing Data Transfer (Dow to/from Azure)
- 8. 8 Rescale Cloud Bursting Proof of Concept at Dow Computational Chemistry • 1,400 molecules from ChEMBL database • Each molecule calculated with 7 CPU using Azure H16r instance • Entire ~9,800 CPU cluster running in <30 minutes • 0 queue time (no queue exists) • Cluster was shut down as jobs finished • Charges stopped automatically • Goal met in 5 hours Computational Fluid Dynamics • Ansys FLUENT simulation • Scaled up 20 -> 200 -> 2,000 CPUs • Entire 2,000 CPU run on Azure H16r instances • One Infiniband fabric • Cluster was stood up – job was verified and then stopped • Goal met in 15 minutes Work performed by Dr. Peter Margl, The Dow Chemical Company Work performed by Dr. Chang Kai (Lance) Wu, The Dow Chemical Company ~2x larger than the biggest parallel job at Dow~2x the total CPUs on premise at Dow
- 10. 10 Q&A