Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Docker Workshop: Containers for Scientific & Engineering Applications

1,661 views

Published on

In this deck from ISC 2015, Wolfgang Gentzsch from The UberCloud presents: HPC Workloads Using Docker.

"The UberCloud Project provides an environment to run HPC workloads on a variety of cloud providers. Wolfgang elaborates on the projects and provides his experience about the roadblocks towards a more flexible cloud-backed HPC future."

Watch the video presentation: http://insidehpc.com/2015/07/video-hpc-workloads-using-docker/

Sign up for our insideHPC Newsletter: http://insideHPC.com/newsletter

Published in: Technology
  • Be the first to comment

Docker Workshop: Containers for Scientific & Engineering Applications

  1. 1. Intel Confidential, Copyright © 2015, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. Optimization Notice Wolfgang Gentzsch & Burak Yenier ISC Frankfurt, July 16, 2015 Docker Workshop Containers for Scientific & Engineering Applications
  2. 2. 2012: UberCloud started its free voluntary Cloud Experiments HPC as a Service, on demand, in a team experiment to explore the end-to-end process using cloud computing resources as a service, on demand, at your fingertips and learning how to resolve the roadblocks. TODAY: 172 experiments, 2500 companies, 72 countries, 50 case studies Supported by our sponsor INTEL
  3. 3. Major Cloud Roadblocks Supply Cloud providers ISVs Consultants Trainers Demand Engineers Scientists Data analysts Experts . . . . . Complexity Data Transfer SecurityLicensing Uncertain Cost Roadblocks
  4. 4. 2015: The UberCloud Marketplace Supply Cloud providers ISVs Consultants Trainers … Demand Engineers Scientists Data analysts Experts Marketplace And HPC Docker
  5. 5. Marketplace
  6. 6. UberCloud application containers based on Docker
  7. 7. UberCloud application containers  Ready-to-execute packages of software, designed to deliver the tools that an engineer needs  Ready to execute, in an instant. No need to install software, deal with complex OS commands, or configure.  The ISV or Open Source tools are pre-installed, configured, and tested, and are running on bare metal, without loss of performance.  Based on open Linux container technology  Enhanced for engineering & scientific applications  ANSYS, CD-adapco, OpenFOAM, Gromacs, Scilab, and more
  8. 8. Virtual Machines vs Docker VM: Each virtualized application includes not only the application and the necessary binaries and libraries, but also an entire guest operating system. Docker Engine container comprises application and its dependencies. Isolated process in user space on host operating system, sharing kernel with other containers. Enjoys resource isolation and allocation benefits ofVMs but more portable and efficient. 9
  9. 9. Builder Launcher Controller ISV DataTools Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data. Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed. Collect granular usage data, logs. Monitor, alert, report. Any Workstation Any Cluster Any Cloud Run Time Run Time Run Time Containers: Build once, run anywhere
  10. 10. Builder Launcher Controller ISV DataTools Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data. Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed. Collect granular usage data, logs. Monitor, alert, report. Any Workstation Any Cluster Any Cloud Run Time Run Time Run Time Containers: Build once, run anywhere
  11. 11. Builder Launcher Controller ISV DataTools Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data. Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed. Collect granular usage data, logs. Monitor, alert, report. Any Workstation Any Cluster Any Cloud Run Time Run Time Run Time Containers: Build once, run anywhere
  12. 12. Builder Launcher Controller ISV DataTools Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data. Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed. Collect granular usage data, logs. Monitor, alert, report. Any Workstation Any Cluster Any Cloud Run Time Run Time Run Time Containers: Build once, run anywhere
  13. 13. Containers remove Portability & Standard related challenges. By integrating 3rd party tools we tackle the following:
  14. 14. UberCloud multi-container environment Docker Runtime Docker Runtime Docker Runtime . . . . NFS Host Host Host Container Container Container MPI Verticalscaling Horizontal scaling Dramatically simplifyingPackaging, Porting, Access, Use, Scaling, with Low Overhead
  15. 15. OpenFOAMTest: Bare Metal vs Docker PerformanceTest Results demonstrating comparison between the same OpenFOAM run (with 1.4 million cells) repeated on bare metal and in a Docker container.
  16. 16. DockerizedTechnical Computing Ecosystem Experts CFDsupport CAEtech Dacolt Any Inhouse & Open Source ISV ANSYS CD-adapco OpenFOAM Resource AWS Azure LLNL
  17. 17. UberCloud multi-container environment  Intel MPI libraries pre-setup for multi-node high performance  Intel compilers achieve architecture specific high performance  Networking between containers  Setting up shared file system access (NFS)  Secure communication (ssh) between containers  Benefits from single-node transferred to multi-node containers:  No setup of the environment  Portability and packaging  Easy access and use  Low overhead +
  18. 18. Hot Spot Question for Jérôme Petazzoni  Docker recently announced that it acquired SocketPlane to help with container networks. I am very interested in some more background and details and how this could support HPC multi-container approach.  There are multiple container implementations, Docker, CoreOS and now a collective effort, runC. Can you talk about what this all means for the community.  Virtualization has been around for a long time, why re-invent the wheel?  In HPC clusters shared file systems are quite commonly used. Can you comment on how to secure a Docker installation in conjunction with a shared file system. Wouldn't the shared file system be vulnerable when Docker is in the mix?
  19. 19. ANSYS Container Demo  ANSYS multi-physics (5 s/w) running inside Docker container  Pre- and post-processing in the cloud works the same way that it would work on the engineer’s desktop  User experience in the cloud matches that on the desktop  GPU support for high-resolution remote visualization  Automated monitoring keeps you connected, under control  4-min demo: https://goo.gl/YIrTr4  6-min demo: http://goo.gl/pDXiTq
  20. 20. Containers as basic technology for the UberCloud SaaS Factory UberCloud SaaS Factory Software SaaS Commercial ISV Open Source In-house Public Cloud Private Cloud On-Premise $$$ We turn high-TCO, complex software into user-friendly pay-per-use services: Development Services (Docker) -------------------- Deployment & Management Services ------------------ Marketplace Providers Consumers $ UberCloud SaaS Factory
  21. 21. Intel Confidential, Copyright © 2015, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. Optimization Notice Thank You ! Please register at http://www.TheUberCloud.com It’s free

×