PARENG Parallel, Distributed & Grid Computing for Engineering
                          Pecs, April 6-8, 2009



         ...
Contents


The Engineering Challenge: Goodyear Example

The Tools: HPC Clusters, Grids and Clouds

Middleware, Services, A...
The Engineering Challenge
            Example: Goodyear




Courtesy: Loren K. Miller, President, Datametric Innovations
 ...
Prototype-Based Design at Goodyear

                    Since 1898, Goodyear had
                    developed new product...
Simulation-Based Engineering Science (SBES)

                        – “Scope of SBES includes much more than
            ...
SBES Vision




              1
            Road
            Tes t
            10
       Pred i c t i ve
          Tes t s...
Technical Complexity

   Tires are surprisingly complex.
   – Geometry.
   – Materials.
   – Service conditions.

   1992:...
LR



       Technical Complexity: Structures

“The pneumatic tire represents one of the most formidable
      challenges ...
Result: Model Fidelity & Speed




Axisymmetric models.     Detailed, treaded models.
                                    ...
Assurance™ TripleTred™ – 2004




                                                              The Goodyear Tire & Rubber...
Bottom Line Results

  Expenditures on prototype building and testing
  dropped 62% (from 40% of the R&D budget to 15%).
 ...
Our Tools Today:
HPC Clusters, Grids, and Clouds
              and
 Middleware, Services, Portals




                    ...
HPC Clusters
HPC Systems: provide “services“ for the past 30 years
Computing, storage, applications, and data
They serve (...
Grids

1998: The Grid: Blueprint for a New Computing Infrastructure:
      “A computational grid is a hardware and softwar...
Example: DEISA UNICORE Infrastructure
         CINECA user                                                           Gatew...
DEISA Service Layers

 Multiple                   Common         Presen-
                Workflow
 ways to                ...
DEISA Global File System

                                                       IBM P6 & BlueGene/P
                     ...
Clouds

IT resources provisioned outside of corporate data center
Resources accessed over the internet
SaaS, PaaS, IaaS, H...
Relationship between Grids and Clouds                                  *)


Different main drivers
    Grids: sharing reso...
A Closer Look at HPC Centers’ Load *. . .

    Single, cpu-intensive, tightly-coupled, highly
    scalable computational e...
. . . and their Suitability for Clouds

    Single, cpu-intensive, tightly-coupled, highly
    scalable computational engi...
An HPC Checklist
       When is your HPC app ready for the Cloud ?
 If there are no issues with licenses, IP, secrecy,
 se...
Finally:
The Cluster, Grid, and Cloud Portal

      Example: EnginFrame




                              Wolfgang Gentzsc...
Engineering today…


               Scripts         Aliases
                               Aliases             FTP        ...
Productive Grid and Cloud Solutions

                        Grid and Cloud Portal

                       Multi-site Mana...
What Issues are Addressed

Complex IT infrastructure
– Difficult to optimally leverage resources
– Different programs, app...
Use of Portals
                     Enterprise                                 Open Grid
                       Grid      ...
The Grid Portal Gateway




  Partners

                                                                            Grid /...
Benefits for the Engineer

Evolutionary deployment
– Preserve all investments in scripting
– Painless roll-out side-by-sid...
Benefits for the IT Manager

Reduced costs
– Menu-based, intuitive, application-centric interface
– Broaden and maximizes ...
Portal Services, e.g. EnginFrame

                                                    Portlet             Client          ...
Interactive job submission



  User friendly,
  Application-oriented
  Job submission


Flexible and efficient
Input file...
Monitoring & control



Global Job
monitoring



Cluster & host
monitoring




                 Job details &
            ...
Job and service notification




                           Wolfgang Gentzsch, PARENG 2009
Output management



  Data lifecycle
  management


Comprehensive output
File manipulation
(view, edit, delete, zip, …)

...
License / Job / Queue monitoring




                            Wolfgang Gentzsch, PARENG 2009
Seamless Interactive Application Integration




VNC, Citrix,
X-Windows




                                   Wolfgang Ge...
Integration of 3D Preview




                            Wolfgang Gentzsch, PARENG 2009
Interactive applications (3D)




IBM
DCV



                                  Wolfgang Gentzsch, PARENG 2009
SOA-enabled job submission
                 WS-I interface
                 Java / .NET client library and
               ...
Enterprise Portal integration




                            Wolfgang Gentzsch, PARENG 2009
Data exchange, sharing and versioning




                               Wolfgang Gentzsch, PARENG 2009
Workflow integration




                                                        Tools
 HTML/HTTP
             Extranet   ...
PARENG Parallel, Distributed & Grid Computing for Engineering
                        Pecs, April 6-8, 2009




          ...
Upcoming SlideShare
Loading in …5
×

Clusters, Grids & Clouds for Engineering Design, Simulation, and Collaboration

1,532 views
1,466 views

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,532
On SlideShare
0
From Embeds
0
Number of Embeds
34
Actions
Shares
0
Downloads
68
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Clusters, Grids & Clouds for Engineering Design, Simulation, and Collaboration

  1. 1. PARENG Parallel, Distributed & Grid Computing for Engineering Pecs, April 6-8, 2009 Clusters, Grids & Clouds for Engineering Design, Simulation, Collaboration (not only portals…) Wolfgang Gentzsch The DEISA Project & The Open Grid Forum Thanks to Loren K. Miller, Datametric Innovations, for Goodyear example Beppe Ugolotti, NICE-Italy, for EnginFrame example Wolfgang Gentzsch, PARENG 2009
  2. 2. Contents The Engineering Challenge: Goodyear Example The Tools: HPC Clusters, Grids and Clouds Middleware, Services, Applications Finally: HPC Cluster, Grid, and Cloud Portals Example: EnginFrame Wolfgang Gentzsch, PARENG 2009
  3. 3. The Engineering Challenge Example: Goodyear Courtesy: Loren K. Miller, President, Datametric Innovations “The Intersection of Science, Engineering, and IT” loren.miller@mac.com +1 330 310 3341 Wolfgang Gentzsch, PARENG 2009
  4. 4. Prototype-Based Design at Goodyear Since 1898, Goodyear had developed new products: design/build prototypes/test methodology: – Significant resources were capitalized and dedicated to tire building and testing. – Processes and release procedures were written assuming the design/build/test process. Design methodology rooted in building/testing prototypes. Wolfgang Gentzsch, PARENG 2009
  5. 5. Simulation-Based Engineering Science (SBES) – “Scope of SBES includes much more than the modeling of physical phenomena. • “[SBES] develops new methods, devices, procedures, processes, and planning strategies. • “We hope to solve the most stubborn problems of modeling, engineering design, manufacturing, and scientific inquiry.” – “Modeling and simulation will enable us to design and manufacture materials and products on a more scientific basis with less trial and error and shorter design cycles.” “Simulation-Based Engineering Science. Revolutionizing Engineering Science through Simulation.” NSF Blue Ribbon Panel, May, 2006, pp. 3 Wolfgang Gentzsch, PARENG 2009
  6. 6. SBES Vision 1 Road Tes t 10 Pred i c t i ve Tes t s 100 0 S imu l a t i on s Sc i ent i f i c F ounda t i o n Wolfgang Gentzsch, PARENG 2009
  7. 7. Technical Complexity Tires are surprisingly complex. – Geometry. – Materials. – Service conditions. 1992: state-of-the-art processes for creating the models, running the analyses, and analyzing the results took months for skilled and dedicated finite element analysts. By the time designers got answers, they’d forgotten their questions. Wolfgang Gentzsch, PARENG 2009
  8. 8. LR Technical Complexity: Structures “The pneumatic tire represents one of the most formidable challenges in computational mechanics today.” Professor A. Noor, Journal of Computers and Structures Modeling Challenges – Incompressible, non-linear visco- elastic material with high (~100%) cyclic strains (rubber) – Inextensible fibers (steel belts & polyester ply) – Flexible structures (sidewall) ~ 60 Million Cycles – Detailed tread patterns – Wide eigenvalue spectrum During an 80,000 – Expensive, low fidelity solutions Mile Tire Lifetime Wolfgang Gentzsch, PARENG 2009
  9. 9. Result: Model Fidelity & Speed Axisymmetric models. Detailed, treaded models. Wolfgang Gentzsch, PARENG 2009
  10. 10. Assurance™ TripleTred™ – 2004 The Goodyear Tire & Rubber Company, First product developed entirely using simulation-based engineering Press Photos science. Optimized for wet, dry, and ice. Most successful new product introduction in Goodyear’s history. Wolfgang Gentzsch, PARENG 2009
  11. 11. Bottom Line Results Expenditures on prototype building and testing dropped 62% (from 40% of the R&D budget to 15%). – ~$100 million annually that has been directed to other R&D projects. Product design times were reduced 67% (from three years to less than one). – Key enabler of corporate new product leadership strategy. Unprecedented string of award-winning new products resulted from the ability to evaluate many more new product alternatives. Results far exceeded what Goodyear dreamed possible in 1992. Wolfgang Gentzsch, PARENG 2009
  12. 12. Our Tools Today: HPC Clusters, Grids, and Clouds and Middleware, Services, Portals Wolfgang Gentzsch, PARENG 2009
  13. 13. HPC Clusters HPC Systems: provide “services“ for the past 30 years Computing, storage, applications, and data They serve (local) research, education, and industry (e.g. HLRS in Stuttgart serving Bosch, Daimler, Porsche) Very professional: to their end-users, they appear almost like a set of Cloud services (Amazon definition: easy, secure, flexible, on demand, pay per use, self serve) But: no virtualization, semi-automatic, operating in static mode (increase of performance…) That’s where HPC centers themselves can become a Cloud customer, adding dynamic scaling and adopting to changing business and user demands Wolfgang Gentzsch, PARENG 2009
  14. 14. Grids 1998: The Grid: Blueprint for a New Computing Infrastructure: “A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities.” 2002: The Anatomy of the Grid: “. . . coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations.” 2002: Grid Checklist: 1) coordinates resources that are not subject to centralized control … 2) … using standard, open, general-purpose protocols and interfaces 3) … to deliver nontrivial qualities of service. Quotes: Ian Foster, Carl Kesselman, Steve Tuecke Wolfgang Gentzsch, PARENG 2009
  15. 15. Example: DEISA UNICORE Infrastructure CINECA user Gateway Gateway IDRIS Gateway Gateway FZJ HLRS Gateway job ECMWF HPCX Gateway Gateway job CSC LRZ NJS Gateway IDRIS IBM P6 LRZ user Gateway CINECA NJS NJS RZG FZJ IBM HLRS NEC SX8 IDB UUDB Gateway Gateway BSC IDB UUDB IDB UUDB SARA NJS ECMWF IBM P5 NJS AIX LL-MC HPCX Cray XT4 IDB UUDB Super-UX AIX NQS II IDB UUDB LL-MC P FT G rid AIX LL UNICOS/lc NJS PBS Pro NJS CSC Cray XT4/5 LRZ SGI ALTIX IDB UUDB UNICOS/lc PBS Pro IDB UUDB LINUX PBS Pro NJS CINECA IBM P5 AIX LL-MC AIX IDB UUDB LL-MC LINUX LINUX NJS Maui/Slurm LL RZG IBM NJS NJS IDB UUDB BSC IBM PPC SARA IBM IDB UUDB IDB UUDB Wolfgang Gentzsch, PARENG 2009
  16. 16. DEISA Service Layers Multiple Common Presen- Workflow ways to production tation managemnt access environmnt layer Co- Single Job manag. Job reservation monitor layer and rerouting and co- system monitor. allocation Data WAN Data Data staging transfer shared manag. tools tools File system layer Network Unified DEISA Network and AAA Sites connectivity AAA layers Wolfgang Gentzsch, PARENG 2009
  17. 17. DEISA Global File System IBM P6 & BlueGene/P IBM P6 & BlueGene/P NEC SX8 AIX, Linux LL-MC Super-UX AIX, Linux NQS II LL-MC P FT IBM P6 G rid Cray XT4 AIX LL UNICOS/lc PBS Pro Cray XT4/5 UNICOS/lc PBS Pro SGI ALTIX LINUX PBS Pro AIX LL-MC IBM P5 AIX, Linux LL-MC LINUX LINUX Maui/Slurm LL IBM P5+ / P6 IBM P6 & BlueGene/P IBM PPC Global transparent file system based on the Multi-Cluster General Parallel File System (MC-GPFS of IBM) Wolfgang Gentzsch, PARENG 2009
  18. 18. Clouds IT resources provisioned outside of corporate data center Resources accessed over the internet SaaS, PaaS, IaaS, HaaS Virtualization: abstraction of the hardware from the service Build and deliver, always-on, pay-per-use IT services Near infinite-scale computing, storage, database, related Web services, AND users Scaling resources and services up and down No need on-premises servers and software Wolfgang Gentzsch, PARENG 2009
  19. 19. Relationship between Grids and Clouds *) Different main drivers Grids: sharing resources, collaborating in teams Clouds: financial and business flexibility, time to market, fast and low-risk experimentation Commonalities Sharing technologies: distributed systems, virtualization Grid owners are taking advantage of Clouds Grids and Clouds run on shared infrastructured Access is via network, often remotely Similar challenges, major impediments Portability of applications, services, and data Secure access to and operation of services Secure movement and storage of data Unified management for internal and external platforms *) OGF Statement on Grids & Clouds, April 2009 Wolfgang Gentzsch, PARENG 2009
  20. 20. A Closer Look at HPC Centers’ Load *. . . Single, cpu-intensive, tightly-coupled, highly scalable computational engineering & science parallel jobs Single, cpu-intensive, computational, weakly- scalable, engineering & science parallel jobs Capacity computing, throughput, parameter jobs Managing massive data sets, possibly geographically distributed Analysis and visualization of data sets * According to the analysis of T.Sterling and D.Stark, LSU, in a recent HPCwire article Wolfgang Gentzsch, PARENG 2009
  21. 21. . . . and their Suitability for Clouds Single, cpu-intensive, tightly-coupled, highly scalable computational engineering & science No Not yet parallel jobs Single, cpu-intensive, computational, weakly- scalable, engineering & science parallel jobs Yes Capacity computing, throughput, parameter jobs Yes Managing massive data sets, possibly Yes geographically distributed Yes Analysis and visualization of data sets * According to the analysis of T.Sterling and D.Stark, LSU, in a recent HPCwire article Wolfgang Gentzsch, PARENG 2009
  22. 22. An HPC Checklist When is your HPC app ready for the Cloud ? If there are no issues with licenses, IP, secrecy, sensitive data, privacy, legal or regulatory issues, . . . If your app is (almost) architecture independent, not optimized for specific architecture (i.e. single process, loosely-coupled low-level parallel, I/O-robust) If it’s just one app and zillions of parameters If latency and bandwidth are not an issue If time (wait, wall, run) doesn’t really matter If your job is low-priority, simple SLAs, can re-run, . . . Ideally, your HPC Center’s meta-scheduler knows all the details, schedules automatically, and hides all complexity underneath a portal ☺ Wolfgang Gentzsch, PARENG 2009
  23. 23. Finally: The Cluster, Grid, and Cloud Portal Example: EnginFrame Wolfgang Gentzsch, PARENG 2009
  24. 24. Engineering today… Scripts Aliases Aliases FTP NFS Scripts FTP NFS Engineers enhance the quality of the products Restart Repository DOE Restart Repository DOE innovation in the product line Engineers foster Teamwork Teamwork Versioning Engineers build reusable knowledge for core business Versioning LSF LSF Library Library Disk Disk Windows So each CRASH! spent by your quota minute engineers is of great value quota Windows CRASH! Queue for your company, besides being greatly self-motivating Queue Linux Linux IP IP Convert Convert Protection Protection Resource Resource Working Working Password Password Execution directory Execution directory UNIX ID UNIX ID host host Wolfgang Gentzsch, PARENG 2009
  25. 25. Productive Grid and Cloud Solutions Grid and Cloud Portal Multi-site Management Security / Authorization License Management ROI Analysis - BI Flow/Process Management Workload Management Data Management Application Management Wolfgang Gentzsch, PARENG 2009
  26. 26. What Issues are Addressed Complex IT infrastructure – Difficult to optimally leverage resources – Different programs, applications, GUIs, OS, SAN, SOA Data management and security – Timely, consistent, transparent data access – Controlled access for IP protection Teamworking and collaboration – Complex, slow, ad-hoc collaboration – Identity management New business opportunities – ASP, compute-on-demand, HPC consolidation – Experience sharing and leveraging Wolfgang Gentzsch, PARENG 2009
  27. 27. Use of Portals Enterprise Open Grid Grid ASP Desktop Scavenging l aS aaaS rtaal or t S C S . .PPo Commercial HPPC p H Appp HPC ASP C A PPC HPC Clusters HH ign ssign e rm faarm eedde n f tiv titoon raativ a i boor liz aaliza lllaab isu CCo l o VVisu Wolfgang Gentzsch, PARENG 2009
  28. 28. The Grid Portal Gateway Partners Grid / Compute Farm Standard protocols Managers Win LX Batch Applications Mac UX Licenses Internal Users Intranet Clients Grid Portal / Gateway Interactive Applications Home Users Storage and Data Enterprise Portal Wolfgang Gentzsch, PARENG 2009
  29. 29. Benefits for the Engineer Evolutionary deployment – Preserve all investments in scripting – Painless roll-out side-by-side with terminal or remote desktop – Handles complexity preserving user-friendly approach Integrated with ISVs and mainstream middleware – Transparent data management capabilities – Reduce errors and misuse of the Grid / applications – Cut training costs and improve users’ productivity Integrated with engineering workflow engines – Accelerate supply chain collaboration – Bottom-up and top-down engineering process automation – Standardize and enrich data management Wolfgang Gentzsch, PARENG 2009
  30. 30. Benefits for the IT Manager Reduced costs – Menu-based, intuitive, application-centric interface – Broaden and maximizes the exploitation of the IT infrastructure – Lower client TCO Reduced risks – Evolves with your IT infrastructure and Grid – Align with company’s IT security policies – Controlled access to data and information Exploitation of Server Consolidation/Virtualization – Black-box, application-level virtualization – One-stop-shop for computing, visualization, data – Only one customization for multiple access media / patterns Wolfgang Gentzsch, PARENG 2009
  31. 31. Portal Services, e.g. EnginFrame Portlet Client News Containers Applications Feeds JSR168 WSDL/SOAP HTTP Plugins Skins / Themes Portlet GW WS GW RSS GW ISV 1 - XML Template-based dynamic presentation engine with AJAX support Application Kit Single-Sign-On ACL manager Auth. delegation Channel security ISV n - XML Session manager User mapping Usage acct./billing engine GUI Application Kit Service chaining Distributed file manager Virtualization Custom XML Multi-language services Data life-cycle manager Workflow Application Kits App. virtualization GridML virtualization Data virtualization Engine Compute Grid Data (Compute Cluster Pack, LSF, PBS, …) Internal Utility Distributed HW/SW Services Storage Wolfgang Gentzsch, PARENG 2009
  32. 32. Interactive job submission User friendly, Application-oriented Job submission Flexible and efficient Input file management Hide complexity of Underlying scheduler Wolfgang Gentzsch, PARENG 2009
  33. 33. Monitoring & control Global Job monitoring Cluster & host monitoring Job details & control Wolfgang Gentzsch, PARENG 2009
  34. 34. Job and service notification Wolfgang Gentzsch, PARENG 2009
  35. 35. Output management Data lifecycle management Comprehensive output File manipulation (view, edit, delete, zip, …) Follow-up actions support Wolfgang Gentzsch, PARENG 2009
  36. 36. License / Job / Queue monitoring Wolfgang Gentzsch, PARENG 2009
  37. 37. Seamless Interactive Application Integration VNC, Citrix, X-Windows Wolfgang Gentzsch, PARENG 2009
  38. 38. Integration of 3D Preview Wolfgang Gentzsch, PARENG 2009
  39. 39. Interactive applications (3D) IBM DCV Wolfgang Gentzsch, PARENG 2009
  40. 40. SOA-enabled job submission WS-I interface Java / .NET client library and command line interface Simplifies integration with client-side applications (optimization, workflow, etc.) for power-users Wolfgang Gentzsch, PARENG 2009
  41. 41. Enterprise Portal integration Wolfgang Gentzsch, PARENG 2009
  42. 42. Data exchange, sharing and versioning Wolfgang Gentzsch, PARENG 2009
  43. 43. Workflow integration Tools HTML/HTTP Extranet Workflow Engine collaborate Portal (Process Manager) r n ito o / m it Storage and Data b m HTML/HTTP Intranet Su Portal Grid EnginFrame Computational Power Wolfgang Gentzsch, PARENG 2009
  44. 44. PARENG Parallel, Distributed & Grid Computing for Engineering Pecs, April 6-8, 2009 Thank You ! And thanks to: Loren K. Miller, Datametric Innovations, for the Goodyear example Loren.miller@mac.com Beppe Ugolotti, NICE-Italy, for the EnginFrame example Beppe@nice-italy.com Wolfgang Gentzsch, PARENG 2009

×