Your SlideShare is downloading. ×

Seminario Paolo Maggi, 24-05-2012

444

Published on

Il seminario mostra come le moderne tecnologie di Cloud Computing possono essere applicate per migliorare l'utilizzabilità e l'efficacia di strumenti di simulazione e di analisi di dati ambientali, …

Il seminario mostra come le moderne tecnologie di Cloud Computing possono essere applicate per migliorare l'utilizzabilità e l'efficacia di strumenti di simulazione e di analisi di dati ambientali, con grandi vantaggi dal punto di vista della semplificazione dei flussi di lavoro.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
444
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Tecnologie Cloud per la fornitura di applicazioni ambientali HPC in modalità SaaS Paolo Maggi <paolo.maggi@nice-software.com> R&D ManagerCopyright © NICE - 2012
  • 2. Tecnologie Cloud per la fornitura di applicazioni ambientali HPC in modalità SaaS Software High as Performance a Computing ServiceCopyright © NICE - 2012
  • 3. What is High Performance Computing? Refers to the use of supercomputers or clusters of computers to solve difficult computational problems that typically arise through scientific inquiry. Computer users turn to HPC when a problem is too large to solve on a conventional laptop or workstation or runs too slowly – it requires too much memory or disk space – the algorithm is complex – the dataset is large – data access is slowCopyright © NICE - 2012
  • 4. When we need HPC? To do a time-consuming operation in less time – I am an aircraft engineer – I need to run a simulation to test the stability of the wings at high speed – I’d rather have the result in 5 minutes than in 5 days so that I can complete the aircraft final design soonerCopyright © NICE - 2012
  • 5. When we need HPC? To do an operation before a tighter deadline – I am a weather prediction agency – I am getting input from weather stations/sensors – I’d like to make the forecast for tomorrow before tomorrowCopyright © NICE - 2012
  • 6. When we need HPC? To do a high number of operations per seconds – I am an engineer of ACME.com – My Web server gets 1,000 hits per seconds – I’d like my web server and my databases to handle 1,000 transactions per seconds so that customers do not experience bad delaysCopyright © NICE - 2012
  • 7. How to get my answer faster? Work harder – Get faster hardware Work smarter – Use optimized algorithms (libraries!) – Write faster code (adapt to match hardware) – Trade convenience for performance (e.g. compiled program vs. script program) Delegate parts of the work – Parallelize code – Use accelerators (GPGPU/CUDA)Copyright © NICE - 2012
  • 8. Leveraging parallelismCopyright © NICE - 2012
  • 9. Leveraging parallelism Functional parallelism Different people are performing different tasks at the same timeCopyright © NICE - 2012
  • 10. Leveraging parallelism Data parallelism Different people are performing the same task, but on different (equivalent) objectsCopyright © NICE - 2012
  • 11. What a cluster is… A cluster needs: – Several computers, nodes, often in special cases for easy mounting in a rack – One or more networks (interconnections) to hook the nodes together – Software that allows the nodes to communicate with each other (e.g. MPI) – Software, called Distributed Resource Manager (DRM), that manage resources and assign them to individual users and computational jobs (e.g. LSF, PBS, SGE, …) A cluster is: all of those components working together to form one big computer (to give you answers in a fast way)Copyright © NICE - 2012
  • 12. How a cluster looks like?Copyright © NICE - 2012
  • 13. What is Cloud Computing?Copyright © NICE - 2012
  • 14. A working definition of Cloud Computing (NIST) Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model promotes availability Composed of – five essential characteristics – three service models – four deployment modelsCopyright © NICE - 2012
  • 15. The NIST Cloud definition framework On Demand Self-ServiceEssential Broad Network Access Rapid ElasticityCharacteristics Resource Pooling Measured ServiceService Software as a Platform as a Infrastructure as aModels Service (SaaS) Service (PaaS) Service (IaaS) Hybrid CloudsDeploymentModels Private Community Public Cloud Cloud CloudCopyright © NICE - 2012
  • 16. Cloud Computing OfferingsCopyright © NICE - 2012
  • 17. About NICE Pioneer in Technical and Engineering Cloud solutions 14 years experience with enterprise Grid/Cloud solutions throughout all industries Worked closely with ISV and HW vendors Headquarters: Italy since the beginning Offices: USA, Germany Strong relationship with Research and Academia Core business: Access to Grid / HPC solutions and Remote Visualization EnginFrame Grid Portal product line DCV Remote Visualization technology Other relevant competencies Distributed Resource Management Grid Intelligence Visualization technology integrationCopyright © NICE - 2012
  • 18. NICE Customers and Market SegmentsEnergy Aerospace & ManufacturingAddax Petroleum, AECL, Hess, Bayerngas, AIRBUS, Air Products and Chemicals,BHP Billiton, Beicip, British Gas, Centrica, Procter&Gamble, SelexGalileo, GoodrichChevron, Conoco-Phillips, Dong, Dowell, Aerospace, Hamilton Sunstrand, KimberlyDSC-Libya, ENI/Agip, GazPromNeft, GDF, Clarke, Magellan Aerospace, Nordam,Logelco, Maersk Oil, Marathon Oil, Nexen, Northrop Grumman, Raytheon, ThalesNovatek, Papuan Oil, Rosneft,Schlumberger, Shengli Oil, Sonatrach, Automotive & Industrial EquipmentStatoil, Talisman Energy, TNK-BP, TNNC, Audi, ARRK, Bridgestone, Bosch, Daimler,TOTAL, WesternGeco, Xinjiang Oil Delphi, Dow Chemical, FIAT, Ferrari, Honda, Hyundai, Jaguar-Land Rover, Lear, MagnetiLife Sciences Marelli, McLaren, P+Z, PSA, Tata Steel,Bayer, Biolab, DEISA project, Swiss Institute Toyota, TRW, Volkswagenfor Bioinformatics, Partners Healthcare,Pharsight, M.D. Anderson Cancer Center Research & Education ASSC, Beihang Uni, CCLRC, CILEA, CNR,Others CNRS/IN2P3, ENEA, Georgia State Uni,Accent, Samsung SDI, SensorDynamics, INFN, RMSC, Harvard Uni, Messina Uni,Bank of Italy, Deutsche Bank Huazhong Normal Uni, Yale UniCopyright © NICE - 2012
  • 19. The NICE Grand Vision Internal Resources Administrators Partner & Managers Supplier Customer Multi-Tenancy & Collaboration services HPC Centers, Public Clouds, … UserCopyright © NICE - 2012
  • 20. NICE Products & Solutions NICE EnginFrame NICE DCV NICE EnginFrame Views Solutions for vertical markets markets: Life Science Oil&Gas AutomotiveCopyright © NICE - 2012
  • 21. NICE Products Overview Project Mgr End Users Developers Developers End Users End Users Customer 1 (Corporate network) Customer 2 (VPN) ... Customer N Service A Service B Service C Service X (HPC) (3D application) (SaaS) SaaS) ... (workflow) workflow) Service Presentation Layer Access Control (Web, API, Command-line, ...) AD, LDAP, NIS, … Vertical solutions Views DataGate SLB-RE HPC … Desktop Cloud Visualization Infrastructure model abstraction GridML ViewsML CloudML Accounting driver driver driver & Billing LSF, Lava, SGE, DCV, VNC, RGS, VMWARE, EC2, RDBMS PBS, MOAB, … TAW, … MOAB, ISF, …Copyright © NICE - 2012
  • 22. Our architecture Access Self-Service Offering Resources Thin viewer DCV protocol Linux &Collaborators, WindowsSupport staff 3D Visualization sessions Servers HTTP(S) HPC schdulers End Users HPC jobs Command-line Orchestration & Provisioning SOAP Storage Developers, IntegratorsCopyright © NICE - 2012
  • 23. User Experience in the Technical Cloud Enterprise Open Grid Grid Portal Desktop Scavenging Commercial HPC Cloud HPCClustersCopyright © NICE - 2012
  • 24. NICE EnginFrame key features Lightweight Web front-end for HPC systems Best in class data management WebServices API and command-line interface Controlled, user friendly application submission Supports all mainstream job schedulers Workload, resource, license monitoringCopyright © NICE - 2012
  • 25. Batch job / workflow submission User friendly, Application-oriented Job submission Hide complexity of Underlying scheduler Flexible and efficient Input file managementCopyright © NICE - 2012
  • 26. Application Data Management Application data can be organized into projects Application data can be marked as starred Metadata can be associated to application dataCopyright © NICE - 2012
  • 27. Monitoring Jobs, Hosts, Queues, Licenses, …Copyright © NICE - 2012
  • 28. Data transfers & file management The file manager component allows to seamless navigate and access server-side files from the web browserCopyright © NICE - 2012
  • 29. SOA-enabled job submission WS-I interface Java / .NET client library and command line interface Simplifies integration with client-side applications (optimization, workflow, etc.) for power-usersCopyright © NICE - 2012
  • 30. What is a 3D Cloud? "Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.” (NIST) 3D Clouds will enable on-demand network access to interactive 3D applications (like visualization applications for scientific data, CAD applications, etc.) Why NICE customers need a 3D Cloud?Copyright © NICE - 2012
  • 31. A typical scenario in the CAE computing sector 1 - Simulation submission Win 2 - Data Transfer to visualize results Server/Data Center 3 - Collaboration Needs Data exchange CAE Workstations • Can run small to medium serial analysis • Requires high-end GPU for rendering Win • May represent IT management challenges (tens to hundreds of seats)Copyright © NICE - 2012
  • 32. Problems associated to this model NETWORK Network overload leading to poor performance and response times all round COST Expensive, dedicated workstations (GPU, memory, …) with short lifecycle IT MANAGEMENT Support, update and replace tens to tens of thousands of workstations WORKSTATION SIZING Workstations have to be sized for the largest expected models SECURITY Moving sensitive data around (in/out organization) is always risky WORKFORCE Current models do not support a diverse mobile workforceCopyright © NICE - 2012
  • 33. Moving technical applications closer to the data WS PC HPC job Collaboration Remote submission and management Visualization Linux Job submissionCopyright © NICE - 2012 33
  • 34. Benefits of the HPC Cloud model NETWORK Network is no more a bottleneck and data loads faster COST Centralized & Shared servers are less expensive to buy & manage IT MANAGEMENT Support, update and replacement are more efficient & do not affect users WORKSTATION SIZING Resources are dynamically sized based on users needs SECURITY Sensitive data remain within protected data center with full access control WORKFORCE Users can virtually connect & collaborate from anywhere with any clientCopyright © NICE - 2012
  • 35. Enabling Technologies for the 3D Clouds Technologies providing secure remote network access to interactive 3D applications leveraging server-side graphic hardware acceleration (GPUs) A software stack that allows the end-user to easily launch and access remote interactive applications and takes care of managing and load balancing applications and desktop sessions running within a Visualization FarmCopyright © NICE - 2012
  • 36. NICE EnginFrame Views NICE EnginFrame (Portal and Gateway) Job Scheduler & Distributed Resource Manager 2D & 3D Applications VirtualGL RealVNC NICE DCV HP RGS NX, RDP, ... LINUX WINDOWS SOLARIS Computing Infrastructure Storage InfrastructureCopyright © NICE - 2012
  • 37. How Does It Work? The user connects to EnginFrame to create a new Session EnginFrame Job sch. selects the appropriate node and starts the session 1 Win 2 3 User gets connected to the session on the selected node Job Sch. 64GBytes, Linux 4GB, Win/Lin blades 8GB, Windows Heterogeneous infrastructure: HW, OS, middle-wares middle-Copyright © NICE - 2012
  • 38. Benefits for the End Users Access to Applications as a Service – Hide infrastructure details (site, platform, etc…) – Session allocation can be influenced by memory requirements, data location affinity and other customer-specific parameters – Intelligent load balancing of sessions, based on the Job Scheduler • Memory-aware, Memory reservation, Application license-aware – Fewer or better optimized data transfers Collaboration, session sharing – The session owner can generate a “URL” that can be sent by email / instant message to invite a collegue to join a given session, without disclosing the user’s password Easy management of active session – Create new sessions with user-specific preferences (resolution, etc...) – List, reconnect and kill existing sessions Seamless Access to the sessions and Single Sign-On – Automation of session-level password create/destroy • E.g. login via NTLM / ActiveDirectory credentials and map to Linux userCopyright © NICE - 2012
  • 39. Benefits for the Administrators Increased level of service provided to users – Sessions are load balanced by the Job Scheduler to match user needs – Memory reservation and Data locality scheduling – Reduce help desk calls – Exposed services can be personalized per user/group/project Accounting – Sessions are jobs, so the resource usage accounting by user, group, project can be collected through any Analytics tool Monitoring – The load and usage of the login farm is monitored via EnginFrame – Node loading conditions, active sessions – Administrators can control and manage users’ idle or stuck sessions Support – Administrators can connect to user’s sessions to provide support Security – Easy integration into identity services, SSO, Enterprise portalsCopyright © NICE - 2012
  • 40. NICE DCV (Desktop Cloud Visualization) Originated in IBM Research in 2004, acquired by NICE in summer 2010 Provides efficient remote access to graphic-intensive, professional OpenGL applications A central (graphics-enabled) server is accessed by remote users Users only need low-end machines with network connectivity to the server in order to view and interact with remote application One or more users (collaborators) can simultaneously access the server Is the key technology for delivery of real-time 3D graphics but relies on other software to provide the collaborative environment (RealVNC Visualization Edition)Copyright © NICE - 2012
  • 41. NICE DCV – Main Features Works on Windows and Linux, leveraging high-end NVIDIA GPUs Support GPU sharing across multiple users, multiple OS – First product on the market to allow the sharing of physical GPUs between multiple Windows VMs while maintaining full OpenGL acceleration and workstation-class performance Bandwidth optimized balancing of quality Vs. frame rate High network latency tolerance Validated and optimized for Technical Computing applications Perfectly integrated with NICE EnginFrame for session managementCopyright © NICE - 2012
  • 42. Windows application deliveryCopyright © NICE - 2012
  • 43. Linux application delivery 43Copyright © NICE - 2012
  • 44. Web-based Thin client accessCopyright © NICE - 2012
  • 45. Built-in collaboration 45Copyright © NICE - 2012
  • 46. Thank you!Copyright © NICE - 2012

×