Downloaded 91 times
![Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
OpenNebula
Statement on homepage
Standard-based open-source toolkit to build private, public and
hybrid clouds. [...] provides flexible architecture, interfaces and
components that fit into any existing data center.
Supports Xen, KVM and VMware (libvirt)
Interfaces also with Amazon EC2 and ElasticHosts
Other users
CERN (cloud infrastructure consisting of 3,200 cores)
NIKHEF
China Mobile (
”
Big Cloud“ project)](https://image.slidesharecdn.com/slides-101028055008-phpapp02/85/Cloud-Computing-in-D-Grid-14-320.jpg)
![Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Worker Node Template
NAME = udo−wn300
VCPU = 8
MEMORY = 13312
OS = [ bootloader = ”/ root / bin /domUloader . py” ]
DISK = [ source = <path >/wn sl54 x86 64 . img ” ,
t a r g e t = ”xvda ” , readonly = ”no” ]
DISK = [ type = swap , s i z e = 1024 ,
t a r g e t = ”xvdb ” , readonly = ”no”
]
DISK = [ type = ” block ” , clone = ” yes ” ,
source = ”/ dev/ c c i s s /c0d0p4 ” ,
t a r g e t = ” xvdc ” , readonly = ”no”
]
NIC = [ NETWORK = ”DGRZRWorkernodes ” ,
IP = 129.217.242.166
]](https://image.slidesharecdn.com/slides-101028055008-phpapp02/85/Cloud-Computing-in-D-Grid-17-320.jpg)
Uploaded byStefan Freitag
Cloud Computing in D-Grid
The document discusses the integration of cloud computing into the D-Grid infrastructure in Germany. It provides background on D-Grid, including its goals and projects. It then describes the setup of cloud computing at the Dortmund Computing Center, including the use of OpenNebula software to create a private cloud. Key challenges of integrating clouds into D-Grid are identified as user management, authorization, accounting, monitoring, and legal issues regarding virtual organizations. Overall, the document analyzes initial efforts to deploy cloud technologies in D-Grid and important open questions around full-scale integration.
More Related Content
Similar to Cloud Computing in D-Grid
Cloud Computing in D-Grid
- 1. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Cloud Computing in D-Grid Science Days 2010 Stefan Freitag Robotics Research Institute Dortmund University of Technology 03. November 2010
- 2. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Collaboration of Mad Rocket Scientists Site B Site A Site C Site D
- 3. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing (D-) Grid Grid . . . coordinated resource sharing and problem solving in dynamic multi-institutional virtual organizations (Foster/ Kesselmann/ Tuecke, 2001) . . . enables resource virtualization, on-demand provisioning, and service (resource) sharing between organizations (Plaszczak/ Wellner, 2005) Aim of the German Grid Initiative D-Grid Establish a national e-infrastructure Sustainable & longterm Grid infrastructure in Germany Targeted towards public research and private-public partnerships involving SME
- 4. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Time line D-Grid started late as other countries like UK or USA started Grid initiatives in the late 1990s or early 2000s.
- 5. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Time line Cloud Computing
- 6. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Projects in D-Grid In total: 35 projects in D-Grid Infrastructure & Services DGI, IVOM, D-MON, SuGI, SLA4D-Grid, . . . Academic HEP-CG, C3-Grid, AstroGrid-D, MediGrid, TextGrid, . . . Academic & SME Cooperation AeroGrid (T-Systems SfR), BIS-Grid, Biz2Grid, FinGrid (Deutsche Bank, Dresdener Bank), GDI-Grid, . . .
- 7. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Projects in D-Grid
- 8. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Compute and Storage Resources ≥ 30.000 CPU cores ≥ 5 PByte of storage 2009: ≥ 140 Mio CPUh consumed by users
- 9. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Utilization 2008 – 2010 9000000 AstroGrid-D ad BWGrid bw 7000000 8000000 9000000 AstroGrid-D ad BWGrid bw MediGrid md Total Linear (Total) 6000000 7000000 8000000 9000000 AstroGrid-D ad BWGrid bw MediGrid md Total Linear (Total) 4000000 5000000 6000000 7000000 8000000 9000000CPUh AstroGrid-D ad BWGrid bw MediGrid md Total Linear (Total) 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000CPUh AstroGrid-D ad BWGrid bw MediGrid md Total Linear (Total) 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000CPUh AstroGrid-D ad BWGrid bw MediGrid md Total Linear (Total) 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 Jul‐08 Aug‐08 Sep‐08 Okt‐08 Nov‐08 Dez‐08 Jan‐09 Feb‐09 Mrz‐09 Apr‐09 Mai‐09 Jun‐09 Jul‐09 Aug‐09 Sep‐09 Okt‐09 Nov‐09 Dez‐09 Jan‐10 Feb‐10 Mrz‐10 Apr‐10 Mai‐10 Jun‐10 CPUh AstroGrid-D ad BWGrid bw MediGrid md Total Linear (Total) 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 Jul‐08 Aug‐08 Sep‐08 Okt‐08 Nov‐08 Dez‐08 Jan‐09 Feb‐09 Mrz‐09 Apr‐09 Mai‐09 Jun‐09 Jul‐09 Aug‐09 Sep‐09 Okt‐09 Nov‐09 Dez‐09 Jan‐10 Feb‐10 Mrz‐10 Apr‐10 Mai‐10 Jun‐10 CPUh AstroGrid-D ad BWGrid bw MediGrid md Total Linear (Total)
- 10. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Current Software Stack Compute middleware gLite 3.2 UNICORE 6 Globus Toolkit 4 Storage middleware dCache 1.9 OGSA-DAI
- 11. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Software Stack OGSA DAI Databases SE dCache VSite UNICORE CE gLite GRAM WS Compute Frontends dCache Storage Frontends PoolToolkitGlobus Torque & MAUI File Systems Online StorageWorker nodes Functionality of this stack is ensured by running a test bed containing all services
- 12. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Software Stack Extension by Cloud VSite UNICORE CE gLite GRAM WS Torque & MAUI Worker Nodes Platform Virtualization Globus Toolkit Cloud Middleware New: Platform virtualization and Cloud middleware
- 13. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Hardware and Service Setup at DGRZR Compute Middleware gLite 3.1 and 3.2 UNICORE 5 and 6 Globus Toolkit 4.0.8 Storage Middleware dCache 1.9.5 OGSA-DAI WSRF 2.2 Hardware 2048 CPU cores, 108 TByte storage SLES10 SP3 with Xen, all services deployed in VMs
- 14. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing OpenNebula Statement on homepage Standard-based open-source toolkit to build private, public and hybrid clouds. [...] provides flexible architecture, interfaces and components that fit into any existing data center. Supports Xen, KVM and VMware (libvirt) Interfaces also with Amazon EC2 and ElasticHosts Other users CERN (cloud infrastructure consisting of 3,200 cores) NIKHEF China Mobile ( ” Big Cloud“ project)
- 15. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing OpenNebula at Dortmund Slightly modified ONE 2.0 RC1 (October 2010) Xen infrastructure and ssh image transfer User interaction via CLI and management console
- 16. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Executions Hosts & Virtual Networks
- 17. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Worker Node Template NAME = udo−wn300 VCPU = 8 MEMORY = 13312 OS = [ bootloader = ”/ root / bin /domUloader . py” ] DISK = [ source = <path >/wn sl54 x86 64 . img ” , t a r g e t = ”xvda ” , readonly = ”no” ] DISK = [ type = swap , s i z e = 1024 , t a r g e t = ”xvdb ” , readonly = ”no” ] DISK = [ type = ” block ” , clone = ” yes ” , source = ”/ dev/ c c i s s /c0d0p4 ” , t a r g e t = ” xvdc ” , readonly = ”no” ] NIC = [ NETWORK = ”DGRZRWorkernodes ” , IP = 129.217.242.166 ]
- 18. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Further plans at DGRZR Present situation Deployment of worker nodes (Private Cloud) Access to Cloud interfaces for D-Grid/ external users Split physical resources in Cloud & Grid partition Dynamic change of partition sizes Preemption or suspension of virtual machines → priority? Job types: simple batch jobs, MPI jobs, services Customer classes: bronze, silver, gold
- 19. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing (How) Should D-Grid deal with Clouds? Situation in 2009 Utilization of compute resources ≥ 140 Mio CPUh consumed, but this was only around 50% of the available CPUh 1/2 of D-Grid compute resources are idle Not really acceptable for a sustainable operation Introduce compute cloud interface as new pillar Not only my opinion. . . Grid is too complex and hence not attractive to SMEs
- 20. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Grid complexity – Resource Access Grid Application for X.509 certificate (1-2 days) Contact Certification authority Contact Registration authority (f2f required) Application for VO membership (hours - days) Updating of the authorization information on the Grid resources (hours - days) Cloud Register once via e. g. web and specify name, credit card information, . . .
- 21. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Grid complexity – Service Interaction
- 22. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Cloud-Enabling D-Grid Project start was on 01.06.2010, duration: 2 years fluid Operations’ eCloudManager (eCM) Self-service component for e. g. application deployment and resource monitoring D-Grid operators can use eCM for resource provisioning and data center operations Accounting & billing modules integrated in eCM Supports professional storage (NetApp, EMC, . . . ) Supports VMware, XenServer and Hyper-V Supports EC2 (external) SAP, LaaS
- 23. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Open issues Technical level User management Authorization Accounting & Billing Monitoring Information system Legal issues (e. g. liability) Legal status of a virtual organization? Company constituted under Civil law (Gesellschaft b¨urgerlichen Rechts) Joint and several liability
- 24. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing User management D-Grid VO central user management Resources query VOMRS servers and generate a mapping VO member → local user ID Attribute-based authorization Cloud (OpenNebula) Local user management, no attributes To Do Enable Cloud middleware to query VOM(R)S servers Can the concept of VOs be supported by cloud middleware?
- 25. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Accounting D-Grid Jobs pass along the compute middleware and reach the LRMS LRMS job accounting information published via DGAS 1 Cloud ” jobs“ do not start at LRMS level Cloud (OpenNebula) Basic accounting features To Do Evaluation of collectable metrics Translation of this metrics to a D-Grid compatible format 1 Distributed Grid Accounting System
- 26. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Information System Each Grid middleware uses a separate information system: BDII, CIS, MDS D-MON2 gathers and aggregates the information Development of a Cloud middleware adapter To Do Which information to publish? Available appliances, hardware limits (#CPU cores, RAM, . . . ) Cloud Resource Broker 2 http://www.d-grid.de/index.php?id=401
- 27. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Conclusion and Future Challenges Conclusion Cloudification of D-Grid Resource Center Ruhr Prototypic integration of a cloud middleware Open issues for integration of compute clouds in D-Grid Future Challenges Virtual Organizations in Clouds Cloudification of D-Grid Why should Grid providers get cloudy? Service Level Agreements
- 28. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloudat DGRZR D-Grid and Cloud Computing Questions? Slides available: http://www.slideshare.net/stefanfreitag Blog ” Grid at Dortmund“ http://gridatdortmund.blogspot.com/