Your SlideShare is downloading. ×
0
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Cloud Computing in D-Grid
Scie...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Collaboration of Mad Rocket Sc...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
(D-) Grid
Grid
. . . coordinat...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Time line
D-Grid started late ...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Time line
Cloud
Computing
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Projects in D-Grid
In total: 3...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Projects in D-Grid
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Compute and Storage Resources
...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Utilization 2008 – 2010
900000...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Current Software Stack
Compute...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Software Stack
OGSA
DAI
Databa...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Software Stack Extension by Cl...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Hardware and Service Setup at ...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
OpenNebula
Statement on homepa...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
OpenNebula at Dortmund
Slightl...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Executions Hosts & Virtual Net...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Worker Node Template
NAME = ud...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Further plans at DGRZR
Present...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
(How) Should D-Grid deal with ...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Grid complexity – Resource Acc...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Grid complexity – Service Inte...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Cloud-Enabling D-Grid
Project ...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Open issues
Technical level
Us...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
User management
D-Grid
VO cent...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Accounting
D-Grid
Jobs pass al...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Information System
Each Grid m...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Conclusion and Future Challeng...
Cloud
Computing in
D-Grid
S. Freitag
D-Grid
DGRZR
Cloud at
DGRZR
D-Grid and
Cloud
Computing
Questions?
Slides available:
h...
Upcoming SlideShare
Loading in...5
×

Cloud Computing in D-Grid

2,650

Published on

Published in: Education, Technology, Business
1 Comment
1 Like
Statistics
Notes
  • thank you soo much
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
2,650
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
87
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Cloud Computing in D-Grid"

  1. 1. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Cloud Computing in D-Grid Science Days 2010 Stefan Freitag Robotics Research Institute Dortmund University of Technology 03. November 2010
  2. 2. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Collaboration of Mad Rocket Scientists Site B Site A Site C Site D
  3. 3. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing (D-) Grid Grid . . . coordinated resource sharing and problem solving in dynamic multi-institutional virtual organizations (Foster/ Kesselmann/ Tuecke, 2001) . . . enables resource virtualization, on-demand provisioning, and service (resource) sharing between organizations (Plaszczak/ Wellner, 2005) Aim of the German Grid Initiative D-Grid Establish a national e-infrastructure Sustainable & longterm Grid infrastructure in Germany Targeted towards public research and private-public partnerships involving SME
  4. 4. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Time line D-Grid started late as other countries like UK or USA started Grid initiatives in the late 1990s or early 2000s.
  5. 5. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Time line Cloud Computing
  6. 6. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Projects in D-Grid In total: 35 projects in D-Grid Infrastructure & Services DGI, IVOM, D-MON, SuGI, SLA4D-Grid, . . . Academic HEP-CG, C3-Grid, AstroGrid-D, MediGrid, TextGrid, . . . Academic & SME Cooperation AeroGrid (T-Systems SfR), BIS-Grid, Biz2Grid, FinGrid (Deutsche Bank, Dresdener Bank), GDI-Grid, . . .
  7. 7. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Projects in D-Grid
  8. 8. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Compute and Storage Resources ≥ 30.000 CPU cores ≥ 5 PByte of storage 2009: ≥ 140 Mio CPUh consumed by users
  9. 9. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Utilization 2008 – 2010 9000000 AstroGrid-D ad BWGrid bw 7000000 8000000 9000000 AstroGrid-D ad BWGrid bw MediGrid md Total Linear (Total) 6000000 7000000 8000000 9000000 AstroGrid-D ad BWGrid bw MediGrid md Total Linear (Total) 4000000 5000000 6000000 7000000 8000000 9000000CPUh AstroGrid-D ad BWGrid bw MediGrid md Total Linear (Total) 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000CPUh AstroGrid-D ad BWGrid bw MediGrid md Total Linear (Total) 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000CPUh AstroGrid-D ad BWGrid bw MediGrid md Total Linear (Total) 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 Jul‐08 Aug‐08 Sep‐08 Okt‐08 Nov‐08 Dez‐08 Jan‐09 Feb‐09 Mrz‐09 Apr‐09 Mai‐09 Jun‐09 Jul‐09 Aug‐09 Sep‐09 Okt‐09 Nov‐09 Dez‐09 Jan‐10 Feb‐10 Mrz‐10 Apr‐10 Mai‐10 Jun‐10 CPUh AstroGrid-D ad BWGrid bw MediGrid md Total Linear (Total) 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 Jul‐08 Aug‐08 Sep‐08 Okt‐08 Nov‐08 Dez‐08 Jan‐09 Feb‐09 Mrz‐09 Apr‐09 Mai‐09 Jun‐09 Jul‐09 Aug‐09 Sep‐09 Okt‐09 Nov‐09 Dez‐09 Jan‐10 Feb‐10 Mrz‐10 Apr‐10 Mai‐10 Jun‐10 CPUh AstroGrid-D ad BWGrid bw MediGrid md Total Linear (Total)
  10. 10. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Current Software Stack Compute middleware gLite 3.2 UNICORE 6 Globus Toolkit 4 Storage middleware dCache 1.9 OGSA-DAI
  11. 11. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Software Stack OGSA DAI Databases SE dCache VSite UNICORE CE gLite GRAM WS Compute Frontends dCache Storage Frontends PoolToolkitGlobus Torque & MAUI File Systems Online StorageWorker nodes Functionality of this stack is ensured by running a test bed containing all services
  12. 12. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Software Stack Extension by Cloud VSite UNICORE CE gLite GRAM WS Torque & MAUI Worker Nodes Platform Virtualization Globus Toolkit Cloud Middleware New: Platform virtualization and Cloud middleware
  13. 13. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Hardware and Service Setup at DGRZR Compute Middleware gLite 3.1 and 3.2 UNICORE 5 and 6 Globus Toolkit 4.0.8 Storage Middleware dCache 1.9.5 OGSA-DAI WSRF 2.2 Hardware 2048 CPU cores, 108 TByte storage SLES10 SP3 with Xen, all services deployed in VMs
  14. 14. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing OpenNebula Statement on homepage Standard-based open-source toolkit to build private, public and hybrid clouds. [...] provides flexible architecture, interfaces and components that fit into any existing data center. Supports Xen, KVM and VMware (libvirt) Interfaces also with Amazon EC2 and ElasticHosts Other users CERN (cloud infrastructure consisting of 3,200 cores) NIKHEF China Mobile ( ” Big Cloud“ project)
  15. 15. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing OpenNebula at Dortmund Slightly modified ONE 2.0 RC1 (October 2010) Xen infrastructure and ssh image transfer User interaction via CLI and management console
  16. 16. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Executions Hosts & Virtual Networks
  17. 17. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Worker Node Template NAME = udo−wn300 VCPU = 8 MEMORY = 13312 OS = [ bootloader = ”/ root / bin /domUloader . py” ] DISK = [ source = <path >/wn sl54 x86 64 . img ” , t a r g e t = ”xvda ” , readonly = ”no” ] DISK = [ type = swap , s i z e = 1024 , t a r g e t = ”xvdb ” , readonly = ”no” ] DISK = [ type = ” block ” , clone = ” yes ” , source = ”/ dev/ c c i s s /c0d0p4 ” , t a r g e t = ” xvdc ” , readonly = ”no” ] NIC = [ NETWORK = ”DGRZRWorkernodes ” , IP = 129.217.242.166 ]
  18. 18. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Further plans at DGRZR Present situation Deployment of worker nodes (Private Cloud) Access to Cloud interfaces for D-Grid/ external users Split physical resources in Cloud & Grid partition Dynamic change of partition sizes Preemption or suspension of virtual machines → priority? Job types: simple batch jobs, MPI jobs, services Customer classes: bronze, silver, gold
  19. 19. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing (How) Should D-Grid deal with Clouds? Situation in 2009 Utilization of compute resources ≥ 140 Mio CPUh consumed, but this was only around 50% of the available CPUh 1/2 of D-Grid compute resources are idle Not really acceptable for a sustainable operation Introduce compute cloud interface as new pillar Not only my opinion. . . Grid is too complex and hence not attractive to SMEs
  20. 20. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Grid complexity – Resource Access Grid Application for X.509 certificate (1-2 days) Contact Certification authority Contact Registration authority (f2f required) Application for VO membership (hours - days) Updating of the authorization information on the Grid resources (hours - days) Cloud Register once via e. g. web and specify name, credit card information, . . .
  21. 21. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Grid complexity – Service Interaction
  22. 22. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Cloud-Enabling D-Grid Project start was on 01.06.2010, duration: 2 years fluid Operations’ eCloudManager (eCM) Self-service component for e. g. application deployment and resource monitoring D-Grid operators can use eCM for resource provisioning and data center operations Accounting & billing modules integrated in eCM Supports professional storage (NetApp, EMC, . . . ) Supports VMware, XenServer and Hyper-V Supports EC2 (external) SAP, LaaS
  23. 23. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Open issues Technical level User management Authorization Accounting & Billing Monitoring Information system Legal issues (e. g. liability) Legal status of a virtual organization? Company constituted under Civil law (Gesellschaft b¨urgerlichen Rechts) Joint and several liability
  24. 24. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing User management D-Grid VO central user management Resources query VOMRS servers and generate a mapping VO member → local user ID Attribute-based authorization Cloud (OpenNebula) Local user management, no attributes To Do Enable Cloud middleware to query VOM(R)S servers Can the concept of VOs be supported by cloud middleware?
  25. 25. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Accounting D-Grid Jobs pass along the compute middleware and reach the LRMS LRMS job accounting information published via DGAS 1 Cloud ” jobs“ do not start at LRMS level Cloud (OpenNebula) Basic accounting features To Do Evaluation of collectable metrics Translation of this metrics to a D-Grid compatible format 1 Distributed Grid Accounting System
  26. 26. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Information System Each Grid middleware uses a separate information system: BDII, CIS, MDS D-MON2 gathers and aggregates the information Development of a Cloud middleware adapter To Do Which information to publish? Available appliances, hardware limits (#CPU cores, RAM, . . . ) Cloud Resource Broker 2 http://www.d-grid.de/index.php?id=401
  27. 27. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Conclusion and Future Challenges Conclusion Cloudification of D-Grid Resource Center Ruhr Prototypic integration of a cloud middleware Open issues for integration of compute clouds in D-Grid Future Challenges Virtual Organizations in Clouds Cloudification of D-Grid Why should Grid providers get cloudy? Service Level Agreements
  28. 28. Cloud Computing in D-Grid S. Freitag D-Grid DGRZR Cloud at DGRZR D-Grid and Cloud Computing Questions? Slides available: http://www.slideshare.net/stefanfreitag Blog ” Grid at Dortmund“ http://gridatdortmund.blogspot.com/
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×