2004 2007 20102009 20112008 2012
Public/ private projects
Pre-project Phase 1
Project structure of D-Grid
Academic service and
D – Grid GmbH
Industrial service and
Representation structure in D-Grid
High Energy Physics
Building & Construction Industry
Geospatial Information Systems
Meteorology for regenerative energy industry
Dynamic Service Value Networks
Backup & Recovery
Sustainable Grid Infrastructure
Networks of Knowledge in the Grid
Gap & Service Projects
Situation in 2005
High energy physicists
● advanced knowledge (CERN experiments)
● gLite middleware
● not really any prior knowledge
How to make all communities happy?
● Let them choose!
● Community requirements still unclear
● Offer a set of commonly used middlewares
D-Grid Reference System
● Template for a D-Grid compute and storage resource
● Targeted at small and medium resource centers
● Technical documentation & software packages available to system
● Validation of packages and documentation by installation “from
● Release Cycle: six months
● Reference system is only a recommendation
● Support available
> 30.000 CPU cores
> 5 PByte of storage
In 2009 >= 140 Mio.
CPUh consumed by
Technology is not really a problem!
Focus on existing concepts
● Support for different Grid middlewares
Bridging the gap to services and applications
● Workflow engines
● Commercial software
Addressing new requirements
● Dynamic virtual organizations
Cooperation between different projects
Heterogeneous Provider Community
Some large hardware providers play a vital role in D-Grid
● Development of a business model for basic services
Some providers used D-Grid to acquire new hardware
● Investments to support a transitional phase to Grid systems
No known concepts to provide IaaS type of services
● Obstacle for commercial users
● Competition to cloud computing?
There are not enough users!
Prominent figures of the various disciplines have not yet joined
● Imbalance between computer science and application science
Most users are not willing to provide the required additional effort.
● Potential lack of focus in the application area
Industrial users are pushed off by bureaucratic obstacles
● Use of D-Grid hardware for commercial purpose
Grid computing is a solution, isn’t it?
Funding agencies cannot be wrong!
More than 100 Mio € in D-Grid
Industry cannot be wrong!
Science cannot be wrong!
LHC production Grid
Due to the weather conditions...
Photo by: Paolo Malgari
Cloud Computing in D-Grid ?
Situation in 2009
● Utilization of compute resources ≥ 140 Mio CPUh
● only around 50% of the available CPUh
½ of D-Grid compute resources idling
● Turn them off → GreenIT
● Not really acceptable for a sustainable operation
→ Introduce compute cloud interface as new pillar
● Grid is too complex and hence not attractive to SMEs
Attractivity of Cloud Interface I
● Register once via web. Specify name, credit card
information,... (few minutes)
● Application for X.509 certificate (1-2 days)
Contact Certification authority
Contact Registration authority (f2f required)
● Application for VO membership (hours - days)
● Updating of the authorization information on the Grid
resources (hours - days)
Cloud Computing in D-Grid
Extension of the
What to add?
D-Grid Cloud Prototype
Located at Dortmund University of Technology
● Compute middlewares
– gLite 3.2
– UNICORE 6
– Globus Toolkit 4.0.8
● Storage middlewares
● dCache 1.9.5
● OGSA-DAI WSRF 2.2
● 2048 CPU cores, 108 TByte storage
All services deployed in Xen virtual machines
Statement on homepage
● Standard-based open-source toolkit to build private, public and
hybrid clouds. [...] provides flexible architecture, interfaces and
components that fit into any existing data center.
Supports Xen, KVM and VMware
Interfaces with Amazon EC2 and ElasticHosts
● CERN (cloud infrastructure consisting of 3,200 cores)
● NIKHEF, Netherlands
Accounting and Billing
→ Interfaces to all D-Grid core services
● VO central user management
● Resources query VOMRS servers and generate a mapping
● VO member → local user ID
● Attribute-based authorization
● Local user management,
● No Attribute-based authorization
Can the concept of VOs and Cloud middleware go together?
Reference System Acceptance
Only a few sites
Follow the recommended setup
Use the technical documentation
Why should a resource install all middlewares?
System administrators are not qualified to support all
Lack of interest at resource provider level
Migration/ Upgrade path unspecified
Legal Issues – Dual Use
Dual-use technology: a technology which can be used for
both peaceful and military aims.
Most industrial countries have export controls on certain
types of designated dual-use technologies […] These
controls restrict the export of certain commodities and
technologies without the permission of the government.
● Is Grid a dual-use technology?
● Export embargo for dual-use technologies to Iran
● Technical solution for a legal requirement
Legal Issues – Liability
With the development of business models, the liability of single
VO members and of the whole VO is of interest
What is the legal status of a virtual organization?
● Company constituted under Civil law
→ Joint and several liability
● In joint and several agreements, the liability for default is
enforceable against all of the signatories as a group[...], or against
any one of them as an individual at the choice of the enforcing party
Can business models and virtual organizations co-exist?
Avoid Parallel Universes
Before D-Grid German HEPs used WLCG resources
In 2010 German HEPs still use WLCG resources
● and not a single one uses D-Grid resources
● Accessibility to compute resources, data and experiment software is
limited by the boundaries of a virtual organization
● In WLCG VOs for ALICE, ATLAS, CMS and LHCb were already
● In D-Grid a HEP-CG VO was established with the aim to absorb all
HEPs independent from their origin
● Even if HEPs joined the HEP-CG VO, they were not able to access
the experiment software
Future Challenge: D-Grid/ NGI-DE
EGI was still EGEE as D-Grid started
With the end of EGEE-III, NGI-DE was established
Since then D-Grid and NGI-DE co-exist in Germany
Merge to only one initiative
● Avoid competition
● Combine endeavors for creating a sustainable infrastructure
● Removal of duplicity of service (GRRS/ GOC DB)
● Integration of D-Grid services (D-Mon)
● Operations concept (1:1 mapping, pool accounts)