Community Clouds from Scratch Cloud Computing represents a radical change in the way we organize and use computing resources and storage. The scientific and academic communities face the challenge of not only adapting their procedures to this new paradigm, but also contributing Cloud Computing development and leading its evolution towards open, secure and interoperable computing infrastructures, which will playing a key role in the community clouds paradigm. The Spanish MEGHA initiative promotes and coordinates contributions to cloud computing R&D, education and management made by institutions affiliated with RedIRIS [7] in Spain. In the first phase (2010–2012), MEGHA validated federated cloud platforms using Opennebula and OCCI [10] to streamline the use of cloud technologies among R&E service centers. Representative infrastructure providers (CESCA, CESGA, PIC), middleware providers (OpenNebula, RedIRIS, OSAmI-Commons) and users (UAB, UOC, UM) together with intermediate/identity/brokers resources (RedIRIS) joined efforts to demonstrate the viability of this approach. The results stimulated the development of use cases including e-learning platforms on demand (Learning Apps project), a distributed HPC platform (e-Science), and Virtual Labs (VDI) in a hybrid scenario (Academic services). Next Steps? As next goal, the Spanish research and academic community is working to assess the possibilities of creating a productive Infrastructure Cloud Computing service within member institutions. With this new approach new challenges appear: Federated user authentication and authorization mechanisms. Brokering architecture scenario. Secure VM image distribution and validation. A federated cloud accounting system integrating the accounting records of multiple cloud managers and supporting federated cloud governance. Monitoring and notification of unpredictable changes in availability and readability status. Security Policies and Service Level Agreements (SLA’s).

2. Community
Clouds from Scratch
A Real World Introduction
Jordi Guijarro - jguijarro@cesca.cat
@cloudadms
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

4. About

5. What Cloud
Computing really
means for R&E
A strategy, a tactical solution...
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

6. R&E Networks
A Research and Education Network is a
non-profit
Internet
service
provider
dedicated to supporting the needs of the
research and education communities.
It has a high-speed backbone network.
Research and Education Networks are usually
the places where new Internet protocols and
architectures are introduced before their
public deployment:
Protocols: IPv6 and multicast.
Architecture: Client/Server and Cloud Computing
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

7. R&E Networks
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

8. What Cloud Computing really means for R&E Community ?
JUST A BUZZWORD?
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

9. What Cloud Computing really means
SELFSERVICE(s)?
jguijarro@cesca..cat Berlin 2013
jguijarro@cesca,.cat – Reykjavik 2012
jguijarro@cesca,.cat
OpenNebula TNC 2013
Conf 2012

10. What Cloud Computing really mean for R&E Community?
RESPONSIBILITIES...
jguijarro@cesca..cat Berlin 2013
jguijarro@cesca,.cat – Reykjavik 2012
jguijarro@cesca,.cat
OpenNebula TNC 2013
Conf 2012

11. MEGHA Working
Group
Let's pull all together...
jguijarro@cesca..cat Berlin 2013
jguijarro@cesca,.cat – Reykjavik 2012
jguijarro@cesca,.cat
OpenNebula TNC 2013
Conf 2012

12. MEGHA Working Group
MISSION
http://wiki.rediris.es/megha
TO PROMOTE
the development of
innovation in Cloud
Computing technologies
within the scope of the
institutions affiliated to
RedIRIS
OPEN
USE CASES
INTEROPERABLE
TESTBED
PROMOTION
IDENTITY
TO ACT AS A CATALYST
for the application of
open and interoperable
Cloud technologies in
other areas in which it
can influence, such as
business in government.
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

13. MEGHA Working Group
MAIN OBJECTIVES
Deploy a federated Cloud infrastructure
as a platform for innovation and
evaluation.
Explore
the
possibilities
for
the
application of Cloud computing has in
research, academic and management
environments.
Facilitate the creation of an ecosystem of
open-source middleware components to
deploy
interoperable
Clouds
and
coordinate
training
and
education
activities for its deployment.
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

14. MEGHA Intercloud
project
From myth to reality...
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

15. MEGHA Intercloud Project
INITIAL STAGE
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

16. CESCA Cloud Infrastructure

17. CESGA Cloud Infrastructure

18. MEGHA Intercloud Project
INTEROPERABILITY
A “Virtual Poka Yoke” ?
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

19. MEGHA Intercloud Project
OPEN PLEASE!
Director,...
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

20. MEGHA Intercloud Project
INTEROPERABLE
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

21. MEGHA Intercloud Project
FEDERATED
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

22. MEGHA Intercloud Project
FEDERATED
OpenNebula 3.8
PKIRISGRID
X.509 Cert
UI
jguijarro@cesca..cat – Berlin 2013
r
OpenNebula Conf 2013

23. MATURING TOGETHER
Images, instances and networks management.
Public high speed connectivity.
SSH access (root).
User manual & Service Provider Howtos
(ONE 3.x)
“Terms of Service”.
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

24. MEGHA Intercloud Project
MAIN PLAYERS
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

25. Some Real World
User Cases
Cloudified...
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

26. REAL WORLD USER CASES
http://www.learningappsstore.org/
UOC: Learning Apps
A store on the
internet inspired by
Apple's App Store
which focuses on
educational
applications
The provision of the
educational
environments is
automated (OCCI 0.8)
and the amount of infr.
depends on parameters
such as a preconfigured
concurrency.
jguijarro@cesca..cat – Berlin 2013
RESTFUL API
AUTOMATION
ON DEMAND
EDUCATION
ACADEMIC
CONTEXTUALIZATION
ACADEMIC
OpenNebula Conf 2013

27. jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

28. REAL WORLD USER CASES
PIC: Grid & Cloud
DIRAC (Interware)
allows homogeneous
access to
heterogeneous
computing resources.
INTERWARE
DIRAC
INTEGRATION
SCIENTIFIC
Providing transparent
access and interoperability
between Grids and Clouds
OCCI 1.1 interface.
CONTEXTUALIZATION
Víctor Méndez (www.pic.es)
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

29. REAL WORLD USER CASES
PIC: Grid & Cloud
Integration Design: VM Job Running
DIRAC WMS
Task
Queue
Virtual
Machine
DIRAC CS
Command-line
DIRAC API
MeghaCloud
Infrastructure
VM Scheduler
Admin
User
Web Portal
VM Director
Cloud Driver
Cloud Manager
Image Repo
Víctor Méndez (www.pic.es)
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

30. REAL WORLD USER CASES
PIC: Grid & Cloud
Integration Design: Occi VM Submission
DIRAC WMS
Task
Queue
Virtual
Machine
MeghaCloud
Infrastructure
VM Scheduler
?
OCCI Director
(VM Director)
OCCI Image
OCCI / OpenNebula
OCCI RESTful
Client
Image Repo
Víctor Méndez (www.pic.es)
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

31. REAL WORLD USER CASES
PIC: Grid & Cloud
Integration Design: VM Job Running
Virtual Machine
VM
VM
VM
Job Agent
MeghaCloud
Infrastructure
DIRAC WMS
Task
Queue
Job
Virtual
Machine
Job Run
CPU statistics
Job Matcher
Job State Update
Cloud Manager
VM Monitor
VM Manager
Víctor Méndez (www.pic.es)
jguijarro@cesca..cat Berlin 2013
jguijarro@cesca,.cat – Reykjavik 2012
jguijarro@cesca,.cat
OpenNebula TNC 2013
Conf 2012

32. Next Steps...
This is only the tip of the iceberg
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

33. NEXT STEPS...
DETECTED PROBLEMS
Small and medium Scientific Research centers
don ́t have the manpower or know-how to
build a 24/365 data center
Server deployment takes a long time
Can t solve peak problems
́
Usually can ́t have more than one data center
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

34. NEXT STEPS FOR...
TEST PLATFORM
COMPATIBILITY AND FEATURES
TO BUILD A MARKETPLACE ?
VO's ? SUNSTONE FEDERATION ?
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

35. NEXT STEPS...
REAL SCENARIO
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

36. Some
conclusions...
Options to cater to all tastes!
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

37. NEXT STEPS...
NEW CHALLENGES
Federated user authentication and authorization
mechanisms and user management between
different cloud managers.
Secure VM image distribution and validation.
among heterogeneous cloud managers.
Federated Cloud accounting system integrating
the accounting records of multiple cloud managers
and supporting the Federated Cloud governance.
Monitoring and notification of unpredictable
change of availability.
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

38. FUTURE COMMUNITY FEDERATED CLOUD MODEL
Scalability (Commercial prov. included)
Distributed scenario
Resource sharing
High Availability
Efficient use of infrastructure
jguijarro@cesca..cat – Berlin 2013
OpenNebula Conf 2013

39. Gràcies!
Gracias!
Thanks!
Dedicated to those who made ​MEGHA possible.
Strategy, strategy, strategy, ...
Jordi Guijarro
jguijarro@cesca.cat
@cloudadms