SUCRE CLOUDSOURCE MAGAZINE

1

Scaling Software Challenges
Alvaro Simón

Carlos Fernández

Victor Mendez

CESGA
asimon@ces...
2

academia communities: TERENA [17] (TFMSP, TF­Stor­
age,   TF­EMC2),   OpenNebula   Interoperability   Working 
Group [5...
SUCRE CLOUDSOURCE MAGAZINE
3

RIA, Institut Télécom, Institut Polytechnique de Greno­
ble, Joseph Fourier Laboratory of In...
Upcoming SlideShare
Loading in...5
×

Scaling software challenges

162

Published on

Megha, the Federated Cloud pilot running in Spain (2010 - 2013), and previous successful projects such as OSMOSE (Open Source Middleware for Open Systems in Europe 2003-2005), OSIRIS (Open Source Infrastructure fun Run-Time Integration of Services  2005-2008) and OSAmI-Commons (Open Source Ambient Intelligence Commons 2008-2011) have constituted excellent foundations for further steps. However, while software is becoming more open, more software will be needed. Managing the increasing amount of software requires new approaches beyond technical scalability issues.  Previous running initiatives and recent initiated projects are presented as well as their benefits in the development of an expanding digital universe.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
162
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Scaling software challenges

  1. 1. SUCRE CLOUDSOURCE MAGAZINE 1 Scaling Software Challenges Alvaro Simón Carlos Fernández Victor Mendez CESGA asimon@cesga.es CESGA carlosf@cesga.es   PIC vmendez@pic.es Jordi Guijarro Jesús Bermejo CESCA jguijarro@cesca.cat Telvent jesus.bermejo@telvent.com Abstract—Megha, the Federated Cloud pilot running in Spain (2010 - 2013), and previous successful projects such as OSMOSE (Open Source Middleware for Open Systems in Europe 2003-2005), OSIRIS (Open Source Infrastructure fun Run-Time Integration of Services 2005-2008) and OSAmI-Commons (Open Source Ambient Intelligence Commons 2008-2011) have constituted excellent foundations for further steps. However, while software is becoming more open, more software will be needed. Managing the increasing amount of software requires new approaches beyond technical scalability issues. Previous running initiatives and recent initiated projects are presented as well as their benefits in the development of an expanding digital universe. Index Terms— Cloud Computing, Federated Cloud, Intercloud, Open Source, Scalability, Software Engineering ——————————  —————————— 1 INTRODUCTION O   nly recently the explosion of the digital universe  has   been   recognized.  Black   Duck   KnowledgeBase  [1]  includes  information from 800.000 projects  in more  than 5.500 sites and 2.200 software licenses (GPL, LGPL,  Apache,   etc).   On   the   other   hand,   information   in   the  world is doubling every two years [2] and every day, 2.5  quintillion   bytes   of   data   are   created.   Over   90%   of   the  data in the world today has been created in the last three  years [3].  ous challenge.  3 OSMOSE, OSIRIS AND OSAMI-COMMONS PROJECTS In   the   technical   dimension   a   remarkable   effort   for  tackling scaling software challenges has been carried out  during   the   last   years   through   several   projects   such   as  OSMOSE (Open Source Middle­ware for Open Systems  in Europe, 2003­2005) [10], OSIRIS (Open Source Infra­ structure   for   Run­Time   Integration   of   Services,   2005­ 2008) [11] and OSAmI­Commons (Open Source Ambient  Intelligence   Commons,   2008­2011)   [12]   .   Open   source  modular and dynamic middleware foundations, service  bus implementations, federated identity and reusability  frameworks   have  contributed   importantly   for  reducing  development efforts demonstrating the benefits for social  relevant   areas   such   as   sustainability,   health,   smart  homes, city services and  education. The links between  composite   and   virtualization   cloud   approaches   have  been   identified   leading   to   the   setting­up   of   MEGHA  Federated Cloud (2010) or Intercloud initiative (i.e. inter­ connected cloud of clouds vs. Internet as network of net­ works [13]). The software is also growing dramatically out of the  traditional software sector. The ITEA [4] Report on Soft­ ware   Intensive   Systems   (2005)   [5]   analyzed   software  growth in six industrial sectors (automotive, aerospace,  medical   equipment,   telecom   equipment   and   consumer  electronics) concluding that software is leading innova­ tion in all these market segments. In fact, currently, it is  difficult to identify a domain in which innovation does  not rely on software.  2 EXPLORING THE ECONOMIC RELEVANCE OF THE SOFTWARE Nicholas   Negropote   used   the   “Digital   Economy”   term  for referring to the shift from processing atoms to pro­ cessing bits [6]. The difference between bits and atoms is  fundamental to the  consequences of living  in a  digital  world   [7].   An   OECD   effort   in   understanding   the   eco­ nomic relevance of the software was initiated in 2007 [8].  Two conferences supported this OECD Software Innova­ tion Study; the first in Cáceres, Spain (November 2007),  and the second in Tokyo (October 2008). The study ad­ dressed innovation themes such as security, privacy, mo­ bility, interoperability, accessibility and reliability from a  user perspective [9].  It also raised the fact that identify­ ing the boundaries of the software industry is a continu­ 4 MEGHA FEDERATED CLOUD Research   and   academic   communities   face   common  challenges not only for adapting their procedures to the  new computing paradigms, but also for contributing to  its development. MEGHA Working Group was originally  initiated to coordinate this within the scope of the insti­ tutions   affiliated   to   RedIRIS   [14]   in   Spain   addressing  R&D, education and management aspects. From the be­ ginning MEGHA established direct links with initiatives  such as e­Science [15] and CRUE­TIC [16] in Spain and  internationally   with   projects   in   the   research   and  March 2013
  2. 2. 2 academia communities: TERENA [17] (TFMSP, TF­Stor­ age,   TF­EMC2),   OpenNebula   Interoperability   Working  Group [5], GÉANT [19], EGI [20] and OGF [21]. SUCRE CLOUDSOURCE MAGAZINE . MEGHA authentication is based in x509 users and robot  certificates issued by Spanish pkIRISGrid CA. This new  feature was used by PIC developers to enhance DIRAC  software framework [25] originally developed by LHCb  [26] to manage computing operations on Grid Comput­ ing infrastructures. The new cloud plugin developed by  PIC and USC teams integrates a cloud broker, user au­ thentication and supports different cloud managers such  as OpenNebula or CloudStack [27]. Currently MEGHA  members are working to integrate Virtual Organizations  (VOs) or a dynamic set of users to share federated cloud  resources.  5 ADDRESSING SCALING SOFTWARE DIMENSIONS Fig 1 Megha concept validation testbed 4.1 Concept validation In the first phase (2010­2011) MEGHA validated feder­ ated cloud platforms using OCCI [22] to streamline the  use   of  cloud   technologies   among   R&E   service   centers.  Representative   infrastructure   providers   (CESCA,  CESGA,   PIC),   middle­ware   providers:   OpenNebula,  RedIRIS, OSAmI­Commons and users (UAB, UOC, UM)  together   with   Intermediate/Identity/brokers   resources  (RedIRIS) joined efforts to demonstrate the viability of  the approach. The results stimulated the development on  use cases among users such as: E­learning platforms on  demand (Academic Services), Distributed HPC platform  (e­Science) and Virtual Labs (VDI) in a hybrid scenario  (Academic services). 4.2 On-going developments The   federation   of   clouds   for   the   research   and   aca­ demic communities could be defined as an aggregation  of multiple Community Clouds. In this context, federa­ tion   requirements   must   address   new   technical   chal­ lenges:  • Federated user authentication and authorization  mechanisms and user management between dif­ ferent cloud managers  • Secure   VM   image   distribution   and   validation  among heterogeneous cloud managers • Federated Cloud accounting system integrating  the accounting records of multiple cloud man­ agers and supporting the Federated Cloud gov­ ernance • Monitoring   and   notification   of   unpredictable  change of availability and readability status   MEGHA is currently working in different solutions to  solve these challenges.   The new rOCCI [23] server and  OCCI [24] clients tested by CESGA and PIC teams with  OpenNebula 3.8.x are able to use x509 user certificates as  authentication method.  While the digital universe is exploding, information  technology staff is growing at a much lower rate. Master­ ing scaling software is not only related to software tech­ nologies rather it is a consequence of managing relevant  business models, organizations, methods, processes and  tools.   For   supporting   this   objective   previous   web   and  SOA workshops (JSWEB) and the one targeting Business  Processes   and   Services   Engineering  (PNIS)   joined   in   a  single event integrating Science and Engineering of Ser­ vices  (JCIS) [28]. On the other hand, SCALARE project  (Scaling Software) [29] is being initiated among several  European partners for addressing scaling software chal­ lenges across its several dimensions.  6 CONCLUSION The digital technologies are transforming our society  faster   than   ever   while  the   increasing   relevance   of   the  software in non­software markets, and in the society in  general, makes difficult to identify the boundaries of the  software industry. The efforts presented in the article are  only a few of many others tackling scaling software chal­ lenges   (including   those   derived   from   the   explosion   of  available data).  Cloud federation has been validated as a suitable ap­ proach for building upon while more software needs are  leading to the technology staff becoming the future bot­ tleneck. Therefore, increasing software development effi­ ciency is needed to address future challenges as well as  further research to fully understand the implications of  merging two worlds with very different behavior rules. ACKNOWLEDGMENT The authors wish to thank the partners contributing  to the mentioned projects: Aicia, Autonomous University  of Barcelona, Bantry technologies, Bull S.A.A., Cenatic,  Capricode,   CARE   Technologies,   Carlos   III   University,  CESCA,   CESGA,   Corscience,   Charles   University   in  Prague,   Dortmund   University,   École   polytechnique  fédérale de Lausanne EPFL, Electricité De France, Espo­ tel, Eteration, Fidetia, France Télécom, ICT­Norway, IN­
  3. 3. SUCRE CLOUDSOURCE MAGAZINE 3 RIA, Institut Télécom, Institut Polytechnique de Greno­ ble, Joseph Fourier Laboratory of Informatics of Greno­ ble,   Laboratoire  d'Informatique   Fondamentale   de   Lille,  Laboratoire   LSR,   Lynx,   Malaga   University,   Materna  GmbH,   mFabrik   Research   Oy,   OpenNebula   Project,  Open University of Catalonia, Paderborn University, PIC,  Politechnic   University   of   Valencia,   Philips,   Prodevelop,  Prosyst   Software   GmbH,   RedIRIS,   Rostock   University,  Sampas, Seville University, Siemens, Schneider Electric,  Schüchtermann­Schiller'sche   Kliniken,   SRDC,   Technical  University   of   Madrid   (UPM),   Telefónica   I+D,   Tecnalia,  Telvent, Thales, Vigo University, Vodafone, VTT Techni­ cal Research Centre of Finland, Whitestein Technologies  A.G., Wisur The   work  in  the  mentioned  projects  was  supported   in  part by grants from the Ministries of Austria, Czech Re­ public,   Finland,   France,   Germany,   Ireland,   Norway,  Spain, Sweden, Switzerland, The Netherlands and Tur­ key. REFERENCES [1] Black Duck KnowledgeBase [Online]. Available:                                                             http://www.blackducksoftware.com/products/knowledg-base  [2] EMC Corporation [Online]. Available: http://www.emc.com/leadership/programs/digital­universe.htm [3] IBM [Online].  Available:  http://www-01.ibm.com/software/data/bigdata/ [4] ITEA 2 [Online]. Available: www.itea2.org [5] TNO/IDATE Softwasre Intensive Systems in the Future September 2005V5 [6] Digital Economy Definition [Online]. Available: http://en.wikipedia.org/wiki/Digital_economy [7] Phoenix University [Online]. Available: https://www.phoenix.edu/lectures/nicholas-negroponte/bits-and-atoms.html [8] Organization for Economic Co­operation and Development. OECD. [Online]. Available:  http://www.oecd.org/sti/inno/innovationinthesoftwaresector.htm#background [9] Organization   for   Economic   Co­operation   and   Development.   OECD.   [Online].   Available:   [Online].   Available:  http://www.oecd.org/sti/ind/44131881.pdf [10] Open Source Middleware for Open Systems in Europe (OSMOSE) Project [Online]. Available:   http://www.itea2.org/project/index/view/?project=46 [11] Open Source Infrastructure for Run­tieme Integration of Services (OSIRIS) Project [Online].Available:  http://www.itea2.org/project/index/view/?project=135 [12] Open Source Ambient Intelligence Commons(OSAmI­Commons) Project [Online]. Available:  http://www.itea2.org/project/index/view/?project=230 [13] Intercloud Definition [Online]. Available: http://en.wikipedia.org/wiki/Intercloud [14] Megha Working Group [Online]. Available: http://wiki.rediris.es/megha/MainPage [15] Spanish e­Science Network [Online]. Available: http://www.e­ciencia.es/ [16] ICT Comission of Spanish University Chancellors Conference. CRUE­TIC[Online]. Available: http://www.crue.org/TIC/ [17] Trans-European Research and Education Networking Association [Online]. Available: http://www.terena.org/ [18] OpenNebula Interoperability Group [Online]. Available: http://blog.opennebula.org/?p=1893 [19] GÉANT Network [Online].Available: http://www.geant.net/pages/home.aspx [20] European Grid Infrastructure. EGI [Online].Available: http://www.egi.eu/ [21] Open Grid Forum [Online].Available: http://www.gridforum.org/ [22] Open Cloud Computing Interface (OCCI) [Online]. Available: http://occi-wg.org/ [23] rOCCI server [Online]. Available: https://github.com/gwdg/rOCCI-server [24] Thijs Metsch, Andy Edmonds: Open Cloud Computing Interface. OGF.org (2010) [Online]. Available:  http://goo.gl/MxX19 [25] Distributed Infrastructure with Remote Agent Control (DIRAC) [Online]. Available: http://diracgrid.org/ [26] Large Hadron Collider beauty (LHCb) http://lhcb-comp.web.cern.ch/lhcb-comp/DIRAC/. [27] Méndez, V., Fernández, V., Graciani, R., Casajus, A., Fernández, T., Merino, G., Saborido, J.J.: The integra­ tion of Cloudstack and OCCI/Opennebula with DIRAC. Journal of Physics Conference Series (2013) [28] IX Conference  on Science and Service Engineering [Online].Available: http://www.kybele.etsii.urjc.es/jcis2013/ [29] Scaling Software (SCALARE) Project [Online]. Available: http://scalare.org/about-scalare/

×