Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

QuESo: a Quality Model for Open Source Software Ecosystems

1,811 views

Published on

Open source software has witnessed an exponential growth in the last two decades and it is playing an increasingly
important role in many companies and organizations leading to the formation of open source software
ecosystems. In this paper we present a quality model that will allow the evaluation of those ecosystems in
terms of their relevant quality characteristics such as health or activeness.

Published in: Software
  • Be the first to comment

QuESo: a Quality Model for Open Source Software Ecosystems

  1. 1. QuESo: a Quality Model for Open Source Software EcosystemsICSOFT-EA 2014 Oscar Franco-Bedoya ohernan@essi.upc.edu David Ameller dameller@essi.upc.edu Dolors Costal dolors@essi.upc.edu Xavier Franch franch@essi.upc.edu
  2. 2. 2 Related work Method QuESo Application example Discussion Future work Conclusions Outline Overview
  3. 3. 3 0 500.000 1.000.000 1.500.000 2.000.000 2.500.000 2007 2009 2011 2013 2015 The number of open source software projects is growing Source: Black Duck Management webinar 2014 in IEEE OSS webcasts series
  4. 4. 4 A challenge arises How can we measure the quality of these OSS projects? Way to preventbad decisions Avoidproblems Verifythe compliance with requirements and business goals Systematic monitoring to provide feedback
  5. 5. 5 The ecosystem metaphor is used to understand software relationships A systemformed by the interactionof a community of organisms with their environment Developers Adopters Software Projects Contributors Software products
  6. 6. 6 Software ecosystems definitions(Messerschmitt and Szyperski, 2005) Business perspective “Setof actorsfunctioning as a unit and interacting with a shared market for software and services.” (Jansen and Cusumano, 2013) Technical perspective “ Acollection of software projects which are developedandevolve together in the same environment.” (Lunguet al., 2009).
  7. 7. 7 We use both definitions in our work In order to assesssoftware ecosystem quality in its broader sense TechnicalperspectiveBusinesspersepctiveOSSecosystem
  8. 8. 8 From the challenge: How can we measure the quality of these OSS projects? To the research goal of this work: How can we measure the quality of these OSS ecosystems?
  9. 9. 9 Proposal: measure OSS ecosystem quality using a quality model “ quality model is the set of characteristicsand the relationshipsbetween them which provide the basis for specifying quality requirements and evaluating quality” (ISO/IEC 9126, 2001) Our working principle: Collect metrics proposed in the state of the art and structurethem in a quality model
  10. 10. 10 Overview Related work Method QuESo Application example Discussion Future work Conclusions Outline
  11. 11. 11 Related Work
  12. 12. 12 Overview Related work Method QuESo Application example Discussion Future work Conclusions Outline
  13. 13. 13 Gathering measures Systematic literature review 1 Constructing Model Bottom up Top down 2 The methodology have two steps
  14. 14. 14 Gathering measures from the available literature. Systematic literature review (Kitchenham and Charters, 2007). Measure inclusion criteria: (Hartighet al., 2013; Neuet al., 2011) User-friendlyand operationalizable. Non-redundant 68 measures! BookManual searchSecondary studiesDigital libraries351116Papers with measures of OSS ecosystems17
  15. 15. 15 Method for quality model construction. RadulovicandGarcia-Castro(2011) Sub-characteristics Relationships Quality measures Derived measures Basic measures Align quality sub- characteristics with a quality model QuESo SLR measures QualOSS ISO Quality model
  16. 16. 16 Overview Related work Method QuESo Application example Discussion Future work Conclusions Outline
  17. 17. 17 Quality characteristics in QuESo have been organized into three dimensions QuESo Community quality Ecosystem network quality Platform quality Software platform in which the projects of the ecosystem are built. OSS communities that grow inside the ecosystem and ecosystem’s projects OSS ecosystem as a network of interrelated elements # 11 54 8 20
  18. 18. 18 Maintenance capacity :Ability to provide the resources necessary for maintaining community products Dimension Characteristic Sub-characteristic From QualOSS Measure QuESo Maintenance capacity Community quality Size Activeness number authors Gnome: 3.500 1997-2012 number of commits Gnome: 480.000 1997-2007
  19. 19. 19 Network health: How well members are connected and their impact Ability of the nodes to establish connections between them Capacity of the ecosystem nodes to be classified around its projects. QuESoEcosystemnetwork qualityNetwork healthClusteringInterrelatednessnumber community projectsGnome: 1.200 1997-2012Contributor activity graphGnome:499 projects / 15.000 changes1998-2011
  20. 20. 20 Our proposal: QuESo QuESo Maintenance capacity Sustainability Process maturity Community quality Size Cohesion Activeness Heterogeneity Regeneration ability Effort balance Expertise balance Visibility Ecosystem network quality Resource health Network health Information consistency Financial Vitality Clustering Trustworthiness Interrelatedness Synergetic evolution Platform quality Dimension Characteristic Sub-characteristic From QualOSS 4 7 15 1 2 11 3 11 2 3 7 4 3 1
  21. 21. 21 Overview Related work Method QuESo Application example Discussion Future work Conclusions Outline
  22. 22. 22 Using QuESOfor risk analysisKenett, R et al. (2014) Managing Risk and Costs in OSS Adoption
  23. 23. 23 Software ecosystem model
  24. 24. 24 Relations between measures and quality characteristics QuESo measures QuESo characteristics Mail per day Activeness
  25. 25. 25 SALMonOSS: AssessingOpen Source Communities’ Health(Oriol , Franco-Bedoya, 2014)
  26. 26. 26 : Ecosystem Quality Object goals-tasks-resources-softgoals : Ecosystem Quality Parameter QuESo Quality characteristics & measures : Ecosystem Measure Directive Measures metrics process : Ecosystem Quality Objectives Logic expressions : Ecosystem Quality Evaluation Event Logic expressions : Ecosystem Action Garantees actions for OSS Actors Has Define Trigger EQLA: Ecosystem Quality Level Agreement
  27. 27. 27 Overview Related work Method QuESo Application example Discussion Future work Conclusions Outline
  28. 28. 28 Discussion Some observations Completeness:To remarkthat QuESo model may not be complete Quantitative vs. qualitative: QuESo measures are mostly quantitative. Unbalanced distribution of measures:more research needed Measure names:room for improvement
  29. 29. 29 Overview Related work Method QuESo Application example Discussion Futurework Conclusions Outline
  30. 30. 30 Futurework: Perform a complete quality assessment Define Quality assessment process How are the valuesof each measure interpreted? Good and bad values? How can the measures be merged to provide the assessment for a particular sub-characteristic of the quality model? What are the principles to perform the assessment with missing, incorrect, and/or inconsistent measure data?
  31. 31. 31 Overview Related work Method QuESo Application example Discussion Futurework Conclusions Outline
  32. 32. 32 Conclusions We have presented QuESo, a quality model for assessing the quality of OSS ecosystems. This quality model has been constructed following bottom-up – Top down strategies. QuESo coversthe platform, the community, and the ecosystem network dimensions. This quality model can be used as a starting pointfor the quality assessment of an OSS ecosystem
  33. 33. Thanks for your attention Comments and Questions
  34. 34. 34 Bibliography Lungu,M.,Lanza,M.,Gˆırba,T.,andRobbes,R.(2010).TheSmallProjectObservatory:Visualizingsoftwareecosystems. ScienceofComputerProgramming,75(4):264–275.Lungu,M.,Malnati,J. Jansen,S.andCusumano,M.(2013).SoftwareEcosystems:AnalyzingandManagingBusinessNetworksinthe SoftwareIndustry,chapterDefiningsoftwareecosystems:asurveyofsoftwareplatformsandbusinessnetwork governance,pages13–28.Volume1of(Jansenetal.,2013). ISO/IEC9126(2001).Productquality–Part1:Qualitymodel. ISO/IEC25010:2011Systemsandsoftwareengineering--SystemsandsoftwareQualityRequirementsandEvaluation(SQuaRE) --Systemandsoftwarequalitymodels. Soto,M.andCiolkowski,M.(2009).TheQualOSSopensourceassessmentmodelmeasuringtheperformance ofopensourcecommunities.InProceedingsofthe3rdESEM,pages498–501. Jansen,S.,Brinkkemper,S.,andMichaelCusumano(2013).SoftwareEcosystems:AnalyzingandManagingBusinessNetworksintheSoftwareIndustry,volume1.EdwardElgarPublishing. Kitchenham,B.andCharters,S.(2007).GuidelinesforperformingSystematicLiteraturereviewsinSoftware EngineeringVersion2.3.Technicalreport,KeeleUniversityEBSE. Hartigh,E.,Visscher,W.,Tol,M.,andSalas,A.J.(2013).SoftwareEcosystems:AnalyzingandManagingBusiness NetworksintheSoftwareIndustry,chapterMeasuringthehealthofabusinessecosystem,pages221–245.Volume1of(Jansenetal.,2013). Radulovic,F.andGarcia-Castro,R.(2011).ExtendingSoftwareQualityModels-ASampleInTheDomain ofSemanticTechnologies.InProceedingsofthe23rdSEKE,pages25–30. Kenett,R,Franch,X.,Susi,A.,Galanis,N.(2014).AdoptionofFreeLibreOpenSourceSoftware(FLOSS):ARiskManagementPerspective.InProceedingsofthe38thCOMPSAC. Oriol,M.Franco-Bedoya,O.Franch,X.,Marco,J.AssessingOpenSourceCommunities’HealthusingServiceOrientedComputingConcepts.Inproceedingsofthe8thRCIS. Messerschmitt,D.G.,Szyperski,C.:Softwareecosystem:understandinganindispensabletechnologyandindustry.TheMITPress, Cambridge(2005)
  35. 35. 35 Backup
  36. 36. 36 Backup
  37. 37. 37 Backup
  38. 38. 38

×