Your SlideShare is downloading. ×
Defining an Open Source Software Trustworthiness Model
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Defining an Open Source Software Trustworthiness Model

659
views

Published on

This presentation show the results of my PhD thesis. …

This presentation show the results of my PhD thesis.

Modern society depends on large-scale software systems of astonishing complexity. Because the consequences of their possible failure are so high, it is vital that software systems should exhibit a trustworthy behavior.
Trustworthiness is a major issue when people and organizations are faced with the selection and the adoption of new software. Although some ad-hoc methods have been proposed (see for instance OpenBQR, OpenBRR and QSOS), there is not yet general agreement about the software characteristics contributing to its trustworthiness.
Therefore, this work focuses on defining an adequate notion of trustworthiness of Open Source Software products and artifacts and identifying a number of factors that influence it to provide both developers and users with an instrument that guides them when deciding whether a given program (or library or other piece of software) is “good enough” and can be trusted in order to be used in an industrial or professional context.

More details on www.taibi.it

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
659
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
37
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • On the web exists many slightly different definitions about the trustworthiness and trustworthy related concepts as is the trustworthy computing concept. We present here just the most relevant and similar definitions to our own nderstanding of the concept of the trustworthy element that is used inside this research. Some of the definitions found > on the web are the following: > > Merriam-webster's online dictionary defines the concept > trustworthy as something being worthy of confidence; > dependable "a trustworthy guide" > "trustworthy information". > > Other definitions found on the web are: > > worthy of trust or belief; "a trustworthy report"; "an > experienced and trustworthy travelling companion" > (wordnet.princeton.edu) > > taking responsibility for one's conduct and obligations; > "trustworthy public servants" (wordnet.princeton.edu) > > The National Security Agency (NSA) defines a trusted system > or component as one "whose failure can break the security > policy", and a trustworthy system or component as one "that > will not fail". (Wikipedia) > > The Committee on Information Systems Trustworthiness' > publication, Trust in Cyberspace, defines a Trustworthy > computing system as one which: > does what people expect it to do - and not something else - > despite environmental disruption, human user and operator > errors, and attacks by hostile parties. Design and > implementation errors must be avoided, eliminated or somehow > tolerated. It is not sufficient to address only some of these > dimensions, nor is it sufficient simply to assemble > components are themselves trustworthy. Trustworthiness is > holistic and multidimensional. > (Wikipedia) > > Our definition of trustworthiness and of the trustworthy > element are closer to the Wordnet's definition since it > depends on the personal beliefs or generic trust that people, > users of FLOSS systems and all the stakeholders share about a > specific software product. We adopted the term element for > describing all the components and aspects influencing the > development and functioning of a software system. > > Therefore we define the trustworthy element, in the scope of > the research done on the FLOSS development process inside the > Qualipso project, as a specific component or aspect of a > software product that influences the belief and trust of the > stakeholders in the overall quality of the software product. >
  • On the web exists many slightly different definitions about the trustworthiness and trustworthy related concepts as is the trustworthy computing concept. We present here just the most relevant and similar definitions to our own nderstanding of the concept of the trustworthy element that is used inside this research. Some of the definitions found > on the web are the following: > > Merriam-webster's online dictionary defines the concept > trustworthy as something being worthy of confidence; > dependable "a trustworthy guide" > "trustworthy information". > > Other definitions found on the web are: > > worthy of trust or belief; "a trustworthy report"; "an > experienced and trustworthy travelling companion" > (wordnet.princeton.edu) > > taking responsibility for one's conduct and obligations; > "trustworthy public servants" (wordnet.princeton.edu) > > The National Security Agency (NSA) defines a trusted system > or component as one "whose failure can break the security > policy", and a trustworthy system or component as one "that > will not fail". (Wikipedia) > > The Committee on Information Systems Trustworthiness' > publication, Trust in Cyberspace, defines a Trustworthy > computing system as one which: > does what people expect it to do - and not something else - > despite environmental disruption, human user and operator > errors, and attacks by hostile parties. Design and > implementation errors must be avoided, eliminated or somehow > tolerated. It is not sufficient to address only some of these > dimensions, nor is it sufficient simply to assemble > components are themselves trustworthy. Trustworthiness is > holistic and multidimensional. > (Wikipedia) > > Our definition of trustworthiness and of the trustworthy > element are closer to the Wordnet's definition since it > depends on the personal beliefs or generic trust that people, > users of FLOSS systems and all the stakeholders share about a > specific software product. We adopted the term element for > describing all the components and aspects influencing the > development and functioning of a software system. > > Therefore we define the trustworthy element, in the scope of > the research done on the FLOSS development process inside the > Qualipso project, as a specific component or aspect of a > software product that influences the belief and trust of the > stakeholders in the overall quality of the software product. >
  • The model will use a number of trustworthiness factors as independent variables An assessment of trustworthiness by practitioners and users as dependant variables
  • The model will use a number of trustworthiness factors as independent variables An assessment of trustworthiness by practitioners and users as dependant variables
  • Transcript

    • 1. Davide Taibi Università degli Studi dell’Insubria Defining an Open Source Software Trustworthiness Model Advisor: Prof. Sandro Morasca Reviewer: Prof. Alberto SIllitti
    • 2.
      • Motivations and Research Goals
      • What Trustworthiness is
      • How to measure trustworthiness
      • The Approach
        • Trustworthiness Factors
        • Tool Definition and Building
        • Model Building
      • Conclusions
      Outline 15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 3.
      • Who is behind Open Source?
      • Why to be confident in OSS?
      • How can I make users confident in my software?
      Motivation Open Source Trustworthiness Model 15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 4. Research Goals
      • OSS Trustworthiness Evaluation
        • Evidence-based approach
          • experiments
          • static and dynamic measures
          • testing
          • tools
          • validated models
      • Tools for evaluating OSS trustworthiness
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 5. What Trustworthiness is 15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 6. http://www.hwupgrade.it/articoli/stampa/portatili/1160/peso.jpg How to measure trustworthiness 15-09-2010 Defining an Open Source Software Trustworthiness Model TRUSTWORTHINESS
    • 7. The Approach 15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 8. Trustworthiness Factors identification
      • Factors Identification  151 Interviews
      • Interviews to understand
        • The confidence parameters of trustworthiness
        • The roles of the involved individuals
        • The problem domains
        • The correlations between the first 3 aspects
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 9. Trustworthiness Factors
      • Interviews - Roles
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 10. Trustworthiness Factors
      • Economics
        • ROI
        • TCO
      • Quality
        • functional requirements
        • reliability
        • performance
        • usability
        • maintainability
        • portability
        • size
        • complexity
        • modularity
        • standard architecture
        • patterns
        • standard compliance
        • self containedness
        • interoperability
        • localization
      • Development
        • type of licenses
        • tools
        • best practices
        • documentation
        • environment
        • training / guidelines
        • user community
        • maintainer organization
        • short term support
        • reputation of vendor
        • distribution channel
        • language uniformity
        • user community that witness quality
        • benchmarks / test suites
      • Customer
        • customer satisfaction
        • interoperability issues
        • law conformance
        • standard imposed
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 11. Analysis of relevant projects
      • Objectives:
        • finding what kind of information is out there to help “users” choose
        • finding what kind of information is missing
        • checking if there is a gap between “demand” and “supply”
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 12. Analysis of relevant projects
      • Results:
        • Some factors are not directly assessable
          • Proxy-measures defined
        • Some factors need some tools to be developed
        • Other factors can not assessed unless developers provide the information (e.g., the number of downloads)
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 13. Analysis of relevant projects
      • Example:
      • The degree to which an OSS product satisfies / covers functional requirements
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 14. Analysis of relevant projects
      • Results:
      • Open Product Portal Assessment Model
      • www.op2a.tk
      • 44 Portals analyzed
      • Apache Tomcat Portal refactoring (proposal)
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 15. Model Building
      • Objectives:
        • Definition of measures, starting from the factors we identified
        • Reuse/define sensible measures
      An initial set of measures has been defined , to capture these dimensions from different viewpoints in a quantitative way Use of a goal-oriented approach : Goal/Question/Metric paradigm 15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 16. Model Building
      • Product related factors
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 17. Model Building
      • Process related factors
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 18. Tools identification and Building
      • Objectives:
        • Definition and building of the tools required for the assessment of GQM metrics
      • Steps:
        • Check existing OSS tools
        • Build new tools
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 19. Toos Identification and Building
      • Developed
        • MacXim ( static code analysis tool)
          • qualipso.dscpi.uninsubria.it
      • Reused
        • Spago4Q
        • STATsvn
        • FOSSology
        • Junit
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 20. Macxim 15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 21.
      • Class level
        • # public attributes
        • # methods per class
        • # eLOC per class
        • # comment lines per class
        • # public, private, protected methods
      • Method level
        • # eLOC
        • # comment lines
        • McCabe complexity
        • # params per method
        • # interfaces per application
        • # dependencies on other methods of the same class
        • # dependencies on attributes of the same class
        • # dependencies on other internal classes
        • # dependencies on attributes of other internal classes
        • # dependencies on methods of other internal classes
      Macxim
      • Application level
        • # eLOC
        • # classes
        • # methods
        • # packages
        • # class attributes
        • # comment LOCs
        • # Abstract Classes
        • # implemented interfaces
        • # classes with defined methods
        • # classes with defined attributes
        • # classes out of packages (root package)
        • # methods on internal app. (not library)
      • … …
      • Java specific
        • # interfaces per class / per application
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 22. Experimentation
      • Objective:
        • build OSS trustworthiness models that are
          • goal-oriented
          • evidence-based
          • customizable
      • Steps
        • a specific measure repository was designed
        • a suitable tool for statistical analysis was selected and suitable scripts were coded
        • a questionnaire for collecting users’ opinions on OSS trustworthiness and other qualities
          • 44 OSS projects (22 Java and 22 C++ projects)
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 23. Experimentation
      • Experiments:
        • 565 questionnaires were collected
        • 3750 product evaluations collected with the questionnaire
          • (6,63 evaluations per questionnaire)
        • Correlations between objective and subjective measures
      • Results:
        • a set of statistically significant models (MOSST: Model of Open Source Software Trustworthiness)
          • between measurable code attributes (the X's) and the evaluated trustworthiness of OSS products (the Y), evaluated reliability (the Y)
          • a few correlational models between the measurable internal characteristics of OSS .
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 24. Experimentation Good Not good Acceptable 15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 25. Experimentation Trustworthiness 15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 26. Experimentation 15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 27. Experimentation Subjective qualities vs. measures 15-09-2010 Defining an Open Source Software Trustworthiness Model Subjective quality Objective measure Outcome Reliability CBO  Reliability LCOM  Reliability McCabe (class average)  Reliability Size (total eLOC)  Reliability Total num. methods  Reliability Total num. classes  Trustworthiness CBO  Trustworthiness LCOM  Trustworthiness McCabe (class average)  Trustworthiness Size (total eLOC)  Trustworthiness Total num. methods  Trustworthiness Total num. classes  Trustworthiness Size (total) & McCabe (class average)  Trustworthiness Num methods & McCabe  Trustworthiness Num classes & McCabe 
    • 28. Experimentation Correlations between subjective qualities 15-09-2010 Defining an Open Source Software Trustworthiness Model Subjective quality Subjective quality Outcome Trustworthiness Reliability  (logistic) Trustworthiness Reliability  (linear) Trustworthiness Reliability  (non-parametric) Trustworthiness ReliabilityGood, ReliabilityBad  (linear) TrustworthinessGood ReliabilityGood  (non-parametric) Trustworthiness Reusability  (linear) Trustworthiness Interoperability  (linear) Trustworthiness Efficiency  (linear) Trustworthiness Documentation  (linear) Trustworthiness Usability  Trustworthiness Portability  Trustworthiness Functionality  Trustworthiness Security  Trustworthiness Efficiency  Trustworthiness Community support 
    • 29. Experimentation Correlations between measures 15-09-2010 Defining an Open Source Software Trustworthiness Model Objective var Objective var Outcome Size (total eLOC) Total num. methods  (log-log) Size (total eLOC) Total num. classes  (log-log) Size (total eLOC) Total num. classes & methods  Total num. methods Total num. classes  (linear)
    • 30. Experimentation Trustworthiness vs popularity 15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 31. Conclusions
      • MOSST: Model for Open Source Trustworthiness
      • Macxim: Static code analysis tool
      • OP2A: OSS Product Portal Assessment model
      15-09-2010 Defining an Open Source Software Trustworthiness Model
    • 32. Thanks 15-09-2010 Defining an Open Source Software Trustworthiness Model