Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
116/06/2015 FITMAN Webinars, Online
FITMAN Verification & Validation
Methodology
FITMAN Webinars
June 16th, 2015
Dr. Fenar...
What is Verification & Validation?
Verification Validation
Definition
[IEEE Std
1012-
2004]
The process of providing objec...
Bringing together agile
and waterfall software
engineering philosophies
FITMAN V&V Method purpose
Infusing a crowd
assessm...
FITMAN WP2 D2.1 4
FITMAN V&V Methodology – Overall Picture
Trial Specific Perspective: Steps T-1 & T-2
 Business Validation (T-2) to assess whether the overall trial solution
event...
6
Product Specific Perspective: Steps P-(1-5)
 Product Validation (P-5) to examine whether the
product satisfies intended...
Assessing Business Impact
Business Performance Indicators (BPI) are identified for each
trial through a simplified ECOGRAI...
Initial Technical Indicators (filled in by trials)
8
Fulfilment of
requirements
“The solution fulfils the Trial
requiremen...
Additional Technical Indicators
Technical
Indicator
Definition Value (in 3 levels)
Correctness The degree to which the sof...
Trial Journals
10
Unstructured
Information
Unstructured information for the experience of each Trial is collected.
Technic...
Overall V&V in practice
11
Developer Acceptance
Questionnaires
Validation of SEs by anyone who reuses
them
4+
16/06/2015 F...
An example of Self-certification for SEs
1216/06/2015 FITMAN Webinars, Online
An Example for “Release Verification” documentation
1316/06/2015 FITMAN Webinars, Online
An example for “Product Validation” documentation
1416/06/2015 FITMAN Webinars, Online
Lessons Learnt so far…
• Experience in FITMAN was collected from trial owners and
technical partners:
– The users mainly c...
FITMAN V&V Decalogue
I. Have a deep knowledge of the requirements and the ends sought in each
trial.
II. Infuse a V&V cult...
Thanks for your attention!
Dr. Fenareti Lampathaki (NTUA)
Join Us!
17
fenareti.lampathaki @fenaretiflamp@epu.ntua.gr
16/06...
Upcoming SlideShare
Loading in …5
×

Fitman webinar 2015 06 Verification and Validation methodology

507 views

Published on

A webinar on the Verification and Validation (V&V) Methodology developed in the FITMAN project. The V&V methodology is a general, holistic method for verifying, validating and evaluating a software product from its conception to final release and implementation. Presented by Fenareti Lampathaki from the National Technical University of Athens (NTUA).

Published in: Science
  • Be the first to comment

  • Be the first to like this

Fitman webinar 2015 06 Verification and Validation methodology

  1. 1. 116/06/2015 FITMAN Webinars, Online FITMAN Verification & Validation Methodology FITMAN Webinars June 16th, 2015 Dr. Fenareti Lampathaki National Technical University of Athens (NTUA)
  2. 2. What is Verification & Validation? Verification Validation Definition [IEEE Std 1012- 2004] The process of providing objective evidence that the software and its associated products conform to requirements (e.g., for correctness, completeness, consistency, accuracy) for all life cycle activities during each life cycle process (acquisition, supply, development, operation, and maintenance); satisfy standards, practices, and conventions during life cycle processes; and successfully complete each life cycle activity and satisfy all the criteria for initiating succeeding life cycle activities (e.g., building the software correctly). The process of providing evidence that the software and its associated products satisfy system requirements allocated to software at the end of each life cycle activity, solve the right problem (e.g., correctly model physical laws, implement business rules, use the proper system assumptions), and satisfy intended use and user needs. Question Are we building the product right? Are we building the right product? Objective To ensure that the product is being built according to the requirements and design specifications. To ensure that the product actually meets the user’s needs, the specifications were correct in the first place and the product fulfils its intended use when placed in its intended environment. 216/06/2015 FITMAN Webinars, Online
  3. 3. Bringing together agile and waterfall software engineering philosophies FITMAN V&V Method purpose Infusing a crowd assessment mentality in the V&V activities Including both business and technical perspectives Providing initial practical guidelines An all-inclusive framework for verifying, validating and evaluating a software product from its conception to final release and implementation in real-life Key message:  A general, holistic method including techniques to be applied for each step and recommendations for its application by the trials  A method reusable beyond FITMAN: • Each trial and development team can streamline the method according to their own needs 316/06/2015 FITMAN Webinars, Online
  4. 4. FITMAN WP2 D2.1 4 FITMAN V&V Methodology – Overall Picture
  5. 5. Trial Specific Perspective: Steps T-1 & T-2  Business Validation (T-2) to assess whether the overall trial solution eventually offers sufficient added value to the trial.  Demonstrating that the software developed has clear benefit to the trials, allowing them to operate more efficiently (usually in terms of cost, time or quality) than before, or supporting them to do something they couldn't do before. User Acceptance Testing Simplified ECOGRAI Methodology  Trial Solution Validation (T-1) to guarantee that the overall trial solution satisfies intended use and user needs.  Validation of each trial solution from a technical and functional point of view only. 5 5 TIs BPIs 16/06/2015 FITMAN Webinars, Online
  6. 6. 6 Product Specific Perspective: Steps P-(1-5)  Product Validation (P-5) to examine whether the product satisfies intended use and user needs. User Acceptance Testing  Release Verification (P-4) to determine whether the requirements of the final product release are met.  Backlog Verification (P-3) to determine whether the requirements of the product after each sprint are met.  Model Verification (P-2) to coordinate the alignment between design and requirements, as well as between design and code.  Code Verification (P-1) to ensure functionality, correctness, reliability, and robustness of code. Alpha Testing Regression Testing Traceability Analysis White Box Testing Developer Acceptance Survey 7+2 TIs Test Cases Documentation 16/06/2015 FITMAN Webinars, Online
  7. 7. Assessing Business Impact Business Performance Indicators (BPI) are identified for each trial through a simplified ECOGRAI Method. Characteristics of a Business Performance Indicator (BPI): • easy to be interpreted, to put in work, to use or to exploit • easily measurable, quantifiable • representative of the objective which it measures For each Business Performance Indicator, the trials are required to report: • The current value (AS-IS) • The target value they want to achieve (Target) and • The actual achieved values after the solution implementation (TO-BE) 716/06/2015 FITMAN Webinars, Online
  8. 8. Initial Technical Indicators (filled in by trials) 8 Fulfilment of requirements “The solution fulfils the Trial requirements” Learnability “It is easy to start to use the solution and learn functionalities” Understandability “The solution is easy and self- clear to understand and the concepts and terminology are understandable” User’s attraction level “The solution is attractive to the user. I feel satisfied and comfortable when using it” Efficiency “The time and resources required to achieve the objectives of the solution are reasonable, the solution is fast enough and does not require too many steps” Openness “Ensuring that specific people groups may access the software for free with specified rights” Interoperability maturity “The capability of the software to interact with other systems” Ease of application “A measure of the applicability of the software in the particular environment in terms of amount of work and extra actions or means” SpecificEnablersTrialsSolution • Strongly agree • Agree • Neither agree nor disagree • Disagree • Strongly disagree DifferentLevelsperTI 16/06/2015 FITMAN Webinars, Online
  9. 9. Additional Technical Indicators Technical Indicator Definition Value (in 3 levels) Correctness The degree to which the software is free from defects in its specification and implementation High (no defects detected) / Medium (relatively minor defects detected) / Low (severe defects detected) Efficiency The capability of the software to provide appropriate performance, relative to the amount of resources used High (performance beyond expectations, with reasonable resources consumption) / Medium (expected performance, even with small increase of resources utilization) / Low (Marginally accepted performance and/or overuse of available resources) Reliability The capability of the software to maintain a specified level of performance when used in the factory settings High (totally stable performance) / Medium (marginal performance divergences) / Low (the performance of the software varies significantly depending on the conditions) Sustainability How easy it is to maintain the software and apply required modifications for adapting to changes in the operating environment High (software maintenance and modification are very easy) / Medium (software maintenance requires reasonable effort) / Low (software maintenance is very complex and modifications difficult to be applied) Additional calculated indicators (not to be assessed by trials nor inserted in SurveyMonkey): Versatility: to be estimated according to the formulas provided in WP2 Portability: to be estimated on a cross-trial basis using existing indicators Existing indicators (already assessed for GEs/SEs): Openness / Ease of application / Interoperability maturity
  10. 10. Trial Journals 10 Unstructured Information Unstructured information for the experience of each Trial is collected. Technical Journal: Registration of the implementation issues encountered in the implementation of the Trial system Registration of the operational resilience of the Trial (e.g. major bugs, blocking errors, etc.) Business Journal: Collection and analysis of the most important operational issues faced in the implementation of the system in the Trial, e.g. organizational and business difficulties, degradation of the business system. 16/06/2015 FITMAN Webinars, Online
  11. 11. Overall V&V in practice 11 Developer Acceptance Questionnaires Validation of SEs by anyone who reuses them 4+ 16/06/2015 FITMAN Webinars, Online
  12. 12. An example of Self-certification for SEs 1216/06/2015 FITMAN Webinars, Online
  13. 13. An Example for “Release Verification” documentation 1316/06/2015 FITMAN Webinars, Online
  14. 14. An example for “Product Validation” documentation 1416/06/2015 FITMAN Webinars, Online
  15. 15. Lessons Learnt so far… • Experience in FITMAN was collected from trial owners and technical partners: – The users mainly considered the method sufficient – Training, support and the online V&V tool are crucial – Definition of BPIs was not trivial, but it was worth the effort • Challenges with the definition of objectives and the BPIs, initially not well formulated / measurable • Confidentiality issues • V&V from technical and business perspective must be an ongoing and iterative process… 1516/06/2015 FITMAN Webinars, Online
  16. 16. FITMAN V&V Decalogue I. Have a deep knowledge of the requirements and the ends sought in each trial. II. Infuse a V&V culture to your trial. III. Create a concrete V&V plan to be followed in each step, tailored to your real needs and competences. IV. Engage from the very beginning the required stakeholders, assigning them with appropriate responsibilities and decision power. V. Tap the power of data to effectively conduct V&V activities. VI. Proceed with each V&V step on the basis of at least 1 recommended technique. VII. Keep the balance between business and technical V&V aspects. VIII. Invest on crowd-sourcing techniques aligned to the trial philosophy. IX. Keep a complete V&V log and document in detail the V&V findings. X. Treat V&V as a continuous, iterative procedure. 1616/06/2015 FITMAN Webinars, Online
  17. 17. Thanks for your attention! Dr. Fenareti Lampathaki (NTUA) Join Us! 17 fenareti.lampathaki @fenaretiflamp@epu.ntua.gr 16/06/2015 FITMAN Webinars, Online

×