• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Test designandmanagementfreenest1dot4
 

Test designandmanagementfreenest1dot4

on

  • 223 views

Test Design and FreeNest 1.4

Test Design and FreeNest 1.4

Draft Material

Statistics

Views

Total Views
223
Views on SlideShare
223
Embed Views
0

Actions

Likes
1
Downloads
3
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Test designandmanagementfreenest1dot4 Test designandmanagementfreenest1dot4 Presentation Transcript

    • Test Design and Managementin context of IFDK reference product
    • About this course material - This material if for general training for Test Design and management - Material is more supportive in class room - Material will be updated during courses - FreeNest Portable Project Platform is used to demonstrate things only in practice. This is not limiting usage for material for other training environments (I hope )About
    • Example of IFDK product concept IFDK = Internal Flame Drum KitRequirement
    • Different aspects to product QA Tester Customer Seller DeveloperYläotsikko
    • Different requirement levels Customer/Business/Stake Holder Requirements System Requirements Scalability Stability Performance Design Requirements Security Performance Stress Component Requirements Usabilty Implementation
    • V-Model in testing Verification and Validation Customer Requirements Acceptance Testing System Requirements System Testing Sub System Requirements Integration Testing Component Requirements Component/Unit Testing Verification = Are we building the product right? Validation = Are we building the right product?Yläotsikko
    • Why we need requirements from testing point of view? ”Traditional Testing Levels”„Test Engineers Area“ Customer/Business/Stake Acceptance Testing Holder Requirements System Requirements System Testing Design Requirements Integration Testing„Developers Area“ Component Unit Testing Requirements Implementation
    • IFDK System Verification and ValidationIFDK Product Ideas VALIDATION Product Features Customer/Business Acceptance Requirements Testing Validation = Are we building the right product? Use Cases Verification = Are we building the product right? System Requirements System Testing User Storys Sub System Integration Requirements Testing Architecture& Component Design& Component / Unit VERIFICATION Requirements Requirements Implementation Testing VALIDATION vs VERIFICATION?Yläotsikko
    • You have vision of product What means testing in brief? Product Design & Testing & Quality Assurance Implementation Ready Features to test Test Case Use Cases Test Case Can we deliver Ready to Product Deliver User Storys Test Case Not ready to deliver ? Customer Requirements Test CaseYläotsikko
    • Verification vs Validation? Verification = Are we building the product right? Validation = Are we building the right product? Verification ValidationProduct Design & Testing & Quality AssuranceImplementation Features Test Case Ready to test Use Cases Test Case Can we deliver Ready to Product Deliver User Storys Test Case Not ready ? Customer Requirements Test Case to deliver
    • SW Development Process (Waterfall) RequirementGathering/Evaluatio n Design Implementation Verification & Validation Error Managment Process Maintenance Mile Stone 1 Mile Stone 2 Mile Stone 3 Task1 Task1 Task1 Task1 Task1 Task1
    • Testing Orientation “SYSTEM TESTING” Black Box Testing Perspective Grey Box Testing “CODE LEVEL TESTING” White Box Testing Perspectivehttp://en.wikipedia.org/wiki/Software_testing
    • Black Box vs White Box Testing ? ? Black Box Testing for selected Unit Testing is White Box testing componentYläotsikko
    • TESTING LEVELS
    • Unit/Module/Compone nt Testing VALIDATION Customer/Business Product Acceptance Requirements Testing System Requirements System Architecture& Testing Sub System Design& Integration Requirements Implementatio Testing n Component Component / Unit Requirements Testing VERIFICATIONYläotsikko
    • How to Test?What should be tested?How ?
    • How to test? What should be tested? How ?
    • How to verify component implementation -Unit Testing -Code Coverage -Branch Coverage -Complexity AnalyseYläotsikko
    • Component /Unit Testing DeveloperImplemented Class Unit Test Frame Work TestClass Class Test Method Call Attributes Attributes TestMethodCall Methods Method Result A=1 B=2MethodCountValues( int x, int y) C=Class.TestMethodCoz=x+y+1 untValues(A+B)Return z C<>3 FAIL C=3 PASS
    • Code Coverage An analysis method that determines which parts of the software have been executed (covered) by the test suite and which parts have not been executed, e.g. statement coverage, decision coverage or condition coverage. http://en.wikipedia.org/wiki/Code_coverage http://www.atlassian.com/software/clover/Yläotsikko
    • Branch coverage The percentage of branches that have been exercised by a test suite. 100% branch coverage implies both 100% decision coverage and 100% statement coverage.Yläotsikko
    • Code Complexity Example tool CCCC https://wiki.jenkins- ci.org/display/JENKINS/CCCC+Plugin http://sourceforge.net/projects/codeanalyze- gpl/?source=recommendedYläotsikko
    • Integration Testing VALIDATION Customer/Business Product Acceptance Requirements Testing System Requirements System Architecture& Testing Sub System Design& Integration Requirements Implementatio Testing n Component Component / Unit Requirements Testing VERIFICATIONYläotsikko
    • Integration Testing
    • How To Test ? What should be tested? How ?http://prosentti.vero.fi/veropros_tietojen_syotto2011.asp
    • Why Integrate first? Avoid Big Bang! Web Service HW Component Data Base Component/Application 10% tested Tested Component/ApplicationYläotsikko
    • System Testing VALIDATION Customer/Business Product Acceptance Requirements Testing System Requirements System Architecture& Testing Sub System Design& Integration Requirements Implementatio Testing n Component Component / Unit Requirements Testing VERIFICATIONYläotsikko
    • How to test?What should be tested?How ?
    • System Testing in Large DB ? Application Sales & Gateways DB Application & GatewaysWhat should be tested? ApplicationHow ? CRM DB Room Reserva tion DB
    • System Acceptance Testing VALIDATION Customer/Business Product Acceptance Requirements Testing System Requirements System Architecture& Testing Sub System Design& Integration Requirements Implementatio Testing n Component Component / Unit Requirements Testing VERIFICATIONYläotsikko
    • What to test? http://www.123rf.com http://www.123rf.comWhat should be tested before so customer could be so happy ?How ?
    • IFDK Verification/Validation (Testing Organization) Product Release Acceptance Test Engineer IFDK System Acceptance Testing System Test Engineer Test ManagerIFDK System Testing Project ManagerSystem Testing Designer/Coder IntegrationTest Automation Test Engineer Error Manager Feature Unit/Integration Testing Feature Pack Validation Verification Report Test Error Staus Management Database Database
    • What should be tested? Input ? Output?
    • What is Test Design? REQ-X REQ-O What I should REQ-Z check ? REQ-Y Implementation IDEALYläotsikko
    • Why we need test design?• Discuss about reasons for test design?• Why we need to do design?• Stupid work  ! I wan’t to progress!??
    • Test Case Design
    • UNDERSTAND YOUR TEST LEVEL Product Release Acceptance Test Engineer IFDK System Acceptance Testing System Test Engineer Test ManagerIFDK System Testing Project ManagerSystem Testing Designer/Coder IntegrationTest Automation Test Engineer Error Manager Feature Unit/Integration Testing Feature Pack Validation Verification Report Test Error Staus Management Database Database
    • What Information Test Case should contain?Add Information about case • Test Case Id • Test Case owner/writer • Date • comments • Verify drum track player pause mode functionality. • Do this with IFDK software release X and playing song ”Show must go on by Freddy Mercury” Verify what? • Test should be done using android emulator Using configuration? environment and using your hands, ears and eyes” With tools? • Pre State: • Android emulator is running • Release X is installed on emulator • Test Case Steps: • 1. Open drum kit player application • 2. Select song ”Show must go on” • 3. Start to play • 4. Press Pause and check song is paused • 5. Check memory usage from system application • 6. Press Play Define pre-state • 7. jump to 4 several time (<10) Define Steps • 8. Listen song to the end Define end-state • 9. Exit player using ”exit button” • End State: • IFDK Kit in main screen mode What is verdict? • If Pause is working result is PASS. If Pause mode failed result is FAIL
    • Why we need test design again!• Stupid work! This takes ages! This Test Case documentation is old as soon I have changed some implementation? Why you need to do so hard documentation? Give me a one good reason!
    • Checklist? Check UI is working Working? Check color change Working? Check Counter value after 50 logins Working? Check disable mode for counter Working?Checklist can be working great in smallteam!  What happens if team is disbanded to other projects? And you are new maintainer for this project?
    • Agile Thinking?• We have to automate all tests!! No sense to create documentation ?• Who does automation without a design?
    • Where I find sources for test design? Specification based Testing design Negative Testing Design based test desing Test CaseTest Design Method M Requirement based test design i Test Defect based test design x Engineer’s Write Step e Daily Job? Step Functional test design d Step • Customers Idea • Brainstorm NonFunctional test design • Intitution • Exploratory
    • How to create Test Case??? Acceptance Test Case Functional Test Case Check Non-Functional Test Cases different sources & WRITE What is strategy Choose Field Test Case A Testing level for Test Case Case Type Interoperability Test Case Test design Case! Conformance Test Case Regression Test Case
    • Test Case Design in agile frameworkUser Story: As a user I would like to use my google account for login Implementation Done Definition for Acceptance Criteria Definition for test cases Check list Tested 1. Verify Test using test 2. Verify Test automation? 3. Verify Test 4. Verify Test
    • Test Driven Development TDD in all levels! Define Architecture &Design draft Tests Case Design? Design Tests Implement Code
    • Unit Testing
    • Test Driven Development and Unit Testing DeveloperDEFINE IMPLEMENT TEST CODE CASES AGAINST Unit Test Frame Work Implemented Class TESTSFIRST!!! TestClass Class Attributes Attributes Test Method Call TestMethodCall Methods Method Result A=1 B=2 MethodCountValues( int x, int y) C=Class.TestMethodCo z=x+y+1 untValues(A+B) Return z C<>3 FAIL C=3 PASS
    • Ideal project team and unit testing Software Product Integration test engineer #1 Integration Integration test engineer #2 Test Sand Box Test Sand Box Test Sand Box Test Sand Box Developer 1 Developer 2 Developer 3 Developer 4 TESTS TESTS TESTS TESTS Implemented Implemented Implemented Implemented Software Software Component Software Component Software Component Component #3 #4 #2 #1
    • Integration Test with stubs/mocs STUB/MOCK Component STUB/MOCK Component Simulated Interface Log Tested Component/ApplicationControl Interface Messages/Events Control Configure Scripted STUB Interface
    • Integration Testing with simulated Interfaces Simulated Bluetooth Interface Fake Fake Spotify Facebook Server Server Simulated MIDI Interface
    • Integration Test with stubs STUB/MOCK Component STUB/MOCK Component Simulated Interface Log Tested Component/Application Control Interface Messages/Events Control Configure Scripted STUB InterfaceYläotsikko
    • in practice #1 IFDK android setup WEB SERVER simulating STUB/MOCK Component Service interface Simulated Interface Tested Component Application Trace/Log Control Interface Activate/Control Messages/Events Control Configure Scripted STUB InterfaceYläotsikko
    • in practice #2 server component testing Mock Server/Daemon WEB SERVER Simulated Interface Tested Component Trace/Log Application Needed Fake Application Control Interface Activate/Control Messages/Events Operating System Automated Test Interface Junit Scripted Interface Scripted STUB InterfaceYläotsikko
    • Integration TestingImplementa Testtion Framework Error Database Test Plan Component 2Implementa Testtion Framework
    • Regression TestTesting of a previously tested program followingmodification to ensure that defects have not beenintroduced or uncovered in unchanged areas of thesoftware, as a result of the changes made. It isperformed when the software or its environment isChanged.(ISTQB Syllabus)
    • Regression testing reason forREGRESSION automation?TEST PLANTest Case Execute Regression ? #1 Test when needed?Test Case #3 Test Automation SolutionTest Case #4 Export Results asTest Case Report #7Test Case #8
    • How to create Test Case??? Functional Test Case: Check also..... ●Requirement ●Use Case ●Feature Verify functionality of ● Write a Case ●Correct functionality ●User Story XXXX path ●Miss-usage of functionality Functional? ●Boundary Check Your Create a Test Sources Which Case! For Test Case Type? Regression Test Case?? Non-Functional? ●Customers Idea ●Brainstorm Non-Functional Test ●Intitution Cases Check also..... ●Exploratory Check Possiblity to ● automated testing? ●Verify Stability of XXXX Write a Case ●Verify Performance of XXX ●Verify Security of XXXX ●Verify Usability of XXXX ●Verify Scalability of XXXX ●etc...Yläotsikko
    • IFDK Verification/Validation (Testing Organization) Product Release Acceptance Test Engineer IFDK System Acceptance Testing System Test Engineer Test ManagerIFDK System Testing Project ManagerSystem Testing Designer/Coder IntegrationTest Automation Test Engineer Error Manager Feature Unit/Integration Testing Feature Pack Validation Verification Report Test Error Staus Management Database Database
    • System Testing in Small Appcliation DBWhat should be tested?How ?
    • Application Functional testing Application What should be tested? How ?
    • Application code level testing
    • Test Case Management Basics• Using Testlink• How to write test case?• Creating a test plan
    • About TestlinkTestLink is a web based Test Management tool. The application provides Testspecification, Test plans and execution, Reporting, Requirements specification andcollaborate with well-known bug trackers.
    • Test Management – Testlink PASSFeature TestCaseImplementation TestCase TestCase TestCase TestCase FAIL Test Plan/Suite Bug Report SourceCode Test Error Version Management Database Control Database TESTLINK
    • Test Plan Life Cycle Test Plan 1.0 Test Plan 2.x Needs Still Routine Work Test Effort REL 0.1 REL 0.2 REL 0.5 REL 1.0 Week 26 Week 52Week 0
    • Selected Test Cases = Test Plan/Suite Test Plan V 0.1 Test Case 1 Test Case 1 Test Case 2 Test Case 2 V2.0 V2.0 Test Case 3 Test Case 3 V2.0 V2.0
    • Test Case Pool and version control Test Case 2 Test Case 3 Test Case 4Test Case 1 V1.0 Draft V1.0 Test Case 2 Test Case 2 V2.0 V2.0 Test Case 2 V3.0
    • Test ManagementWhat we can do with TestlinkTool
    • Who executes test cases and when? Test Plan V 0.1 Test Case 1 Test Case 2 V2.0 Test Case 2 V2.0
    • What should be tested? Priority setting Test Case 2 Test Case 3 Test Case 1 V2.0 V2.0P4 P1 P3
    • What is a target? Product RELEASE/BUILDTest Plan V 0.1 VERSION 1.1 Executes Test Case 1 Product Executes RELEASE/BUILD Test Case 2 V2.0 VERSION 1.2 PASS / FAIL Test Case 3 V2.0 Executes Product RELEASE/BUILD VERSION 1.3 PASS / FAIL
    • Setting ConfigurationTestedRelease/configuration Version 0.4 Error/De fect Report Test Plan Version Test Error/De 0.2.2 Cases fect For Report Features Error/De Version 0.2.2.2 fect Report
    • Testlink use….
    • Needed ToolsTESTLINK: http://sourceforge.net/projects/testlink/BUGZILLA: http://www.bugzilla.org/
    • Execution of Test Case Test Engineer daily job ?Yläotsikko
    • What is generated as results from test case execution TOOLS SUT/DUT LOG FILE TEST CASE Test Case ”IFDK- Indications Events unit” NOTIFICATIONS ENVIRONMENT/ EMULATOR SUT = System Under Test DUT = Device Under TestYläotsikko
    • Regression TestTesting of a previously tested program followingmodification to ensure that defects have not beenintroduced or uncovered in unchanged areas of thesoftware, as a result of the changes made. It isperformed when the software or its environment isChanged.(ISTQB Syllabus)
    • Test Plan & Regression TEST PLAN 1 REGRESSION TEST PLAN 1.1 TEST PLANTest Case Test Case Test Case Test Case #1 #2 #9 #2 Test Case #1 NewTest Case Test Case TCases Test Case Test Case #3 #4 Added! #10 #11 Test Case #3Test Case Test Case Test Case Test Case #5 #6 #5 #6 Test Case #4Test Case Test Case Test Case Test Case #7 #8 #9 #12 Test Case #7Test Case Test Case Test Case Test Case #9 #10 #13 #10 Test Case #8
    • Regression testing reason forREGRESSION automation?TEST PLANTest Case Execute Regression ? #1 Test when needed?Test Case #3 Test Automation SolutionTest Case #4 Export Results asTest Case Report #7Test Case #8
    • Product verification/validation and test automation A Product Release Acceptance Test EngineerSystem Acceptance Testing System Acceptance Testing Test Engineer Test ManagerSystem Testing Project Manager Functional System Testing A Load, Stress, Designer/Coder performance Integration Regression Testing A Test Engineer Test AutomationFeature Unit/Integration Testing Engineer Integration Testing Component Integration Testing A Validation Feature Component Verification A Unit Testing
    • Heading - Sed posuere interdum sem. - Quisque ligula eros ullamcorper quis, lacinia quis facilisis sed sapien. - Mauris varius diam vitae arcu. Sed arcu lectus auctor vitae, consectetuer et venenatis eget velit. - Sed augue orci, lacinia eu tincidunt et eleifend nec lacus.Yläotsikko
    • Component Design Class Attributes MethodsYläotsikko
    • Feature example 1 (Invented on course 2009-2010) Calory Counter: Player can measure calories during training session. This can be seen as exercise result in web service eg. Facebook application Feature X * n Energy usageYläotsikko
    • SW Development Process (Agile) Product Backlog Task4 User Story X Task5 Task1 Task6 User Story Y Task7 Task2 Task8 User Story Z Task3 Task9 User Story Z Sprint Sprint Sprint Sprint Design Design Design Design Implementation Implementation Implementation Implementation Verification Verification Verification VerificationYläotsikko
    • Yläotsikko