Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Case Procountor: Zephyr test tool deployment

707 views

Published on

JCN Conference 2015

Published in: Technology
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1url.pw/ahKZe ◀ ◀ ◀ ◀
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Case Procountor: Zephyr test tool deployment

  1. 1. Zephyr Test Tool Deployment | Procountor Knowit Oy, Jussi Rautio 6.10.2015 PAGE 2 [CONFIDENTIAL] THIS IS KNOWIT6.10.2015 www.knowit.fi
  2. 2. 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.03 Senior Quality Consultant, Service Productization Responsible, Service Manager, Test Tool Technical support, Test Tool Deployments and Migrations… • Speaks, train, coach and mentor about • How to take new testing tool in use and best practices. • User trainings for testers, test managers, product owners and quality managers • Metrics and KPI`s benchmarking and design • Data Quality • Process enhancements • Etc. JUSSI RAUTIO, KNOWIT www.knowit.fi
  3. 3. KNOWIT Scandinavian company that is listed on Nordic Exchange Stockholm Knowit offering in Finland: • Test and quality management • Solutions for digital business and information management • Development and maintenance of client- specific applications 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.04 www.knowit.fi
  4. 4. 02.09.15 [CONFIDENTIAL] THIS IS KNOWITPAGE 5
  5. 5. • Leading provider of cloud accounting software • Used by over 10 000 companies and over 400 accounting offices • Available in Finland, Norway, Sweden and Denmark 02.09.15 [CONFIDENTIAL] THIS IS KNOWITPAGE 6
  6. 6. PROCOUNTOR ATLASSIAN ENVIRONMENT 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.07 Crowd JIRA FisheyeConfluence Refinedwiki JIRAagile ZephyrforJIRA Timesheetreports Crucible Confluence • Social Intranet • Knowledge Management JIRA • Requirement management • Product development (scrum) • Test management Fisheye/Crucible • Source code browsing • Code review Crowd • User management *500 *50*100*250 Hosted by: www.ambientia.net www.knowit.fi
  7. 7. 10/6/2015 © Copyright Knowit Oy 2014 | Confidential | Version 1.08 Problem Description www.knowit.fi
  8. 8. PROCOUNTOR NEED • Procountor is a rapidly growing firm • New people are hired constantly • Testing tool and processes need to be updated to meet the new demands • Knowit was asked to enhance Procountor testing 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.09 www.knowit.fi
  9. 9. TEST PROCESS ANALYSIS Knowit test process analysis for Procountor Means of analysis were: • Workshops • Interviews Two weeks after the release development teams are very busy on fixing bugs. Why? • Too tight schedule • Fixed content • Not enough time for regression testing 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.010 www.knowit.fi
  10. 10. TEST PROCESS ANALYSIS CONCLUSION Insufficient regression testing time -> Whole team approach should be introduced -> Testing tasks are in the different testing tool -> Better tool to enforce task sharing -> Jira ensures already a common way of working -> Test tool Zephyr was chosen to be used in regression testing because of seamless integration to Jira among other things 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.011 www.knowit.fi
  11. 11. © Copyright Knowit Oy 2014 | Confidential | Version 1.012 Introducing the Test Tool www.knowit.fi
  12. 12. TEST TOOL DEPLOYMENT PRINCIPLES Fail fast • If tool is not suited for you, don’t waste your time, look other way Start light • Put only necessary test data in to system in pilot phase • Do only minimum tweaking to the tool at the beginning Embrace the change • Give tool a chance 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.013 www.knowit.fi
  13. 13. www.knowit.fi Enhanced Quality and Effectivity Create a usage model for test tool Test tool in pilot use for regression testing Usage model tweaking Test tool in use for wider testing Test tool in use internationally Continuous improvement activities Integrations to other systems Test tool used through organization if Finland Virtualization & automation
  14. 14. GENERAL ABOUT ZEPHYR • Brings test management and execution functionalities to Jira New issue type Test • Possibility to have as an add on for Jira or as an own product • First Version released in the beginning of 2012 • Latest version 2.6.0 Released on 09.2015 Characteristics • Easy to use for Jira users • Customizable • No test case version control • If lots of add-on’s are in use the update process of Jira and Zephyr becomes challenging • No folder structure 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.015 www.knowit.fi
  15. 15. THINGS TO DO BEFORE DEPLOYMENT 1/3 • Fields for tests must be agreed • default fields can be used at start • Screens must be agreed • Tests workflow should be done when testing process with the new tool is mature enough • User groups and rights should be put in order • For example who can modify test executions should be restricted • Procountor has one tester per group so there is no need to restrict user rights 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.016 www.knowit.fi
  16. 16. THINGS TO DO BEFORE DEPLOYMENT 2/3 Labels must be decided • Label is users self chosen tag for tests • Labels should be agreed so that their amount would not explode • Labels could be for example: Functional testing, Regression testing, Usability, Load… 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.017 www.knowit.fi
  17. 17. THINGS TO DO BEFORE DEPLOYMENT 3/3 Components must be decided • Components are user defined parts of software • For example: Accounting, User Management, Billing.. 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.018 www.knowit.fi
  18. 18. TEST STATUSES One part of test process definition is specifying test statuses • Statuses can be modified based on test process Default test statuses of Zephyr are • PASS • FAIL • WIP • BLOCKED • UNEXECUTED 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.019 www.knowit.fi
  19. 19. LINKING TESTS TO USER STORIES Tests can be linked to User Stories • “Is blocked by” -link statuses are NOT affecting if the test can be run or not Linking should be done always so changes in User Story can be followed easily • Changes in user story are NOT sending signal for re-testing 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.020 www.knowit.fi
  20. 20. TEST CASE DIVISION 1. By Project • Project is a way to division test cases in a very high level. Procountor has only one Project 2. By Version • Fix Versions –field can be used to division tests. Test Summary Screen shows tests by version automatically 3. By Component • Test can be attached to a component (multivalue field.) Test Summary shows also these. 4. By Label • Anyone can create new labels so this should be agreed in organization to keep it controlled 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.021 www.knowit.fi
  21. 21. CYCLES – TEST ROUNDS Tests are executed in Cycles and one test can be in many Cycle Tests can be taken to test Cycles either by: • One at a time • By Filter • By Component • By Label 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.022 www.knowit.fi
  22. 22. NAMING AND DIVISION OF CYCLES Cycle name should present their division and support reporting. Cycle names can be for example: • Regression test for version xx (Procountor) • Sprint • Week • Tester • Group • Combination of above … 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.023 www.knowit.fi
  23. 23. © Copyright Knowit Oy 2014 | Confidential | Version 1.024 Monitoring Test Tool Deployment Success www.knowit.fi
  24. 24. TEST TOOL CONCEPT OWNER Someone from the organization should be named as a test tool concept owner after the test tool deployment Test tool concept owner tasks: • Follow test tool efficiency continuously • Follow test tool development roadmap • Introduce new test tool features to organization • Gather user requests for test tool customization and integrations 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.025 www.knowit.fi
  25. 25. MEASURING OF TEST TOOL EFFICIENCY Test tool efficiency should be followed in during the long period of time Has the quality of the product enhanced? • Number of bugs found in production • Customer satisfaction Has the efficiency increased? • Is the tool really used? • What is the gut feeling? Has the tool helped testing? Is regression testing easier now? • How wide has the tool usage spread? • Has the hurry after the release decreased? • What is the usability of the tool? 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.026 www.knowit.fi
  26. 26. FUTURE SCENARIO FOR PROCOUNTOR Testing during development could be done with exploratory testing • Link exploratory testing session to User Story • Leaves a trail what has been already tested 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.027 • Actor: intended user of the system • Purpose: the theme of the charter including what particular objective the actor wants to achieve, i.e., the test conditions • Setup: what needs to be in place in order to start the test execution • Priority: relative importance of this charter, based on the priority of the associated user story or the risk level • Reference: specifications (e.g., user story), risks, or other information sources • Data: whatever data is needed to carry out the charter • Activities: a list of ideas of what the actor may want to do with the system and what would be interesting to test (both positive and negative tests) • Oracle notes: how to evaluate the product to determine correct results • Variations: alternative actions and evaluations to complement the ideas described under activities www.knowit.fi
  27. 27. EXPLORATORY TESTING – TEST SESSION 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.028 Session based testing aims to combine accountability and exploratory testing Test sessions: • Based on charter • Uninterrupted session from 60 to 120 minutes • Evaluation of the functionality or characteristics • Each session is reported in detail Test Design Test Execution Test Results Learn Test Charter Test Session Test Design Test Execution Test Results Learn Test Charter Test Session Test Design Test Execution Test Results Learn Test Charter Test Session Session-Based Test Management www.knowit.fi
  28. 28. TOOLS FOR EXPLORATORY TESTING Record Steps to Reproduce a Problem • Windows Out Of the Box tool to take screenshots • Creates screenshots on each click and typing and creates one file from the recording • Free in all Windows versions JIRA Capture • Web –application to create issues easily • Records Test Session length, issues and tester notes • Needs to be purchased 6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.029 www.knowit.fi
  29. 29. THANK YOU! QUESTIONS? www.knowit.fi
  30. 30. Knowit Oy Tehtaankatu 27-29 D 00150 Helsinki tel. +358 40 3400600 www.knowit.fi
  31. 31. What Test Management challenges and possibilities are there in the agile/DevOps model? www.ambientia.net Let’s talk! © Copyright Knowit Oy 2015 | Confidential | Version 1.0www.knowit.fi

×