Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Software Testing
Presentation
October 2017
Introduction
• Founded in 2001
• Test Lab in Central London
• Specialise in testing web-based applications and mobile appl...
Page 3 Software Testing Presentation - October 2017
Today’s Topic
Point 8 of the local government digital service standard:
“Be able to test the end-to-end service in an envi...
Today’s Topic
I will talk about:
• Comparison of the benefits and shortcomings of manual testing and
automated checking.
•...
Today’s Topic
I will not talk about:
• Unit testing.
• Automated checking (with one small exception).
• Security and load ...
Key Requirements
From the standard:
• Test not only during development but throughout the life of the service.
• As servic...
Comparison of the benefits and shortcomings of manual testing and automated
checking
Software Testing Presentation - Octob...
Comparison of the benefits and shortcomings of manual testing and automated
checking
Software Testing Presentation - Octob...
Comparison of the benefits and shortcomings of manual testing and automated
checking
Software Testing Presentation - Octob...
Comparison of the benefits and shortcomings of manual testing and automated
checking
Software Testing Presentation - Octob...
Comparison of the benefits and shortcomings of manual testing and automated
checking
Software Testing Presentation - Octob...
Comparison of the benefits and shortcomings of manual testing and automated
checking
Software Testing Presentation - Octob...
Comparison of the benefits and shortcomings of manual testing and automated
checking
Software Testing Presentation - Octob...
Comparison of the benefits and shortcomings of manual testing and automated
checking
Software Testing Presentation - Octob...
Comparison of the benefits and shortcomings of manual testing and automated
checking
Software Testing Presentation - Octob...
Context-driven Testing
Your context includes factors such as:
• Objectives
• Timescale
• Available budget and other resour...
Context-driven Testing
• Start by identifying your context.
• Consider how your context should influence your approach to ...
Different Types of Testing
• Give you different things.
• Automated checking is not better than manual testing. It’s total...
Scripted and Exploratory Testing
• Scripted testing focuses on verifying the documented requirements.
• Cannot test withou...
Scripted and Exploratory Testing
Software Testing Presentation - October 2017Page 21
Criterion Scripted Testing Explorator...
Scripted and Exploratory Testing
Software Testing Presentation - October 2017Page 22
Criterion Scripted Testing Explorator...
Scripted and Exploratory Testing
Software Testing Presentation - October 2017Page 23
Criterion Scripted Testing Explorator...
Scripted and Exploratory Testing
Software Testing Presentation - October 2017Page 24
Criterion Scripted Testing Explorator...
Scripted and Exploratory Testing
Software Testing Presentation - October 2017Page 25
Criterion Scripted Testing Explorator...
Scripted and Exploratory Testing
Software Testing Presentation - October 2017Page 26
Criterion Scripted Testing Explorator...
Client-side Test Environments
If you want accurate, repeatable results:
• Test on dedicated “clean” test machines i.e. new...
Client-side Test Environments
SaaS services include:
• BrowserStack - https://www.browserstack.com/
• Perfecto Mobile - ht...
Choosing Which Browsers and Devices to Test
The standard says “analyse current user behaviour to determine the device and
...
Choosing Which Browsers and Devices to Test
There are plenty of free sources but the best data usually has to be paid for:...
Manual Regression Testing
There is no “right” answer. Possible strategies include:
• Always do the same tests with the sam...
Manual Regression Testing
More options:
• Execute all the same “small” tests that you initially did.
• Very time-consuming...
Test Data
When entering data in forms:
• Don’t just “bang on the keyboard”.
• Create data that helps you identify if a bug...
Test Data
Example of typical data used when registering accounts for 10 users of an
e-commerce system.
Use Excel’s concate...
Steve Green
Managing Director
Test Partners Ltd
68-72 Queen Victoria Street, London EC4N 4SJ
020 3002 4176 (direct)
0800 6...
Upcoming SlideShare
Loading in …5
×

1

Share

Download to read offline

GLA Testing Presentation by Test Partners Ltd v1

Download to read offline

Software testing presentation

Related Books

Free with a 30 day trial from Scribd

See all

GLA Testing Presentation by Test Partners Ltd v1

  1. 1. Software Testing Presentation October 2017
  2. 2. Introduction • Founded in 2001 • Test Lab in Central London • Specialise in testing web-based applications and mobile applications • Functionality, compatibility and accessibility • 2000+ projects • 500+ clients Software Testing Presentation - October 2017Page 2
  3. 3. Page 3 Software Testing Presentation - October 2017
  4. 4. Today’s Topic Point 8 of the local government digital service standard: “Be able to test the end-to-end service in an environment similar to that of the live version, including all common browsers and devices.” http://localgovdigital.info/localgov-digital-makers/outputs/local-government- digital-service-standard/standard/8/ Software Testing Presentation - October 2017Page 4
  5. 5. Today’s Topic I will talk about: • Comparison of the benefits and shortcomings of manual testing and automated checking. • Context-driven testing. • Scripted and exploratory testing. • Client-side test environments. • Online services such as BrowserStack and Perfecto. • Choosing the devices and browsers to test. • Approaches to manual regression testing. • Test data. Software Testing Presentation - October 2017Page 5
  6. 6. Today’s Topic I will not talk about: • Unit testing. • Automated checking (with one small exception). • Security and load testing. • Infrastructure and test environments. • Development approaches such as TDD and BDD. • Accessibility testing (but perhaps another time). Software Testing Presentation - October 2017Page 6
  7. 7. Key Requirements From the standard: • Test not only during development but throughout the life of the service. • As services will be part of a continual improvement review, it is important to be able to test changes. • It is important that the service is tested on the devices and internet browsers that we know are in use within the user base. Software Testing Presentation - October 2017Page 7
  8. 8. Comparison of the benefits and shortcomings of manual testing and automated checking Software Testing Presentation - October 2017Page 8 Criterion Manual Testing Automated Checking Objective Investigate the application to verify expected behaviours, identify unexpected behaviours and identify any other information of interest to stakeholders. Verify expected behaviours.
  9. 9. Comparison of the benefits and shortcomings of manual testing and automated checking Software Testing Presentation - October 2017Page 9 Criterion Manual Testing Automated Checking Speed (initial test) Slow, but produces results immediately from the start. Slower than manual testing due to scripting time. Typically no results for some time.
  10. 10. Comparison of the benefits and shortcomings of manual testing and automated checking Software Testing Presentation - October 2017Page 10 Criterion Manual Testing Automated Checking Speed (initial test) Slow, but produces results immediately from the start. Slower than manual testing due to scripting time. Typically no results for some time. Speed (subsequent tests) Much the same as the initial test. Much faster than the initial test.
  11. 11. Comparison of the benefits and shortcomings of manual testing and automated checking Software Testing Presentation - October 2017Page 11 Criterion Manual Testing Automated Checking Speed (initial test) Slow, but produces results immediately from the start. Slower than manual testing due to scripting time. Typically no results for some time. Speed (subsequent tests) Much the same as the initial test. Much faster than the initial test. Ability to accommodate change Exploratory testing easily accommodates change. Scripted tests need to be updated. Poor - requires script maintenance.
  12. 12. Comparison of the benefits and shortcomings of manual testing and automated checking Software Testing Presentation - October 2017Page 12 Criterion Manual Testing Automated Checking Thoroughness (test scenarios) Typically includes both simple and complex scenarios. Typically limited to simple "happy path" scenarios.
  13. 13. Comparison of the benefits and shortcomings of manual testing and automated checking Software Testing Presentation - October 2017Page 13 Criterion Manual Testing Automated Checking Thoroughness (test scenarios) Typically includes both simple and complex scenarios. Typically limited to simple "happy path" scenarios. Thoroughness (test data) Very limited. Can check a huge number of data variations.
  14. 14. Comparison of the benefits and shortcomings of manual testing and automated checking Software Testing Presentation - October 2017Page 14 Criterion Manual Testing Automated Checking Thoroughness (test scenarios) Typically includes both simple and complex scenarios. Typically limited to simple "happy path" scenarios. Thoroughness (test data) Very limited. Can check a huge number of data variations. Performs "what if…" tests Exploratory testing does but scripted tests typically do not. No. Automated tools cannot ask questions or apply intelligence.
  15. 15. Comparison of the benefits and shortcomings of manual testing and automated checking Software Testing Presentation - October 2017Page 15 Criterion Manual Testing Automated Checking Notices behaviour that was not the focus of the test. It is an important feature of exploratory testing. People doing scripted tests could but typically do not. No. Automated checks only verify the assertions they have been programmed to do.
  16. 16. Comparison of the benefits and shortcomings of manual testing and automated checking Software Testing Presentation - October 2017Page 16 Criterion Manual Testing Automated Checking Notices behaviour that was not the focus of the test. It is an important feature of exploratory testing. People doing scripted tests could but typically do not. No. Automated checks only verify the assertions they have been programmed to do. Ability to test without design documentation No problem at all for exploratory testing. Scripted tests require design documentation. Automated checks require design documentation.
  17. 17. Context-driven Testing Your context includes factors such as: • Objectives • Timescale • Available budget and other resources • Risk • Scope and nature of changes • Regulatory environment • Lifespan • Constraints • Organisational culture Software Testing Presentation - October 2017Page 17
  18. 18. Context-driven Testing • Start by identifying your context. • Consider how your context should influence your approach to testing. • Be aware when your context changes. Do you need to change your testing? • Be wary of “best practices” that are implemented without consideration – these cause: • Waste. • Risk. Sounds obvious but it rarely happens in practice. People tend to do what they have always done. http://context-driven-testing.com/ Software Testing Presentation - October 2017Page 18
  19. 19. Different Types of Testing • Give you different things. • Automated checking is not better than manual testing. It’s totally different. • Exploratory testing is not necessarily better than scripted testing (although it is for most purposes). • Choose the types of testing that are most appropriate in your context. Software Testing Presentation - October 2017Page 19
  20. 20. Scripted and Exploratory Testing • Scripted testing focuses on verifying the documented requirements. • Cannot test without documented requirements. • Vulnerable to intentional undocumented design changes. • Exploratory testing focuses on what users can do to the system. • Verifies expected behaviours. • Identifies unexpected behaviours. • Little or no design documentation required. • Can accommodate incomplete functionality and undocumented design changes. Software Testing Presentation - October 2017Page 20
  21. 21. Scripted and Exploratory Testing Software Testing Presentation - October 2017Page 21 Criterion Scripted Testing Exploratory Testing Skill level required Minimal. Anyone can do it but requires skill to do well.
  22. 22. Scripted and Exploratory Testing Software Testing Presentation - October 2017Page 22 Criterion Scripted Testing Exploratory Testing Skill level required Minimal. Anyone can do it but requires skill to do well. Speed Very slow. 2 to 3 times faster.
  23. 23. Scripted and Exploratory Testing Software Testing Presentation - October 2017Page 23 Criterion Scripted Testing Exploratory Testing Skill level required Minimal. Anyone can do it but requires skill to do well. Speed Very slow. 2 to 3 times faster. Good for finding bugs Not if your developers are any good. Finds heaps of bugs.
  24. 24. Scripted and Exploratory Testing Software Testing Presentation - October 2017Page 24 Criterion Scripted Testing Exploratory Testing Skill level required Minimal. Anyone can do it but requires skill to do well. Speed Very slow. 2 to 3 times faster. Good for finding bugs Not if your developers are any good. Finds heaps of bugs. Records test coverage Yes. Can do.
  25. 25. Scripted and Exploratory Testing Software Testing Presentation - October 2017Page 25 Criterion Scripted Testing Exploratory Testing Skill level required Minimal. Anyone can do it but requires skill to do well. Speed Very slow. 2 to 3 times faster. Good for finding bugs Not if your developers are any good. Finds heaps of bugs. Records test coverage Yes. Can do. Provides detailed audit trail Yes. Can do but adds overhead.
  26. 26. Scripted and Exploratory Testing Software Testing Presentation - October 2017Page 26 Criterion Scripted Testing Exploratory Testing Skill level required Minimal. Anyone can do it but requires skill to do well. Speed Very slow. 2 to 3 times faster. Good for finding bugs Not if your developers are any good. Finds heaps of bugs. Records test coverage Yes. Can do. Provides audit trail Yes. Can do but adds overhead. Fun level Tedious beyond words. Highly engaging.
  27. 27. Client-side Test Environments If you want accurate, repeatable results: • Test on dedicated “clean” test machines i.e. new installation of the operating system and nothing else. • Avoid emulators. • Avoid virtual machines such as Parallels, VMWare and VirtualBox. • Don’t use tools that run multiple browser versions in one browser. • Check the provenance of all sources of browsers and plug-ins. Build a library while new versions are still available. If you value speed, cost and convenience over accuracy and repeatability, feel free to ignore all the above. Software Testing Presentation - October 2017Page 27
  28. 28. Client-side Test Environments SaaS services include: • BrowserStack - https://www.browserstack.com/ • Perfecto Mobile - https://www.perfectomobile.com/ Advantages • Cheap compared with buying devices. • Easier than maintaining a software library. • Probably ok for automated checking. Disadvantages • My personal experience was that they were terrible for manual testing. Software Testing Presentation - October 2017Page 28
  29. 29. Choosing Which Browsers and Devices to Test The standard says “analyse current user behaviour to determine the device and browser types in use”. • OK up to a point. But if the website does not work well with that browser or device, its browser or device usage statistics will be disproportionally low. • Use other sources of statistics but: • Check the demographics. Do they match yours? • Check the methodology. Is it valid? • Check the time period the statistics were collected over. • A narrow time period may not be statistically valid. • A wide time period will give emphasis to older browsers. • I usually advise one week to one month. Software Testing Presentation - October 2017Page 29
  30. 30. Choosing Which Browsers and Devices to Test There are plenty of free sources but the best data usually has to be paid for: • http://marketshare.hitslink.com/ • https://mixpanel.com/trends/#report/ios_frag/from_date:0,to_date:0 • http://www.w3schools.com/browsers/browsers_stats.asp (techie-biased) • https://www.fiksu.com/resources/ios_trackers • http://opensignal.com/reports/2015/08/android-fragmentation/ • https://www.browserstack.com/test-on-the-right-mobile-devices • http://gs.statcounter.com/ • https://www.appbrain.com/stats • https://david-smith.org/iosversionstats/ Software Testing Presentation - October 2017Page 30
  31. 31. Manual Regression Testing There is no “right” answer. Possible strategies include: • Always do the same tests with the same test data. • Always do the same tests with different test data. • Do different tests each time. • A mixture of these. • You stand the best chance of finding a bug if you do something you have not done before. • You might have a core set of tests, but introduce as much variation as you can. Software Testing Presentation - October 2017Page 31
  32. 32. Manual Regression Testing More options: • Execute all the same “small” tests that you initially did. • Very time-consuming. • Potentially wasteful. • Execute a subset of the same “small” tests that you initially did. • Saves time but is potentially risky. • Write a few “mega” tests that touch all the important features and areas. • Efficient. Frees up time to do more targeted testing. • One bug can block all subsequent tests. • Complex scenarios can make fault diagnosis difficult. Software Testing Presentation - October 2017Page 32
  33. 33. Test Data When entering data in forms: • Don’t just “bang on the keyboard”. • Create data that helps you identify if a bug has occurred. • Enter unique data wherever possible. • Make the data self-describing. • Keep a record of the data you enter, with time and date stamps. Benefits • Wherever or whenever you see a piece of data, you know where and when you entered it. • Corrupted and misplaced data is easily identified. Software Testing Presentation - October 2017Page 33
  34. 34. Test Data Example of typical data used when registering accounts for 10 users of an e-commerce system. Use Excel’s concatenation function to automatically create more complex data. Automatic data creation reduces probability of errors. Software Testing Presentation - October 2017Page 34 Date Time Title First Name Last Name Email Address 1 Address 2 Address 3 Town County 12 June 2017 09:37 Mr first1 last1 1sdg@testpartners.co.uk Address1-1 Address2-1 Address3-1 Town1 County1 12 June 2017 10:12 Mrs first2 last2 2sdg@testpartners.co.uk Address1-2 Address2-2 Address3-2 Town2 County2 12 June 2017 10:18 Miss first3 last3 3sdg@testpartners.co.uk Address1-3 Address2-3 Address3-3 Town3 County3 13 June 2017 14:34 Ms first4 last4 4sdg@testpartners.co.uk Address1-4 Address2-4 Address3-4 Town4 County4 13 June 2017 16:05 Dr first5 last5 5sdg@testpartners.co.uk Address1-5 Address2-5 Address3-5 Town5 County5 13 June 2017 16:17 Mr first6 last6 6sdg@testpartners.co.uk Address1-6 Address2-6 Address3-6 Town6 County6 13 June 2017 17:01 Mrs first7 last7 7sdg@testpartners.co.uk Address1-7 Address2-7 Address3-7 Town7 County7 14 June 2017 10:33 Miss first8 last8 8sdg@testpartners.co.uk Address1-8 Address2-8 Address3-8 Town8 County8 14 June 2017 12:55 Ms first9 last9 9sdg@testpartners.co.uk Address1-9 Address2-9 Address3-9 Town9 County9 14 June 2017 13:42 Dr first10 last10 10sdg@testpartners.co.uk Address1-10 Address2-10 Address3-10 Town10 County10
  35. 35. Steve Green Managing Director Test Partners Ltd 68-72 Queen Victoria Street, London EC4N 4SJ 020 3002 4176 (direct) 0800 612 2780 (switchboard) 07957 246 276 (mobile) 020 7692 5517 (fax) Skype: testpartners Twitter: testpartners http://uk.linkedin.com/in/stevegreen2 steve.green@testpartners.co.uk www.testpartners.co.uk Page 35 Contact Software Testing Presentation - October 2017
  • SagarHeddurshetti1

    Oct. 2, 2018

Software testing presentation

Views

Total views

394

On Slideshare

0

From embeds

0

Number of embeds

103

Actions

Downloads

4

Shares

0

Comments

0

Likes

1

×