Automated Testing vs Manual Testing By Bhavin Turakhia CEO, Directi (shared under  Creative Commons Attribution Share-alik...
Manual Tests <ul><li>Coding Process with Manual Tests </li></ul><ul><ul><li>Write code </li></ul></ul><ul><ul><li>Uploadin...
Automated Tests <ul><li>Coding Process with Automated Unit Tests </li></ul><ul><ul><li>Write one or more test cases </li><...
Automated Tests vs Manual Tests <ul><li>Effort and Cost </li></ul><ul><ul><li>Lets assume 6 test cases </li></ul></ul><ul>...
Automated Tests vs Manual Tests <ul><li>Effort and Cost </li></ul><ul><ul><li>Adding incremental Unit test cases is cheape...
Automated Tests vs Manual Tests <ul><li>Manual Testing is boring </li></ul><ul><ul><li>Noone wants to keep filling the sam...
Automated Tests vs Manual Tests <ul><li>Manual Testing is not reusable </li></ul><ul><ul><li>The effort required is the sa...
Automated Tests vs Manual Tests <ul><li>Manual Tests provide limited Visibility and have to be Repeated by all Stakeholder...
Automated Tests vs Manual Tests <ul><li>Manual Testing ends up being an Integration Test </li></ul><ul><ul><li>In a typica...
Automated Tests vs Manual Tests <ul><li>Manual Testing requires complex Manual Setup and Tear Down </li></ul><ul><ul><li>C...
Automated Tests vs Manual Tests <ul><li>Manual Testing has a high risk of missing out on something </li></ul><ul><ul><li>E...
Automated Tests vs Manual Tests <ul><li>Manual Tests do not drive design </li></ul><ul><ul><li>Manual tests are run post-f...
Automated Tests vs Manual Tests <ul><li>Manual Tests do not provide a safety-net </li></ul><ul><ul><li>Manual tests are ru...
Automated Tests vs Manual Tests <ul><li>Manual Tests have no training value </li></ul><ul><li>Automated Tests act as docum...
Automated Tests vs Manual Tests <ul><li>Manual Tests create crazy code clutter </li></ul><ul><ul><li>Most manual testing i...
Summary <ul><li>Manual Tests take more Effort and Cost more than Automated Test to write and run </li></ul><ul><li>Manual ...
Summary <ul><li>Automated Testing ensures you dont miss out on running a test </li></ul><ul><li>Automated Testing can actu...
Why do people not write Automated Tests <ul><li>Initial learning curve </li></ul><ul><ul><li>Understanding Unit Testing Fr...
Why do people not write Automated Tests <ul><li>Solution </li></ul><ul><ul><li>Spend time during First Release to freeze /...
Why do people not write Automated Tests <ul><li>Don’t give up </li></ul><ul><ul><li>If you come across a hurdle, pair </li...
What to Test <ul><li>Unit Tests </li></ul><ul><ul><li>Ideally do not cross class boundaries </li></ul></ul><ul><ul><li>Def...
Best Practices <ul><li>You must use a unit testing frameworks (there’s one for every platform) </li></ul><ul><li>You must ...
Best Practices <ul><li>Organize the tests properly </li></ul><ul><li>If you do not commit Tests they are not reusable and ...
Visit our Websites http://careers.directi.com  |  http://www.directi.com
Upcoming SlideShare
Loading in...5
×

Automated Testing vs Manual Testing

72,072

Published on

From http://wiki.directi.com/x/AgAa - This is a 24 slide internal presentation covering virtues of Automated Testing vs Manual Testing. Inkeeping with our agile adoption this presentation covers various advantages (11 to be specific) obtained in using TDD and Automated Testing as opposed to Manual Testing

Published in: Business, Technology
4 Comments
19 Likes
Statistics
Notes
  • In software testing, test automation is the use of special software (separate from the software being tested) to control the execution of tests and the comparison of actual outcomes with predicted outcomes.[1] Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or add additional testing that would be difficult to perform manually
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • hey, this includes testing software manually. that is without using any tool or script..
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hi,



    Technology Specific Guide for QTP is a new book that has got great feedback from all the readers. It will get you started on the path of automation, introducing solutions, roadblocks, patches and ideas that will give you a solid foundation in the field of automation. Everyone from a novice user to an automation expert will have something they would have never heard of or used.



    http://bluemagic-automation.blogspot.in/



    Best Regards,

    Aditya.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Visit http://wiki.directi.com/x/AgAa for additional information and downloading the slides
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
72,072
On Slideshare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
1,132
Comments
4
Likes
19
Embeds 0
No embeds

No notes for slide

Automated Testing vs Manual Testing

  1. 1. Automated Testing vs Manual Testing By Bhavin Turakhia CEO, Directi (shared under Creative Commons Attribution Share-alike License incorporated herein by reference) ( http://creativecommons.org/licenses/by-sa/3.0/ )
  2. 2. Manual Tests <ul><li>Coding Process with Manual Tests </li></ul><ul><ul><li>Write code </li></ul></ul><ul><ul><li>Uploading the code to some place </li></ul></ul><ul><ul><li>Build it </li></ul></ul><ul><ul><li>Running the code manually (in many cases filling up forms etc step by step) </li></ul></ul><ul><ul><li>Check Log files, Database, External Services, Values of variable names, Output on the screen etc </li></ul></ul><ul><ul><li>If it does not work, repeat the above process </li></ul></ul>Creative Commons Attribution Share-alike
  3. 3. Automated Tests <ul><li>Coding Process with Automated Unit Tests </li></ul><ul><ul><li>Write one or more test cases </li></ul></ul><ul><ul><li>Auto-compile and run to see the tests fail </li></ul></ul><ul><ul><li>Write code to pass the tests </li></ul></ul><ul><ul><li>Auto-compile and run </li></ul></ul><ul><ul><li>If tests fail -> make appropriate modifications </li></ul></ul><ul><ul><li>If tests pass -> repeat for next method </li></ul></ul><ul><li>Coding Process with Automated Functional Tests </li></ul><ul><ul><li>Finish writing code (with all unit tests passing) </li></ul></ul><ul><ul><li>Write a Functional Test using any tool </li></ul></ul><ul><ul><li>Auto-compile and run </li></ul></ul><ul><ul><li>If tests fail -> make appropriate modifications </li></ul></ul><ul><ul><li>If tests pass -> move ahead </li></ul></ul>Creative Commons Attribution Share-alike
  4. 4. Automated Tests vs Manual Tests <ul><li>Effort and Cost </li></ul><ul><ul><li>Lets assume 6 test cases </li></ul></ul><ul><ul><li>Effort required to run all 6 manually => 10 min </li></ul></ul><ul><ul><li>Effort required to write unit tests for all 6 cases => 10 min </li></ul></ul><ul><ul><li>Effort required to run unit tests for all 6 cases => < 1 min </li></ul></ul><ul><ul><li>Number of testing iterations => 5 </li></ul></ul><ul><ul><li>Total manual testing time => 50 min </li></ul></ul><ul><ul><li>Total unit testing time => 10 min </li></ul></ul>Creative Commons Attribution Share-alike Release Manual Test Auto Test Manual Test Cumulative 1 10 10 10 2 10 0 20 3 10 0 30 4 10 0 40 5 10 0 50
  5. 5. Automated Tests vs Manual Tests <ul><li>Effort and Cost </li></ul><ul><ul><li>Adding incremental Unit test cases is cheaper than adding incremental Manual Test Cases </li></ul></ul><ul><ul><ul><li>Eg registerDomain </li></ul></ul></ul><ul><ul><ul><ul><li>Case 1: Register a .com domain with all correct fields </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Case 2: Register a .com domain with an invalid nameserver </li></ul></ul></ul></ul>Creative Commons Attribution Share-alike
  6. 6. Automated Tests vs Manual Tests <ul><li>Manual Testing is boring </li></ul><ul><ul><li>Noone wants to keep filling the same forms </li></ul></ul><ul><ul><li>There is nothing new to learn when one tests manually </li></ul></ul><ul><ul><li>People tend to neglect running manual tests </li></ul></ul><ul><ul><li>Noone maintains a list of the tests required to be run if they are manual tests </li></ul></ul><ul><li>Automated Tests on the other hand are code </li></ul><ul><ul><li>They are fun and challenging to write </li></ul></ul><ul><ul><li>One has to carefully think of design for reusability and coverage </li></ul></ul><ul><ul><li>They require analytical and reasoning skills </li></ul></ul><ul><ul><li>They represent contribution that is usable in the future </li></ul></ul>Creative Commons Attribution Share-alike
  7. 7. Automated Tests vs Manual Tests <ul><li>Manual Testing is not reusable </li></ul><ul><ul><li>The effort required is the same each time </li></ul></ul><ul><ul><li>One cannot reuse a Manual Test </li></ul></ul><ul><li>Automated Tests are completely reusable </li></ul><ul><ul><li>IMPORTANT: One needs to setup a Continuous Integration Server, a common Code Repository and a organization structure </li></ul></ul><ul><ul><li>Once written the Automated Tests form a part of the codebase </li></ul></ul><ul><ul><li>They can be reused without any additional effort for the lifetime of the Project </li></ul></ul>Creative Commons Attribution Share-alike
  8. 8. Automated Tests vs Manual Tests <ul><li>Manual Tests provide limited Visibility and have to be Repeated by all Stakeholders </li></ul><ul><ul><li>Only the developer testing the code can see the results </li></ul></ul><ul><ul><li>Tests have to be repeated by each stakeholder </li></ul></ul><ul><ul><ul><li>For eg Developer, Tech Lead, GM, Management </li></ul></ul></ul><ul><li>Automated Tests provide global visibility </li></ul><ul><ul><li>Developers, Tech Leads and Management can login and see Test Results </li></ul></ul><ul><ul><li>No additional effort required by any of them to see the software works!! </li></ul></ul>Creative Commons Attribution Share-alike Release Manual Testing by Dev Manual Testing by Team Leads Manual Testing by Mgmt Total Manual Testing Auto Test Dev Manual Test Cumulative Total Manual Test Cumulative 1 10 5 3 18 10 10 18 2 10 5 3 18 0 20 36 3 10 5 3 18 0 30 54 4 10 5 3 18 0 40 72 5 10 5 3 18 0 50 90
  9. 9. Automated Tests vs Manual Tests <ul><li>Manual Testing ends up being an Integration Test </li></ul><ul><ul><li>In a typical manual test it is very difficult to test a single unit </li></ul></ul><ul><ul><li>In most circumstances you end up checking the unit alongwith backend services </li></ul></ul><ul><ul><li>Introduces fragility – if something else breaks the manual test breaks </li></ul></ul><ul><li>Automated Tests can have varying scopes </li></ul><ul><ul><li>One can test a unit (class / method), a module, a system etc </li></ul></ul>Creative Commons Attribution Share-alike
  10. 10. Automated Tests vs Manual Tests <ul><li>Manual Testing requires complex Manual Setup and Tear Down </li></ul><ul><ul><li>Can involve frequently running db queries </li></ul></ul><ul><ul><li>Can involve making changes to backend servers </li></ul></ul><ul><ul><li>Steps become more complex with multiple dependent test cases </li></ul></ul><ul><li>Automated Tests can have varying scopes and require less complex setup and teardown </li></ul><ul><ul><li>Unit Tests have external dependencies mocked – so no setup / teardown required </li></ul></ul><ul><ul><li>Setup and Tear down are automated in Functional Tests using framework support </li></ul></ul>Creative Commons Attribution Share-alike
  11. 11. Automated Tests vs Manual Tests <ul><li>Manual Testing has a high risk of missing out on something </li></ul><ul><ul><li>Each time a developer runs manual tests it is likely he will miss out on an important test case </li></ul></ul><ul><ul><li>New developers may have no clue about the battery of tests to be run </li></ul></ul><ul><li>Automated Tests have zero risk of missing out a pre-decided test </li></ul><ul><ul><li>Once a Test becomes a part of Continuous Integration – it will run without someone having to remember to run it </li></ul></ul>Creative Commons Attribution Share-alike
  12. 12. Automated Tests vs Manual Tests <ul><li>Manual Tests do not drive design </li></ul><ul><ul><li>Manual tests are run post-facto and hence only drive bug-patching </li></ul></ul><ul><li>Automated Tests and TDD / Test-First development drive design </li></ul><ul><ul><li>Writing a Unit test first clarifies the requirement and influences design </li></ul></ul><ul><ul><li>Writing Unit Tests with Mock Objects etc forces clean design and segregation through abstraction / interfaces / polymorphism etc </li></ul></ul>Creative Commons Attribution Share-alike
  13. 13. Automated Tests vs Manual Tests <ul><li>Manual Tests do not provide a safety-net </li></ul><ul><ul><li>Manual tests are run post-facto and hence only drive bug-patching </li></ul></ul><ul><li>Automated Tests provide a safety-net for refactoring / additions </li></ul><ul><ul><li>Even New developers who have never touched the code can be confident about making changes </li></ul></ul>Creative Commons Attribution Share-alike
  14. 14. Automated Tests vs Manual Tests <ul><li>Manual Tests have no training value </li></ul><ul><li>Automated Tests act as documentation </li></ul><ul><ul><li>Reading a set of Unit Tests clarifies the purpose of a codebase </li></ul></ul><ul><ul><li>They provide a clear contract and define the requirement </li></ul></ul><ul><ul><li>They provide visibility into different use cases and expected results </li></ul></ul><ul><ul><li>A new developer can understand a piece of code much more by looking at Unit Tests than by looking at the code </li></ul></ul><ul><ul><li>Unit Tests define the expected behavior of the code </li></ul></ul>Creative Commons Attribution Share-alike
  15. 15. Automated Tests vs Manual Tests <ul><li>Manual Tests create crazy code clutter </li></ul><ul><ul><li>Most manual testing involves – </li></ul></ul><ul><ul><ul><li>System.outs to check values of variable names </li></ul></ul></ul><ul><ul><ul><li>Useless log file entries in app server, db server etc </li></ul></ul></ul><ul><ul><ul><li>Cause code / log / console clutter </li></ul></ul></ul><ul><ul><ul><ul><li>if then(s), flag based logging, event based log entries etc </li></ul></ul></ul></ul><ul><ul><ul><li>Slows down the application </li></ul></ul></ul><ul><li>Automated Tests reduce code clutter to zero </li></ul><ul><ul><li>Log file entries / System.outs are replaced by assertions in test code </li></ul></ul><ul><ul><li>Even if specific console / log entries are needed they can reside in the test and not in the code </li></ul></ul><ul><ul><li>Keep a live application / logs / console clutter-free and fast </li></ul></ul>Creative Commons Attribution Share-alike
  16. 16. Summary <ul><li>Manual Tests take more Effort and Cost more than Automated Test to write and run </li></ul><ul><li>Manual Testing is boring </li></ul><ul><li>Automated Tests are reusable </li></ul><ul><li>Manual Tests provide limited Visibility and have to be Repeated by all Stakeholders </li></ul><ul><li>Automated Tests can have varying scopes and can test single units of code by Mocking the dependencies </li></ul><ul><li>Automated tests may require less complex setup and teardown </li></ul>Creative Commons Attribution Share-alike
  17. 17. Summary <ul><li>Automated Testing ensures you dont miss out on running a test </li></ul><ul><li>Automated Testing can actually enforce and drive clean design decisions </li></ul><ul><li>Automated Tests provide a Safety Net for refactoring </li></ul><ul><li>Automated Tests have Training value </li></ul><ul><li>Automated Tests do not create clutter in code/console/logs </li></ul>Creative Commons Attribution Share-alike
  18. 18. Why do people not write Automated Tests <ul><li>Initial learning curve </li></ul><ul><ul><li>Understanding Unit Testing Frameworks and Functional Testing Frameworks </li></ul></ul><ul><ul><li>Understanding Continuous Integration and effective usage of it </li></ul></ul><ul><ul><li>Understanding and learning Code Coverage Tools </li></ul></ul><ul><ul><li>Figuring out how to organize the tests </li></ul></ul><ul><ul><li>How to create Mock Objects? </li></ul></ul><ul><ul><li>How to automate the running of the tests each time? </li></ul></ul><ul><ul><li>Where to commit the tests? </li></ul></ul><ul><li>Am I really going to be working on this same module again? </li></ul><ul><li>Will my tests be re-used? If not what is the point? </li></ul>Creative Commons Attribution Share-alike
  19. 19. Why do people not write Automated Tests <ul><li>Solution </li></ul><ul><ul><li>Spend time during First Release to freeze / design / implement - </li></ul></ul><ul><ul><ul><li>A Code Repository structure that incorporates Unit Tests and Functional Tests </li></ul></ul></ul><ul><ul><ul><li>A CI Server integrated with the release </li></ul></ul></ul><ul><ul><ul><li>Unit Testing Framework (any xUnit framework) </li></ul></ul></ul><ul><ul><ul><li>Functional Testing Tools (Sahi / Watir / Selenium / QTP etc) </li></ul></ul></ul><ul><ul><ul><li>Code Coverage Tools (Clover) </li></ul></ul></ul><ul><ul><ul><li>Testing guidelines and principles </li></ul></ul></ul><ul><ul><li>Designate Responsibility </li></ul></ul><ul><ul><ul><li>Each developer MUST write Unit tests for multiple use cases per unit </li></ul></ul></ul><ul><ul><ul><li>Designate a specific Developer to write Functional Tests </li></ul></ul></ul><ul><ul><ul><li>The developer who writes the tests is also responsible for organizing them, committing them and linking them in CI </li></ul></ul></ul>Creative Commons Attribution Share-alike
  20. 20. Why do people not write Automated Tests <ul><li>Don’t give up </li></ul><ul><ul><li>If you come across a hurdle, pair </li></ul></ul><ul><ul><li>Make sure you complete your testing responsibility </li></ul></ul><ul><li>Check Code Coverage </li></ul><ul><ul><li>Use code coverage tools while coding and post-coding to check parts of your code that are covered by tests </li></ul></ul>Creative Commons Attribution Share-alike
  21. 21. What to Test <ul><li>Unit Tests </li></ul><ul><ul><li>Ideally do not cross class boundaries </li></ul></ul><ul><ul><li>Definitely do not cross process-boundaries </li></ul></ul><ul><ul><li>Write a unit test with multiple cases </li></ul></ul><ul><li>Functional Tests </li></ul><ul><ul><li>UI Tests using specific tools (Watir / Selenium / QTP / White etc) </li></ul></ul><ul><ul><li>Tests one layer below the UI (Using APIs) </li></ul></ul>Creative Commons Attribution Share-alike
  22. 22. Best Practices <ul><li>You must use a unit testing frameworks (there’s one for every platform) </li></ul><ul><li>You must have an auto-build process, a CI server, auto-testing upon commits etc </li></ul><ul><li>Unit Tests are locally during the day, and upon commit by CI Server </li></ul><ul><li>Over a period of time you may want to have your CI Server run tests selectively </li></ul><ul><li>Tests must be committed alongwith code </li></ul>Creative Commons Attribution Share-alike
  23. 23. Best Practices <ul><li>Organize the tests properly </li></ul><ul><li>If you do not commit Tests they are not reusable and the reduced effort advantage is lost </li></ul>Creative Commons Attribution Share-alike
  24. 24. Visit our Websites http://careers.directi.com | http://www.directi.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×