Improved software testing using Visual Studio and TFS 2010

6,394 views
6,143 views

Published on

How to use the power of Team Foundation Server 2010 to improve your software quality

Published in: Technology

Improved software testing using Visual Studio and TFS 2010

  1. 1. Improved software testing<br />using Visual Studio and TFS 2010<br />Alexander Vanwynsberghe<br />Software Engineer<br />@vanwynsberghea<br />
  2. 2. What is this presentation about?<br />Software testing, why?<br />Use structured testing<br />The capabilities of TFS 2010<br />Summary<br />Alexander Vanwynsberghe – 09/05/2011<br />
  3. 3. Software testing, why?<br />It’s all about the quality<br />Lets you measure how well software is designed<br />Find the bugs!<br />Checks how the product meets the requirements<br />Testing does not make software perfect<br />Alexander Vanwynsberghe – 09/05/2011<br />
  4. 4. Use structured testing<br />What does that mean?<br />http://www.flickr.com/photos/59068359@N04/5563821201<br />
  5. 5. 1. Make test plans<br />A summary of the test activities that will be performed<br />Shows how the tests will be organized<br />A list of tests which, if performed, will identify all of the requirements that have been set<br />Alexander Vanwynsberghe – 09/05/2011<br />
  6. 6. 2. Define test cases<br />Description of a specific interaction that a tester will have in order to test a single behavior of the software<br />Step-by-step guide for the tester<br />Consists of:<br />Preconditions<br />To-do<br />Result<br />Alexander Vanwynsberghe – 09/05/2011<br />
  7. 7. 3. Execute tests<br />Run all test cases from the appropriate test plan<br />First iteration test focuses on new features<br />Regression testing usually involves executing all test cases which have previously been executed<br />Test parallel with the developers, not only just before a release<br />Alexander Vanwynsberghe – 09/05/2011<br />
  8. 8. 4. Automate tests<br />User interfaces can be test-automated<br />Eliminate repetitive tasks<br />Can save the testers a lot of time if many iterations of testing are required<br />Do not automate one-time-only tests<br />Alexander Vanwynsberghe – 09/05/2011<br />
  9. 9. 5. Report bugs<br />And assign them to developer(s)<br />http://www.flickr.com/photos/harmony19490/362411276/<br />
  10. 10. The ‘how-not-to-do-it’ process<br />http://http://www.flickr.com/photos/frau_k/3984560746//<br />
  11. 11. The developer..<br />All coding done<br />Some small ‘debugger-tests’<br />Check-in/commit<br />Deploy to a development environment<br />“Test please” to tester…<br />Alexander Vanwynsberghe – 09/05/2011<br />
  12. 12. The tester tests..<br />No plan available (no user stories)<br />Lucky to find bugs<br />Waste a lot of time on irrelevant things<br />This is called “Exploratory testing”<br />Alexander Vanwynsberghe – 09/05/2011<br />
  13. 13. ..and reports the bug..<br />Email subject: Mayor bug found!!<br />Email content: “I can’t quite remember what I did, but my data is gone !?”<br />Where is my data?<br />Alexander Vanwynsberghe – 09/05/2011<br />
  14. 14. The dev tests and replies..<br />Email subject: RE: mayor bug found!!<br />Alexander Vanwynsberghe – 09/05/2011<br />
  15. 15. Problems<br /><ul><li>Tester had no plan – Exploratory testing
  16. 16. Tester tests everything
  17. 17. “Unable to reproduce” = “Works on my machine”
  18. 18. Bug never gets fixed
  19. 19. Ship a buggy product</li></ul>Alexander Vanwynsberghe – 09/05/2011<br />
  20. 20. There is a better way<br />http://http://www.flickr.com/photos/frau_k/3984560746//<br />
  21. 21. Use the power of TFS 2010<br />All data is stored in TFS<br />Alexander Vanwynsberghe – 09/05/2011<br />
  22. 22. Manual testing<br />http://www.flickr.com/photos/avalsitarb/403407215/<br />
  23. 23. Manual testing with MTM 2010<br />Alexander Vanwynsberghe – 09/05/2011<br />
  24. 24. What can a tester do with MTM?<br /><ul><li>Create test plans
  25. 25. Define test cases
  26. 26. Execute test plans
  27. 27. Step-by-step execution
  28. 28. Screen recording
  29. 29. IntelliTrace logging
  30. 30. Inline bug creation with attachments </li></ul>Alexander Vanwynsberghe – 09/05/2011<br />
  31. 31. What does the developer receive?<br /><ul><li>A work-item of type ‘bug’ assigned to him/her
  32. 32. An email to inform about the bug</li></ul> (configured using TFS Alert Explorer)<br /><ul><li>The performed testcase with completed steps
  33. 33. Necessary information to reproduce the bug
  34. 34. IntelliTrace to get to the ‘bug point’ in code
  35. 35. Video recording
  36. 36. Comments of the tester</li></ul>Alexander Vanwynsberghe – 09/05/2011<br />
  37. 37. Ok, bug solved, what’s next?<br /><ul><li>Check-in the changes
  38. 38. Link the bug to the changeset
  39. 39. The changes for a particular bug are traceable
  40. 40. Developer sets the bug work-item to ‘Resolved’
  41. 41. Tester can test again and check the influence of the code change on other tests (Test Impact Analysis)</li></ul>Alexander Vanwynsberghe – 09/05/2011<br />
  42. 42. Some interesting things<br /><ul><li>When the developer finds the bug:
  43. 43. Don’t fix it at first
  44. 44. Create an automated test from the recorded test
  45. 45. Let the test fail
  46. 46. Fix the bug
  47. 47. Run the test again
  48. 48. TDD</li></ul>Will be explained later<br />Alexander Vanwynsberghe – 09/05/2011<br />
  49. 49. Demo MTM 2010<br />http://www.flickr.com/photos/avalsitarb/403407215/<br />
  50. 50. Automatic testing<br />http://www.flickr.com/photos/slimwhitman/2317727338<br />
  51. 51. Automatic testing with VS2010<br />Alexander Vanwynsberghe – 09/05/2011<br />
  52. 52. What can be tested with VS2010?<br /><ul><li>Code from the developer
  53. 53. Unit testing
  54. 54. Integration testing
  55. 55. User Interface = Coded UI testing
  56. 56. Database (stored procedures) = Database testing
  57. 57. Web Performance testing
  58. 58. Load testing</li></ul>Alexander Vanwynsberghe – 09/05/2011<br />
  59. 59. Unit tests<br /><ul><li>A piece of code that tests another piece of code
  60. 60. Readable
  61. 61. Understand!
  62. 62. What? Scenario? What will happen?
  63. 63. Maintainable
  64. 64. Trustworthy
  65. 65. If all unit tests passed = bug free</li></ul>Alexander Vanwynsberghe – 09/05/2011<br />
  66. 66. The structure of a unit test<br />Arrange<br />Act<br />Assert<br />[TestMethod]<br />public void Add_TwoPlusThree_EqualsFive()<br />{<br />// arrange<br /> int expected = 5; int first = 2; int second = 3;<br /> var calculator = new Calculator();<br />// act<br /> int actual = calculator.Add(first, second);<br />// assert<br /> Assert.AreEqual(expected, actual, “addition failed”);<br />}<br />Alexander Vanwynsberghe – 09/05/2011<br />
  67. 67. Integration test<br /><ul><li>Integrates different pieces:
  68. 68. UI
  69. 69. Business Logic
  70. 70. Database
  71. 71. Touches:
  72. 72. File system
  73. 73. Database
  74. 74. Other shared resources
  75. 75. Not in memory</li></ul>Alexander Vanwynsberghe – 09/05/2011<br />
  76. 76. Coded UI test<br /><ul><li>Test that the user interface functions correctly
  77. 77. Created using recording, or using code
  78. 78. For web, Win-forms and Silverlight
  79. 79. Can check control behaviors (like label values)</li></ul>Alexander Vanwynsberghe – 09/05/2011<br />
  80. 80. Database testing<br /><ul><li> Unit test for database objects like stored procedures
  81. 81. Can execute T-SQL statements like:
  82. 82. SELECT * FROM Content.Workspaces WHERE WorkspaceId = 1 --will fail, because it exists</li></ul>Alexander Vanwynsberghe – 09/05/2011<br />
  83. 83. Web performance testing<br /><ul><li>Simulate a user performing a set of operations
  84. 84. Validate the responses to see if the application is working as expected
  85. 85. Created by recording a human user clicking through a Web application
  86. 86. Web tests can be made smarter by using Validation Rules, Extraction Rules, and Test Context </li></ul>Alexander Vanwynsberghe – 09/05/2011<br />
  87. 87. Web performance testing<br />Alexander Vanwynsberghe – 09/05/2011<br />
  88. 88. Load testing<br /><ul><li>Contains web performance tests and unit tests
  89. 89. Simulate virtual users by setting the number of iterations
  90. 90. Collects all sorts of information like:
  91. 91. Response times of the executed requests,
  92. 92. Performance counters of the tested application infrastructure (like CPU, Memory, I/O, …)
  93. 93. The health of your load-testing infrastructure (load controller and agents)</li></ul>Alexander Vanwynsberghe – 09/05/2011<br />
  94. 94. Load testing<br />Alexander Vanwynsberghe – 09/05/2011<br />
  95. 95. Demo automatic testing<br />http://www.flickr.com/photos/avalsitarb/403407215/<br />
  96. 96. What you may not forget<br />http://www.flickr.com/photos/niffgurd/3324865930<br />
  97. 97. Summary<br /><ul><li>Testing is important, and easy with TFS
  98. 98. It gives confidence to those selling the product
  99. 99. The earlier a defect is found, the less development rework and re-test is required
  100. 100. Use proven high performance as a selling point
  101. 101. Testing needs discipline</li></ul>Alexander Vanwynsberghe – 09/05/2011<br />
  102. 102. Alexander Vanwynsberghe<br />@vanwynsberghea<br />Engelse Wandeling 2, K18<br />B-8500 Kortrijk, Belgium<br />alexander.vanwynsberghe@cognistreamer.comwww.cognistreamer.com<br />

×