Os Testes são cada vez mais uma necessidade nos projectos de desenvolvimento de software... Sejam eles unitários, de carga ou de "User Interface", uma boa framework de testes ajuda a resolver os problemas mais cedo, de forma mais eficaz e mais barata. No final da sessão vamos perceber não só para que servem, como são feitos e como o Visual Studio 2010 pode ajudar.

1. http://netponto.org 1ª Reunião em Coimbra - 09/04/2011 Testes? Mas isso não aumenta o tempo de projecto? Não quero...Pedro Rosa

2. Patrocinadores desta reunião

3. Pedro Rosa LinkedIn http://pt.linkedin.com/in/pedrobarraurosa Twitter http://twitter.com/pedrorosa

4. Agenda Why? Code Coverage Static / Dynamic Analysis Unit Testing WebTesting Load (Performance) Testing

5. Também disponível em vídeo... Assista! http://www.vimeo.com/22446417

7. Projects cost 50% more then budgeted

8. Lack of software quality cost US businesses $59B / year.

9. Bugs account for 55% of downtime Source: Standish Group, 2006 Third Quarter Research Report, CHAOS Research Results

10. Have you heard any of these? “my testers are spending too long testing the same thing” “tooling is expensive (time, licenses, people)” “the developers say the defects are useless” “when is my software ready to ship?” “developers and testers work in silos and don’t communicate/speak the same language” “we have a changed requirement, what do I need to test?”

11. 68%

12. What if you could… reduce the time it takes to determine the root cause of a bug enable users to easily run acceptance tests and track their results reduce the time it takes to configure and manage test environments reduce the time it takes to verify the status of a reported bug fix repair a bug without having to reproduce it

13. software bugs cost the US economy an estimated $59.5 billion every year on average professional coders make 100 to 150 errors in every 1000 lines of code they write last year, canceled projects cost firms $67 billion; overruns on the other projects racked up another $21billion

14. Need something a little more concrete?

15. USS Yorktown, SmartShip crew member entered 0 in a data entry field, caused a “divide by 0” error that shut down propulsion dead in the water for 2hrs 45mins

16. Ariane 5 Flight 501 re-used code from ariane 4, but took a different flight path because of different booster design conversation from 64bit float to 16bit signed int caused overflow (exception handler was disabled for perf reasons) cost: > $370 million @ 1996 prices

17. F-22 Raptor deploying overseas to japan the first time crossed international dateline, computers crashed losing all navigation and communications systems clear weather allowed them to follow tankers back to hawaii Raptor is likely the most advanced manned aircraft that will ever fly…

18. Need something a little more visual?

23. Still Not Convinced?

24. Climbing Cost of Failure Conditioning Training Training Phase

25. Release Cost of Bugs Test Development Software Phase

26. Push Quality Upstream Release Cost of Bugs Test Development Software Phase

27. Where does testing happen? 70% of testing happens here Majority of test tools target here Black Box Testing White Box Testing API Testing

28. Visual Studio 2010

29. Test Manager 2010 UML Modeling Manual Testing Layer Diagram Load Testing Web Testing Test Case Management IntelliTrace™ Architecture Explorer Logical Class Designer Cloud Development Office Development Windows Development New WPF Editor Customizable IDE Multi-core Development Silverlight Tools Web Development SharePoint Development Generate from Usage Static Code Analysis Database Deployment Code Metrics Database Unit Testing Test Data Generation Test Impact Analysis UI Test Automation Code Coverage Performance Profiling Database Change Mgmt Fast Forward for Manual Testing

31. Database Unit Test

32. Web PerformanceTest

33. Load Test

34. Manual Test (TestCase)

35. Generic Test

36. Ordered Test

38. Unit Test – Definition A unit test is a piece of code written by a developer that exercises a very small, specific area of functionality of the code being tested. “Program testing can be used to show the presence of bugs, but never to show their absence!” EdsgerDijkstra, [1972]

40. Manually, by hand

41. Manual tests are less efficient

42. Not structured

43. Not repeatable

44. Not on all your code

46. Unit Testing Some Facts Tests are specific pieces of code Unit testing framework is needed Visual Studio Team Test NUnit MbUnit Unit tests are written by developers, not by QA engineers Unit tests are released into the code repository along with the code they test

47. Unit Testing – More Facts All classes should be tested Test anything that could have bugs All methods should be tested Trivial code may be omitted E.g. property getters and setters Ideally all unit tests should pass before check-in into the source control repository

48. Why Unit Tests? Unit tests dramatically decrease the number of defects in the code Unit tests improve design Unit tests are good documentation Unit tests reduce the cost of change Unit tests allow refactoring Unit tests decrease the defect-injection rate due to refactoring / changes

49. Code and Test vs. Test Driven Development

50. Unit Testing Approaches "Code and Test" approach Classical approach "Test First" approach Test driven development (TDD)

51. Code and Test Approach Write code Write unit test Run and succeed Time flow

52. Create a testlist Pick а test Compile and fail Write enough code to compile Run test and fail Write code to pass test Test Driven Development Write test Time flow Remove duplication

53. Why Test Driven Development? Helps find design issues early and avoids rework Writing code to satisfy a test is a focused activity – less chance of error Tests will be a more comprehensive than when written after code

54. Unit Testing Frameworks and Visual Studio Team Test

55. Unit Testing Frameworks JUnit The first popular unit testing framework Based on Java Similar frameworks have been developed for a broad range of computer languages NUnit – for C# and all .NET languages cppUnit, jsUnit, PhpUnit, PerlUnit, ... Visual Studio Team Test (VSTT) Developed by Microsoft, integrated in VS

56. Visual Studio Team Test – Features Team Test (TT) is very well integrated with Visual Studio Create test projects and unit tests Execute unit tests View execution results View code coverage Located in the assembly Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll

58. [TestMethod]– denotes a unit test method

59. [ExpectedException]– test causes an exception

60. [Timeout]– sets a timeout for test execution

61. [Ignore]– temporary ignored test case

62. [ClassInitialize], [ClassCleanup]– setup / cleanup logic for the testing class

64. VSTT – Assertions Assertions check condition and throw exception if condition is not satisfied Comparing values AreEqual(expected value, calculated value [,message]) – compare two values for equality Comparing objects AreSame(expected object, current object [,message])– compare object references

65. VSTT – Assertions (2) Checking for null value IsNull(object [,message]) IsNotNull(object [,message]) Conditions IsTrue(condition) IsFalse(condition) Forced test fail Fail(message)

66. The 3A Pattern Arrange all necessary preconditions and inputs Act on the object or method under test Assert that the expected results have occurred [TestMethod] public void TestDeposit() { BanckAccount account = new BanckAccount(); account.Deposit(125.0); account.Deposit(25.0); Assert.AreEqual(150.0, account.Balance, "Balance is wrong."); }

67. VSTT – Example public class Account { private decimal balance; public void Deposit(decimal amount) { this.balance += amount; } public void Withdraw(decimal amount) { this.balance -= amount; } public void TransferFunds( Account destination, decimal amount) { ... } public decimal Balance { ... } }

68. VSTT – Example (2) using Microsoft.VisualStudio.TestTools.UnitTesting; [TestClass] public class AccountTest { [TestMethod] public void TransferFunds() { Account source = new Account(); source.Deposit(200.00M); Account dest = new Account(); dest.Deposit(150.00F); source.TransferFunds(dest, 100.00F); Assert.AreEqual(250.00F, dest.Balance); Assert.AreEqual(100.00F, source.Balance); } }

69. VSTT – Screenshot

70. Visual Studio Team Test Live Demo

71. Unit Testing Best Practices

72. Naming Standards for Unit Tests The test name should express a specific requirement that is tested Usually prefixed with [Test] E.g. TestAccountDepositNegativeSum() The test name should include Expected input or state Expected result output or state Name of the tested method or class

73. Naming Standards for Unit Tests Example Given the method: with requirement to ignore numbers greater than 100 in the summing process The test name should be: Sum_NumberIgnoredIfGreaterThan100 public int Sum(params int[] values)

74. When Should a Test be Changed or Removed? Generally, a passing test should never be removed These tests make sure that code changes don’t break working code A passing test should only be changed to make it more readable When failing tests don’t pass, it usually means there are conflicting requirements

75. When Should a Test be Changed or Removed? (2) Example: New features allows negative numbers [ExpectedException(typeof(Exception), "Negatives not allowed")] void Sum_FirstNegativeNumberThrowsException() { Sum (-1,1,2); }

76. When Should a Test be Changed or Removed?(3) New developer writes the following test: Earlier test fails due to a requirement change void Sum_FirstNegativeNumberCalculatesCorrectly() { int sumResult = sum(-1, 1, 2); Assert.AreEqual(2, sumResult); }

77. When Should a Test be Changed or Removed? (4) Two course of actions: Delete the failing test after verifying if it’s valid Change the old test: Either testing the new requirement Or test the older requirement under new settings

78. Tests Should Reflect Required Reality What’s wrong with the following test? A failing test should prove that there is something wrong with the production code Not with the unit test code int Sum(int a, int b) –> returns sum of a and b public void Sum_AddsOneAndTwo() { int result = Sum(1,2); Assert.AreEqual(4, result, "Bad sum"); }

79. What Should Assert Messages Say? Assert message in a test is one of the most important things Tells us what we expected to happen but didn’t, and what happened instead Good assert message helps us track bugs and understand unit tests more easily Example: "Withdrawal failed: accounts are not supposed to have negative balance."

80. What Should Assert Messages Say? (2) Express what should have happened and what did nothappen “Verify() did not throw any exception” “Connect() did not open the connection before returning it” Do not: Provide empty or meaningless messages Provide messages that repeat the name of the test case

81. Avoid Multiple Asserts in a Single Unit Test void Sum_AnyParamBiggerThan1000IsNotSummed() { Assert.AreEqual(3, Sum(1001, 1, 2); Assert.AreEqual(3, Sum(1, 1001, 2); Assert.AreEqual(3, Sum(1, 2, 1001); } Avoid multiple asserts in a single test case If the first assert fails, the test execution stops for this test case Affect future coders to add assertions to test rather than introducing a new one

82. Unit Testing – The Challenge The concept of Unit Testing has been around for many years New methodologies in particular XP, have turned unit testing into a cardinal foundation of software development Writing good & effective Unit Tests is hard! This is where supporting integrated tools and suggested guidelines enter the picture The ultimate goal is tools that generate unit tests automatically

84. Load tests run software unit tests, database unit tests, and web performance tests

86. Create and debug Web Performance Tests

87. Use Coded UI tests to measure end user performance

88. Run performance tests

89. Analyze results

90. Report to stakeholders

92. Throttles bandwidth, introduces latency and errors at the network layer

93. Scenarios

94. Single user testing over slow network

96. Web Test Recorder Plugin WebTestRecorderPlugins Visual Studio Launches Internet Explorer RecorderListener Browser Recorder Wininet Recorder Merging Recorder Internet Explorer Visual Studio Recorder BHO Browser events Recorded Web Test Record events Browser Control Wininet Shim Recorder Result Wininet Record events Intercept s wininet calls Record events

97. Load Test Agent Architecture Test agents run tests and collect data Test agent #1 Test agent #2 Test agent #3 Test Agent Run Tests Visual Studio Ultimate 2010 Test Controller Target Server Collector Agent Asp.Net Profiler Test Controller manages test agents

98. Visual Studio Team Test Live Demo

99. ? ? ? ? ? Questions? ? ? ? ? ?

100. Patrocinadores desta reunião

101. Próximas reuniões presenciais 09/04/2011 - Abril (Coimbra) 16/04/2011 - Abril (Lisboa) 21/05/2011 - Maio (Lisboa) 18/06/2011 - Junho (Lisboa)Reserva estes dias na agenda! :)

102. Obrigado! Pedro Rosa http://pt.linkedin.com/in/pedrobarraurosa http://twitter.com/pedrorosa