Lessons Learned in Testability            Scott McMaster                Google        Kirkland, Washington USA
About Me• Software Design Engineer @ Google.    – Building high-traffic web frontends and services in Java.    – AdWords, ...
Testing and Me• Doing automated testing since 1995.• Ph.D. work in test coverage and test suite maintenance.• Champion of ...
Agenda•   What is Testability?•   Testability Sins     – Statics and singletons     – Mixing Business and Presentation Log...
Testability: Formal Definition• Wikipedia: “the degree to which a software  artifact (i.e. a software system, software  mo...
Some Aspects of Testability• Controllable: We can put the software in a state to begin  testing.• Observable: We can see t...
•         Testability: your testingPractical    Testability is a function of                                 More goals.• ...
Testers and Testability• At Google, test engineers:  – Help ensure that developers build testable    software.  – Provide ...
Example: Weather App
Weather App Architecture  Rich Browser UI (GWT)    Frontend Server (GWT RPC             servlet)     Remote Web  Service (...
Original Weather App Design
Testability Problems?• Can’t test without calling the cloud service.  – Slow to run, unstable.• Can’t test any client-side...
Mission #1: Unit Tests for•   Problem:WeatherServiceImplto            Uses static singleton reference    GlobalWeatherServ...
WeatherServiceImpl: Before   private static GlobalWeatherService service = new GlobalWeatherService();                    ...
Proposed Solution• First we need to get rid of the static singleton.• Then we need something that:  – Behaves like GlobalW...
A Word About Static Methods and• Never use them! Singletons• They are basically global variables (and we’ve all  been taug...
Scott’s Rules About Static     Methods and Singletons1. Avoid static methods.2. For classes that are logically “singleton”...
Singleton Removal    public class WeatherServiceImpl extends RemoteServiceServlet implements        WeatherService {      ...
We’ve Broken Our Service!• The servlet container does not understand how to create WeatherServiceImpl anymore.  – Its cons...
Dependency Injection• Can be a little complicated, but here is what you  need to know here:• Accept your dependencies, don...
• When properly set up, it will create your objects and pass  them to the appropriate constructors at runtime, freeing you...
Fixing WeatherServiceImpl (1)•Configure our servlet to use Guice and tell it about our objects:     public class WeatherAp...
Fixing WeatherServiceImpl (2)• At runtime, Guice will create our servlet and the object(s) it  needs:    @Singleton    pub...
After Testability Refactoring #1
Finally! We Can Test!• But how?• We want to test WeatherServiceImpl in isolation.  ⇒For GlobalWeatherService, we only care...
• The mock object framework verifies these interactions occur as expected.    – A useful consequence of this: If appropria...
Using Mock Objects1. Create a mock object.2. Set up expectations:  1. How we expect the class-under-test to call it.  2. W...
Testing with a Mock Object    private GlobalWeatherService globalWeatherService;                                 globalWea...
• Problem: Talks to external web service, does non-trivial XML          Mission #2: Unit Tests for    processing that we w...
After Testability Refactoring #2
–   Contains a simplified implementation of the real thing (perhaps using static data, an in-memory         database, etc....
Use a Fake, or Use a Mock?• Problem: Setting up a mock object for  GlobalWeatherDataAccess that returns static XML is  pos...
Implementing the Fake Objectpublic class FakeGlobalWeatherDataAccess implements GlobalWeatherDataAccess {  // Try http://w...
Testing with a Fake Objectprivate GlobalWeatherServiceImpl globalWeatherService;                                 globalWea...
• Problem: UI and business logic / service calls all mixed        Mission #3: Unit Tests for  together.                   ...
Mixing Business and Presentation            @UiHandler("login")            void onLogin(ClickEvent e) {                 on...
Model-View-Presenter (MVP)• UI pattern that separates business and  presentation logic.• Makes the View easier to modify.•...
Model-View-Presenter Overview
Model-View-Presenter          Responsibilities• Presenter uses the View interface to  manipulate the UI.• View delegates U...
•             Passive Viewthe View is    A particular style of MVP where                                    MVP    complet...
After Testability Refactoring #3
Presenter Unit Test Using                      EasyMock@Testpublic void testOnLogin_unknownUser() {            testOnLogin...
Question• Why does the View make the Presenter do  this:                                         ;  weatherView.setUnknown...
AnswerweatherView.getUnknownUser().setVisible(true)• Is hard to test because it is hard to mock:–To mock this, we would ha...
• Also known as the “Don’t talk to strangers” rule.• It says:               Law of friends.   Demeter   –Only have knowled...
• To write good unit tests, we need to be able to insert mocks and fakes   into the code.                 What’s the Point...
What Can I Do?• Developers:   – Follow these practices!• Testers:   – Educate your developers.   – Jump into the code and ...
Questions?       Scott McMaster            Google  Kirkland, Washington USAscott.d.mcmaster (at) gmail.com
Bonus Slides
Model-View-Controller (MVC)
Model-View-Controller (MVC)• View directly accesses the Model and fires  events to the Controller.• Controller performs op...
Which to Use?• Many web MVC frameworks exist (Struts, Rails, ASP.NET MVC).• But these days, we work more with MVP.
Manually Created Mockpublic class MockWeatherServiceAsync implements WeatherServiceAsync {    private List<String> expectG...
2012-12-20
Upcoming SlideShare
Loading in …5
×

谷歌 Scott-lessons learned in testability

474 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
474
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

谷歌 Scott-lessons learned in testability

  1. 1. Lessons Learned in Testability Scott McMaster Google Kirkland, Washington USA
  2. 2. About Me• Software Design Engineer @ Google. – Building high-traffic web frontends and services in Java. – AdWords, Google Code• Ph.D. in Computer Science, U. of Maryland.• Formerly of Amazon.com (2 years), Lockheed Martin (2 years), Microsoft (7 years), and some small startups.• Frequent adjunct professor @ Seattle University, teaching software design, architecture, and OO programming.• Author of technical blog at http://www.scottmcmaster365.com.
  3. 3. Testing and Me• Doing automated testing since 1995.• Ph.D. work in test coverage and test suite maintenance.• Champion of numerous unit, system, and performance testing tools and techniques.• Co-founder of WebTestingExplorer open-source automated web testing framework (www.webtestingexplorer.org).
  4. 4. Agenda• What is Testability?• Testability Sins – Statics and singletons – Mixing Business and Presentation Logic – Breaking the Law of Demeter• Testability Solutions – Removing singletons. – Asking for Dependencies – Dependency Injection – Mocks and Fakes – Refactoring to UI Patterns
  5. 5. Testability: Formal Definition• Wikipedia: “the degree to which a software artifact (i.e. a software system, software module, requirements- or design document) supports testing in a given test context.” http://en.wikipedia.org/wiki/Software_testability
  6. 6. Some Aspects of Testability• Controllable: We can put the software in a state to begin testing.• Observable: We can see things going right (or wrong).• Isolatable: We can test classes/modules/systems apart from others.• Automatable: We can write or generate automated tests. – Requires each of the previous three to some degree.
  7. 7. • Testability: your testingPractical Testability is a function of More goals.• Definition automated tests. Our primary goal is to write or generate• Therefore, testability is the ease with which we can write: – Unit tests – System tests – End-to-end tests
  8. 8. Testers and Testability• At Google, test engineers: – Help ensure that developers build testable software. – Provide guidance to developers on best practices for unit and end-to-end testing. – May participate in refactoring production code for testability.
  9. 9. Example: Weather App
  10. 10. Weather App Architecture Rich Browser UI (GWT) Frontend Server (GWT RPC servlet) Remote Web Service (XML-over- User Database HTTP)
  11. 11. Original Weather App Design
  12. 12. Testability Problems?• Can’t test without calling the cloud service. – Slow to run, unstable.• Can’t test any client-side components without loading a browser or browser emulator. – Slow to develop, slow to run, perhaps unstable.
  13. 13. Mission #1: Unit Tests for• Problem:WeatherServiceImplto Uses static singleton reference GlobalWeatherService, can’t be tested in isolation.• Solution: – Eliminate the static singleton. – Pass a mock or stub to the WeatherServiceImpl constructor at test-time.
  14. 14. WeatherServiceImpl: Before private static GlobalWeatherService service = new GlobalWeatherService(); GlobalWeatherService(); public List<String> getCitiesForCountry(String countryName) { getCitiesForCountry(String countryName) try { if (countryName == null || countryName.isEmpty()) { countryName.isEmpty()) return new ArrayList<String>(); ArrayList<String>(); } return service.getCitiesForCountry(countryName); service.getCitiesForCountry(countryName) } catch (Exception e) { throw new RuntimeException(e); RuntimeException(e); } }What if we try to test this in its current form?1. GlobalWeatherService gets loaded at classload-time. 1. This itself could be slow or unstable depending on the implementation.2. When we call getCititesForCountry(“China”), a remote web service call gets made.3. This remote web service call may: 1. Fail. 2. Be really slow. 3. Not return predictable results. � Any of these things can make our test “flaky”.
  15. 15. Proposed Solution• First we need to get rid of the static singleton.• Then we need something that: – Behaves like GlobalWebService. – Is fast and predictable. – Can be inserted into WeatherServiceImpl at test- time.
  16. 16. A Word About Static Methods and• Never use them! Singletons• They are basically global variables (and we’ve all been taught to avoid those).• They are hard to replace with alternative implementations, mocks, and stubs/fakes. – They make automated unit testing extremely difficult.
  17. 17. Scott’s Rules About Static Methods and Singletons1. Avoid static methods.2. For classes that are logically “singleton”, make them non-singleton instances and manage them in a dependency injection container (more on this shortly).
  18. 18. Singleton Removal public class WeatherServiceImpl extends RemoteServiceServlet implements WeatherService { private final GlobalWeatherService service; public WeatherServiceImpl(GlobalWeatherService service) { WeatherServiceImpl( this.service = service; } ...• Also, make GlobalWeatherService into an interface.• Now we can pass in a special implementation for unit testing.• But we have a big problem…
  19. 19. We’ve Broken Our Service!• The servlet container does not understand how to create WeatherServiceImpl anymore. – Its constructor takes a funny parameter.• The solution?
  20. 20. Dependency Injection• Can be a little complicated, but here is what you need to know here:• Accept your dependencies, don’t ask for them. – Then your dependencies can be replaced (generally, with simpler implementations) at test time. – In production, your dependencies get inserted by a dependency injection container. • In Java, this is usually Spring or Google Guice.
  21. 21. • When properly set up, it will create your objects and pass them to the appropriate constructors at runtime, freeing you Dependency Injection with Google Guice up to do other things with the constructors at test-time.• Setting up Guice is outside the scope of this talk. – This will get you started: http://code.google.com/p/google- guice/wiki/Servlets
  22. 22. Fixing WeatherServiceImpl (1)•Configure our servlet to use Guice and tell it about our objects: public class WeatherAppModule extends AbstractModule { @Override protected void configure() { bind(WeatherServiceImpl.class); class class); bind(GlobalWeatherService.class).to(GlobalWeatherServiceImpl.class); class ).to(GlobalWeatherServiceImpl.class class).to( GlobalWeatherServiceImpl.class); }•When someone asks for a “GlobalWeatherService”, Guice will }give it an instance of GlobalWeatherServiceImpl.
  23. 23. Fixing WeatherServiceImpl (2)• At runtime, Guice will create our servlet and the object(s) it needs: @Singleton public class WeatherServiceImpl extends RemoteServiceServlet implements WeatherService { private final GlobalWeatherService service; @Inject public WeatherServiceImpl(GlobalWeatherService service) { WeatherServiceImpl( this.service = service; } ...The “@Inject” constructor parameters is how we ask Guice forinstances.
  24. 24. After Testability Refactoring #1
  25. 25. Finally! We Can Test!• But how?• We want to test WeatherServiceImpl in isolation. ⇒For GlobalWeatherService, we only care about how it interacts with WeatherServiceImpl. ⇒To create the proper interactions, a mock object is ideal
  26. 26. • The mock object framework verifies these interactions occur as expected. – A useful consequence of this: If appropriate, you can verify that an application is not making more remote calls than expected. Mock Object Testing – Another useful consequence: Mocks make it easy to test exception handling.• Common mocking frameworks (for Java): – Mockito – EasyMock • I will use this.
  27. 27. Using Mock Objects1. Create a mock object.2. Set up expectations: 1. How we expect the class-under-test to call it. 2. What we want it to return.3. “Replay” the mock.4. Invoke the class-under-test.5. “Verify” the mock interactions were as-expected.
  28. 28. Testing with a Mock Object private GlobalWeatherService globalWeatherService; globalWeatherService; private WeatherServiceImpl weatherService; weatherService; @Before public void setUp() { setUp() globalWeatherService = EasyMock.createMock(GlobalWeatherService.class ); EasyMock.createMock( GlobalWeatherService.class); weatherService = new WeatherServiceImpl(globalWeatherService); WeatherServiceImpl( globalWeatherService); } @Test public void testGetCitiesForCountry_nonEmpty() throws Exception { testGetCitiesForCountry_nonEmpty() EasyMock.expect(globalWeatherService.getCitiesForCountry ("china")) EasyMock.expect( globalWeatherService.getCitiesForCountry("china")) .andReturn(ImmutableList.of("beijing", "shanghai")); .andReturn ImmutableList.of(" beijing", andReturn( ("beijing EasyMock.replay(globalWeatherService); EasyMock.replay( globalWeatherService); List<String> cities = weatherService.getCitiesForCountry ("china"); List<String> weatherService.getCitiesForCountry("china"); assertEquals(2, cities.size()); assertEquals(2, cities.size()); assertTrue(cities.contains("beijing")); assertTrue( cities.contains(" beijing")); ("beijing assertTrue(cities.contains("shanghai")); assertTrue( cities.contains("shanghai")); EasyMock.verify(globalWeatherService); EasyMock.verify( globalWeatherService); }Observe:• How we take advantage of the new WeatherServiceImpl constructor.• How we use the mock GlobalWeatherService.
  29. 29. • Problem: Talks to external web service, does non-trivial XML Mission #2: Unit Tests for processing that we want to test.• Solution:GlobalWeatherService – Split the remote call from the XML processing. – Wrap external web service in an object with a known interface. – Pass an instance to the GlobalWeatherServiceImpl constructor. – Use dependency injection to create the real object at runtime, use a fake at test-time.
  30. 30. After Testability Refactoring #2
  31. 31. – Contains a simplified implementation of the real thing (perhaps using static data, an in-memory database, etc.). – Implementation usually generated by hand. – Often reusable across test cases and test suites if carefully designed.• Mocks and Fakes – Fakes vs. Mocks Either can often be used in a given situation. – But some situations lend themselves to one more than the other.
  32. 32. Use a Fake, or Use a Mock?• Problem: Setting up a mock object for GlobalWeatherDataAccess that returns static XML is possible, but ugly (and perhaps not very reusable).• Idea: Create a fake implementation of GlobalWeatherDataAccess. – We can give the fake object the capability to return different XML in different test circumstances.
  33. 33. Implementing the Fake Objectpublic class FakeGlobalWeatherDataAccess implements GlobalWeatherDataAccess { // Try http://www.htmlescape.net/javaescape_tool.html to generate these. // http://www.htmlescape.net www.htmlescape.net/ private static final String CHINA_CITIES = "<?xml version="1.0" encoding="utf-8"?>n<string xml private static final String BEIJING_WEATHER = "<?xml version="1.0" encoding="utf-8"?>n<string&lt;Visibility&gt; greater than 7 mile(s):0&lt;/ Visibility&gt;n &lt;Temperature&gt; 39 F (4 C)&lt;/ lt;Visibility&gt; mile(s):0&lt;/Visibility&gt Visibility&gt;n &lt;Temperature&gt lt;Temperature&gt; C)&lt lt;/ private static final String NO_CITIES = "<?xml version="1.0" encoding="utf-8"?>n<string xmlns= xmlns= @Override @Override public String getCitiesForCountryXml(String countryName) throws Exception { getCitiesForCountryXml(String countryName) if ("china".equals(countryName.toLowerCase())) { ("china".equals countryName.toLowerCase())) china".equals( return CHINA_CITIES; } return NO_CITIES; } @Override @Override public String getWeatherForCityXml(String countryName, String cityName) getWeatherForCityXml(String countryName, cityName) throws Exception { return BEIJING_WEATHER; }}
  34. 34. Testing with a Fake Objectprivate GlobalWeatherServiceImpl globalWeatherService; globalWeatherService;private FakeGlobalWeatherDataAccess dataAccess; dataAccess;@Beforepublic void setUp() { setUp() dataAccess = new FakeGlobalWeatherDataAccess(); FakeGlobalWeatherDataAccess(); globalWeatherService = new GlobalWeatherServiceImpl(dataAccess); GlobalWeatherServiceImpl(dataAccess);}@Testpublic void testGetCitiesForCountry_nonEmpty () throws Exception { testGetCitiesForCountry_nonEmpty() List<String> cities = globalWeatherService.getCitiesForCountry("china"); assertEquals(2, cities.size()); assertTrue(cities.contains("beijing")); assertTrue(cities.contains("shanghai"));}@Testpublic void testGetCitiesForCountry_empty () throws Exception { testGetCitiesForCountry_empty() List<String> cities = globalWeatherService.getCitiesForCountry("nowhere"); assertTrue(cities.isEmpty());}The fake keeps the tests short, simple, and to-the-point!
  35. 35. • Problem: UI and business logic / service calls all mixed Mission #3: Unit Tests for together. WeatherHome – The view layer is difficult and slow to instantiate at unit test-time. – But we need to unit test the business logic.• Solution: – Refactor to patterns -- Model-View-Presenter (MVP). – Write unit tests for the Presenter using a mock or stub View.
  36. 36. Mixing Business and Presentation @UiHandler("login") void onLogin(ClickEvent e) { onLogin( weatherService.getWeatherForUser(userName.getText(), new AsyncCallback<Weather>() { AsyncCallback<Weather>() @Override public void onFailure(Throwable caught) { onFailure( Window.alert("oops"); } @Override public void onSuccess(Weather weather) { onSuccess( weather) if (weather != null) { fillWeather(weather); false); unknownUser.setVisible(false); } else { true); unknownUser.setVisible(true); } } }); }How NOT to write a UI event handler for maximum testability:• Have tight coupling between the UI event, processing a remote service call, and updating the UI.
  37. 37. Model-View-Presenter (MVP)• UI pattern that separates business and presentation logic.• Makes the View easier to modify.• Makes the business logic easier to test by isolating it in the Presenter.
  38. 38. Model-View-Presenter Overview
  39. 39. Model-View-Presenter Responsibilities• Presenter uses the View interface to manipulate the UI.• View delegates UI event handling back to the Presenter via an event bus or an interface.• Presenter handles all service calls and reading/updating of the Model.
  40. 40. • Passive Viewthe View is A particular style of MVP where MVP completely passive, only defining and layout and exposing its widgets for manipulation by the controller. – In practice, you sometimes don’t quite get here, but this is the goal.• Especially if you use this style, you can skip testing the View altogether.
  41. 41. After Testability Refactoring #3
  42. 42. Presenter Unit Test Using EasyMock@Testpublic void testOnLogin_unknownUser() { testOnLogin_unknownUser() weatherService.expectGetWeatherForUser("unknown"); EasyMock.expect(weatherView.getUserName()).andReturn("unknown"); true); weatherView.setUnknownUserVisible(true); EasyMock.expectLastCall(); weatherView.setEventHandler(EasyMock.anyObject(WeatherViewEventHandler.class)); class class)); EasyMock.expectLastCall(); EasyMock.replay(weatherView); WeatherHomePresenter presenter = new WeatherHomePresenter(weatherService, weatherView); WeatherHomePresenter( weatherService, weatherView); presenter.onLogin(); EasyMock.verify(weatherView); weatherService.verify();} This test uses a manually created mock to make handling the async callback easier.
  43. 43. Question• Why does the View make the Presenter do this: ; weatherView.setUnknownUserVisible(true);• Instead of this: weatherView.getUnknownUser().setVisible(true)
  44. 44. AnswerweatherView.getUnknownUser().setVisible(true)• Is hard to test because it is hard to mock:–To mock this, we would have to mock not only theWeatherView, but also the UnknownUser Label insideof it.• The above code is “talking to strangers”.
  45. 45. • Also known as the “Don’t talk to strangers” rule.• It says: Law of friends. Demeter –Only have knowledge of closely collaborating objects. –Only make calls on immediate• Look out for long chained “get”-style calls (and don’t do them: a.getB().getC().getD()• Your system will be more testable (and maintainable, because you have to rework calling objects less often).
  46. 46. • To write good unit tests, we need to be able to insert mocks and fakes into the code. What’s the Point?• Some things that help us do that: – Eliminating static methods and singletons. – Asking for dependencies instead of creating them. – Using design patterns that promote loose coupling, especially between business and presentation logic. – Obeying the Law of Demeter.• Code that does not do these things will often have poor test coverage.
  47. 47. What Can I Do?• Developers: – Follow these practices!• Testers: – Educate your developers. – Jump into the code and drive testability improvements. • A good way to motivate this is to track test coverage metrics.
  48. 48. Questions? Scott McMaster Google Kirkland, Washington USAscott.d.mcmaster (at) gmail.com
  49. 49. Bonus Slides
  50. 50. Model-View-Controller (MVC)
  51. 51. Model-View-Controller (MVC)• View directly accesses the Model and fires events to the Controller.• Controller performs operations on the Model.• Controller doesn’t really know about the View other than selecting the View to render.
  52. 52. Which to Use?• Many web MVC frameworks exist (Struts, Rails, ASP.NET MVC).• But these days, we work more with MVP.
  53. 53. Manually Created Mockpublic class MockWeatherServiceAsync implements WeatherServiceAsync { private List<String> expectGetWeatherForUserCalls = Lists.newArrayList(); newArrayList(); private List<String> observeGetWeatherForUserCalls = Lists.newArrayList(); newArrayList(); // More @Overrides not shown on the slide. @Override public void getWeatherForUser(String userName, AsyncCallback<Weather> callback) { getWeatherForUser(String userName, AsyncCallback<Weather> observeGetWeatherForUserCalls.add(userName); if ("scott".equals(userName)) { ("scott ".equals(userName scott".equals( userName)) new callback.onSuccess(new Weather()); } else { callback.onSuccess(null); null null); } } public void expectGetWeatherForUser(String userName) { expectGetWeatherForUser( userName) expectGetWeatherForUserCalls.add(userName); } public void verify() { verify() assertEquals(expectGetWeatherForUserCalls, observeGetWeatherForUserCalls); expectGetWeatherForUserCalls.clear(); observeGetWeatherForUserCalls.clear(); }}
  54. 54. 2012-12-20

×