Your SlideShare is downloading. ×
0
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
谷歌 Scott-lessons learned in testability
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

谷歌 Scott-lessons learned in testability

191

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
191
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Lessons Learned in Testability Scott McMaster Google Kirkland, Washington USA
  • 2. About Me• Software Design Engineer @ Google. – Building high-traffic web frontends and services in Java. – AdWords, Google Code• Ph.D. in Computer Science, U. of Maryland.• Formerly of Amazon.com (2 years), Lockheed Martin (2 years), Microsoft (7 years), and some small startups.• Frequent adjunct professor @ Seattle University, teaching software design, architecture, and OO programming.• Author of technical blog at http://www.scottmcmaster365.com.
  • 3. Testing and Me• Doing automated testing since 1995.• Ph.D. work in test coverage and test suite maintenance.• Champion of numerous unit, system, and performance testing tools and techniques.• Co-founder of WebTestingExplorer open-source automated web testing framework (www.webtestingexplorer.org).
  • 4. Agenda• What is Testability?• Testability Sins – Statics and singletons – Mixing Business and Presentation Logic – Breaking the Law of Demeter• Testability Solutions – Removing singletons. – Asking for Dependencies – Dependency Injection – Mocks and Fakes – Refactoring to UI Patterns
  • 5. Testability: Formal Definition• Wikipedia: “the degree to which a software artifact (i.e. a software system, software module, requirements- or design document) supports testing in a given test context.” http://en.wikipedia.org/wiki/Software_testability
  • 6. Some Aspects of Testability• Controllable: We can put the software in a state to begin testing.• Observable: We can see things going right (or wrong).• Isolatable: We can test classes/modules/systems apart from others.• Automatable: We can write or generate automated tests. – Requires each of the previous three to some degree.
  • 7. • Testability: your testingPractical Testability is a function of More goals.• Definition automated tests. Our primary goal is to write or generate• Therefore, testability is the ease with which we can write: – Unit tests – System tests – End-to-end tests
  • 8. Testers and Testability• At Google, test engineers: – Help ensure that developers build testable software. – Provide guidance to developers on best practices for unit and end-to-end testing. – May participate in refactoring production code for testability.
  • 9. Example: Weather App
  • 10. Weather App Architecture Rich Browser UI (GWT) Frontend Server (GWT RPC servlet) Remote Web Service (XML-over- User Database HTTP)
  • 11. Original Weather App Design
  • 12. Testability Problems?• Can’t test without calling the cloud service. – Slow to run, unstable.• Can’t test any client-side components without loading a browser or browser emulator. – Slow to develop, slow to run, perhaps unstable.
  • 13. Mission #1: Unit Tests for• Problem:WeatherServiceImplto Uses static singleton reference GlobalWeatherService, can’t be tested in isolation.• Solution: – Eliminate the static singleton. – Pass a mock or stub to the WeatherServiceImpl constructor at test-time.
  • 14. WeatherServiceImpl: Before private static GlobalWeatherService service = new GlobalWeatherService(); GlobalWeatherService(); public List<String> getCitiesForCountry(String countryName) { getCitiesForCountry(String countryName) try { if (countryName == null || countryName.isEmpty()) { countryName.isEmpty()) return new ArrayList<String>(); ArrayList<String>(); } return service.getCitiesForCountry(countryName); service.getCitiesForCountry(countryName) } catch (Exception e) { throw new RuntimeException(e); RuntimeException(e); } }What if we try to test this in its current form?1. GlobalWeatherService gets loaded at classload-time. 1. This itself could be slow or unstable depending on the implementation.2. When we call getCititesForCountry(“China”), a remote web service call gets made.3. This remote web service call may: 1. Fail. 2. Be really slow. 3. Not return predictable results. � Any of these things can make our test “flaky”.
  • 15. Proposed Solution• First we need to get rid of the static singleton.• Then we need something that: – Behaves like GlobalWebService. – Is fast and predictable. – Can be inserted into WeatherServiceImpl at test- time.
  • 16. A Word About Static Methods and• Never use them! Singletons• They are basically global variables (and we’ve all been taught to avoid those).• They are hard to replace with alternative implementations, mocks, and stubs/fakes. – They make automated unit testing extremely difficult.
  • 17. Scott’s Rules About Static Methods and Singletons1. Avoid static methods.2. For classes that are logically “singleton”, make them non-singleton instances and manage them in a dependency injection container (more on this shortly).
  • 18. Singleton Removal public class WeatherServiceImpl extends RemoteServiceServlet implements WeatherService { private final GlobalWeatherService service; public WeatherServiceImpl(GlobalWeatherService service) { WeatherServiceImpl( this.service = service; } ...• Also, make GlobalWeatherService into an interface.• Now we can pass in a special implementation for unit testing.• But we have a big problem…
  • 19. We’ve Broken Our Service!• The servlet container does not understand how to create WeatherServiceImpl anymore. – Its constructor takes a funny parameter.• The solution?
  • 20. Dependency Injection• Can be a little complicated, but here is what you need to know here:• Accept your dependencies, don’t ask for them. – Then your dependencies can be replaced (generally, with simpler implementations) at test time. – In production, your dependencies get inserted by a dependency injection container. • In Java, this is usually Spring or Google Guice.
  • 21. • When properly set up, it will create your objects and pass them to the appropriate constructors at runtime, freeing you Dependency Injection with Google Guice up to do other things with the constructors at test-time.• Setting up Guice is outside the scope of this talk. – This will get you started: http://code.google.com/p/google- guice/wiki/Servlets
  • 22. Fixing WeatherServiceImpl (1)•Configure our servlet to use Guice and tell it about our objects: public class WeatherAppModule extends AbstractModule { @Override protected void configure() { bind(WeatherServiceImpl.class); class class); bind(GlobalWeatherService.class).to(GlobalWeatherServiceImpl.class); class ).to(GlobalWeatherServiceImpl.class class).to( GlobalWeatherServiceImpl.class); }•When someone asks for a “GlobalWeatherService”, Guice will }give it an instance of GlobalWeatherServiceImpl.
  • 23. Fixing WeatherServiceImpl (2)• At runtime, Guice will create our servlet and the object(s) it needs: @Singleton public class WeatherServiceImpl extends RemoteServiceServlet implements WeatherService { private final GlobalWeatherService service; @Inject public WeatherServiceImpl(GlobalWeatherService service) { WeatherServiceImpl( this.service = service; } ...The “@Inject” constructor parameters is how we ask Guice forinstances.
  • 24. After Testability Refactoring #1
  • 25. Finally! We Can Test!• But how?• We want to test WeatherServiceImpl in isolation. ⇒For GlobalWeatherService, we only care about how it interacts with WeatherServiceImpl. ⇒To create the proper interactions, a mock object is ideal
  • 26. • The mock object framework verifies these interactions occur as expected. – A useful consequence of this: If appropriate, you can verify that an application is not making more remote calls than expected. Mock Object Testing – Another useful consequence: Mocks make it easy to test exception handling.• Common mocking frameworks (for Java): – Mockito – EasyMock • I will use this.
  • 27. Using Mock Objects1. Create a mock object.2. Set up expectations: 1. How we expect the class-under-test to call it. 2. What we want it to return.3. “Replay” the mock.4. Invoke the class-under-test.5. “Verify” the mock interactions were as-expected.
  • 28. Testing with a Mock Object private GlobalWeatherService globalWeatherService; globalWeatherService; private WeatherServiceImpl weatherService; weatherService; @Before public void setUp() { setUp() globalWeatherService = EasyMock.createMock(GlobalWeatherService.class ); EasyMock.createMock( GlobalWeatherService.class); weatherService = new WeatherServiceImpl(globalWeatherService); WeatherServiceImpl( globalWeatherService); } @Test public void testGetCitiesForCountry_nonEmpty() throws Exception { testGetCitiesForCountry_nonEmpty() EasyMock.expect(globalWeatherService.getCitiesForCountry ("china")) EasyMock.expect( globalWeatherService.getCitiesForCountry("china")) .andReturn(ImmutableList.of("beijing", "shanghai")); .andReturn ImmutableList.of(" beijing", andReturn( ("beijing EasyMock.replay(globalWeatherService); EasyMock.replay( globalWeatherService); List<String> cities = weatherService.getCitiesForCountry ("china"); List<String> weatherService.getCitiesForCountry("china"); assertEquals(2, cities.size()); assertEquals(2, cities.size()); assertTrue(cities.contains("beijing")); assertTrue( cities.contains(" beijing")); ("beijing assertTrue(cities.contains("shanghai")); assertTrue( cities.contains("shanghai")); EasyMock.verify(globalWeatherService); EasyMock.verify( globalWeatherService); }Observe:• How we take advantage of the new WeatherServiceImpl constructor.• How we use the mock GlobalWeatherService.
  • 29. • Problem: Talks to external web service, does non-trivial XML Mission #2: Unit Tests for processing that we want to test.• Solution:GlobalWeatherService – Split the remote call from the XML processing. – Wrap external web service in an object with a known interface. – Pass an instance to the GlobalWeatherServiceImpl constructor. – Use dependency injection to create the real object at runtime, use a fake at test-time.
  • 30. After Testability Refactoring #2
  • 31. – Contains a simplified implementation of the real thing (perhaps using static data, an in-memory database, etc.). – Implementation usually generated by hand. – Often reusable across test cases and test suites if carefully designed.• Mocks and Fakes – Fakes vs. Mocks Either can often be used in a given situation. – But some situations lend themselves to one more than the other.
  • 32. Use a Fake, or Use a Mock?• Problem: Setting up a mock object for GlobalWeatherDataAccess that returns static XML is possible, but ugly (and perhaps not very reusable).• Idea: Create a fake implementation of GlobalWeatherDataAccess. – We can give the fake object the capability to return different XML in different test circumstances.
  • 33. Implementing the Fake Objectpublic class FakeGlobalWeatherDataAccess implements GlobalWeatherDataAccess { // Try http://www.htmlescape.net/javaescape_tool.html to generate these. // http://www.htmlescape.net www.htmlescape.net/ private static final String CHINA_CITIES = "<?xml version="1.0" encoding="utf-8"?>n<string xml private static final String BEIJING_WEATHER = "<?xml version="1.0" encoding="utf-8"?>n<string<Visibility> greater than 7 mile(s):0</ Visibility>n <Temperature> 39 F (4 C)</ lt;Visibility> mile(s):0</Visibility&gt Visibility>n <Temperature&gt lt;Temperature> C)&lt lt;/ private static final String NO_CITIES = "<?xml version="1.0" encoding="utf-8"?>n<string xmlns= xmlns= @Override @Override public String getCitiesForCountryXml(String countryName) throws Exception { getCitiesForCountryXml(String countryName) if ("china".equals(countryName.toLowerCase())) { ("china".equals countryName.toLowerCase())) china".equals( return CHINA_CITIES; } return NO_CITIES; } @Override @Override public String getWeatherForCityXml(String countryName, String cityName) getWeatherForCityXml(String countryName, cityName) throws Exception { return BEIJING_WEATHER; }}
  • 34. Testing with a Fake Objectprivate GlobalWeatherServiceImpl globalWeatherService; globalWeatherService;private FakeGlobalWeatherDataAccess dataAccess; dataAccess;@Beforepublic void setUp() { setUp() dataAccess = new FakeGlobalWeatherDataAccess(); FakeGlobalWeatherDataAccess(); globalWeatherService = new GlobalWeatherServiceImpl(dataAccess); GlobalWeatherServiceImpl(dataAccess);}@Testpublic void testGetCitiesForCountry_nonEmpty () throws Exception { testGetCitiesForCountry_nonEmpty() List<String> cities = globalWeatherService.getCitiesForCountry("china"); assertEquals(2, cities.size()); assertTrue(cities.contains("beijing")); assertTrue(cities.contains("shanghai"));}@Testpublic void testGetCitiesForCountry_empty () throws Exception { testGetCitiesForCountry_empty() List<String> cities = globalWeatherService.getCitiesForCountry("nowhere"); assertTrue(cities.isEmpty());}The fake keeps the tests short, simple, and to-the-point!
  • 35. • Problem: UI and business logic / service calls all mixed Mission #3: Unit Tests for together. WeatherHome – The view layer is difficult and slow to instantiate at unit test-time. – But we need to unit test the business logic.• Solution: – Refactor to patterns -- Model-View-Presenter (MVP). – Write unit tests for the Presenter using a mock or stub View.
  • 36. Mixing Business and Presentation @UiHandler("login") void onLogin(ClickEvent e) { onLogin( weatherService.getWeatherForUser(userName.getText(), new AsyncCallback<Weather>() { AsyncCallback<Weather>() @Override public void onFailure(Throwable caught) { onFailure( Window.alert("oops"); } @Override public void onSuccess(Weather weather) { onSuccess( weather) if (weather != null) { fillWeather(weather); false); unknownUser.setVisible(false); } else { true); unknownUser.setVisible(true); } } }); }How NOT to write a UI event handler for maximum testability:• Have tight coupling between the UI event, processing a remote service call, and updating the UI.
  • 37. Model-View-Presenter (MVP)• UI pattern that separates business and presentation logic.• Makes the View easier to modify.• Makes the business logic easier to test by isolating it in the Presenter.
  • 38. Model-View-Presenter Overview
  • 39. Model-View-Presenter Responsibilities• Presenter uses the View interface to manipulate the UI.• View delegates UI event handling back to the Presenter via an event bus or an interface.• Presenter handles all service calls and reading/updating of the Model.
  • 40. • Passive Viewthe View is A particular style of MVP where MVP completely passive, only defining and layout and exposing its widgets for manipulation by the controller. – In practice, you sometimes don’t quite get here, but this is the goal.• Especially if you use this style, you can skip testing the View altogether.
  • 41. After Testability Refactoring #3
  • 42. Presenter Unit Test Using EasyMock@Testpublic void testOnLogin_unknownUser() { testOnLogin_unknownUser() weatherService.expectGetWeatherForUser("unknown"); EasyMock.expect(weatherView.getUserName()).andReturn("unknown"); true); weatherView.setUnknownUserVisible(true); EasyMock.expectLastCall(); weatherView.setEventHandler(EasyMock.anyObject(WeatherViewEventHandler.class)); class class)); EasyMock.expectLastCall(); EasyMock.replay(weatherView); WeatherHomePresenter presenter = new WeatherHomePresenter(weatherService, weatherView); WeatherHomePresenter( weatherService, weatherView); presenter.onLogin(); EasyMock.verify(weatherView); weatherService.verify();} This test uses a manually created mock to make handling the async callback easier.
  • 43. Question• Why does the View make the Presenter do this: ; weatherView.setUnknownUserVisible(true);• Instead of this: weatherView.getUnknownUser().setVisible(true)
  • 44. AnswerweatherView.getUnknownUser().setVisible(true)• Is hard to test because it is hard to mock:–To mock this, we would have to mock not only theWeatherView, but also the UnknownUser Label insideof it.• The above code is “talking to strangers”.
  • 45. • Also known as the “Don’t talk to strangers” rule.• It says: Law of friends. Demeter –Only have knowledge of closely collaborating objects. –Only make calls on immediate• Look out for long chained “get”-style calls (and don’t do them: a.getB().getC().getD()• Your system will be more testable (and maintainable, because you have to rework calling objects less often).
  • 46. • To write good unit tests, we need to be able to insert mocks and fakes into the code. What’s the Point?• Some things that help us do that: – Eliminating static methods and singletons. – Asking for dependencies instead of creating them. – Using design patterns that promote loose coupling, especially between business and presentation logic. – Obeying the Law of Demeter.• Code that does not do these things will often have poor test coverage.
  • 47. What Can I Do?• Developers: – Follow these practices!• Testers: – Educate your developers. – Jump into the code and drive testability improvements. • A good way to motivate this is to track test coverage metrics.
  • 48. Questions? Scott McMaster Google Kirkland, Washington USAscott.d.mcmaster (at) gmail.com
  • 49. Bonus Slides
  • 50. Model-View-Controller (MVC)
  • 51. Model-View-Controller (MVC)• View directly accesses the Model and fires events to the Controller.• Controller performs operations on the Model.• Controller doesn’t really know about the View other than selecting the View to render.
  • 52. Which to Use?• Many web MVC frameworks exist (Struts, Rails, ASP.NET MVC).• But these days, we work more with MVP.
  • 53. Manually Created Mockpublic class MockWeatherServiceAsync implements WeatherServiceAsync { private List<String> expectGetWeatherForUserCalls = Lists.newArrayList(); newArrayList(); private List<String> observeGetWeatherForUserCalls = Lists.newArrayList(); newArrayList(); // More @Overrides not shown on the slide. @Override public void getWeatherForUser(String userName, AsyncCallback<Weather> callback) { getWeatherForUser(String userName, AsyncCallback<Weather> observeGetWeatherForUserCalls.add(userName); if ("scott".equals(userName)) { ("scott ".equals(userName scott".equals( userName)) new callback.onSuccess(new Weather()); } else { callback.onSuccess(null); null null); } } public void expectGetWeatherForUser(String userName) { expectGetWeatherForUser( userName) expectGetWeatherForUserCalls.add(userName); } public void verify() { verify() assertEquals(expectGetWeatherForUserCalls, observeGetWeatherForUserCalls); expectGetWeatherForUserCalls.clear(); observeGetWeatherForUserCalls.clear(); }}
  • 54. 2012-12-20

×