Ensuring quality in cloud and mobile applications
Upcoming SlideShare
Loading in...5
×
 

Ensuring quality in cloud and mobile applications

on

  • 585 views

by Gerard Meszaros on QCon 2012 Hangzhou.

by Gerard Meszaros on QCon 2012 Hangzhou.

Statistics

Views

Total Views
585
Views on SlideShare
585
Embed Views
0

Actions

Likes
1
Downloads
9
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Ensuring quality in cloud and mobile applications Ensuring quality in cloud and mobile applications Presentation Transcript

  • Ensuring Quality in Cloud and Mobile Applications Gerard Meszaros infoq2012hz@gerardm.comInfoQ Hangzhou 2012 1 Copyright 2012 Gerard Meszaros
  • My Background • Software developer80’s • Development manager Embedded----- • Project Manager Telecom • Software architect90’s • OOA/OOD Mentor • Requirements (Use Case) Mentor----- I.T. • XP/TDD Mentor • Agile PM Mentor00’s • Test Automation Consultant & Trainer Gerard Meszaros • Lean/Agile Coach/Consultant infoq2012hz@gerardm.com Product & I.T. InfoQ Hangzhou 2012 2 Copyright 2012 Gerard Meszaros
  • Agenda •  What’s Special About Cloud and Mobile Apps? •  What is Quality? •  A Quality Strategy •  Testing Cloud-Based Apps •  Testing Mobile Apps •  No DOH! •  Ensuring Design for Testability •  Tools, Frameworks and PatternsInfoQ Hangzhou 2012 3 Copyright 2012 Gerard Meszaros
  • Current Challenges The Good Old Days Current Reality •  Captive Audience •  Users Have Real Choice –  Have no real choice –  Many alternatives •  Stable requirements •  Rapid change –  Can have a single release –  Highly iterative; multi releases •  Long lead times •  Rapid change –  Can have long manual test –  No time for extensive manual cycle testing •  Relatively simple •  Highly Complex environment Environment –  Just a few inputs (user, DB) –  Many sources of inputs –  Mostly synchronous –  Sync and AsyncInfoQ Hangzhou 2012 4 Copyright 2012 Gerard Meszaros
  • Typical Desktop Application•  UI Logic and Business Logic run on desktop Other System•  Server hosts Other System the database. Desktop DB Server User Business Database Interface Logic OS Services Other SystemInfoQ Hangzhou 2012 5 Copyright 2012 Gerard Meszaros
  • Typical Cloud-Based Application•  UI Logic runs in (A.K.A. Software-as-a-Service or SaaS) browser•  Business Logic is split Other System between Server and Desktop Browser Browser•  Both use browser Browser Server services –  Widgets, Cookies, SSL,DOM rendering UI Business Business Logic Logic Logic Database•  Business logic may also interact with other systems. Browser Services –  E.g. IP-based location Container Services•  Browser-based BL may receive data OS Services from server: Other System –  synchronous (request-reply) –  asynchronous Dynamic scripting languages (event-based) •  won’t report compile errors until run.InfoQ Hangzhou 2012 6 Copyright 2012 Gerard Meszaros
  • Typical Web-Based Mobile Application•  UI Logic runs in (A.K.A. Software-as-a-Service or SaaS) browser•  Business Logic is split Other System between Server and Device Browser Browser•  Both use browser Browser Server services –  Widgets, Cookies, SSL,DOM rendering UI Business Business Logic Logic Logic Database•  Business logic may also interact with other systems. Browser Services –  E.g. IP-based location Container Services•  Browser-based BL may receive data Platform Services from server: Other System –  synchronous Sensors OthrApps (request-reply) GPS Acc Contacts –  asynchronous Msg’ing (event-based) Gyro Batt GyroInfoQ Hangzhou 2012 7 Copyright 2012 Gerard Meszaros
  • Typical Mobile Application•  UI Logic runs in device•  Business Logic is split Other System between Device and Device Servers•  Both use Platform Server services –  Config, Prefs, Widgets,SSL, UI Business Business Logic Logic Logic Database•  Business logic may also interact with other systems. Platform Services –  E.g. IP-based location Container Services Config Time•  Browser-based BL may receive data Prefs Prefs from server: Other System –  synchronous Sensors OthrApps (request-reply) GPS Acc Contacts –  asynchronous Msg’ing (event-based) Gyro Batt GyroInfoQ Hangzhou 2012 8 Copyright 2012 Gerard Meszaros
  • Outputs / Controlables •  Screen (LCD) •  Other Applications •  LED indicators •  Add event to •  Vibration motor Calendar •  Camera •  Add/edit/read contact •  Wifi, Bluetooth, 3G, NFC Radios (on/off)InfoQ Hangzhou 2012 9 Copyright 2012 Gerard Meszaros
  • Other static inputs Physical Configuration User Preferences •  Screen size •  Language / locale •  Screen density •  Keyboard prefs •  Roles of Physical •  Application prefs buttons •  Camera(s) capabilitiesInfoQ Hangzhou 2012 10 Copyright 2012 Gerard Meszaros
  • Sources of Events Sensors Services •  Proximity •  Send message (e-mail, SMS, MMS, IAM) •  Accelerometer •  Voice interpretation •  GPS •  Output Audio •  Signal Strength (Wifi, •  IP Location 3g, BT, NFC) •  Wifi Location •  Compass •  GPS Location •  Get/Show Advertising •  In-App purchasingInfoQ Hangzhou 2012 11 Copyright 2012 Gerard Meszaros
  • Events & Reponses •  Low Battery •  Valid Responses •  Lost Data •  Data returned sync Connection •  Data returned async •  Change in data •  Error responses capabilities •  Low resources •  Geofence crossed •  Can’t communicateInfoQ Hangzhou 2012 12 Copyright 2012 Gerard Meszaros
  • Example – Screen Layout •  Large number of available screen sizes •  Need to verify app uses screen space well •  Four different orientations •  For each screen size and orientation: – need to verify how various screen elements are sized and placed •  May be possible to do manually for IOS –  iPhone 1, iPhone 4, iPhone 5, iPad1/2, iPad3, iPod Touch •  Not possible to do well manually for Android – Automation is essential; best done at component levelInfoQ Hangzhou 2012 13 Copyright 2012 Gerard Meszaros
  • Example – Screen Layout •  What about changes to orientation of the device? •  That’s 4P2 or 12 transitions we need to test North ->South South -> North East -> North West -> North North -> East South -> East East -> West West -> East North -> West South -> West East -> South West -> South •  And we have to run this test at various states of the app •  So, 12 tests in 10’s or 100’s of states •  On 10’s to 100’s of device configurationsInfoQ Hangzhou 2012 14 Copyright 2012 Gerard Meszaros
  • Agenda •  What’s Special About Cloud and Mobile Apps? •  What is Quality? •  A Quality Strategy •  Testing Cloud-Based Apps •  Testing Mobile Apps •  No DOH! •  Ensuring Design for Testability •  Tools, Frameworks and PatternsInfoQ Hangzhou 2012 15 Copyright 2012 Gerard Meszaros
  • Google Quality Checklists Core App Quality Tablet App Quality Quality Criteria 1. Test for Core App Quality •  Design and Interaction 2. Optimize your layouts •  Functionality 3. Use the extra screen area •  Performance and Stability 4. Use assets designed for •  Google Play tablets Testing 5. Adjust fonts and touch targets •  Setting Up a Test 6. Adjust HomeScreen widgets Environment 7. Offer the apps full feature set •  Test Procedures 8. Don’t require hardware You Should Also Read features •  Tablet App Quality Checklist 9. Declare tablet screen support •  Improving App Quality 10. Follow best practices for publishing in Google Play http://developer.android.com/distribute/googleplay/quality/core.html http://developer.android.com/distribute/googleplay/quality/tablet.htmlInfoQ Hangzhou 2012 16 Copyright 2012 Gerard Meszaros
  • What Exactly is Quality? Building the Right Thing Building the Thing Right •  Fitness-for-purpose •  Good architecture •  Understand the users •  Well implemented in •  And their usage env’ts code •  Well tested at all levels Practices: Practices: •  UxD •  Automated Unit Testing •  Usability testing •  Test-Driven Development •  Lean Startup stuff •  ATDD •  Exploratory TestingInfoQ Hangzhou 2012 17 Copyright 2012 Gerard Meszaros
  • Traditional Test Cycle e S oftwarRequirements Requirements Verification & Development Acceptance InfoQ Hangzhou 2012 18 Copyright 2012 Gerard Meszaros
  • Agenda •  What’s Special About Cloud and Mobile Apps? •  What is Quality? •  A Quality Strategy •  Testing Cloud-Based Apps •  Testing Mobile Apps •  No DOH! •  Ensuring Design for Testability •  Tools, Frameworks and PatternsInfoQ Hangzhou 2012 19 Copyright 2012 Gerard Meszaros
  • How to Achieve Quality?InfoQ Hangzhou 2012 20 Copyright 2012 Gerard Meszaros
  • What Should We Test? •  Product X consists of components A, B, and C •  Components A and B are well tested but have never been used together. •  Component C is not well tested •  We need to Component-test C, •  Interaction test A vs. B •  Interaction test A vs CInfoQ Hangzhou 2012 21 Copyright 2012 Gerard Meszaros
  • A Quality Strategy •  Use automated tests at unit and component level – to maximize confidence in quality of components. •  Use system-level tests focusing on the logic not already tested – mostly integration tests – automated if possible, manual otherwise •  Use Exploratory testing to find out what else could possibly go wrong. – Let testers follow their hunches to find problem areasInfoQ Hangzhou 2012 22 Copyright 2012 Gerard Meszaros
  • A word about Terminology •  Testing vs. Checking – Checking is verifying that a behaviour occurs as expected – Testing is seeing what happens when I do something – Automated Testing is therefor an oxymoron; it should be automated checking. •  Direct vs Indirect Inputs (and outputs) Depended-on Client Objects (Users) System Under Test Components Direct Indirect DOC Client Input Output Input SUT methodA( ) DOC Client Output methodB( ) Client Direct Indirect DOC Output InputInfoQ Hangzhou 2012 23 Copyright 2012 Gerard Meszaros
  • Test Automation Pyramid •  Tools to support effective exploratory testing Exploratory Tests •  A small number of tests for the entire application System & workflow Tests – Ensure application(s) support Component users’ requirements Tests •  Medium number of functional tests for major components Unit Tests – Verify integration of units •  Large numbers of very small unit tests – Ensures integrity of codeInfoQ Hangzhou 2012 Pyramid originally proposed by Mike Cohn 24 Copyright 2012 Gerard Meszaros
  • Agenda •  What’s Special About Cloud and Mobile Apps? •  What is Quality? •  A Quality Strategy •  Testing Cloud-Based Apps •  Testing Mobile Apps •  No DOH! •  Ensuring Design for Testability •  Tools, Frameworks and PatternsInfoQ Hangzhou 2012 25 Copyright 2012 Gerard Meszaros
  • Testing Cloud-Based Application•  Natural interface (A.K.A. Software-as-a-Service or SaaS) between Browser & Server facilitates Other System testing –  Server testing –  Browser testing with Browser Server Server stubbed out UI Business Business•  Component tests for Logic Logic Logic Database Browser Business Logic bypassing UI –  Requires API on BL Browser Services Container Services•  Also need unit tests for Business Logic OS Services Other System codeInfoQ Hangzhou 2012 26 Copyright 2012 Gerard Meszaros
  • Testing Cloud-Based Application•  Natural interface (A.K.A. Software-as-a-Service or SaaS) between Browser & Server facilitates Other System testing –  Server testing –  Browser testing with Browser Server Server stubbed out Server UI Business Business•  Component tests for Logic LogicTest Logic Database Browser Business Logic bypassing UI –  Requires API on BL Browser Services Container Services•  Also need unit tests for Business Logic OS Services Other System codeInfoQ Hangzhou 2012 27 Copyright 2012 Gerard Meszaros
  • Testing Cloud-Based Application•  Natural interface (A.K.A. Software-as-a-Service or SaaS) between Browser & Server facilitates Other System testing –  Server testing –  Browser testing with Browser Server Whole Server Server stubbed out Client UI Business Test Business•  Component tests for Logic Test Logic StubLogic Database Browser Business Logic bypassing UI –  Requires API on BL Browser Services Container Services•  Also need unit tests for Business Logic OS Services Other System codeInfoQ Hangzhou 2012 28 Copyright 2012 Gerard Meszaros
  • Testing Cloud-Based Application•  Natural interface (A.K.A. Software-as-a-Service or SaaS) between Browser & Server facilitates Other System testing –  Server testing –  Browser testing with Browser Server Server stubbed out Unit Unit Test Unit Unit Test UI Test Stub Test Business Stub Business•  Component tests for Logic Unit Logic Test Unit Database Logic Test Browser Business Unit Unit Test Stub Test Stub Logic bypassing UI –  Requires API on BL Browser Services Container Services•  Also need unit tests for Business Logic OS Services Other System codeInfoQ Hangzhou 2012 29 Copyright 2012 Gerard Meszaros
  • Testing Cloud-Based Application•  Natural interface (A.K.A. Software-as-a-Service or SaaS) between Browser & Server facilitates Other System testing –  Server testing –  Browser testing with Browser Server Server Server stubbed out Business Test UI Business Stub Business•  Component tests for Logic Logic Database Logic Logic Browser Business Test Logic bypassing UI Platform Services Browser Services –  Requires API on BL Container Services Test Stub•  Also need unit tests for Business Logic OS Services Other System codeInfoQ Hangzhou 2012 30 Copyright 2012 Gerard Meszaros
  • Testing Cloud-Based Application•  Natural interface (A.K.A. Software-as-a-Service or SaaS) between Browser & Server facilitates Other System testing –  Server testing –  Browser testing with Browser Server DB Server stubbed out Server Test UI Business Business Stub•  Component tests for Logic Database Logic Logic Logic Browser Business Test Logic bypassing UI –  Requires API on BL Browser Services Platform Services Container Services•  Also need unit tests Test Stub for Business Logic OS Services Other System codeInfoQ Hangzhou 2012 31 Copyright 2012 Gerard Meszaros
  • Testing Cloud-Based Application•  Natural interface (A.K.A. Software-as-a-Service or SaaS) between Browser & Server facilitates Other System testing –  Server testing –  Browser testing with Browser Server Server stubbed out Whole Business System for UI•  Component tests Business Database Logic Logic Logic Test Browser Business Logic bypassing UI –  Requires API on BL Browser Services Container Services•  Also need unit tests for Business Logic OS Services Other System codeInfoQ Hangzhou 2012 32 Copyright 2012 Gerard Meszaros
  • Special Considerations •  Different Browsers and Browser-Versions – Eg. Rendering behaviour, widget behavior, etc. •  Different Host-OSes – For OS-based services such as file choosers, etc. •  Different languages & locales •  Browser resizingInfoQ Hangzhou 2012 33 Copyright 2012 Gerard Meszaros
  • Testing Browser-Based MobileSimilar to testing Application Cloud-based apps but: Other System•  Different browsers & Screen sizes Device Server•  Need to test with events from UI Business Business relevant sensors Logic Logic Logic Database –  E.g. Orientations and changes Platform Services Container Services Config Time Prefs Prefs Other System Sensors OthrApps GPS Acc Contacts Msg’ing Gyro Batt GyroInfoQ Hangzhou 2012 34 Copyright 2012 Gerard Meszaros
  • Testing Browser-Based MobileSimilar to testing Application Cloud-based apps but: Other System•  Different browsers & Screen sizes Device Server Server•  Need to test with Business Test events from UI Business Stub Business Logic Database relevant sensors Logic Logic Logic Test –  E.g. Orientations and changes Platform Services Platform Services Container Services Test Config Stub Time Prefs Prefs Other System Sensors OthrApps GPS Acc Contacts Msg’ing Gyro Batt GyroInfoQ Hangzhou 2012 35 Copyright 2012 Gerard Meszaros
  • Testing Native Mobile Application•  To test device BL, need to: –  Provide API Other System bypassing UI –  Trap outgoing reqests to sensors and services Device Server –  Inject responses from same UI Business Business Logic Logic Logic Database•  Need ability to: –  Stub out platform services, sensors and other apps Platform Services Container Services Config Time•  Need unit tests for Prefs Prefs all code Other System Sensors OthrApps GPS Acc Contacts Msg’ing Gyro Batt GyroInfoQ Hangzhou 2012 36 Copyright 2012 Gerard Meszaros
  • Testing Native Mobile Application•  To test device BL, need to: –  Provide API Other System bypassing UI –  Trap outgoing reqests to sensors Server and services Device Server Business Test –  Inject responses from same UI Business Stub Business Logic Database Logic Logic Logic•  Need ability to: Test –  Stub out platform services, sensors Platform Services Platform Services and other apps Container Services Test Config Stub Time•  Need unit tests for Prefs Prefs all code Other System Sensors OthrApps GPS Acc Contacts Msg’ing Gyro Batt GyroInfoQ Hangzhou 2012 37 Copyright 2012 Gerard Meszaros
  • Testing Native Mobile Application•  To test device BL, need to: –  Provide API Other System bypassing UI –  Trap outgoing reqests to sensors Dependency and services Device Characterization Server –  Inject responses Tests from same UI Business Business Logic Logic Logic Database•  Need ability to: –  Stub out platform services, sensors and other apps Platform Services Container Services Config Time•  Need unit tests for Prefs Prefs all code Other System•  Characterization Sensors OthrApps Tests to detect GPS Acc Contacts changes in Msg’ing dependencies Gyro Batt GyroInfoQ Hangzhou 2012 38 Copyright 2012 Gerard Meszaros
  • Agenda •  What’s Special About Cloud and Mobile Apps? •  What is Quality? •  A Quality Strategy •  Testing Cloud-Based Apps •  Testing Mobile Apps •  No DOH! •  Ensuring Design for Testability •  Tools, Frameworks and PatternsInfoQ Hangzhou 2012 39 Copyright 2012 Gerard Meszaros
  • DOH! •  DOH! = Debugging on Hardware •  Should be avoided because •  Slow, tedious, non-deterministic •  Complete code coverage is impossible because we cannot cause on-demand: – Hardware errors – “Impossible” return codes or exceptions – All but most common events from sensors You have to ask yourself the following question: Do I feel lucky?InfoQ Hangzhou 2012 40 Copyright 2012 Gerard Meszaros
  • James Grenning on DOH! •  Reasons why DOH is not feasible: – There is no hardware, it’s being concurrently developed. – The hardware is expensive, so team members have to share it. – DOH is slow, with tools that may be less capable than the development system native tools. – The hardware has bugs, as if DOH! isn’t slow enough; buggy hardware makes it extra difficult. – Building code for the target is slow. – Downloading to the target is slow. – The embedded processor is probably slow too.InfoQ Hangzhou 2012 41 Copyright 2012 Gerard Meszaros
  • How No DOH? •  How do we avoid Debugging on Hardware? Test in IDE Test in Test on Test in (Automated) Emulator Hardware Acceptance (Automated) (Automated) (manual) When: Day 1 As soon as Available Weekly Which tests: Unit Tests Component Tests System TestsInfoQ Hangzhou 2012 42 Copyright 2012 Gerard Meszaros
  • No DOH How •  How do we avoid Debugging on Hardware? Test in IDE Test in Test on Test in (Automated) Emulator Hardware Acceptance (Automated) (Automated) (manual) Frequency: Every few minutes Daily Weekly+ Each code change Per component test Per user storyInfoQ Hangzhou 2012 43 Copyright 2012 Gerard Meszaros
  • Agenda •  What’s Special About Cloud and Mobile Apps? •  What is Quality? •  A Quality Strategy •  Testing Cloud-Based Apps •  Testing Mobile Apps •  No DOH! •  Ensuring Design for Testability •  Tools, Frameworks and PatternsInfoQ Hangzhou 2012 44 Copyright 2012 Gerard Meszaros
  • How Do We Ensure Testability? “Design for Testability”, then code •  Write tests last •  And hope it’s easy Or •  Code for Testability •  Via Test-Driven Development (TDD)InfoQ Hangzhou 2012 45 Copyright 2012 Gerard Meszaros
  • TDD Rhythm Test Code Product Code •  First Test •  Hard-coded method •  Second Test •  Introduce variable •  Third Test •  Introduce conditional •  Fourth Test •  Surround with a loop Just Like Double-Entry Booking: •  An entry on the test side for each entry on the Prod sideInfoQ Hangzhou 2012 46 Copyright 2012 Gerard Meszaros
  • Test - After Architecture•  Must test through User Interface System Under Test Configuration Configure User Notification NotificationWorkflow Interface Threshold Rules Test Should we Transaction Process Notify? Interface Transaction Do Notification. Notification Log InfoQ Hangzhou 2012 47 Copyright 2012 Gerard Meszaros
  • Test-Driven Architecture •  Need to provide API’s to invoke functionality directly System Under Test Configuration Configure User Notification NotificationWorkflow Interface Threshold Rules Test Should we Test Stub Transaction Process Notify? Interface Transaction Do Notification. Notification •  And ways to stub out Log dependencies InfoQ Hangzhou 2012 48 Copyright 2012 Gerard Meszaros
  • Test-Driven Architecture •  “What kind of tests do I want to be able to automate?” •  “Which component would be responsible for that part of the behavior?” ConfigurationConfiguration Configure User Notification Notification TX Test Interface Threshold Rules Should we Transaction Process Notify? Interface Transaction Do Notification. Notification Log InfoQ Hangzhou 2012 49 Copyright 2012 Gerard Meszaros
  • Test-Driven Architecture •  “What other components would that component depend on?” •  “How can I break that dependency when component testing this component?” Configuration Configure Notification Notification User Notification Rules Fake Rules Interface Threshold Notification Should we Rule Test Transaction Process Notify? Interface Transaction Do Notification Method Test Notification. Notification Log •  With the right architecture, automating these tests is trivialInfoQ Hangzhou 2012 50 Copyright 2012 Gerard Meszaros
  • Behavior Specification at Right Level •  Specify broad scope at minimum detail – E.g. Use least detail when specifying workflow •  Specify most detailed req’ts at narrowest scope – E.g. Don’t use workflow when specifying business rules Too much detail Workflow Low Unmaintainable System Tests Transactions Make examples /Detail tests easy to Component Business understand and Tests Rules easy to write High Too vague Unit Tests Unit Tests Broad Narrow InfoQ Hangzhou 2012 Scope 51 Copyright 2012 Gerard Meszaros
  • Agenda •  What’s Special About Cloud and Mobile Apps? •  What is Quality? •  A Quality Strategy •  Testing Cloud-Based Apps •  Testing Mobile Apps •  No DOH! •  Ensuring Design for Testability •  Tools, Frameworks and PatternsInfoQ Hangzhou 2012 52 Copyright 2012 Gerard Meszaros
  • Tools, Frameworks & Patterns •  Tools come & go – Different ecosystems (Android, IOS, BBOS, WP, etc.) – New, better tools appear regularly •  Strategies persist – Need to define strategy first •  Then choose tools to support the strategy •  And Patterns for testable designs Avoids “Impedance Mismatch” between tools and strategyInfoQ Hangzhou 2012 53 Copyright 2012 Gerard Meszaros
  • Framework Issues •  Many frameworks not designed for testability •  May need to introduce level of indirection to API (via Proxy or Adapter) •  May need to extract Plain Ordinary Testable Object (POTO) leaving a Humble Object that subclasses framework base classInfoQ Hangzhou 2012 54 Copyright 2012 Gerard Meszaros
  • Extract POTO - Before ActionBase ActionContext getRules() Framework Plug-in +rules MyActionTest tests MyAction Rules testDoAction() doAction()InfoQ Hangzhou 2012 55 Copyright 2012 Gerard Meszaros
  • Extract POTO - After Runs in device; ActionBase ActionContext Tests “wrapping” getRules() (Optional) Framework Plug-in +rules MyActionHOTest tests MyActionHO Rules testDoAction() doAction()Runs in IDE; Tests Humble “business logic” Rules Object Testable MyActionPotoTest tests MyActionPoto Object testDoAction() Fake doAction(rules) Rules Simple design change; Huge testability improvement!InfoQ Hangzhou 2012 56 Copyright 2012 Gerard Meszaros
  • Conclusions •  Mobile and Cloud-Based Applications come with special challenges – While business logic tends to be simpler, constraints and variations make design and testing more challenging •  Testing-in Quality is not an option – There are too many potentially interacting dimensions – Many are not controllable from outside the black box •  Need to avoid Debugging on Hardware – Test as much as possible “off device” – Only deploy code you know works correctlyInfoQ Hangzhou 2012 57 Copyright 2012 Gerard Meszaros
  • Conclusions •  Design-for-Testability is essential for success – Otherwise, tests will be too expensive to execute and/or automate •  Beware tools/frameworks that make DfT difficult – May need to decouple from framework to ensure testability – But extra effort is compensated for by better quality and productivityInfoQ Hangzhou 2012 58 Copyright 2012 Gerard Meszaros
  • What Can You Do? •  Product Owner: – Break work into manageble chunks – Provide examples/tests for each chunk to drive development – Insist that team automates the tests •  Tester/BA: – Provide tests to developers Before they start dev – Insist on seeing the results of the automate tests before you start testing – Focus manual testing on areas not covered by automated testsInfoQ Hangzhou 2012 59 Copyright 2012 Gerard Meszaros
  • Developer: What Can You Do? •  Ask for tests from PO/Testers before agreeing to build •  Discuss the kinds of tests that they will provide •  Include the work to automate the tests in any estimates you provide •  Implement the automation before you develop the code, not after! •  Use the automated tests to drive development •  Don’t recreate provided tests at unit test level; find ways of reusing business rule tests as unit tests •  Think/Do Design-for-Testability •  If you find yourself in an interactive debugger, ask yourself: “What unit tests would tell me the same information and prevent this from happening again?”InfoQ Hangzhou 2012 60 Copyright 2012 Gerard Meszaros
  • References •  James Grenning “Don’t Let Embedded Tool Chain Slow You Down” –  http://www.renaissancesoftware.net/blog/archives/41 •  Michael Bolton “Testing vs. Checking” –  http://www.developsense.com/blog/2009/08/testing-vs-checking/ •  Dion Johnson “Cyber-dueling Over ‘Check’ vs. ‘Test’ and Other Semantics” –  http://www.automatedtestinginstitute.com/home/index.php? option=com_content&task=view&id=1373 •  Gerard Meszaros “Humble Object” –  http://xunitpatterns.com/Humble%20Object.html •  Homer Simpson “Doh!” –  http://en.wikipedia.org/wiki/D%27oh!InfoQ Hangzhou 2012 61 Copyright 2012 Gerard Meszaros
  • Resources •  http://www.mobileapptesting.com/ •  For Success, Build Record/Playback into Your Application - StarEast 2008 – http://builtinrecordandplayback.xunitpatterns.com – http://mobile.testAutomationPatterns.comInfoQ Hangzhou 2012 62 Copyright 2012 Gerard Meszaros
  • Thank You! Gerard Meszaros infoq2012hz@gerardm.com http://www.xunitpatterns.com Slides: http://infoQ2012hz.testautomationpatterns.com Jolt Productivity Award winner - Technical BooksCall me when you: Coming Soon:•  Want to transition to Agile or Lean•  Want to do Agile or Lean better•  Want to teach developers how to test•  Need help with test automation strategy•  Want to improve your test automationInfoQ Hangzhou 2012 63 Copyright 2012 Gerard Meszaros