Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
TERRIBLE TESTING case studies in
are you testing the right things?
I have no idea
PROJECTS SOFTWARE TESTING and what they taught me about 5
toddhgardner Todd H Gardner !
1Big Todd’s Awesome CRM
1 Delivera CustomerRelationshipManagement systemto replacead-hoc processesfor internalsalesteams
I have no idea what I’m doing
Consulting!
They have no idea what they’re doing
70% Code Coverage REQUIRED
Assertion-Free Testing public void Test_MyObj_For_Coverage() { MyObj obj = new MyObj(); try { obj.method1(); } catch (Exce...
Cool Tests Bro
1Big Todd’s Awesome CRM The Aftermath
They all had their rules… 1. Thou shall test everything 2. Thou shalt not ship code 3. Thou shall have more sys 4. Thou sh...
goodness testiness coverage test code ratio
It’s unprofessional to ship untested code “
2Big Corp’s Video Academy
2 Createa online Video Training Platform for crypto-zoologists
unit code coverage 100% 1000
MOCKS
code analysis metrics
BUGS!
2Big Corp’s Video Academy The Aftermath
code coverage % 1000 what do we learn?
code coverage % 1000 what do we learn? not testing
code coverage % 1000 what do we learn? metrics-driven incentives
code coverage % 1000 what do we learn? nothing at all
Testing the Mocks public void Test_MyObj_With_Mocks() { Mock<MyObj> mockObj1 = new Mock<MyObj>(); Mock<OtherObj> mockObj2 ...
Unit Tests
Test-Induced http://david.heinemeierhansson.com/2014/test-induced-design-damage.html Design Damage
3Acme’s Collaboration ComputingPlatform
3 Developa platformforSpiesto Upload ConfidentialData andCollaborateon missions.
Market and Technology FRONTIERS
$2M / Mo Over 200 of the Cleverest Designers, Developers, and Testers Burning
Advanced TECHNOLOGY
System Tests Integration Tests Unit Tests Selenium UI and Services Inter-Module Contract Tests TDD, Conversation-based tes...
WE BUILT IT
NO ONE CAME
3Acme’s Collaboration ComputingPlatform The Aftermath
System Tests Integration Tests Unit Tests Testing Pyramid
System Tests Integration Tests Unit Tests Testing Pyramid Interface Risk Orchestration Risk Functional Risk
System Tests Integration Tests Unit Tests Testing Pyramid Interface Risk Orchestration Risk Functional Risk User Tests Mar...
Interface Risk Orchestration Risk Functional Risk Market Risk Testing Scales
Interface Risk Orchestration Risk Functional Risk Market Risk Testing Scales System Tests Integration Tests User Tests Uni...
4Widget eCommerce Monolith’s Sell-a-majig
4 Expandthe Widget Ecommerce platformwithmoreproducts
yet another ECOMMERCE project
lots of captive USERS
Interface Risk Orchestration Risk Functional Risk Market Risk Testing Scales
Interface Orchestration Functional Market Testing Scales System Tests Integration Tests User Tests Unit Tests
it shipped UGLY
customers kept BUYING
Prints Money the dirty old duct-taped machine that tm
4Widget eCommerce Monolith’s Sell-a-majig The Aftermath
Monolith ECommerce Acme Collaboration Information Technology Uninteresting Fragile Technology Poorly Tested Success! Engin...
Monolith ECommerce Acme Collaboration Success! Failure! High RiskLow Risk
SYSTEM Purpose Criticality Lifespan Frequency CONFIDENCE
5 My Startup
5findandfix JavaScriptbugs in modernweb applications …andbuilda betterweb
{Track:js} http://trackjs.com JavaScript Error Monitoring
User Activity AJAX Requests Console Logging Browser Context Inline Source
Interface Risk Orchestration Risk Functional Risk Market Risk Testing Scales
My Money! This time it’s
Launch Fast and measure
Simple Decoupled Obvious Abstract & &
Plumbing Services
Interface Risk Orchestration Risk Functional Risk Market Risk Testing Scales System Tests Integration Tests User Tests Uni...
Service Model Test Fixture Other Services Wrapper Data Unit Test Integration Test Testing Structure
“beta” Service Dragons be here! Testing Structure Canary Users Analytics Monitoring
FAST TO FIXis almost as good as NEVER BROKEN (and sometimes better)
PROJECTS SOFTWARE TESTING and what they taught me about 5
1 Big Todd’s Awesome CRM METRICS don’t test to hit
2 Big Corp’s Video Academy BREAKING test what is
3 Acme’s Collaboration ComputingPlatform SCARIEST test the thing first
4 Widget eCommerce Monolith’s Sell-a-majig CUSTOMER USE tests can’tsubstitutefor
5 JavaScriptError Tracking TrackJS’s MONITORING can substitutefor tests
{Track:js} http://trackjs.com JavaScript Error Monitoring
TERRIBLE TESTING case studies in toddhgardner! " todd@trackjs.com
Case Studies in Terrible Testing
Case Studies in Terrible Testing
Case Studies in Terrible Testing
Case Studies in Terrible Testing
Case Studies in Terrible Testing
Case Studies in Terrible Testing
Case Studies in Terrible Testing
Case Studies in Terrible Testing
Case Studies in Terrible Testing
Upcoming SlideShare
Loading in …5
×

Case Studies in Terrible Testing

339 views

Published on

Projects fail because they don’t test. Some fail because they test the wrong things. Others fail because they test too much. In this session, an enterprise consultant turned startup entrepreneur will share project case studies in testing atrocities and what can be learned from them. You’ll come away questioning your own testing. Check your dogma and let’s build better software.

Check out TrackJS JavaScript Error Monitoring
https://trackjs.com/javascript?utm_source=slides&utm_medium=slideshare&utm_term=testing

Published in: Technology
License: CC Attribution-NonCommercial-NoDerivs License
no profile picture user

  • Be the first to comment

Case Studies in Terrible Testing

  1. 1. TERRIBLE TESTING case studies in
  2. 2. are you testing the right things?
  3. 3. I have no idea
  4. 4. PROJECTS SOFTWARE TESTING and what they taught me about 5
  5. 5. toddhgardner Todd H Gardner !
  6. 6. 1Big Todd’s Awesome CRM
  7. 7. 1 Delivera CustomerRelationshipManagement systemto replacead-hoc processesfor internalsalesteams
  8. 8. I have no idea what I’m doing
  9. 9. Consulting!
  10. 10. They have no idea what they’re doing
  11. 11. 70% Code Coverage REQUIRED
  12. 12. Assertion-Free Testing public void Test_MyObj_For_Coverage() { MyObj obj = new MyObj(); try { obj.method1(); } catch (Exception e) {} try { obj.method2(); } catch (Exception e) {} try { obj.method3(); } catch (Exception e) {} // 100% Coverage, W00t! }
  13. 13. Cool Tests Bro
  14. 14. 1Big Todd’s Awesome CRM The Aftermath
  15. 15. They all had their rules… 1. Thou shall test everything 2. Thou shalt not ship code 3. Thou shall have more sys 4. Thou shall abstract test f 5. Thou shalt not duplicate
  16. 16. goodness testiness coverage test code ratio
  17. 17. It’s unprofessional to ship untested code “
  18. 18. 2Big Corp’s Video Academy
  19. 19. 2 Createa online Video Training Platform for crypto-zoologists
  20. 20. unit code coverage 100% 1000
  21. 21. MOCKS
  22. 22. code analysis metrics
  23. 23. BUGS!
  24. 24. 2Big Corp’s Video Academy The Aftermath
  25. 25. code coverage % 1000 what do we learn?
  26. 26. code coverage % 1000 what do we learn? not testing
  27. 27. code coverage % 1000 what do we learn? metrics-driven incentives
  28. 28. code coverage % 1000 what do we learn? nothing at all
  29. 29. Testing the Mocks public void Test_MyObj_With_Mocks() { Mock<MyObj> mockObj1 = new Mock<MyObj>(); Mock<OtherObj> mockObj2 = new Mock<OtherObj>(); MyObj obj = new MyObj(mockObj1, mockObj2); mockObj2.expect().toBeCalledWith(mockObj1).andReturn(mockObj1); obj.method(); mockObj1.assertExpectations(); mockObj2.assertExpectations(); // What are we testing again? }
  30. 30. Unit Tests
  31. 31. Test-Induced http://david.heinemeierhansson.com/2014/test-induced-design-damage.html Design Damage
  32. 32. 3Acme’s Collaboration ComputingPlatform
  33. 33. 3 Developa platformforSpiesto Upload ConfidentialData andCollaborateon missions.
  34. 34. Market and Technology FRONTIERS
  35. 35. $2M / Mo Over 200 of the Cleverest Designers, Developers, and Testers Burning
  36. 36. Advanced TECHNOLOGY
  37. 37. System Tests Integration Tests Unit Tests Selenium UI and Services Inter-Module Contract Tests TDD, Conversation-based tests Testing Pyramid
  38. 38. WE BUILT IT
  39. 39. NO ONE CAME
  40. 40. 3Acme’s Collaboration ComputingPlatform The Aftermath
  41. 41. System Tests Integration Tests Unit Tests Testing Pyramid
  42. 42. System Tests Integration Tests Unit Tests Testing Pyramid Interface Risk Orchestration Risk Functional Risk
  43. 43. System Tests Integration Tests Unit Tests Testing Pyramid Interface Risk Orchestration Risk Functional Risk User Tests Market Risk
  44. 44. Interface Risk Orchestration Risk Functional Risk Market Risk Testing Scales
  45. 45. Interface Risk Orchestration Risk Functional Risk Market Risk Testing Scales System Tests Integration Tests User Tests Unit Tests
  46. 46. 4Widget eCommerce Monolith’s Sell-a-majig
  47. 47. 4 Expandthe Widget Ecommerce platformwithmoreproducts
  48. 48. yet another ECOMMERCE project
  49. 49. lots of captive USERS
  50. 50. Interface Risk Orchestration Risk Functional Risk Market Risk Testing Scales
  51. 51. Interface Orchestration Functional Market Testing Scales System Tests Integration Tests User Tests Unit Tests
  52. 52. it shipped UGLY
  53. 53. customers kept BUYING
  54. 54. Prints Money the dirty old duct-taped machine that tm
  55. 55. 4Widget eCommerce Monolith’s Sell-a-majig The Aftermath
  56. 56. Monolith ECommerce Acme Collaboration Information Technology Uninteresting Fragile Technology Poorly Tested Success! Engineering Advancing Latest Technology Well Tested Failure!
  57. 57. Monolith ECommerce Acme Collaboration Success! Failure! High RiskLow Risk
  58. 58. SYSTEM Purpose Criticality Lifespan Frequency CONFIDENCE
  59. 59. 5 My Startup
  60. 60. 5findandfix JavaScriptbugs in modernweb applications …andbuilda betterweb
  61. 61. {Track:js} http://trackjs.com JavaScript Error Monitoring
  62. 62. User Activity AJAX Requests Console Logging Browser Context Inline Source
  63. 63. Interface Risk Orchestration Risk Functional Risk Market Risk Testing Scales
  64. 64. My Money! This time it’s
  65. 65. Launch Fast and measure
  66. 66. Simple Decoupled Obvious Abstract & &
  67. 67. Plumbing Services
  68. 68. Interface Risk Orchestration Risk Functional Risk Market Risk Testing Scales System Tests Integration Tests User Tests Unit Tests
  69. 69. Service Model Test Fixture Other Services Wrapper Data Unit Test Integration Test Testing Structure
  70. 70. “beta” Service Dragons be here! Testing Structure Canary Users Analytics Monitoring
  71. 71. FAST TO FIXis almost as good as NEVER BROKEN (and sometimes better)
  72. 72. PROJECTS SOFTWARE TESTING and what they taught me about 5
  73. 73. 1 Big Todd’s Awesome CRM METRICS don’t test to hit
  74. 74. 2 Big Corp’s Video Academy BREAKING test what is
  75. 75. 3 Acme’s Collaboration ComputingPlatform SCARIEST test the thing first
  76. 76. 4 Widget eCommerce Monolith’s Sell-a-majig CUSTOMER USE tests can’tsubstitutefor
  77. 77. 5 JavaScriptError Tracking TrackJS’s MONITORING can substitutefor tests
  78. 78. {Track:js} http://trackjs.com JavaScript Error Monitoring
  79. 79. TERRIBLE TESTING case studies in toddhgardner! " todd@trackjs.com

×