Testing of distributed                      applications                                 Rails Way Con 2011               ...
le x                                m  p                              co                 Testing of distributed           ...
Agenda                        • Background                        • What & Why & How of Testing                        • D...
why?Freitag, 3. Juni 2011
Freitag, 3. Juni 2011
we move moneyFreitag, 3. Juni 2011
What is behind?                                    1 Rails app                                   5 Sinatra apps           ...
ArchitectureFreitag, 3. Juni 2011
Merchant Web                        Customer Web                     Customer Voice                                       ...
Merchant Web                        Customer Web                     Customer Voice                                       ...
Merchant Web                        Customer Web                        Customer Voice                                    ...
Trichet       Volcker   DuisenbergGreenspan               Keynes      SmithFreitag, 3. Juni 2011
Freitag, 3. Juni 2011
TestingFreitag, 3. Juni 2011
The old days                        • Simple web apps                        • Click through testingFreitag, 3. Juni 2011
Testi                                          g   n     Royce, W. (1970). Managing the development of large       softwar...
Freitag, 3. Juni 2011
Freitag, 3. Juni 2011
Freitag, 3. Juni 2011
In Rails                                 —                        everything is betterFreitag, 3. Juni 2011
Automated Tests                             Unit                          Integration                        Frontend test...
Complexity                  • Modern applications consist of                    many pieces                  • App Servers...
What to test                  Robert „UncleBob“ Martin:                  •Unit Tests                  •Component Tests    ...
Unit Tests                        • Close to 100% code coverage                        • Test Driven Development          ...
Component Tests                        • Test ~50% of the System                        • Cucumber / Selenium / Fitnesse  ...
Integration Tests                        • Cover ~20% of system                        • Test assembly of components      ...
System Tests                        • Cover 10% of the system                        • Test the whole system              ...
Manual Tests                  • Exploratory Manual Tests                  • Manual but not scripted                  • Hum...
Where do we                          test?Freitag, 3. Juni 2011
Testing the                          edgesFreitag, 3. Juni 2011
Merchant Web                        Customer Web                     Customer Voice                                       ...
Merchant Web                        Customer Web                     Customer Voice                                       ...
How do we                          test?Freitag, 3. Juni 2011
http://blog.objectmentor.com/articles/2009/09/29/               ruining-your-test-automation-strategyFreitag, 3. Juni 2011
Tools of the trade                             Selenium                             Cucumber                              ...
only cover a minimal                                part                           the web frontend,                      ...
Decoupling                        Systems diverge                         APIs divergeFreitag, 3. Juni 2011
Web Services                          Many services                        outside our controlFreitag, 3. Juni 2011
Freitag, 3. Juni 2011
Synchronizing                        • Tests require complex state                          across several systems        ...
QuestionsFreitag, 3. Juni 2011
Questions to ask                        • Development                        • Testing                        • Production...
Development                        Is the system I‘m                         building correct?Freitag, 3. Juni 2011
Test                        Do all the pieces of                         the system work                            togeth...
Production                        Are all parts of my                         system working?Freitag, 3. Juni 2011
Development                        • Test Driven Development only                          takes you so far               ...
Development        • Setting up a complete system for          development can be taxing        • pow to the rescue       ...
Testing                        • Integration tests need to                          stress complete application           ...
Outside your                            appFreitag, 3. Juni 2011
Merchant Web                        Customer Web                     Customer Voice                                       ...
Merchant Web                        Customer Web                     Customer Voice                                       ...
Write Mock Services                        • If youitdon‘t control it,                          mock                      ...
Mock Services                        • Allowsspecification against a                          „real“                      ...
Making it                         workFreitag, 3. Juni 2011
Version Control                          Repository setup                         One Big repository?                     ...
Development Servers                  Webrick doesn‘t scale                  but:                  it‘s „easy“ for Rack bas...
Configuration              Configuring half a dozen applications                     Again: Synchronisation              I...
Production                        Is my system working correctly?                        Probes                        •  ...
Freitag, 3. Juni 2011
Freitag, 3. Juni 2011
Car industry                  If car manufacturers would build                  engines the way enterprise IT             ...
OBD         •        1996: The OBD-II specification is made mandatory for                  all cars sold in the United Sta...
Solved ProblemFreitag, 3. Juni 2011
Freitag, 3. Juni 2011
http://the-adam.com/adam/rantrave/ibm_360.jpgFreitag, 3. Juni 2011
Diagnosis Channel      • Known from Car / Planes / Mainframes      • Red lights on your dashboard      • Complete diagnosi...
Rolls Royce                        • InFlight Diagnosis of engines                          remotelyFreitag, 3. Juni 2011
Dashboard                        • Look into state of application                        • Sanity CheckerFreitag, 3. Juni ...
Dashboarding                        • Extract relevant data from                          your system                     ...
SanityChecker                        • API stateallows you to extract                          the                        ...
Chaos Monkey                        • Random failures                        • Better reliabilityFreitag, 3. Juni 2011
Build testable apps        • Expose your internals (through APIs)        • Use SOLID design              • Single Responsi...
Testable apps                        • Obey the abstraction layers                         • keep of abstractionat it‘s ow...
Build probe-able apps                  • Measure APIs                  $	  rails	  console	  productionFreitag, 3. Juni 2011
Build controllable apps                        • Call functionality of the app                         • Create new user  ...
Test user / Magic                • In „Enterprise“ systems:                 • Test Company                 • Test User    ...
Use production data                  in development / testFreitag, 3. Juni 2011
Wrapping upFreitag, 3. Juni 2011
What & Why & How of Testing         Discover the state of your apps               Make testable appsFreitag, 3. Juni 2011
Thank you!Freitag, 3. Juni 2011
Credits                        • http://www.flickr.com/photos/                          myoldpostcards/4020087797/        ...
License      http://creativecommons.org/licenses/by-nc-sa/2.5/ch/Freitag, 3. Juni 2011
Upcoming SlideShare
Loading in...5
×

Testing distributed, complex web applications

2,940

Published on

As applications grow from single Rails applications to complex systems with multiple, interacting applications & web services, testing becomes more and more difficult. While we can test each application independently, we need to be able to test the full stack. This presentation shows methods, tools and tipps & tricks from testing such a complex application.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,940
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
28
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Testing distributed, complex web applications

  1. 1. Testing of distributed applications Rails Way Con 2011 Jens-Christian Fischer with Michael Mahlberg & Keith Bingman jcf@mobino.com @jcfischerFreitag, 3. Juni 2011
  2. 2. le x m p co Testing of distributed applications Rails Way Con 2011 Jens-Christian Fischer with Michael Mahlberg & Keith Bingman jcf@mobino.com @jcfischerFreitag, 3. Juni 2011
  3. 3. Agenda • Background • What & Why & How of Testing • Discover the state of your apps • Make testable appsFreitag, 3. Juni 2011
  4. 4. why?Freitag, 3. Juni 2011
  5. 5. Freitag, 3. Juni 2011
  6. 6. we move moneyFreitag, 3. Juni 2011
  7. 7. What is behind? 1 Rails app 5 Sinatra apps 1 Ruby daemon 1 VOIP Server 1 Speech Synthesizer 1 SQL Database (Postgres) 1 NoSQL Server (Redis) (with Resque)Freitag, 3. Juni 2011
  8. 8. ArchitectureFreitag, 3. Juni 2011
  9. 9. Merchant Web Customer Web Customer Voice Widget Volcker Trichet Freeswitch Duisenberg Keynes Greenspan Redis Smith VAAS Postgres 9 AcapelaFreitag, 3. Juni 2011
  10. 10. Merchant Web Customer Web Customer Voice Widget Volcker Trichet Freeswitch Duisenberg Keynes Greenspan Redis Smith VAAS Postgres 9 AcapelaFreitag, 3. Juni 2011
  11. 11. Merchant Web Customer Web Customer Voice Widget Volcker FrontTrichet Freeswitch Duisenberg Keynes Greenspan Back Voice Redis Smith VAAS Database Postgres 9 AcapelaFreitag, 3. Juni 2011
  12. 12. Trichet Volcker DuisenbergGreenspan Keynes SmithFreitag, 3. Juni 2011
  13. 13. Freitag, 3. Juni 2011
  14. 14. TestingFreitag, 3. Juni 2011
  15. 15. The old days • Simple web apps • Click through testingFreitag, 3. Juni 2011
  16. 16. Testi g n Royce, W. (1970). Managing the development of large software systems. Proceedings of IEEE Wescon.Freitag, 3. Juni 2011
  17. 17. Freitag, 3. Juni 2011
  18. 18. Freitag, 3. Juni 2011
  19. 19. Freitag, 3. Juni 2011
  20. 20. In Rails — everything is betterFreitag, 3. Juni 2011
  21. 21. Automated Tests Unit Integration Frontend testsFreitag, 3. Juni 2011
  22. 22. Complexity • Modern applications consist of many pieces • App Servers, DB, NoSQL, Speech WebServices, Telephony, Synthesizers • HTTP, Line Protocols, JSON, XML, Corba, RMIFreitag, 3. Juni 2011
  23. 23. What to test Robert „UncleBob“ Martin: •Unit Tests •Component Tests •Integration Tests •System Test •Manual Tests http://stackoverflow.com/questions/2965483Freitag, 3. Juni 2011
  24. 24. Unit Tests • Close to 100% code coverage • Test Driven Development • Written by ProgrammersFreitag, 3. Juni 2011
  25. 25. Component Tests • Test ~50% of the System • Cucumber / Selenium / Fitnesse • Written by QA & Analysts • Happy Path & some unhappy pathsFreitag, 3. Juni 2011
  26. 26. Integration Tests • Cover ~20% of system • Test assembly of components • Cucumber / Selenium / Fitnesse • Written by ArchitectsFreitag, 3. Juni 2011
  27. 27. System Tests • Cover 10% of the system • Test the whole system • Cucumber / Selenium / Fitnesse • Written by ArchitectsFreitag, 3. Juni 2011
  28. 28. Manual Tests • Exploratory Manual Tests • Manual but not scripted • Human ingenuity & creativity Bach, J. (n.d.). Exploratory testing explained. Online: http://www. satisfice. com/articles/et-article. pdf.Freitag, 3. Juni 2011
  29. 29. Where do we test?Freitag, 3. Juni 2011
  30. 30. Testing the edgesFreitag, 3. Juni 2011
  31. 31. Merchant Web Customer Web Customer Voice Widget Volcker Trichet Freeswitch Duisenberg Keynes Greenspan Redis Smith VAAS Postgres 9 AcapelaFreitag, 3. Juni 2011
  32. 32. Merchant Web Customer Web Customer Voice Widget Volcker Trichet Freeswitch Duisenberg Keynes Greenspan Redis Smith VAAS Postgres 9 AcapelaFreitag, 3. Juni 2011
  33. 33. How do we test?Freitag, 3. Juni 2011
  34. 34. http://blog.objectmentor.com/articles/2009/09/29/ ruining-your-test-automation-strategyFreitag, 3. Juni 2011
  35. 35. Tools of the trade Selenium Cucumber Steak capybara webkit zombieFreitag, 3. Juni 2011
  36. 36. only cover a minimal part the web frontend, but our apps are so much moreFreitag, 3. Juni 2011
  37. 37. Decoupling Systems diverge APIs divergeFreitag, 3. Juni 2011
  38. 38. Web Services Many services outside our controlFreitag, 3. Juni 2011
  39. 39. Freitag, 3. Juni 2011
  40. 40. Synchronizing • Tests require complex state across several systems • Tests involvebetween systems interactions complexFreitag, 3. Juni 2011
  41. 41. QuestionsFreitag, 3. Juni 2011
  42. 42. Questions to ask • Development • Testing • ProductionFreitag, 3. Juni 2011
  43. 43. Development Is the system I‘m building correct?Freitag, 3. Juni 2011
  44. 44. Test Do all the pieces of the system work together?Freitag, 3. Juni 2011
  45. 45. Production Are all parts of my system working?Freitag, 3. Juni 2011
  46. 46. Development • Test Driven Development only takes you so far • Test a single component • Unit tests in work, but don‘t components multiple „cross the chasm“Freitag, 3. Juni 2011
  47. 47. Development • Setting up a complete system for development can be taxing • pow to the rescue http://pow.cx/ • Foreman https://github.com/ddollar/foremanFreitag, 3. Juni 2011
  48. 48. Testing • Integration tests need to stress complete application stack • Does your CI Server handle this?Freitag, 3. Juni 2011
  49. 49. Outside your appFreitag, 3. Juni 2011
  50. 50. Merchant Web Customer Web Customer Voice Widget Volcker Trichet Freeswitch Duisenberg Keynes Greenspan Redis Smith VAAS Postgres 9 AcapelaFreitag, 3. Juni 2011
  51. 51. Merchant Web Customer Web Customer Voice Widget Volcker Trichet Freeswitch Duisenberg Keynes Greenspan Redis Smith VAAS Postgres 9 AcapelaFreitag, 3. Juni 2011
  52. 52. Write Mock Services • If youitdon‘t control it, mock • „Rails Enterprise ready ™ “Freitag, 3. Juni 2011
  53. 53. Mock Services • Allowsspecification against a „real“ you to code • Allows of said code against failure you to service • Gives you a stick to beat up the developers of said service („Rails-Enterprise ready“ ™ )Freitag, 3. Juni 2011
  54. 54. Making it workFreitag, 3. Juni 2011
  55. 55. Version Control Repository setup One Big repository? One repository for each component? Tagging / SyncingFreitag, 3. Juni 2011
  56. 56. Development Servers Webrick doesn‘t scale but: it‘s „easy“ for Rack based apps pow by 37 Signals (Mac Only)Freitag, 3. Juni 2011
  57. 57. Configuration Configuring half a dozen applications Again: Synchronisation Inject configuration (Example Picky) http://florianhanke.com/picky/Freitag, 3. Juni 2011
  58. 58. Production Is my system working correctly? Probes • DTrace (OSX, Solaris) • Monitoring (Scout, NewRelic) • Your own probesFreitag, 3. Juni 2011
  59. 59. Freitag, 3. Juni 2011
  60. 60. Freitag, 3. Juni 2011
  61. 61. Car industry If car manufacturers would build engines the way enterprise IT systems are built, you‘d need to remove the engine from the car and disassemble it completely to find a fault — @MMahlberg, 2011Freitag, 3. Juni 2011
  62. 62. OBD • 1996: The OBD-II specification is made mandatory for all cars sold in the United States. • 2001: The European Union makes EOBD mandatory for all gasoline (petrol) vehicles sold in the European Union, starting in MY2001 (see European emission standards Directive 98/69/EC [1]). • 2008: All cars sold in the United States are required to use the ISO 15765-4 [2] signaling standard (a variant of the Controller Area Network (CAN) bus). http://en.wikipedia.org/wiki/On-board_diagnosticsFreitag, 3. Juni 2011
  63. 63. Solved ProblemFreitag, 3. Juni 2011
  64. 64. Freitag, 3. Juni 2011
  65. 65. http://the-adam.com/adam/rantrave/ibm_360.jpgFreitag, 3. Juni 2011
  66. 66. Diagnosis Channel • Known from Car / Planes / Mainframes • Red lights on your dashboard • Complete diagnosis of system possible • Complete remote control possibleFreitag, 3. Juni 2011
  67. 67. Rolls Royce • InFlight Diagnosis of engines remotelyFreitag, 3. Juni 2011
  68. 68. Dashboard • Look into state of application • Sanity CheckerFreitag, 3. Juni 2011
  69. 69. Dashboarding • Extract relevant data from your system • How many req / s • How many logins • Delta / TrendsFreitag, 3. Juni 2011
  70. 70. SanityChecker • API stateallows you to extract the that of your app • DB there? • All neighbouring components reachable?Freitag, 3. Juni 2011
  71. 71. Chaos Monkey • Random failures • Better reliabilityFreitag, 3. Juni 2011
  72. 72. Build testable apps • Expose your internals (through APIs) • Use SOLID design • Single Responsibility Principle • Open Closed • Liskov Substitution Principle • Inversion of Control • Dependency InjectionFreitag, 3. Juni 2011
  73. 73. Testable apps • Obey the abstraction layers • keep of abstractionat it‘s own level each methodFreitag, 3. Juni 2011
  74. 74. Build probe-able apps • Measure APIs $  rails  console  productionFreitag, 3. Juni 2011
  75. 75. Build controllable apps • Call functionality of the app • Create new user • Create transactions • Delete stuff • Do stuff • ...Freitag, 3. Juni 2011
  76. 76. Test user / Magic • In „Enterprise“ systems: • Test Company • Test User • Test Accounts • Are those in production system? • Do everything - or not quite?Freitag, 3. Juni 2011
  77. 77. Use production data in development / testFreitag, 3. Juni 2011
  78. 78. Wrapping upFreitag, 3. Juni 2011
  79. 79. What & Why & How of Testing Discover the state of your apps Make testable appsFreitag, 3. Juni 2011
  80. 80. Thank you!Freitag, 3. Juni 2011
  81. 81. Credits • http://www.flickr.com/photos/ myoldpostcards/4020087797/ • http://www.flickr.com/photos/ benncapon/390585495/ • iStockphotoFreitag, 3. Juni 2011
  82. 82. License http://creativecommons.org/licenses/by-nc-sa/2.5/ch/Freitag, 3. Juni 2011
  1. Gostou de algum slide específico?

    Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

×