Testing distributed, complex web applications

Testing of distributed applications Rails Way Con 2011 Jens-Christian Fischer with Michael Mahlberg & Keith Bingman jcf@mobino.com @jcfischerFreitag, 3. Juni 2011

le x m p co Testing of distributed applications Rails Way Con 2011 Jens-Christian Fischer with Michael Mahlberg & Keith Bingman jcf@mobino.com @jcfischerFreitag, 3. Juni 2011

Agenda • Background • What & Why & How of Testing • Discover the state of your apps • Make testable appsFreitag, 3. Juni 2011

why?Freitag, 3. Juni 2011

Freitag, 3. Juni 2011

we move moneyFreitag, 3. Juni 2011

What is behind? 1 Rails app 5 Sinatra apps 1 Ruby daemon 1 VOIP Server 1 Speech Synthesizer 1 SQL Database (Postgres) 1 NoSQL Server (Redis) (with Resque)Freitag, 3. Juni 2011

ArchitectureFreitag, 3. Juni 2011

Merchant Web Customer Web Customer Voice Widget Volcker Trichet Freeswitch Duisenberg Keynes Greenspan Redis Smith VAAS Postgres 9 AcapelaFreitag, 3. Juni 2011

Merchant Web Customer Web Customer Voice Widget Volcker FrontTrichet Freeswitch Duisenberg Keynes Greenspan Back Voice Redis Smith VAAS Database Postgres 9 AcapelaFreitag, 3. Juni 2011

Trichet Volcker DuisenbergGreenspan Keynes SmithFreitag, 3. Juni 2011

TestingFreitag, 3. Juni 2011

The old days • Simple web apps • Click through testingFreitag, 3. Juni 2011

Testi g n Royce, W. (1970). Managing the development of large software systems. Proceedings of IEEE Wescon.Freitag, 3. Juni 2011

In Rails — everything is betterFreitag, 3. Juni 2011

Automated Tests Unit Integration Frontend testsFreitag, 3. Juni 2011

Complexity • Modern applications consist of many pieces • App Servers, DB, NoSQL, Speech WebServices, Telephony, Synthesizers • HTTP, Line Protocols, JSON, XML, Corba, RMIFreitag, 3. Juni 2011

What to test Robert „UncleBob“ Martin: •Unit Tests •Component Tests •Integration Tests •System Test •Manual Tests http://stackoverflow.com/questions/2965483Freitag, 3. Juni 2011

Unit Tests • Close to 100% code coverage • Test Driven Development • Written by ProgrammersFreitag, 3. Juni 2011

Component Tests • Test ~50% of the System • Cucumber / Selenium / Fitnesse • Written by QA & Analysts • Happy Path & some unhappy pathsFreitag, 3. Juni 2011

Integration Tests • Cover ~20% of system • Test assembly of components • Cucumber / Selenium / Fitnesse • Written by ArchitectsFreitag, 3. Juni 2011

System Tests • Cover 10% of the system • Test the whole system • Cucumber / Selenium / Fitnesse • Written by ArchitectsFreitag, 3. Juni 2011

Manual Tests • Exploratory Manual Tests • Manual but not scripted • Human ingenuity & creativity Bach, J. (n.d.). Exploratory testing explained. Online: http://www. satisfice. com/articles/et-article. pdf.Freitag, 3. Juni 2011

Where do we test?Freitag, 3. Juni 2011

Testing the edgesFreitag, 3. Juni 2011

How do we test?Freitag, 3. Juni 2011

http://blog.objectmentor.com/articles/2009/09/29/ ruining-your-test-automation-strategyFreitag, 3. Juni 2011

Tools of the trade Selenium Cucumber Steak capybara webkit zombieFreitag, 3. Juni 2011

only cover a minimal part the web frontend, but our apps are so much moreFreitag, 3. Juni 2011

Decoupling Systems diverge APIs divergeFreitag, 3. Juni 2011

Web Services Many services outside our controlFreitag, 3. Juni 2011

Synchronizing • Tests require complex state across several systems • Tests involvebetween systems interactions complexFreitag, 3. Juni 2011

QuestionsFreitag, 3. Juni 2011

Questions to ask • Development • Testing • ProductionFreitag, 3. Juni 2011

Development Is the system I‘m building correct?Freitag, 3. Juni 2011

Test Do all the pieces of the system work together?Freitag, 3. Juni 2011

Production Are all parts of my system working?Freitag, 3. Juni 2011

Development • Test Driven Development only takes you so far • Test a single component • Unit tests in work, but don‘t components multiple „cross the chasm“Freitag, 3. Juni 2011

Development • Setting up a complete system for development can be taxing • pow to the rescue http://pow.cx/ • Foreman https://github.com/ddollar/foremanFreitag, 3. Juni 2011

Testing • Integration tests need to stress complete application stack • Does your CI Server handle this?Freitag, 3. Juni 2011

Outside your appFreitag, 3. Juni 2011

Write Mock Services • If youitdon‘t control it, mock • „Rails Enterprise ready ™ “Freitag, 3. Juni 2011

Mock Services • Allowsspecification against a „real“ you to code • Allows of said code against failure you to service • Gives you a stick to beat up the developers of said service („Rails-Enterprise ready“ ™ )Freitag, 3. Juni 2011

Making it workFreitag, 3. Juni 2011

Version Control Repository setup One Big repository? One repository for each component? Tagging / SyncingFreitag, 3. Juni 2011

Development Servers Webrick doesn‘t scale but: it‘s „easy“ for Rack based apps pow by 37 Signals (Mac Only)Freitag, 3. Juni 2011

Configuration Configuring half a dozen applications Again: Synchronisation Inject configuration (Example Picky) http://florianhanke.com/picky/Freitag, 3. Juni 2011

Production Is my system working correctly? Probes • DTrace (OSX, Solaris) • Monitoring (Scout, NewRelic) • Your own probesFreitag, 3. Juni 2011

Car industry If car manufacturers would build engines the way enterprise IT systems are built, you‘d need to remove the engine from the car and disassemble it completely to find a fault — @MMahlberg, 2011Freitag, 3. Juni 2011

OBD • 1996: The OBD-II specification is made mandatory for all cars sold in the United States. • 2001: The European Union makes EOBD mandatory for all gasoline (petrol) vehicles sold in the European Union, starting in MY2001 (see European emission standards Directive 98/69/EC [1]). • 2008: All cars sold in the United States are required to use the ISO 15765-4 [2] signaling standard (a variant of the Controller Area Network (CAN) bus). http://en.wikipedia.org/wiki/On-board_diagnosticsFreitag, 3. Juni 2011

Solved ProblemFreitag, 3. Juni 2011

http://the-adam.com/adam/rantrave/ibm_360.jpgFreitag, 3. Juni 2011

Diagnosis Channel • Known from Car / Planes / Mainframes • Red lights on your dashboard • Complete diagnosis of system possible • Complete remote control possibleFreitag, 3. Juni 2011

Rolls Royce • InFlight Diagnosis of engines remotelyFreitag, 3. Juni 2011

Dashboard • Look into state of application • Sanity CheckerFreitag, 3. Juni 2011

Dashboarding • Extract relevant data from your system • How many req / s • How many logins • Delta / TrendsFreitag, 3. Juni 2011

SanityChecker • API stateallows you to extract the that of your app • DB there? • All neighbouring components reachable?Freitag, 3. Juni 2011

Chaos Monkey • Random failures • Better reliabilityFreitag, 3. Juni 2011

Build testable apps • Expose your internals (through APIs) • Use SOLID design • Single Responsibility Principle • Open Closed • Liskov Substitution Principle • Inversion of Control • Dependency InjectionFreitag, 3. Juni 2011

Testable apps • Obey the abstraction layers • keep of abstractionat it‘s own level each methodFreitag, 3. Juni 2011

Build probe-able apps • Measure APIs $ rails console productionFreitag, 3. Juni 2011

Build controllable apps • Call functionality of the app • Create new user • Create transactions • Delete stuff • Do stuff • ...Freitag, 3. Juni 2011

Test user / Magic • In „Enterprise“ systems: • Test Company • Test User • Test Accounts • Are those in production system? • Do everything - or not quite?Freitag, 3. Juni 2011

Use production data in development / testFreitag, 3. Juni 2011

Wrapping upFreitag, 3. Juni 2011

What & Why & How of Testing Discover the state of your apps Make testable appsFreitag, 3. Juni 2011

Thank you!Freitag, 3. Juni 2011

Credits • http://www.flickr.com/photos/ myoldpostcards/4020087797/ • http://www.flickr.com/photos/ benncapon/390585495/ • iStockphotoFreitag, 3. Juni 2011

License http://creativecommons.org/licenses/by-nc-sa/2.5/ch/Freitag, 3. Juni 2011

http://blog.invisible.ch	175
url_unknown	4
http://feeds.feedburner.com	3
http://static.slidesharecdn.com	1
http://webcache.googleusercontent.com	1

Testing distributed, complex web applications

by Jens-Christian Fischer on Jun 03, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

5 Embeds 184

Statistics

Testing distributed, complex web applications — Presentation Transcript