Your SlideShare is downloading. ×
Software Quality and Test Strategies for Ruby and Rails Applications
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Software Quality and Test Strategies for Ruby and Rails Applications


Published on

Presentation of the talk I gave at RubyConf India 2011 ->

Presentation of the talk I gave at RubyConf India 2011 ->

Published in: Technology
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Software Quality and Test Strategies forRuby and Rails Applications - Bhavin Javia May 29th, 2011
  • 2. About Me Ex ThoughtWorkerAgile/RoR Consultant - Ennova, Australia Founder - Ennova, India
  • 3. bhavinjavia
  • 4. Co-Author Dr Adrian SmithManaging Director - Technology Agile Coach @
  • 5. Content Software Quality Test Strategy Ruby/Rails Testing EcosystemQuality and Testing for Rails Projects
  • 6. Software Quality
  • 7. Importance of Quality
  • 8. “Quality is the most important factor for deriving long term value from a software product” - Jim Highsmith (Co-Author of Agile Manifesto)
  • 9. ”Continuous attention to quality can helpmaintain velocity, maintain the ability to deliver, without it the team will slow down” - Robert C. Martin (Author of Clean Code)
  • 10. Why worry ? Needs to be built in (implicitly) Needs to be defined (to prioritize vs cost/scope/time)Agile project delivery relies on continuous attention to quality Begins at the code level
  • 11. code_quality != "App Quality"
  • 12. Internal Quality vsExternal Quality
  • 13. External Quality( Quality as perceived by users ) interface design performance defects user experience
  • 14. Internal Quality( Quality of the implementation and system architecture ) duplication complexity coupling test coverage
  • 15. “Technical Debt may be costing you more than you imagined!” - Jim Highsmith
  • 16. The trade-off
  • 17. Managing Quality
  • 18. Managing Quality Set Goals "Max 3 clicks to any page" Measure "Page X takes 5 clicks" Prioritize "Make page X reachable in <= 3 clicks"
  • 19. How to achieve Quality ?
  • 21. What/When/How to Test ?
  • 22. " Test Strategy "
  • 23. Test Strategy Template
  • 24. Test Strategy Template Not a Waterfall style documentA template to structure the discussion Prompts for things to consider Flexible format
  • 25. PurposeCreate a shared understanding of approach tools targets timing of test activities
  • 26. Guiding Principles Shared Responsibility Everyone is Responsible for Testing and Quality Test AutomationAll types of tests should be automated (except exploratory tests) Data Management Production data must be obfuscated before use Test Management Tests, documents and data treated as production code
  • 27. Quality and Test Objectives Correctness Integrity Maintainability Availability Interoperability Performance
  • 28. Quality and Test Objectives Correctness Features and functions work as intended e.g. 0 Critical Defects Integrity Prevent unauthorized access or information loss e.g. All access via HTTPS, Encrypted passwords Maintainability Easy to add new features or fix defects e.g. Code Complexity < 8, Unit Test Coverage > 80% ...
  • 29. Quality and Test Objectives Availability Planned uptime percentage e.g. 99.99 % availability Interoperability Ease of information exchange with other systemse.g. Published & versioned API, Supports - IE 8, FF 3.5, Chrome 11 etc Performance Responsiveness & Scalability of the system e.g. Apdex Score > 0.9, Response Time < 200ms
  • 30. Quality and Test Objectives Target Measure Prioritize
  • 31. Test Scope Systems and features in and out of scope Test both business processes and the technical solution Specify regions and sub-regions included in testing Identify interfaces with other systemsSystem / feature / integration / region / environment /dependency / service In Scope Things to be tested with automation Things to be tested manually Out of Scope Things we cant test e.g. Third Party systems
  • 32. Test Approach Identify the test types Identify the toolsInclude the timing of execution
  • 33. Test Types Unit Functional Integration Acceptance PerformanceData Conversion
  • 34. Test Approach Identify Tools e.g. rspec, cucumber etc Decide Timing of Executione.g. dev machine, CI server, after each commit etc
  • 35. Test Automation Pyramid
  • 36. Test Preparation Use requirements captured in user stories Include acceptance criteria in user storyDecide approach to prepare and execute tests
  • 37. Example
  • 38. Acceptance Test
  • 39. Environments Development Unit, Functional, Acceptance tests IntegrationContinuous Integration - Unit, Functional, Acceptance tests Code Analysis/Metrics Staging For exploratory testing Demos to customers Production Smoke/Sanity tests & Monitoring
  • 40. Test Executionsteps in preparation for deployment/release Build the system Populate test/reference data Execute automated tests Generate test report/code metrics
  • 41. Test Data Management System and user acceptance tests Use subset of production data Performance/volume/stress test Use full size production files Generate large volumes of data
  • 42. begin database.load! "Sensitive Production Data"ensure database.obfuscate! "Sensitive Data"end
  • 43. Defect Management
  • 44. Why defects ?
  • 45. :missing_tests => "Defects"
  • 46. Defect ManagementDefects only raised when not fixed immediately Capture conditions and severity Capture steps to reproduce Critical Major Minor Trivial
  • 47. Defect LifeCycle Identify Prioritize Assign Severity Analyze Write test to reproduce Resolve Fix code to pass the test Verify Run all tests, verify manually Close
  • 48. Test Strategy Example
  • 49. Principles and Objectives
  • 50. Test Scope
  • 51. Test Approach
  • 52. Ruby/RailsTesting Ecosystem
  • 53. Whats different in Ruby/Rails ?
  • 54. Advantages Testing built right-in Passionate developersVibrant open source community Excellent tools support
  • 55. So what are our options ?
  • 56. There are ... 23 Testing Frameworks 8 Javascript Testing Frameworks 8 Browser Testing Frameworks 8 Distributed Testing Frameworks 6 Object Mocking Frameworks 7 Web Mocking Frameworks 2 Continuous Testing Frameworks7 Continuous Integration Frameworks 12 Code Metrics Frameworks and many more ...
  • 57. raise "OMG !!"
  • 58. What to Test ?Anything that could possibly break Public interface of each layer Whole application stackDepends on what else youre testing
  • 59. What NOT to test ?Cost of test > 10 * Cost of breakage (very rare) Internal implementation Framework code
  • 60. Youre doing it wrong if Fixing tests all the time Tests that always fail together Tests that never fail
  • 61. What to Test ? Models Controllers Views Helpers Libraries APIs
  • 62. Testing Options Models Test::Unit (Ruby 1.8, built-in)MiniTest (Ruby 1.9, built-in, replaces Test::Unit) Unit Tests (ActiveSupport::TestCase) Model Specs (rspec) Controllers Functional Tests (built-in) Controller Specs (rspec)
  • 63. Testing Options Views assert_select (built-in) View Specs (rspec) Helpers ActionView::TestCase Helper Specs (rspec)
  • 64. Testing Options Routes assert_routing (built-in) Routing specs (rspec) Mailers Unit Tests (built-in) Functional Tests (built-in)
  • 65. Testing Options Workflow Integration Tests (built-in) Request Specs (rspec) BDD Cucumber + Capybara Cucumber + Rspec
  • 66. Testing Options Performance Performance Tests (built-in) NewRelic
  • 67. Testing Helpers Data Setup Fixtures (built-in) factory_girl Machinist
  • 68. Testing Helpers Mocks/Stubs Rspec mocks Mocha Flexmock API / Web Services VCR Fakeweb Webmock
  • 69. Testing Helpers Speed NullDB Autotest/Infinity Test/Watchr Spork parallel_tests TLB CI cijoe cruisecontrol.rb/goldberg integrity
  • 70. Quality/Metrics Test Coverage rcov SimpleCov Code Analysis metric_fu Metrical Rails Best Practices
  • 71. Which is the Best ? Even the God(s) Cant Say !!
  • 72. Which is the Best for Me ?
  • 73. IMHOWHATS_BEST = { :newbie => "Start with built-ins", :intermediate => "Checkout alternatives", :expert => "Venture into the obscure" }
  • 74. WHATS_BEST[:god] = "Write your own !"
  • 75. References http://www.jimhighsmith.com
  • 76. Questions ? bhavinjavia @bhavinjavia