Download to read offline
![database.yml test: &test adapter: mysql2 encoding: utf8 reconnect: false database: myapp_test<%= ENV['TEST_ENV_NUMBER'] %> pool: 5 username: root password:](https://image.slidesharecdn.com/vodqareducingbuildtimeamanking-110406060531-phpapp02/75/VodQA_ParallelizingCukes_AmanKing-30-2048.jpg)
![sunspot.yml test: solr: hostname: localhost port: <%= 8982 - ENV['TEST_ENV_NUMBER'].to_i %> log_level: INFO #WARNING log_file: <%= File.join(::Rails.root, 'log', "solr_#{ENV['TEST_ENV_NUMBER'].to_i}.log") %> data_path: <%= File.join(::Rails.root, 'solr', 'data’, "#{ENV['TEST_ENV_NUMBER'].to_i}") %> pid_path: <%= File.join(::Rails.root, 'solr', 'pids', "#{ENV['TEST_ENV_NUMBER'].to_i}") %>](https://image.slidesharecdn.com/vodqareducingbuildtimeamanking-110406060531-phpapp02/75/VodQA_ParallelizingCukes_AmanKing-32-2048.jpg)
![Thank you [email_address] http://www.wikyblog.com/AmanKing](https://image.slidesharecdn.com/vodqareducingbuildtimeamanking-110406060531-phpapp02/75/VodQA_ParallelizingCukes_AmanKing-42-2048.jpg)
Uploaded bypoojaelkunchwar
VodQA_ParallelizingCukes_AmanKing
The document discusses strategies to reduce build time in a Ruby on Rails project from 55 minutes to approximately 5 minutes using parallelization and functional testing. It highlights the implementation of parallel_tests and custom report generation while managing dependencies like Solr and Firefox. Key practices include stubbing external calls, consolidating test scenarios, and maintaining conventions to enhance efficiency.
More Related Content
Similar to VodQA_ParallelizingCukes_AmanKing
![[DevFest Strasbourg 2025] - NodeJs Can do that !!](https://cdn.slidesharecdn.com/ss_thumbnails/devfeststrasbourg2025-nodejscandothat-251127142731-da65b6fd-thumbnail.jpg?width=640&height=640&fit=bounds)
![[BDD 2025 - Mobile Development] Mobile Engineer and Software Engineer: Are we...](https://cdn.slidesharecdn.com/ss_thumbnails/md-mobileengineerandsoftwareengineerarewestillrelevantsidiqpermana-251127010650-55224ef1-thumbnail.jpg?width=640&height=640&fit=bounds)
![[BDD 2025 - Artificial Intelligence] Building AI Systems That Users (and Comp...](https://cdn.slidesharecdn.com/ss_thumbnails/ai-buildingaisystemsthatusersandcompanieslove-251124030845-038f7732-thumbnail.jpg?width=640&height=640&fit=bounds)
![[BDD 2025 - Full-Stack Development] The Modern Stack: Building Web & AI Appli...](https://cdn.slidesharecdn.com/ss_thumbnails/fs-themodernstackbuildingwebaiapplicationswithserverless-251124030844-388cf04f-thumbnail.jpg?width=640&height=640&fit=bounds)
![[BDD 2025 - Full-Stack Development] Agentic AI Architecture: Redefining Syste...](https://cdn.slidesharecdn.com/ss_thumbnails/fs-agenticaiarchitectureredefiningsystemcommunication-251124030838-e6c70cc2-thumbnail.jpg?width=640&height=640&fit=bounds)
VodQA_ParallelizingCukes_AmanKing
- 1. Saturday, 26 th March, 2011
- 2. Reducing build timewhen patience is not a virtue Aman King Application Developer Thought Works
- 3.
- 4.
- 5. Did you choosethis? 1 3 2
- 6. Or did youchoose this? 1 2 3
- 7. Patience is notalways a virtue!
- 8. Fail fast!quick feedback
- 9. Rank these Highfunctional coverage Short build time
- 10. Did you choosethis? High functional coverage Short build time
- 11. Or did youchoose this? Short build time High functional coverage
- 12. You can chooseboth! Short build time High functional coverage
- 13. But before wesee how…
- 14.
- 15. Content driven, community oriented website
- 16. Ruby on RailsCucumber + Capybara + Selenium
- 17. 4 : 1ratio of Dev : QA Everyone writes functional tests in a common automation suite Acceptance Test Driven Development (ATDD)
- 18.
- 19. Long build time~ 55 minutes
- 20.
- 21.
- 22.
- 23.
- 24. Reduced build time~ 5 minutes
- 25.
- 26.
- 27. Parallelization on asingle multicore machine used Ruby library parallel_tests
- 28. Report generation inparallel wrote custom report formatter
- 29. Isolating databases eachprocess needs its own database
- 30. database.yml test: &testadapter: mysql2 encoding: utf8 reconnect: false database: myapp_test<%= ENV['TEST_ENV_NUMBER'] %> pool: 5 username: root password:
- 31. Isolating external dependencies(Solr) each process needs its own Solr instance
- 32. sunspot.yml test: solr:hostname: localhost port: <%= 8982 - ENV['TEST_ENV_NUMBER'].to_i %> log_level: INFO #WARNING log_file: <%= File.join(::Rails.root, 'log', "solr_#{ENV['TEST_ENV_NUMBER'].to_i}.log") %> data_path: <%= File.join(::Rails.root, 'solr', 'data’, "#{ENV['TEST_ENV_NUMBER'].to_i}") %> pid_path: <%= File.join(::Rails.root, 'solr', 'pids', "#{ENV['TEST_ENV_NUMBER'].to_i}") %>
- 33. Handling multiple Firefoxinstances and Selenium’s use of shared ephemeral ports
- 34. Monitoring Memory and CPU usage adjust number of parallel process accordingly
- 35. Auto rerunning failedtests non-deterministic failures are likelier with the CPU under stress
- 36.
- 37. Stubbing external callsreduce dependency-related delays where avoidable
- 38. Consolidating scenarios combinesimilar test scenarios into single runs instead of separate tests
- 39. Maintaining conventions whenwriting automated tests avoid time-based wait statements, use test framework APIs that take lesser time, etc
- 40.
- 41. Resources http://cukes.info http://rubygems.org/gems/capybarahttp://seleniumhq.org/projects/remote-control http://rubygems.org/gems/parallel_tests http://selenium-grid.seleniumhq.org http://test-load-balancer.github.com http://github.com/bblimke/webmock http://testing.thoughtworks.com
- 42. Thank you [email_address]http://www.wikyblog.com/AmanKing
Editor's Notes
- #4 * Green icon represents a stable green CI build * Orange icon represents a CI build in progress * Red icon represents a broken CI build
- #7 * It is good to immediately know where you stand: whether all is good or something is broken * That’d enable you to fix something almost as soon as you break it
- #10 * High functional coverage implies that almost all functionality of the system is tested and any error introduced would be caught * Short CI build time implies that quick feedback is in place and any error introduced in the covered functionality would be immediately highlighted
- #16 * A website with a lot of content that comes from various sources, with a focus on creating an online community of people with similar interests
- #21 * Failures not caused by programmatic errors: possible reasons could include AJAX response delays, unpredictable browser responsiveness, network slowness, etc
- #22 * If a build is broken on what appears to be non-deterministic failure, someone will have to manually trigger another build in hope of the error not reoccurring * A retriggered build will take long again, further delaying feedback
- #24 * One main process will fork multiple sub-processes in parallel, each running only a subset of the original collection of test scenarios * When all sub-processes complete, the union of their results will be the final result of the entire test run * If the sub-processes are run on the same machine, a powerful multicore machine is desirable *Another approach is to run the sub-processes on different machines on a grid
- #26 * Chart of build times vs build numbers: the drastic drop is when parallelization was implemented * Slight variances in build times, including minor drops, also occur based on memory and CPU load at the time
- #28 * Used a Ruby library called parallel_tests to fork multiple Ruby processes, each running a subset of Cucumber features * This also involved coming up with custom rake tasks, including multiple pre-steps and post-steps * Instead of using parallel_tests to parallelize on the same machine, Selenium Grid is an alternate option to leverage grid computing
- #29 * Capture each forked process’s results independently * Generate consolidated HTML report as after-step when all forked processes finish
- #30 * Rails’ database.yml was modified to suffix process number to the name of the database
- #32 * Sunspot gem’s sunspot.yml was modified to suffix process number to the paths that Solr was to use * Monkey-patched Sunspot code to allow above parameterization in sunspot.yml
- #34 * Although each sub-process starts its independent Firefox instance, Selenium uses a shared set of ephemeral ports to communicate with the browsers * Contention occurs for these ephemeral ports which get locked when in use for a particular Firefox instance * Monkey-patched Capybara’s Selenium driver code to retry communicating with Firefox if a couple of attempts fail
- #35 * A particular machine can only allow so much parallelization based on its hardware specifications * Beyond a certain number of sub-processes, the non-deterministic failures will increase * After a few tries, we stabilized on having 6 concurrent sub-processes on the machine we were using
- #36 * Cucumber provides a rerun mode which retries all failed tests one more time before deciding the final status of the test suite run * Customized a rake task to invoke Cucumber’s rerun sequentially, after the parallel results come in
- #38 * We stubbed some external calls to return known values to cut down on communication delays * Used a Ruby library called WebMock * Examples: authentication servers, email servers, Facebook, Twitter, etc
- #39 * It helps to reduce the number of independent tests without reducing the functionality coverage * Example: if during sign up, email and zipcode are mandatory, they don’t need independent tests; there can be one test that begins by leaving email and zipcode empty, verifies that error messages are displayed for email and zipcode, and then adds email and zipcode, and proceeds with valid sign up scenarios * The above reduces browser clearing steps, visiting a certain page repeatedly without it contributing to the test steps, and so on
- #41 * There are multiple benefits of having quick feedback; implementing parallelization is one approach to achieve that * A complementary approach is to introspect and work on improving performance and responsiveness of the app itself, reducing build time by reducing waiting time * Being aware of build times and improving the same continuously is a good habit; even with parallelization in place, as more functionality gets added, build times will creep up