Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Continuous Visual Integration - RailsConf 2016 - Mike Fotinakis - Percy.io

545 views

Published on

Unit testing is mostly a solved problem, but how do you write tests for the visual side of your app—the part that your users actually see and interact with? How do you stop visual bugs from reaching your users?

We will dive deep into visual regression testing, a fast-growing technique for testing apps pixel-by-pixel. We will integrate perceptual diffs in Rails feature specs, and learn how to visually test even complex UI states. We will show tools and techniques for continuous visual integration on every commit, and learn how to introduce team visual reviews right alongside code reviews.

Published in: Software
  • Hello! Who wants to chat with me? Nu photos with me here http://bit.ly/helenswee
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Continuous Visual Integration - RailsConf 2016 - Mike Fotinakis - Percy.io

  1. 1. Continuous visual integration
  2. 2. Mike Fotinakis @mikefotinakis
  3. 3. Percy https://percy.io @percy_io Mike Fotinakis @mikefotinakis
  4. 4. ● jsonapi-serializers ● swagger-blocks Mike Fotinakis @mikefotinakis
  5. 5. Continuous visual integration
  6. 6. - The Problem - The Solution - How it works
  7. 7. - The Problem - The Solution - How it works
  8. 8. “Regression”
  9. 9. “Visual Regression”
  10. 10. How do product teams fix this today?
  11. 11. QA
  12. 12. QA is necessary, but complicated, slow, and manual.
  13. 13. QA is necessary, but complicated, slow, and manual. And impossible to catch everything... Medium sized app 60+ models Hundreds of flows Thousands of possible page states and permutations Constant feature churn
  14. 14. Also, expen$ive.
  15. 15. Regression tests?
  16. 16. describe 'posting a project', type: :feature do it 'creates a new project' do visit '/' fill_in 'Title', with: 'My project' click_button 'Post Project' expect(page).to have_content('Project created!') expect(page).to have_content('My project') end end
  17. 17. describe 'posting a project', type: :feature do it 'creates a new project' do visit '/' fill_in 'Title', with: 'My project' click_button 'Post Project' expect(page).to have_content('Project created!') expect(page).to have_content('My project') end end Doesn’t fail! The button still technically “works”.
  18. 18. describe 'posting a project', type: :feature do it 'creates a new project' do visit '/' fill_in 'Title', with: 'My project' click_button 'Post Project' # ??? expect(page).to have_content('Project created!') expect(page).to have_content('My project') end end
  19. 19. describe 'posting a project', type: :feature do it 'creates a new project' do visit '/' fill_in 'Title', with: 'My project' click_button 'Post Project' # Assert that…CSS computed style…color of… # is not equal… to… ??? expect(page).to have_content('Project created!') expect(page).to have_content('My project') end end
  20. 20. xkcd.com/55/
  21. 21. The Problem: Pixels are changing, but we only test what’s underneath.
  22. 22. The Problem: Even with all our current tests, we still lack confidence in deploys.
  23. 23. - The Problem - The Solution - How it works
  24. 24. What if we could see every pixel changed in any UI state in every PR?
  25. 25. What if we could test our apps pixel-by-pixel?
  26. 26. “Perceptual diffs” “pdiffs” “Visual diffs”
  27. 27. Perceptual diffs
  28. 28. Perceptual diffs
  29. 29. Perceptual diffs
  30. 30. Perceptual diffs
  31. 31. pdiffs in 30 seconds
  32. 32. Let’s write a visual regression test in 2 minutes!
  33. 33. Awesome right?
  34. 34. Simple uses: Catching visual regressions!
  35. 35. Simple uses: Catching visual regressions! Advanced uses: - CSS refactors / deletions. - Testing style guides. - Safe dependency upgrades. - Visual regression tests for emails. - Testing D3 visualizations.
  36. 36. Simple uses: Catching visual regressions! Advanced uses: - CSS refactors / deletions. - Testing style guides. - Safe dependency upgrades. - Visual regression tests for emails. - Testing D3 visualizations. What I really want: - Visual review alongside code review.
  37. 37. So if this is all so easy, why aren’t we all doing visual testing right now?
  38. 38. If it was easy it wouldn’t be hard. - Definitely somebody
  39. 39. 1. Tooling & workflows 2. Performance 3. Non-deterministic rendering
  40. 40. Non-deterministic rendering
  41. 41. Non-deterministic rendering Animations
  42. 42. Non-deterministic rendering Animations
  43. 43. Non-deterministic rendering Animations
  44. 44. Non-deterministic rendering Animations in Percy (see blog.percy.io)
  45. 45. Non-deterministic rendering Dynamic data
  46. 46. Non-deterministic rendering Old test browsers
  47. 47. Non-deterministic rendering http://alienryderflex.com/sub_pixel/ Sub-pixel antialiasing
  48. 48. Whew.
  49. 49. So... pdiffs are only half the battle
  50. 50. Back to our goal: What if we could see every pixel changed in any UI state in every PR?
  51. 51. That would require: - Fast. Really fast. As fast as your test suite. - Handle complex UI states - Continuously integrated with your workflow on any commit.
  52. 52. - The Problem - The Solution - How it works
  53. 53. describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click end end
  54. 54. describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click end end
  55. 55. describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page, name: 'homepage') end end
  56. 56. describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end Percy describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end
  57. 57. describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end Images?? Percy describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end
  58. 58. describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end Noooooo... Percy describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end
  59. 59. describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end DOM / HTML snapshots! Percy describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end
  60. 60. DOM / HTML snapshots! (SHA256 fingerprinted) describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end Percy describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end
  61. 61. percy-worker a mitmproxy percy-worker a mitmproxy percy-worker a mitmproxy describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end describe 'app features', type: :feature do it 'shows the dropdown menu when clicked' do visit '/' first('.dropdown-toggle').click Percy::Capybara.snapshot(page) end end DOM / HTML snapshots percy-hub percy-api percy-worker a mitmproxy
  62. 62. 1,000,000 visual diffs rendered in Percy as of yesterday!
  63. 63. Quick Percy examples...
  64. 64. Visual testing is: - Possible. - A new way to think about testing. - Helps give you deployment confidence.
  65. 65. Visual testing is: - Possible. - A new way to think about testing. - Helps give you deployment confidence. Much more work to be done to make it a mainstream engineering practice!
  66. 66. One. More. Thing...
  67. 67. describe('groups list page', function() { it('shows correct groups', function() { visit('/groups'); click('.show-groups') expect($('.group-list')).to.be.visible; percySnapshot(); }); }); Sneak peak: percy-js & ember-percy
  68. 68. @percy_io @mikefotinakis Thanks! @fotinakis

×