Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
How I stopped worrying  and  loved DumpRenderTree                                                   2011/03/08            ...
How I stopped worrying  and  loved DumpRenderTree                                                          a bit          ...
• WebKit                         Test Smell                    •    .js   Test                    •                 (   )T...
DumpRenderTree (DRT)                 DemoTuesday, March 8, 2011
DumpRenderTree (DRT)                    •                    • HTML                                                (   )  ...
(-2002)                    •                         •   DumpRenderTree, LayoutTest                    •    WebKit      Ma...
Mac      •       JavaScript / DOM      •       ex: ws://      •                                         •   OS/GUI        ...
3 Test Smells in WebKit                    • Obscure Tests                    • Fragile Tests                    • High Te...
(1/3): Obscure Tests                                                   Dump?                    • JavaScript              ...
LayoutTestController.dumpAsText()     if (window.layoutTestController) {         layoutTestController.dumpAsText();       ...
:  Passed           totest_minimum  Passed           totest_minimum (with contentEditable)  Passed           totest_before...
(1/3): Obscure Tests                    • LayoutTestController.dumpAsText()                    • js-test-pre.js/js-test-po...
(2/3): Fragile Tests                Golden File          : Over specification                    •                         ...
Tree Redness                               http://build.webkit.org/waterfall            •                              (  ...
“Skipped” list                Port                    •                             (=           )                    •   ...
Rebaselining Tool                             (Chromium only)                                       script                ...
(3/3): High Test Maintenance Cost                    •                 Golden Files                    •                  ...
Gardener (for Chromium)                 •       Rebaseline, Skip, Revert, ..                 •Tuesday, March 8, 2011
Reftests (work in progress)                    •                HTML                    •    Port                    •    ...
• WebKit                         Test SmellTuesday, March 8, 2011
Backup SlidesTuesday, March 8, 2011
7 Test Smells in WebKit                 • Obscure Tests                 • Fragile Tests                 • Erratic Tests   ...
(x/x): Erratic Tests      aka. Flaky Tests                    •                    •    OS   (           )                ...
(Chromium only)                                   unexpected flaky 1                          http://build.chromium.org/p/c...
(x/x): Manual Intervention                             JS                    •                    •    DOM                ...
DRT      API    // from editing/input/emacs-ctrl-o.html    ...        if (window.eventSender)            eventSender.keyDo...
...Tuesday, March 8, 2011
(x/x): Slow Tests                    •    2                    •        HTML/JSTuesday, March 8, 2011
...                    •    GUI                                   : “Active Window”, “Clipboard”, ...                    •...
$ ./Tools/Scripts/run-webkit-tests --help     Usage: old-run-webkit-tests [options] [testdir|testpath ...]       ....     ...
Upcoming SlideShare
Loading in …5
×

How I stopped worrying about and loved DumpRenderTree

5,525 views

Published on

For the talk at shibuya.js on Test.js

  • Be the first to comment

How I stopped worrying about and loved DumpRenderTree

  1. 1. How I stopped worrying and loved DumpRenderTree 2011/03/08 MORITA Hajime <omo@dodgson.org> http://steps.dodgson.org/Tuesday, March 8, 2011
  2. 2. How I stopped worrying and loved DumpRenderTree a bit 2011/03/08 MORITA Hajime <omo@dodgson.org> http://steps.dodgson.org/Tuesday, March 8, 2011
  3. 3. • WebKit Test Smell • .js Test • ( )Tuesday, March 8, 2011
  4. 4. DumpRenderTree (DRT) DemoTuesday, March 8, 2011
  5. 5. DumpRenderTree (DRT) • • HTML ( ) • “Golden File” • OSTuesday, March 8, 2011
  6. 6. (-2002) • • DumpRenderTree, LayoutTest • WebKit MacTuesday, March 8, 2011
  7. 7. Mac • JavaScript / DOM • ex: ws:// • • OS/GUI Win, Qt, GTK, EFL, Skia ... (Ports) ex: -webkit-transition: • OS, • Mac Tiger, Leopard, SL, Lion ex: drag-n-drop, media, storage • ex: <input type=”speech”> • WYSIWYG contentEditableTuesday, March 8, 2011
  8. 8. 3 Test Smells in WebKit • Obscure Tests • Fragile Tests • High Test Maintenance CostTuesday, March 8, 2011
  9. 9. (1/3): Obscure Tests Dump? • JavaScript • HTML5-ish (File, XHR, WebSocket, etc.) •Tuesday, March 8, 2011
  10. 10. LayoutTestController.dumpAsText() if (window.layoutTestController) { layoutTestController.dumpAsText(); ... } function log(msg) { document.getElementById(log).innerHTML += (msg + “n”); } ... sel = window.getSelection(); actualText = sel.getRangeAt(0).toString(); if (expectedText == actualText) { log("Passed " + testId + " (with contentEditable)"); } else { log("Failed " + testId + " (with contentEditable)"); log(" Expected: " + expectedText); log(" Actual: " + actualText); } ...Tuesday, March 8, 2011
  11. 11. : Passed totest_minimum Passed totest_minimum (with contentEditable) Passed totest_before_after Passed totest_before_after (with contentEditable) Passed totest_linebreak_out Passed totest_linebreak_out (with contentEditable) Passed totest_tab_out Passed totest_tab_out (with contentEditable) Passed totest_multiple_whitespaces_out Passed totest_multiple_whitespaces_out (with contentEditable) Passed totest_linebreak_in Passed totest_linebreak_in (with contentEditable) • • PortTuesday, March 8, 2011
  12. 12. (1/3): Obscure Tests • LayoutTestController.dumpAsText() • js-test-pre.js/js-test-post.js assertion • dump-as-markup.js w/ dumpAsText()) DOM (built contentEditableTuesday, March 8, 2011
  13. 13. (2/3): Fragile Tests Golden File : Over specification • • OS • OS • • GPU • • DRTTuesday, March 8, 2011
  14. 14. Tree Redness http://build.webkit.org/waterfall • ( ) •Tuesday, March 8, 2011
  15. 15. “Skipped” list Port • (= ) • • ( ) • Bug = TODO list • • Port Golden File ...Tuesday, March 8, 2011
  16. 16. Rebaselining Tool (Chromium only) script Download Update Commit (by dev) ... ... expected.txt expected.txt result.zip ... ... Buildbot Local PC SVN ServerTuesday, March 8, 2011
  17. 17. (3/3): High Test Maintenance Cost • Golden Files • • • Port • ...Tuesday, March 8, 2011
  18. 18. Gardener (for Chromium) • Rebaseline, Skip, Revert, .. •Tuesday, March 8, 2011
  19. 19. Reftests (work in progress) • HTML • Port • Mozilla https://developer.mozilla.org/en/Creating_reftest-based_unit_tests • W3CTuesday, March 8, 2011
  20. 20. • WebKit Test SmellTuesday, March 8, 2011
  21. 21. Backup SlidesTuesday, March 8, 2011
  22. 22. 7 Test Smells in WebKit • Obscure Tests • Fragile Tests • Erratic Tests • Manual Intervention • Slow Tests • High Test Maintenance Cost • Developers Not Writing (Unit) TestsTuesday, March 8, 2011
  23. 23. (x/x): Erratic Tests aka. Flaky Tests • • OS ( ) • • • ... ( )Tuesday, March 8, 2011
  24. 24. (Chromium only) unexpected flaky 1 http://build.chromium.org/p/chromium.webkit/waterfallTuesday, March 8, 2011
  25. 25. (x/x): Manual Intervention JS • • DOM • ex: •Tuesday, March 8, 2011
  26. 26. DRT API // from editing/input/emacs-ctrl-o.html ... if (window.eventSender) eventSender.keyDown("o", ["ctrlKey"]); ... // from editing/spelling/.../spelling-attribute-at-child.js ... var marked = layoutTestController.hasSpellingMarker(1, 2); ... // from fast/speech/input-onspeechchange-event.html ... if (window.layoutTestController && window.eventSender) { layoutTestController.addMockSpeechInputResult(....); ...Tuesday, March 8, 2011
  27. 27. ...Tuesday, March 8, 2011
  28. 28. (x/x): Slow Tests • 2 • HTML/JSTuesday, March 8, 2011
  29. 29. ... • GUI : “Active Window”, “Clipboard”, ... • (ex. localStorage) • ( ) • Chromium WebKit • Window-less by design • ...Tuesday, March 8, 2011
  30. 30. $ ./Tools/Scripts/run-webkit-tests --help Usage: old-run-webkit-tests [options] [testdir|testpath ...] .... --slowest Report the 10 slowest tests ....Tuesday, March 8, 2011

×