Your SlideShare is downloading. ×
How I stopped worrying about and loved DumpRenderTree
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

How I stopped worrying about and loved DumpRenderTree

4,428
views

Published on

For the talk at shibuya.js on Test.js

For the talk at shibuya.js on Test.js


0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,428
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
15
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. How I stopped worrying and loved DumpRenderTree 2011/03/08 MORITA Hajime <omo@dodgson.org> http://steps.dodgson.org/Tuesday, March 8, 2011
  • 2. How I stopped worrying and loved DumpRenderTree a bit 2011/03/08 MORITA Hajime <omo@dodgson.org> http://steps.dodgson.org/Tuesday, March 8, 2011
  • 3. • WebKit Test Smell • .js Test • ( )Tuesday, March 8, 2011
  • 4. DumpRenderTree (DRT) DemoTuesday, March 8, 2011
  • 5. DumpRenderTree (DRT) • • HTML ( ) • “Golden File” • OSTuesday, March 8, 2011
  • 6. (-2002) • • DumpRenderTree, LayoutTest • WebKit MacTuesday, March 8, 2011
  • 7. Mac • JavaScript / DOM • ex: ws:// • • OS/GUI Win, Qt, GTK, EFL, Skia ... (Ports) ex: -webkit-transition: • OS, • Mac Tiger, Leopard, SL, Lion ex: drag-n-drop, media, storage • ex: <input type=”speech”> • WYSIWYG contentEditableTuesday, March 8, 2011
  • 8. 3 Test Smells in WebKit • Obscure Tests • Fragile Tests • High Test Maintenance CostTuesday, March 8, 2011
  • 9. (1/3): Obscure Tests Dump? • JavaScript • HTML5-ish (File, XHR, WebSocket, etc.) •Tuesday, March 8, 2011
  • 10. LayoutTestController.dumpAsText() if (window.layoutTestController) { layoutTestController.dumpAsText(); ... } function log(msg) { document.getElementById(log).innerHTML += (msg + “n”); } ... sel = window.getSelection(); actualText = sel.getRangeAt(0).toString(); if (expectedText == actualText) { log("Passed " + testId + " (with contentEditable)"); } else { log("Failed " + testId + " (with contentEditable)"); log(" Expected: " + expectedText); log(" Actual: " + actualText); } ...Tuesday, March 8, 2011
  • 11. : Passed totest_minimum Passed totest_minimum (with contentEditable) Passed totest_before_after Passed totest_before_after (with contentEditable) Passed totest_linebreak_out Passed totest_linebreak_out (with contentEditable) Passed totest_tab_out Passed totest_tab_out (with contentEditable) Passed totest_multiple_whitespaces_out Passed totest_multiple_whitespaces_out (with contentEditable) Passed totest_linebreak_in Passed totest_linebreak_in (with contentEditable) • • PortTuesday, March 8, 2011
  • 12. (1/3): Obscure Tests • LayoutTestController.dumpAsText() • js-test-pre.js/js-test-post.js assertion • dump-as-markup.js w/ dumpAsText()) DOM (built contentEditableTuesday, March 8, 2011
  • 13. (2/3): Fragile Tests Golden File : Over specification • • OS • OS • • GPU • • DRTTuesday, March 8, 2011
  • 14. Tree Redness http://build.webkit.org/waterfall • ( ) •Tuesday, March 8, 2011
  • 15. “Skipped” list Port • (= ) • • ( ) • Bug = TODO list • • Port Golden File ...Tuesday, March 8, 2011
  • 16. Rebaselining Tool (Chromium only) script Download Update Commit (by dev) ... ... expected.txt expected.txt result.zip ... ... Buildbot Local PC SVN ServerTuesday, March 8, 2011
  • 17. (3/3): High Test Maintenance Cost • Golden Files • • • Port • ...Tuesday, March 8, 2011
  • 18. Gardener (for Chromium) • Rebaseline, Skip, Revert, .. •Tuesday, March 8, 2011
  • 19. Reftests (work in progress) • HTML • Port • Mozilla https://developer.mozilla.org/en/Creating_reftest-based_unit_tests • W3CTuesday, March 8, 2011
  • 20. • WebKit Test SmellTuesday, March 8, 2011
  • 21. Backup SlidesTuesday, March 8, 2011
  • 22. 7 Test Smells in WebKit • Obscure Tests • Fragile Tests • Erratic Tests • Manual Intervention • Slow Tests • High Test Maintenance Cost • Developers Not Writing (Unit) TestsTuesday, March 8, 2011
  • 23. (x/x): Erratic Tests aka. Flaky Tests • • OS ( ) • • • ... ( )Tuesday, March 8, 2011
  • 24. (Chromium only) unexpected flaky 1 http://build.chromium.org/p/chromium.webkit/waterfallTuesday, March 8, 2011
  • 25. (x/x): Manual Intervention JS • • DOM • ex: •Tuesday, March 8, 2011
  • 26. DRT API // from editing/input/emacs-ctrl-o.html ... if (window.eventSender) eventSender.keyDown("o", ["ctrlKey"]); ... // from editing/spelling/.../spelling-attribute-at-child.js ... var marked = layoutTestController.hasSpellingMarker(1, 2); ... // from fast/speech/input-onspeechchange-event.html ... if (window.layoutTestController && window.eventSender) { layoutTestController.addMockSpeechInputResult(....); ...Tuesday, March 8, 2011
  • 27. ...Tuesday, March 8, 2011
  • 28. (x/x): Slow Tests • 2 • HTML/JSTuesday, March 8, 2011
  • 29. ... • GUI : “Active Window”, “Clipboard”, ... • (ex. localStorage) • ( ) • Chromium WebKit • Window-less by design • ...Tuesday, March 8, 2011
  • 30. $ ./Tools/Scripts/run-webkit-tests --help Usage: old-run-webkit-tests [options] [testdir|testpath ...] .... --slowest Report the 10 slowest tests ....Tuesday, March 8, 2011