0
Measuring Performance,
  JavaScript Games,
and Distributed Testing
                  John Resig
 http://ejohn.org - http:/...
Measuring
Performance
Analyzing Performance
    Optimizing performance is a huge
✦
    concern: Faster code = happy users!
    Measure execution...
Stack Profiling
    jQuery Stack Profiler
✦

    Look for problematic methods and plugins
✦

    http://ejohn.org/blog/deep-...
Accuracy of
  JavaScript Time
We’re measuring the performance of
 JavaScript from within JavaScript!


  http://ejohn.org/...
15ms intervals ONLY!

     Error Rate of 50-750%!
Performance Tools
    How can we get good numbers?
✦

    We have to go straight to the source: Use
✦
    the tools the br...
Firebug Profiler
Safari 4 Profiler
IE 8 Profiler
FireUnit
    A simple JavaScript test suite embedded in
✦
    Firebug.
    http://fireunit.org/
✦
FireUnit Profile Data

                                            {
 fireunit.getProfile();                              quo...
Complexity Analysis
             Analyze complexity rather than raw time
         ✦

             jQuery Call Count Profile...
Complexity Analysis
               Reducing call count helps to reduce
           ✦
               complexity
            ...
Why JavaScript
Games Are HARD
    to Build
Browser-Based Games
    No installation required
✦

    (Will be able to play at work!)
✦
Why not Flash?
    JavaScript works everywhere
✦
    ✦ Desktop
    ✦ iPhone / Mobile Device
    ✦ Wii
    ✦ OLPC

    Java...
Goals of a game
    Should be Multiplayer
✦

    Can’t be casually cheated
✦

    Work well everywhere
✦

    Addictive
✦
Why Multiplayer?
    Incredibly addictive
✦
    ✦ No longer “alone”
    ✦ Enticed to continue playing
3 Styles of Games
    Strategy
✦

    Intelligence
✦

    Accuracy
✦
Strategy Games
    Require a lot of time to think
✦

    Generally last over a couple hours or days
✦

    Replayability
✦...
WarFish




        http://warfish.net/
    ✦
Nile Online




      http://www.playnileonline.com/
  ✦
Strategy Games
    Very server-side heavy
✦
    ✦ Most logic hidden from the user

    Hard to cheat
✦
    ✦ Casual cheate...
Intelligence Games
    Player’s intelligence/knowledge challenged
✦

    Games could be quick or slow-paced
✦

    Easy to...
Word Twist




      http://wordtwist.org/
  ✦
Babble




      http://playbabble.com/
  ✦
Iron Sudoku




      http://ironsudoku.com/
  ✦
Speed/Accuracy Games
    Require low latency
✦

    Fast-paced and addictive
✦

    JavaScript completely fails
✦
    ✦ Ga...
Guitar Hero




    http://ejohn.org/apps/hero/
Guitar Hero
    Heavily dependent upon accuracy
✦
    ✦ (Hit the right notes at the right time)

    Garbage collection cy...
Failures on All Ends
    Strategy: Slow, secret server-side code
✦

    Intelligence: Easily cheatable
✦

    Accuracy: To...
What can make a fun game?
    Quick play
✦

    Points
✦

    High Scores
✦

    Head-to-head competition
✦
Wordsplay




    Real-time Boggle
✦

    Head-to-head with other players
✦

          http://www.wordsplay.net/
Tringo




    Tetris + Bingo (based on a Second Life
✦
    game)
                              http://ejohn.org/tringo/
DeepLeap



    Fast-paced Scrabble-like game
✦

    Speed + Intelligence + Strategy
✦




                               ...
vs. Cheating
    All words are recorded with exact time
✦
    information
    Game is “played back” on the server to
✦
   ...
DeepLeap
    Works in multiple languages
✦
    ✦ Dictionaries pulled from OpenOffice,
      can build a Spanish version in <...
Distributed Testing
Choose Your Browsers
Cost / Benefit




  IE 7     IE 6          FF 3    Safari 3   Opera 9.5
                  Cost          Benefit


        ...
Graded Support




   Yahoo Browser Compatibility
Browser Support Grid
           IE      Firefox   Safari   Opera Chrome


Previous   6.0       2.0      3.0      9.5


Cur...
Browser Support Grid
           IE       Firefox   Safari   Opera Chrome


Previous                       3.0      9.5
   ...
The Scaling Problem
    The Problem:
✦
    ✦ jQuery has 6 test suites
    ✦ Run in 11 browsers
    ✦ (Not even including m...
Distributed Testing
    Hub server
✦

    Clients connect and help run tests
✦

    A simple JavaScript client that can be...
FF 3.5 FF 3.5 FF 3.5
                                                  IE 6
                                              ...
Manual Testing
    Push tests to users who follow pre-defined
✦
    steps
    Answer ‘Yes’/’No’ questions which are
✦
    p...
TestSwarm.com
    Incentives for top testers (t-shirts, books)
✦

    Will be opening for alpha testing very soon
✦

    H...
Questions
    Contact:
✦
    ✦ John Resig
    ✦ http://ejohn.org/
    ✦ http://twitter.com/jeresig
Performance, Games, and Distributed Testing in JavaScript
Performance, Games, and Distributed Testing in JavaScript
Performance, Games, and Distributed Testing in JavaScript
Performance, Games, and Distributed Testing in JavaScript
Upcoming SlideShare
Loading in...5
×

Performance, Games, and Distributed Testing in JavaScript

9,143

Published on

A talk I gave at JSConf09 in Washington D.C. in April 2009.

1 Comment
12 Likes
Statistics
Notes
No Downloads
Views
Total Views
9,143
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
354
Comments
1
Likes
12
Embeds 0
No embeds

No notes for slide

Transcript of "Performance, Games, and Distributed Testing in JavaScript"

  1. 1. Measuring Performance, JavaScript Games, and Distributed Testing John Resig http://ejohn.org - http://twitter.com/jeresig
  2. 2. Measuring Performance
  3. 3. Analyzing Performance Optimizing performance is a huge ✦ concern: Faster code = happy users! Measure execution time ✦ Loop the code a few times ✦ Measure the difference: ✦ ✦ (new Date).getTime();
  4. 4. Stack Profiling jQuery Stack Profiler ✦ Look for problematic methods and plugins ✦ http://ejohn.org/blog/deep-profiling- ✦ jquery-apps/
  5. 5. Accuracy of JavaScript Time We’re measuring the performance of JavaScript from within JavaScript! http://ejohn.org/blog/accuracy-of-javascript-time/
  6. 6. 15ms intervals ONLY! Error Rate of 50-750%!
  7. 7. Performance Tools How can we get good numbers? ✦ We have to go straight to the source: Use ✦ the tools the browsers provide. Tools: ✦ ✦ Firebug Profiler ✦ Safari Profiler ✦ (Part of Safari 4) ✦ IE 8 Profiler
  8. 8. Firebug Profiler
  9. 9. Safari 4 Profiler
  10. 10. IE 8 Profiler
  11. 11. FireUnit A simple JavaScript test suite embedded in ✦ Firebug. http://fireunit.org/ ✦
  12. 12. FireUnit Profile Data { fireunit.getProfile(); quot;timequot;: 8.443, quot;callsquot;: 611, quot;dataquot;:[ { quot;namequot;:quot;makeArray()quot;, quot;callsquot;:1, quot;percentquot;:23.58, quot;ownTimequot;:1.991, quot;timequot;:1.991, quot;avgTimequot;:1.991, quot;minTimequot;:1.991, quot;maxTimequot;:1.991, quot;fileNamequot;:quot;jquery.js (line 2059)quot; }, // etc. http://ejohn.org/blog/function-call-profiling/ ]}
  13. 13. Complexity Analysis Analyze complexity rather than raw time ✦ jQuery Call Count Profiler (uses FireUnit) ✦ Method Calls Big-O .addClass(quot;testquot;); 542 6n .addClass(quot;testquot;); 592 6n .removeClass(quot;testquot;); 754 8n .removeClass(quot;testquot;); 610 6n .css(quot;colorquot;, quot;redquot;); 495 5n .css({color: quot;redquot;, border: quot;1px 887 9n solid redquot;}); .remove(); 23772 2n+n2 .append(quot;<p>test</p>quot;); 307 3n
  14. 14. Complexity Analysis Reducing call count helps to reduce ✦ complexity Results for 1.3.3: ✦ Method Calls Big-O .remove(); 298 3n .html(quot;<p>test</p>quot;); 507 5n .empty(); 200 2n http://ejohn.org/blog/function-call-profiling/
  15. 15. Why JavaScript Games Are HARD to Build
  16. 16. Browser-Based Games No installation required ✦ (Will be able to play at work!) ✦
  17. 17. Why not Flash? JavaScript works everywhere ✦ ✦ Desktop ✦ iPhone / Mobile Device ✦ Wii ✦ OLPC JavaScript is an open technology ✦
  18. 18. Goals of a game Should be Multiplayer ✦ Can’t be casually cheated ✦ Work well everywhere ✦ Addictive ✦
  19. 19. Why Multiplayer? Incredibly addictive ✦ ✦ No longer “alone” ✦ Enticed to continue playing
  20. 20. 3 Styles of Games Strategy ✦ Intelligence ✦ Accuracy ✦
  21. 21. Strategy Games Require a lot of time to think ✦ Generally last over a couple hours or days ✦ Replayability ✦ Hard to cheat ✦
  22. 22. WarFish http://warfish.net/ ✦
  23. 23. Nile Online http://www.playnileonline.com/ ✦
  24. 24. Strategy Games Very server-side heavy ✦ ✦ Most logic hidden from the user Hard to cheat ✦ ✦ Casual cheaters can’t change values ✦ Dedicated cheaters have to write full AI
  25. 25. Intelligence Games Player’s intelligence/knowledge challenged ✦ Games could be quick or slow-paced ✦ Easy to cheat ✦ ✦ Casual cheaters can open dictionary / encyclopedia for answers
  26. 26. Word Twist http://wordtwist.org/ ✦
  27. 27. Babble http://playbabble.com/ ✦
  28. 28. Iron Sudoku http://ironsudoku.com/ ✦
  29. 29. Speed/Accuracy Games Require low latency ✦ Fast-paced and addictive ✦ JavaScript completely fails ✦ ✦ Garbage Collection cycles freeze the browser None, or few, Accuracy-centric JavaScript ✦ games Experienced coders can easily cheat ✦ ✦ (A bot to hit the keys at the right time)
  30. 30. Guitar Hero http://ejohn.org/apps/hero/
  31. 31. Guitar Hero Heavily dependent upon accuracy ✦ ✦ (Hit the right notes at the right time) Garbage collection cycles absolutely kill ✦ the game Rendering the play area is also difficult ✦ ✦ And impossible in all browsers. ✦ (Use HTML 5 Canvas Element)
  32. 32. Failures on All Ends Strategy: Slow, secret server-side code ✦ Intelligence: Easily cheatable ✦ Accuracy: Too hard to implement ✦ Optimal solution would be a combination ✦ of the above. JavaScript games can’t be like other games, ✦ have to be unique.
  33. 33. What can make a fun game? Quick play ✦ Points ✦ High Scores ✦ Head-to-head competition ✦
  34. 34. Wordsplay Real-time Boggle ✦ Head-to-head with other players ✦ http://www.wordsplay.net/
  35. 35. Tringo Tetris + Bingo (based on a Second Life ✦ game) http://ejohn.org/tringo/
  36. 36. DeepLeap Fast-paced Scrabble-like game ✦ Speed + Intelligence + Strategy ✦ http://deepleap.org/
  37. 37. vs. Cheating All words are recorded with exact time ✦ information Game is “played back” on the server to ✦ verify score integrity using Server-Side JS This data can be used to simulate a multi- ✦ player experience!
  38. 38. DeepLeap Works in multiple languages ✦ ✦ Dictionaries pulled from OpenOffice, can build a Spanish version in < 5sec Players can challenge each other head-to- ✦ head Score multiplier (carry over from Guitar ✦ Hero)
  39. 39. Distributed Testing
  40. 40. Choose Your Browsers
  41. 41. Cost / Benefit IE 7 IE 6 FF 3 Safari 3 Opera 9.5 Cost Benefit Draw a line in the sand.
  42. 42. Graded Support Yahoo Browser Compatibility
  43. 43. Browser Support Grid IE Firefox Safari Opera Chrome Previous 6.0 2.0 3.0 9.5 Current 7.0 3.0 3.2 9.6 1.0 Next 8.0 3.1 4.0 10.0 2.0 jQuery Browser Support
  44. 44. Browser Support Grid IE Firefox Safari Opera Chrome Previous 3.0 9.5 6.0 2.0 Current 7.0 3.0 3.2 9.6 1.0 Next 8.0 3.1 4.0 2.0 10.0 jQuery 1.3 Browser Support
  45. 45. The Scaling Problem The Problem: ✦ ✦ jQuery has 6 test suites ✦ Run in 11 browsers ✦ (Not even including multiple platforms!) All need to be run for every commit, ✦ patch, and plugin. JavaScript testing doesn’t scale well. ✦
  46. 46. Distributed Testing Hub server ✦ Clients connect and help run tests ✦ A simple JavaScript client that can be run ✦ in all browsers ✦ Including mobile browsers! ✦ TestSwarm
  47. 47. FF 3.5 FF 3.5 FF 3.5 IE 6 IE 6 FF 3 IE 6 Op 9 FF 3 IE 7 TestSwarm IE 7 Test Suite Test Suite Test Suite
  48. 48. Manual Testing Push tests to users who follow pre-defined ✦ steps Answer ‘Yes’/’No’ questions which are ✦ pushed back to the server. An effective way to distribute manual test ✦ load to dozens of clients.
  49. 49. TestSwarm.com Incentives for top testers (t-shirts, books) ✦ Will be opening for alpha testing very soon ✦ Help your favorite JavaScript library ✦ become better tested! http://testswarm.com ✦
  50. 50. Questions Contact: ✦ ✦ John Resig ✦ http://ejohn.org/ ✦ http://twitter.com/jeresig
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×