Measuring Performance,
JavaScript Games,
and Distributed Testing
John Resig
http://ejohn.org - http://twitter.com/jeresig

Measuring
Performance

Analyzing Performance
Optimizing performance is a huge
✦
concern: Faster code = happy users!
Measure execution time
✦
Loop the code a few times
✦
Measure the difference:
✦
✦ (new Date).getTime();

Stack Profiling
jQuery Stack Profiler
✦
Look for problematic methods and plugins
✦
http://ejohn.org/blog/deep-profiling-
✦
jquery-apps/

Accuracy of
JavaScript Time
We’re measuring the performance of
JavaScript from within JavaScript!
http://ejohn.org/blog/accuracy-of-javascript-time/

15ms intervals ONLY!
Error Rate of 50-750%!

Performance Tools
How can we get good numbers?
✦
We have to go straight to the source: Use
✦
the tools the browsers provide.
Tools:
✦
✦ Firebug Profiler
✦ Safari Profiler
✦ (Part of Safari 4)
✦ IE 8 Profiler

Firebug Profiler

Safari 4 Profiler

IE 8 Profiler

FireUnit
A simple JavaScript test suite embedded in
✦
Firebug.
http://fireunit.org/
✦

FireUnit Profile Data
{
fireunit.getProfile(); \"time\": 8.443,
\"calls\": 611,
\"data\":[
{
\"name\":\"makeArray()\",
\"calls\":1,
\"percent\":23.58,
\"ownTime\":1.991,
\"time\":1.991,
\"avgTime\":1.991,
\"minTime\":1.991,
\"maxTime\":1.991,
\"fileName\":\"jquery.js (line 2059)\"
},
// etc.
http://ejohn.org/blog/function-call-profiling/ ]}

Complexity Analysis
Analyze complexity rather than raw time
✦
jQuery Call Count Profiler (uses FireUnit)
✦
Method Calls Big-O
.addClass(\"test\"); 542 6n
.addClass(\"test\"); 592 6n
.removeClass(\"test\"); 754 8n
.removeClass(\"test\"); 610 6n
.css(\"color\", \"red\"); 495 5n
.css({color: \"red\", border: \"1px 887 9n
solid red\"});
.remove(); 23772 2n+n2
.append(\"<p>test</p>\"); 307 3n

Complexity Analysis
Reducing call count helps to reduce
✦
complexity
Results for 1.3.3:
✦
Method Calls Big-O
.remove(); 298 3n
.html(\"<p>test</p>\"); 507 5n
.empty(); 200 2n
http://ejohn.org/blog/function-call-profiling/

Why JavaScript
Games Are HARD
to Build

Browser-Based Games
No installation required
✦
(Will be able to play at work!)
✦

Why not Flash?
JavaScript works everywhere
✦
✦ Desktop
✦ iPhone / Mobile Device
✦ Wii
✦ OLPC
JavaScript is an open technology
✦

Goals of a game
Should be Multiplayer
✦
Can’t be casually cheated
✦
Work well everywhere
✦
Addictive
✦

Why Multiplayer?
Incredibly addictive
✦
✦ No longer “alone”
✦ Enticed to continue playing

3 Styles of Games
Strategy
✦
Intelligence
✦
Accuracy
✦

Strategy Games
Require a lot of time to think
✦
Generally last over a couple hours or days
✦
Replayability
✦
Hard to cheat
✦

WarFish
http://warfish.net/
✦

Nile Online
http://www.playnileonline.com/
✦

Strategy Games
Very server-side heavy
✦
✦ Most logic hidden from the user
Hard to cheat
✦
✦ Casual cheaters can’t change values
✦ Dedicated cheaters have to write full AI

Intelligence Games
Player’s intelligence/knowledge challenged
✦
Games could be quick or slow-paced
✦
Easy to cheat
✦
✦ Casual cheaters can open dictionary /
encyclopedia for answers

Word Twist
http://wordtwist.org/
✦

Babble
http://playbabble.com/
✦

Iron Sudoku
http://ironsudoku.com/
✦

Speed/Accuracy Games
Require low latency
✦
Fast-paced and addictive
✦
JavaScript completely fails
✦
✦ Garbage Collection cycles freeze the
browser
None, or few, Accuracy-centric JavaScript
✦
games
Experienced coders can easily cheat
✦
✦ (A bot to hit the keys at the right time)

Guitar Hero
http://ejohn.org/apps/hero/

Guitar Hero
Heavily dependent upon accuracy
✦
✦ (Hit the right notes at the right time)
Garbage collection cycles absolutely kill
✦
the game
Rendering the play area is also difficult
✦
✦ And impossible in all browsers.
✦ (Use HTML 5 Canvas Element)

Failures on All Ends
Strategy: Slow, secret server-side code
✦
Intelligence: Easily cheatable
✦
Accuracy: Too hard to implement
✦
Optimal solution would be a combination
✦
of the above.
JavaScript games can’t be like other games,
✦
have to be unique.

What can make a fun game?
Quick play
✦
Points
✦
High Scores
✦
Head-to-head competition
✦

Wordsplay
Real-time Boggle
✦
Head-to-head with other players
✦
http://www.wordsplay.net/

Tringo
Tetris + Bingo (based on a Second Life
✦
game)
http://ejohn.org/tringo/

DeepLeap
Fast-paced Scrabble-like game
✦
Speed + Intelligence + Strategy
✦
http://deepleap.org/

vs. Cheating
All words are recorded with exact time
✦
information
Game is “played back” on the server to
✦
verify score integrity using Server-Side JS
This data can be used to simulate a multi-
✦
player experience!

DeepLeap
Works in multiple languages
✦
✦ Dictionaries pulled from OpenOffice,
can build a Spanish version in < 5sec
Players can challenge each other head-to-
✦
head
Score multiplier (carry over from Guitar
✦
Hero)

Distributed Testing

Choose Your Browsers

Cost / Benefit
IE 7 IE 6 FF 3 Safari 3 Opera 9.5
Cost Benefit
Draw a line in the sand.

Graded Support
Yahoo Browser Compatibility

Browser Support Grid
IE Firefox Safari Opera Chrome
Previous 6.0 2.0 3.0 9.5
Current 7.0 3.0 3.2 9.6 1.0
Next 8.0 3.1 4.0 10.0 2.0
jQuery Browser Support

Browser Support Grid
IE Firefox Safari Opera Chrome
Previous 3.0 9.5
6.0 2.0
Current 7.0 3.0 3.2 9.6 1.0
Next 8.0 3.1 4.0 2.0
10.0
jQuery 1.3 Browser Support

The Scaling Problem
The Problem:
✦
✦ jQuery has 6 test suites
✦ Run in 11 browsers
✦ (Not even including multiple platforms!)
All need to be run for every commit,
✦
patch, and plugin.
JavaScript testing doesn’t scale well.
✦

Distributed Testing
Hub server
✦
Clients connect and help run tests
✦
A simple JavaScript client that can be run
✦
in all browsers
✦ Including mobile browsers!
✦ TestSwarm

FF 3.5 FF 3.5 FF 3.5
IE 6
IE 6
FF 3 IE 6
Op 9
FF 3
IE 7
TestSwarm
IE 7
Test Suite Test Suite Test Suite

Manual Testing
Push tests to users who follow pre-defined
✦
steps
Answer ‘Yes’/’No’ questions which are
✦
pushed back to the server.
An effective way to distribute manual test
✦
load to dozens of clients.

TestSwarm.com
Incentives for top testers (t-shirts, books)
✦
Will be opening for alpha testing very soon
✦
Help your favorite JavaScript library
✦
become better tested!
http://testswarm.com
✦

Questions
Contact:
✦
✦ John Resig
✦ http://ejohn.org/
✦ http://twitter.com/jeresig