Testing Mobile JavaScript John Resig
June 2010

I’ve been researching and trying
to make sense of the mobile space recently. Started preparations in 2009 (with the creation of TestSwarm).

Cross-browser mobile web development is
crazy. (...but not nearly as crazy as it use to be.) I’ve seen things...

A Simple Goal ✤ Started
with a simple goal: ✤ To make sure that jQuery works on the most popular mobile platforms and browsers. ✤ As it turns out, it’s really hard to define the scope of the problem.

Questions: Answered ✤ Three questions
that need to be answered before doing mobile development: ✤ What platforms and browsers are popular? ✤ What browsers are capable of supporting modern scripting? ✤ What devices and simulators do I acquire to test with? ✤ To answer those questions we need good data.

Best Stats? ✤ Who has
the best statistics on the mobile market? ✤ StatCounter and Gartner seem to be tops. ✤ StatCounter covers billions of hits per month. ✤ Gartner is well-respected and knows sale information well. ✤ AdMob is decent for specific platforms (iPhone, Android). ✤ That being said: Very little actually-useful information leaks out.

Lack of Information ✤ Right
now finding this information is a challenge. ✤ As a result, developers develop for what’s in front of them:

Platforms

Platform Sales

Hard Question ✤ What versions
of those platforms are popular? ✤ No one seems to know, or isn’t talking about it.

Browsers

Hard Question ✤ What versions
of those browsers are popular? ✤ No one seems to know, or isn’t talking about it. ✤ Supposedly Yahoo is going to be releasing some information soon, we’ll see. ✤ Right now it’s considered to be a competitive advantage to keep it private.

Testing Strategy ✤ Draw a
line in the sand ✤ Buy Devices, Download Simulators ✤ Automate testing (TestSwarm!)

Drawing a Line ✤ Yahoo!s
Graded Browser Support technique is optimal. ✤ Qualify which browsers you will support (actively test against) and give them a grade. ✤ A = Full support, C = Fall back to old site, Other = Assume full support.

My God... it’s full of
browsers... Mobile Graded Browser Support: Market share + Browser quality

Testing Browsers ✤ Two options:
✤ Buy devices and/or ✤ Download simulators ✤ Simulators are good for most automated testing. ✤ Any sort of interaction testing you’ll want to do on a physical device. ✤ Always good to do a sanity check on a physical device before going live.

Simulators ✤ Simulators are available
for most platforms and browsers. ✤ Most simulators require Windows to run (some require Perl or Java) ✤ Some browsers even provide standalone executables (Opera, Fennec). ✤ Getting the simulators running can be a real bear.

Automated Testing ✤ Once you
have simulators (or physical devices) up and running you’ll want to interact with them as little as possible. ✤ Automated test execution will be really important. ✤ TestSwarm was developed for this express purpose: Make it easy to push tests out to a large number of clients (even mobile).

Platforms ✤ Symbian (S60 /
UIQ) ✤ iPhone OS ✤ Blackberry OS ✤ Android ✤ Windows Mobile ✤ WebOS ✤ Maemo / Meego

Symbian

Symbian S60 ✤ The most
popular mobile OS - heavily used by Nokia. ✤ v5.0 is for touch screen devices (Equiv. to Safari 3.1) ✤ v3.0 is the latest for ‘regular’ devices (Equiv. to Safari 2.0) ✤ 2 ‘feature packs’ have been released ✤ The oldest, active, mobile browser that should be supported. ✤ Simulators available on Nokia.com.

Symbian S60
B
A B?

Symbian UIQ ✤ A now-dead
implementation of the Symbian OS last released in 2008. ✤ Appears to still be in use, some what, but quickly becoming irrelevant. ✤ Uses Opera Mobile (8.6, 8.65) as its built-in browser. ✤ UIQ is dead and Nokia doesn’t distribute the simulator any more. I found a guy in Russia that has some extra copies (totally legit, heh).

Symbian UIQ
C C

iPhone OS ✤ Apple’s super-popular
mobile OS ✤ Apple is very good about pushing updates, almost all users are on the latest OS version. ✤ Current with Safari 4 ✤ A gotchya: Does not have fixed position support (making it difficult to implement toolbars). Recommend: TouchScroll library. ✤ Simulator is part of the iPhone SDK.

iPhone Simulator No 2.x simulator!
B A A

Blackberry OS ✤ Blackberry continues
to be enormously popular - and growing rapidly. ✤ Ships with a custom browser, will be switching to a WebKit-based one in 6.0. ✤ Anything older than Blackberry 4.6 is really frightening. ✤ Andrew Dupont calls 4.6 the “rubicon”. ✤ 4.6/4.7 have some strange quirks, 5.0 is pretty decent. ✤ Blackberry provides simulator downloads on their dev site.

Blackberry OS
B
B
B

Android ✤ Rapidly growing OS
from Google. ✤ Tons of manufacturers are modifying and shipping it. ✤ Expect Android to grow drastically over the next couple years. ✤ Lack of control has yielded extreme version fracturing. ✤ Simulators are an easy download.

Android
A
A
A

Windows Mobile ✤ Windows Mobile
6.5 is the current release. ✤ Uses IE 6 as its rendering engine. ✤ Window Mobile 7.0 is coming soon. ✤ Uses IE 7 as its rendering engine. :-( :-( :-( :,( ✤ Window Mobile 6.1 is still relatively popular but is based off of IE 4.0 - this is a non-starter. ✤ Windows Mobile 6.5 simulator is an easy download.

Windows Mobile 7.0? :-( F
B

WebOS ✤ Palm’s (HP’s?) Mobile
OS ✤ Has a minimal amount of traction. ✤ Built off of WebKit, easy to support. ✤ 1.4 simulator ships with 1.3 browser (wtf!?) ✤ I’m testing using a physical Palm Pre. ✤ Palm Pre simulator is an easy download. A

Maemo / Meego ✤ Linux-based
OS used for tablets (and potentially phones). Used by Nokia. ✤ Not very popular. ✤ Has a browser called ‘MicroB’ that C uses Gecko. ✤ Mostly interesting as Fennec is capable of running on it. ✤ Simulator requires Linux to run - haven’t figured it out yet.

Platform-Independent Browsers ✤ A number
of browsers target multiple platforms. Some work directly with carriers (Opera, Netfront) whereas others provide downloads (Opera, Fennec). ✤ Opera ✤ Fennec ✤ Netfront ✤ Phonegap (App Platform)

Opera Mobile and Opera Mini
✤ Hugely popular mobile browser (especially Mini). ✤ Available on a large number of platforms. ✤ Even ships as the default browser on some. ✤ Opera Mobile and Opera Mini couldn’t be more different. ✤ Mobile is a full-featured browser. ✤ Mini is a glorified bitmap viewer. ✤ Pulls from a proxy, no JS executing on client.

Opera Mobile and Opera Mini
C A C Simulator is awesome!

Fennec ✤ Codename for Firefox
on mobile devices ✤ Released for Maemo, alpha release out for Android (watch this!) A B ✤ Equivalent to the latest releases of Firefox Simulator is awesome!

Netfront ✤ Used on Playstation
and N-Gage ✤ Download available for Windows Mobile. ✤ Highly crippled, custom, browser. ✤ Not worth supporting. C ✤ Compatibility can be a real pain.

Phonegap ✤ Not a browser,
a way to develop cross-platform applications. ✤ Uses WebKit as the rendering engine. ✤ Use HTML/CSS/JS to develop deployable apps. A ✤ Quite popular, easy to support.

Other Browsers ✤ Not worth
the time for such small market share. ✤ Ozone (Uses WebKit) ✤ Openwave (Dead) ✤ Bolt (Uses WebKit) C ✤ JIL (Uses WebKit, Widget Platform)

Summary ✤ What do we
need to support? Roughly: ✤ Opera 9.5, 10 ✤ Internet Explorer 6, 7 ✤ Firefox 3.5+ ✤ Safari 2, 3, 4 ✤ (and Blackberry... weeee...) ✤ Don’t assume it’ll work, download the simulators and test!

What Devices to Buy? ✤
For A level: ✤ 1 iPhone 3GS: Latest running 3.x (* Watch for new 4.x iPhone) ✤ 1 iPad: Running 3.2 ✤ 1 Nokia N97 Running S60v5 ✤ 1 Palm Pre: Running 1.4 ✤ 2 Android Phones: HTC Magic (1.5/1.6), Droid Incredible (2.1) (* Watch for new 2.2 Android) ✤ And download the Opera and Fennec Simulators and Phonegap

A -Grade Costs ✤ $315
- iPhone 3GS ✤ $499 - iPad ✤ $370 - Nokia N97 ✤ $399 - Palm Pre ✤ $349 - HTC Magic ✤ $637 - Droid Incredible ✤ Total: $2569

What Devices to Buy? ✤
For B level (in addition to A level): ✤ 1 1st Gen iPhone: Running 2.x ✤ 1 Nokia N96: Running 3.0v2 ✤ 3 Blackberry Devices: Curve 8900 (4.6), 2x Tour 9630 (4.7 and 5.0) ✤ 1 HTC HD2: Running Windows Mobile 6.5

B -Grade Costs ✤ $315
- 1st Gen iPhone ✤ $360 - Nokia N96 ✤ $310 - Curve 8900 ✤ $350 x 2 - Tour 9630 ✤ $600 - HTC HD2 ✤ Total: $2285 ✤ A -Grade + B -Grade: $4854

More Information ✤ Coming Soon
at: ✤ http://jquerymobile.com/ ✤ See also: ✤ http://testswarm.com/ and ✤ http://swarm.jquery.org/ ✤ And: ✤ http://quirksmode.org/m/