Testing Mobile JavaScript

Testing Mobile JavaScript John Resig June 2010

I’ve been researching and trying to make sense of the mobile space recently. Started preparations in 2009 (with the creation of TestSwarm).

Cross-browser mobile web development is crazy. (...but not nearly as crazy as it use to be.) I’ve seen things...

A Simple Goal ✤ Started with a simple goal: ✤ To make sure that jQuery works on the most popular mobile platforms and browsers. ✤ As it turns out, it’s really hard to deﬁne the scope of the problem.

Questions: Answered ✤ Three questions that need to be answered before doing mobile development: ✤ What platforms and browsers are popular? ✤ What browsers are capable of supporting modern scripting? ✤ What devices and simulators do I acquire to test with? ✤ To answer those questions we need good data.

Best Stats? ✤ Who has the best statistics on the mobile market? ✤ StatCounter and Gartner seem to be tops. ✤ StatCounter covers billions of hits per month. ✤ Gartner is well-respected and knows sale information well. ✤ AdMob is decent for speciﬁc platforms (iPhone, Android). ✤ That being said: Very little actually-useful information leaks out.

Lack of Information ✤ Right now ﬁnding this information is a challenge. ✤ As a result, developers develop for what’s in front of them:

Platforms

Platform Sales

Hard Question ✤ What versions of those platforms are popular? ✤ No one seems to know, or isn’t talking about it.

Browsers

Hard Question ✤ What versions of those browsers are popular? ✤ No one seems to know, or isn’t talking about it. ✤ Supposedly Yahoo is going to be releasing some information soon, we’ll see. ✤ Right now it’s considered to be a competitive advantage to keep it private.

Testing Strategy ✤ Draw a line in the sand ✤ Buy Devices, Download Simulators ✤ Automate testing (TestSwarm!)

Drawing a Line ✤ Yahoo!s Graded Browser Support technique is optimal. ✤ Qualify which browsers you will support (actively test against) and give them a grade. ✤ A = Full support, C = Fall back to old site, Other = Assume full support.

My God... it’s full of browsers... Mobile Graded Browser Support: Market share + Browser quality

Testing Browsers ✤ Two options: ✤ Buy devices and/or ✤ Download simulators ✤ Simulators are good for most automated testing. ✤ Any sort of interaction testing you’ll want to do on a physical device. ✤ Always good to do a sanity check on a physical device before going live.

Simulators ✤ Simulators are available for most platforms and browsers. ✤ Most simulators require Windows to run (some require Perl or Java) ✤ Some browsers even provide standalone executables (Opera, Fennec). ✤ Getting the simulators running can be a real bear.

Automated Testing ✤ Once you have simulators (or physical devices) up and running you’ll want to interact with them as little as possible. ✤ Automated test execution will be really important. ✤ TestSwarm was developed for this express purpose: Make it easy to push tests out to a large number of clients (even mobile).

Platforms ✤ Symbian (S60 / UIQ) ✤ iPhone OS ✤ Blackberry OS ✤ Android ✤ Windows Mobile ✤ WebOS ✤ Maemo / Meego

Symbian

Symbian S60 ✤ The most popular mobile OS - heavily used by Nokia. ✤ v5.0 is for touch screen devices (Equiv. to Safari 3.1) ✤ v3.0 is the latest for ‘regular’ devices (Equiv. to Safari 2.0) ✤ 2 ‘feature packs’ have been released ✤ The oldest, active, mobile browser that should be supported. ✤ Simulators available on Nokia.com.

Symbian S60 B A B?

Symbian UIQ ✤ A now-dead implementation of the Symbian OS last released in 2008. ✤ Appears to still be in use, some what, but quickly becoming irrelevant. ✤ Uses Opera Mobile (8.6, 8.65) as its built-in browser. ✤ UIQ is dead and Nokia doesn’t distribute the simulator any more. I found a guy in Russia that has some extra copies (totally legit, heh).

Symbian UIQ C C

iPhone OS ✤ Apple’s super-popular mobile OS ✤ Apple is very good about pushing updates, almost all users are on the latest OS version. ✤ Current with Safari 4 ✤ A gotchya: Does not have ﬁxed position support (making it difﬁcult to implement toolbars). Recommend: TouchScroll library. ✤ Simulator is part of the iPhone SDK.

iPhone Simulator No 2.x simulator! B A A

Blackberry OS ✤ Blackberry continues to be enormously popular - and growing rapidly. ✤ Ships with a custom browser, will be switching to a WebKit-based one in 6.0. ✤ Anything older than Blackberry 4.6 is really frightening. ✤ Andrew Dupont calls 4.6 the “rubicon”. ✤ 4.6/4.7 have some strange quirks, 5.0 is pretty decent. ✤ Blackberry provides simulator downloads on their dev site.

Blackberry OS B B B

Android ✤ Rapidly growing OS from Google. ✤ Tons of manufacturers are modifying and shipping it. ✤ Expect Android to grow drastically over the next couple years. ✤ Lack of control has yielded extreme version fracturing. ✤ Simulators are an easy download.

Android A A A

Windows Mobile ✤ Windows Mobile 6.5 is the current release. ✤ Uses IE 6 as its rendering engine. ✤ Window Mobile 7.0 is coming soon. ✤ Uses IE 7 as its rendering engine. :-( :-( :-( :,( ✤ Window Mobile 6.1 is still relatively popular but is based off of IE 4.0 - this is a non-starter. ✤ Windows Mobile 6.5 simulator is an easy download.

Windows Mobile 7.0? :-( F B

WebOS ✤ Palm’s (HP’s?) Mobile OS ✤ Has a minimal amount of traction. ✤ Built off of WebKit, easy to support. ✤ 1.4 simulator ships with 1.3 browser (wtf!?) ✤ I’m testing using a physical Palm Pre. ✤ Palm Pre simulator is an easy download. A

Maemo / Meego ✤ Linux-based OS used for tablets (and potentially phones). Used by Nokia. ✤ Not very popular. ✤ Has a browser called ‘MicroB’ that C uses Gecko. ✤ Mostly interesting as Fennec is capable of running on it. ✤ Simulator requires Linux to run - haven’t ﬁgured it out yet.

Platform-Independent Browsers ✤ A number of browsers target multiple platforms. Some work directly with carriers (Opera, Netfront) whereas others provide downloads (Opera, Fennec). ✤ Opera ✤ Fennec ✤ Netfront ✤ Phonegap (App Platform)

Opera Mobile and Opera Mini ✤ Hugely popular mobile browser (especially Mini). ✤ Available on a large number of platforms. ✤ Even ships as the default browser on some. ✤ Opera Mobile and Opera Mini couldn’t be more different. ✤ Mobile is a full-featured browser. ✤ Mini is a gloriﬁed bitmap viewer. ✤ Pulls from a proxy, no JS executing on client.

Opera Mobile and Opera Mini C A C Simulator is awesome!

Fennec ✤ Codename for Firefox on mobile devices ✤ Released for Maemo, alpha release out for Android (watch this!) A B ✤ Equivalent to the latest releases of Firefox Simulator is awesome!

Netfront ✤ Used on Playstation and N-Gage ✤ Download available for Windows Mobile. ✤ Highly crippled, custom, browser. ✤ Not worth supporting. C ✤ Compatibility can be a real pain.

Phonegap ✤ Not a browser, a way to develop cross-platform applications. ✤ Uses WebKit as the rendering engine. ✤ Use HTML/CSS/JS to develop deployable apps. A ✤ Quite popular, easy to support.

Other Browsers ✤ Not worth the time for such small market share. ✤ Ozone (Uses WebKit) ✤ Openwave (Dead) ✤ Bolt (Uses WebKit) C ✤ JIL (Uses WebKit, Widget Platform)

Summary ✤ What do we need to support? Roughly: ✤ Opera 9.5, 10 ✤ Internet Explorer 6, 7 ✤ Firefox 3.5+ ✤ Safari 2, 3, 4 ✤ (and Blackberry... weeee...) ✤ Don’t assume it’ll work, download the simulators and test!

What Devices to Buy? ✤ For A level: ✤ 1 iPhone 3GS: Latest running 3.x (* Watch for new 4.x iPhone) ✤ 1 iPad: Running 3.2 ✤ 1 Nokia N97 Running S60v5 ✤ 1 Palm Pre: Running 1.4 ✤ 2 Android Phones: HTC Magic (1.5/1.6), Droid Incredible (2.1) (* Watch for new 2.2 Android) ✤ And download the Opera and Fennec Simulators and Phonegap

A -Grade Costs ✤ $315 - iPhone 3GS ✤ $499 - iPad ✤ $370 - Nokia N97 ✤ $399 - Palm Pre ✤ $349 - HTC Magic ✤ $637 - Droid Incredible ✤ Total: $2569

What Devices to Buy? ✤ For B level (in addition to A level): ✤ 1 1st Gen iPhone: Running 2.x ✤ 1 Nokia N96: Running 3.0v2 ✤ 3 Blackberry Devices: Curve 8900 (4.6), 2x Tour 9630 (4.7 and 5.0) ✤ 1 HTC HD2: Running Windows Mobile 6.5

B -Grade Costs ✤ $315 - 1st Gen iPhone ✤ $360 - Nokia N96 ✤ $310 - Curve 8900 ✤ $350 x 2 - Tour 9630 ✤ $600 - HTC HD2 ✤ Total: $2285 ✤ A -Grade + B -Grade: $4854

More Information ✤ Coming Soon at: ✤ http://jquerymobile.com/ ✤ See also: ✤ http://testswarm.com/ and ✤ http://swarm.jquery.org/ ✤ And: ✤ http://quirksmode.org/m/