Qa extreme 2011 eran kinsbruner

Trends and accentuations around mobile application testing eranki@matrix.co.il

Agenda- Testing challenges- Existing mobile Operating systems- Mobile applications testing- Porting- Testing tools- Real life examples- Trends and accentuations in the mobile world- Q&A

Testing ChallengesDevice battery consumptionDevice memory constrainsScreen size/resolution, Touch screenForm factors (Slider, Clamshell, Block, etc.)Performance of the deviceFragmentation among device OS’sPrice of defects

Testing Challenges - ContinueSecurity element in device testing (Apple UDID’s,prompts for unsigned apps etc.)Incoming events during testing, Coex testing(Combinations, complex scenario’s, launch camera etc.)Signal strengthBack/Exit buttons behavior among devicesInstallation method (OTA, Serial cable, SDK’s)Device Access point configurationVarious OS versions complianceVarious languages/locales

Mobile devices operating systems – Trends (Gartner, September 2010)

Existing mobile Operating systems• Symbian (Nokia)• iOS (Apple)• Android (Google)• Blackberry OS (RIM)• Windows Phone 7 (Microsoft)• J2ME proprietary OS (SE/Samsung/LG) ?• Palm/WebOS (Palm Pre) ? http://www.calcalist.co.il/internet/articles/0,7340,L -3519738,00.html)

Symbian OS• Symbian OS was originally developed by Symbian Ltd, which in 2008 was acquired by Nokia.• Symbian OS had almost no UI Layer, Most UI design was made by third parties – S60 by Nokia – UIQ by SonyEricsson – MOAP by NTT DoCoMo• In 2008 with Nokia acquiring Symbian Ltd. the Symbian Foundation was created and the three UIs were contributed to it in order to create a Unified Symbian Platform.

Symbian OS - Continue• Today two streams of Symbian UIs are available. – Symbian^3 is used by Nokia’s High-end smartphones. – S60 is used in Mid-Range smartphones.• It contains associated libraries, user interface, frameworks and reference implementations of common tools.• Symbian runs on ARM processors (x86 unofficial).• Symbian applications are packed in SISX files and requires Symbian Signed certificate to be commercially deployed on a handset.• SEMC recently announced that it is deserting Symbian OS to switch to the more popular OS’s – Android & Windows Phone 7• Nokia recently announced that it is also deserting Symbian and is about to move to Windows Phone 7/8 OS for its future devices.• Note: NTT DocoMo and Fujitsu announced a new Tablet/Smartphone running Windows 7 OS (Tablet) and Symbian (Mobile) on one platform (Details)

Android OS• Android is a mobile operating system developed by Google and is based upon a modified version of the Linux kernel. It was initially developed by Android Inc. (a firm purchased by Google)• Its OS versions are usually incremented alphabetically and named upon famous cookies/cakes • 1.5 cupcake • 1.6 donut • 2.0 éclair • 2.2 Froyo • 2.3 Gingerbread • 3.0 Honeycomb • ? IceCream• What is ADB ?

Android SDK Emulator

Android – Screen resolution method• In Android, Google developed a new way of handling different screen resolutions, in order to save porting time of applications among various of Android phones • Basically Android OS divides its screen size to 4 different sizes and 4 different densities and the platform transparently handles the rendering of the layouts at the actual screen size • Small, Normal, Large and Extra Large • ldpi (low), mdpi (medium), hdpi (high), and xhdpi (extra high)• Detail explanation of how it’s done can be found in a post at TACT/Blogix web page: http://qa.blogix.co.il/2011/04/16/%D7%90%D7%99%D7%9A- %D7%9E%D7%A2%D7%A8%D7%9B%D7%AA- %D7%94%D7%94%D7%A4%D7%A2%D7%9C%D7%94- %D7%90%D7%A0%D7%93%D7%A8%D7%95%D7%90%D7%99%D7%93- %D7%93%D7%95%D7%90%D7%92%D7%AA- %D7%9C%D7%94%D7%A6%D7%92%D7%94/• Top rated Android apps for 2011 - http://www.web-designers-directory.org/articles/top- rated-android-applications-for-2011-20.html

Blackberry OS• BlackBerry OS is a proprietary mobile operating system, developed by Research In Motion for its BlackBerry line of smart phone handheld devices.• BlackBerry applications are developed in Java with RIM’s SDK• The operating system provides multi-tasking, server-side push service and supports specialized input devices: • trackwheel (7100i, OS 4.1.0) • trackball (9630 Tour, OS 5.0) • trackpad (9780 Bold, OS 6.0) • touchscreen (9800 Torch, OS 6.0).• BlackBerry Various Emulators for testing - Download

Apple iOS• iOS formerly known as iPhone OS is Apple’s proprietary mobile operating system. Developed originally for the iPhone, it has since been shipped on the iPod Touch and iPad as well. • iOS 1.0 – iPhone • iOS 2.0 – iPhone 3G • iOS 3.0 – iPhone 3Gs • iOS 4.0 – iPhone 4• iOS 4.3.3 was released on 5-May, 2011 and is the version used on latest iPhone 4 devices• MAC OS tools: • Xcode, Interface Builder, iPhone Simulator

Apple iOS - Continue• There are few free tools which can allow a developer to develop/test his application for iPhone: • For Windows OS there are: • AirPlay https://www.airplaysdk.com/downloadsall.php • C:Airplay SDK4.4examplesgameskartz • Titanium http://www.appcelerator.com/products/download/ (JavaScript developing for iPhone, Android) • Mono Touch Develop Android and iPhone apps with C# on windows - not free (http://www.go-mono.com/mono- downloads/download.html (http://mono-tools.com/download/) • For MAC OS iPhone SDK (Xcode) provided by Apple with full development environment (C++ or Objective C languages)• Top 10 iPhone apps by TIME magazine: http://www.time.com/time/specials/packages/completelist/0,29569,2044480,00.html• Xcode Demo’s • http://www.youtube.com/watch?v=miIBUq4mGSw • http://www.youtube.com/watch?v=y5XYKl3iHDM&feature=relmfu

Apple iOS – Dev. & Testing

Windows Mobile• Windows Mobile is a mobile operating system developed by Microsoft for use in smart phones (non-touch) and PPC (touch) devices.• WM features a suite of basic applications developed using the Microsoft Windows API. It is designed to be somewhat similar to desktop versions of Windows, feature-wise and aesthetically. Additionally, third-party software can be purchased via the Windows Marketplace• The OS is now obsolete and The Last version is "Windows Mobile 6.5.3“ (based on the Windows CE 5.2 kernel) • Sample devices: • HTC HD2 • Sony Ericsson XPERIA X2

Windows Phone 7• Windows Phone 7 is the successor to MicrosoftWindows Mobile platform s (based on Windows CE 6.0 R3 kernel).• Windows phone 7 is built from the ground up and features no backward compatibility to the previous 6.5 version. – Does require certification and signing of app to be publicly deployed• Some of the features are – Live Tiles, – Social networking integration, – multi-touch Support, – internet explorer mobile, – Office 2010 Mobile, – Search & Maps using Bing, – Xbox live gaming – Metro UI L&F(http://www.microsoft.com/windowsphone/en-us/default.aspx)

Windows Phone 7 – Dev. & TestingWindows Phone 7 development tools are distributed free by Microsoft.Important and up to date data is constantly updated at http://blogs.microsoft.co.il/blogs/msdn/• Windows Phone Developer Tools http://www.microsoft.com/express/Downloads/#2010-Visual-Phone• Expression Blend is a more user friendly application used to design only the User Interface Experience• There is a newly developed application market for windows phone 7 platform App Hub http://create.msdn.com/en-US/ • Preliminary application market for Israel (Yalla Apps - http://www.yallaapps.com/ )• A training kit for Windows Phone 7 new comers can be obtained from http://msdn.microsoft.com/en- us/wp7trainingcourse.aspx Zune Sync music and video for WP7 devices**• Note: The tools above require Windows 7 to be installed

Windows Phone 7 – Dev. & Testing Visual Studio Express 2010

Windows Phone 7 – Expression Blend 4Rapid Prototyping without writing code

Windows Phone 7 & SilverLight• Short demo http://isvis.com/work/silverlight/3d/tracking/• Silverlight is the application development platform for Windows Phone 7.• Silverlight for Windows Phone supports core Silverlight capabilities in managed .NET code with XAML including: – High quality video and audio using a wide range of codecs, DRM and IIS Smooth Streaming – Deep Zoom for enhanced reading and photo browsing experiences – Vector and Bitmap Graphics and animation• Note: In order to deploy your app on a real phone, the phone must be registered within MSFT (Emulator is the preferred preliminary way of testing)

Windows Phone 7 & SilverLight• Silverlight can also access the unique capabilities of the phone including: – Hardware acceleration for video and graphics – Accelerometer for motion sensing – Multi-touch – Camera and microphone – Location awareness – Push notifications – Native phone functionality• Silverlight can also utilize the XNA Framework for Audio capture and playback, Media Library Access, and even accessing Xbox LIVE.• Currently Windows Phone 7 uses silverlight 3.0, soon silverlight 4.0 with additional features will be released as part of the next MSFT SDK (MANGO)• Tombstone Vs. Dormant (MANGO Release)

Windows Phone 7 Demos

Windows Phone 7 Leading Phones Samsung Focus - http://www.microsoft.com/windowsphone/en-us/buy/7/phones.aspx?id=1536&id=1536#detail=1536 HTC Arrive - http://www.microsoft.com/windowsphone/en-us/buy/7/phones.aspx?id=1536&id=1536#detail=1684 HTC HD7 - http://www.microsoft.com/windowsphone/en-us/buy/7/phones.aspx?id=1536&id=1536#detail=1569 LG Quantum - http://www.microsoft.com/windowsphone/en-us/buy/7/phones.aspx?id=1536&id=1536#detail=1537 HTC Surround - http://www.microsoft.com/windowsphone/en-us/buy/7/phones.aspx?id=1536&id=1536#detail=1538 Dell Venue Pro - http://www.microsoft.com/windowsphone/en-us/buy/7/phones.aspx?id=1536&id=1536#detail=1568

Mobile Application Testing - Methodologies• The testing methodology mostly depends on the type of application.• There are 3 main mobile applications: 1) Stand alone Taking into account the mobile phone limitations, such applications shall be tested by Functional tests, Usability tests, performance tests. 2) Distributed Testing for both client and server side alone and together is required (e.g- Instant messaging application)

Mobile Application Testing - Methodologies3) Web based In a Web-based application the client is an Internet browser,and the main functionality is built on the server-side. Testing Web basedapplications require the following type of testing: • Functional testing • Compatibility testing • Content testing • Performance testing • Load testing • Security testing • Server-side testing (Web server, application server) • Database testing

Mobile Application Testing – Proposed strategy• Do invest in a device emulator – It’s a well instrumented testing environment, usually more than 1 skin – Cost effective – Bypasses real network – Easy to debug• Take advantage of a real remote device solution (Nokia, Device Anywhere, Perfectomobile) – “Hire” target device/s per hour to perform real device testing as needed• Don’t completely avoid testing on real device on real networkConclusion: Seek a solution which combines remote manual testing on real device, emulated device testing and automation tests wherever possible.

Mobile Application Testing – Porting Concepts 1 of 3 Since there are variety of devices from each OEM, each has different constraints (Memory, screen size, heap size, form factor, camera, touch screen, installation path etc.) – it is a must to plan the application to be as much portable as possible during the design phases. Not taking into account the devices family and the above constrains will require massive porting and long development cycles when starting the deployment Having configurable pre-compiled options which vary between devices would ease the porting and enable to use the same code (Mostly) for all devices, with only per device configuration options

Mobile Application Testing – Porting Concepts 2 of 3 Having more than one “gold” target device during development phase is also an important requirement Using automation solution as mentioned earlier and SDK’s is also an advantage. There are special application which can provide quite easily the list of supported features per device (TasePhone Java application e.g.) Using emulation environments per device is cheap and cost effective during development Taking into account the resources used by the application is important (there are tools who can shrink the entire folder of resources into a small size file which can be used by the application – BoomFs)

Mobile Application Testing – Porting Concepts 3 of 3 When a new or pre-launched phone should be supported by the application developers there are few things which should be taken into account: Phone characteristics (Screen size, OS, memory, form factor, supported features such as camera and others) Can be retrieved for most phones through GSM Arena site How close the phone is to the lead device within a family which is already supported If the phone represents a new family this might have implications on the porting time in opposed to a new member to an existing family Is there a way to upgrade phone OS to overcome firmware defects (Cables, contact with the OEM etc.) It would always be good to have at least 2 devices from this kind to be able to reproduce defects, compare performance and firmwares. How the application works in comparison to other apps on the device (Closing method, events reaction and more)

Mobile Application Testing – Bug Trackers• There are few free/cheap tools (Clients) which can be installed on your mobile device and allow you to submit bugs on the fly as you test your application – JIRA Mobile client for Android Beta (and soon iPhone) http://blogs.atlassian.com/jira/2010/05/jira-android-bug-tracking-mobile-phone- device-20100520b.html – FogBugs https://www.fogcreek.com/fogbugz/mobile/ • Supports iPhone, Android, Blackberry

Mobile Application Testing – Battery Consumption• As mentioned earlier, the battery of the mobile phone is limited and the application developers must take into account the application battery usage, performance and take relevant steps and optimizations in order to assure that the application consumes the minimal battery resources as needed – There are monitoring tools which can be used to check the battery usage (Not always specific per application but this can be narrowed to the specific application vendor needs) For iPhone Battery Life Pro http://itunes.apple.com/us/app/battery-life-pro-all-in-1/id373041141?mt=8 Battery Led

Mobile Application Testing – Battery Consumption - Continue• For Android OS there is e.g. the ‘Power Tutor’ tool – http://gigaom.com/mobile/android-power-consumption-app/• For windows Mobile – http://www.freewarepocketpc.net/ppc-download-wr-tools-resinfo.html

Mobile Application Testing – Leading automation toolsThere are 2-3 main players in the device test automation worldwith proven results and ROI Perfectomobile http://perfectomobile.com/?key=adwords2&gclid=CM_G0eWipaQCFR5 N5QodNUXJ6w Remote Access - http://perfectomobile.com/portal/cms/Products/Web_Access_To_Real_Handsets.html?k ey=adwords2 Automation - http://perfectomobile.com/portal/cms/Products/automated_testing.html?key=adwords2

Mobile Application Testing – PerfectoMobile

Mobile Application Testing – Leading automation toolsDevice Anywhere - http://www.deviceanywhere.com/mobile-application-testing-overview.html

Mobile Application Testing – Device Anywhere

Mobile Application Testing – Leading automation tools/SDK’s• Experitest - http://experitest.com/ • Provides Emulator and real device automation solution (Record/replay and code generation) • Supports the following OS: • Android • iOS • Blackberry • Windows Mobile • Symbian • Exports code to C#, Java, Python, QTP-VBScript, Perl• Robotium - http://code.google.com/p/robotium/ • robust automatic black-box test cases for Android applications (Run your application as an Android Junit test on Android SDK emulator)

Mobile Application Testing – Experitest

Mobile Application Testing – SDK’s & Emulator’s In order to test application developed for mobile, often the vendors would prefer touse as many devices SDK’s and Emulators to get as close to reality L&F andfunctionality before deploying first on device which has limited debugging capabilitiesin opposed to the SDK’sFew SDK’s which are free to use NetBNeans IDE which has simple integration for most common OEM SDK’s (Motorola, Nokia, SEMC) - http://netbeans.org Nokia specific Emulation environment - http://www.forum.nokia.com/Library/Tools_and_downloads/Archive.xhtml SEMC specific Emulation environment 1) http://developer.sonyericsson.com/wportal/devworld/technology/android/developer-tools 2) http://developer.sonyericsson.com/wportal/devworld/technology/windows-mobile/xperia-x2 Motorola specific Emulation environment - http://developer.motorola.com/docstools/sdks/archive/c975ADK/ Sun Java Wirelsss Toolkit (J2ME SDK for application developers) - http://java.sun.com/products/sjwtoolkit/download-2_5_1.html

Mobile Application Testing – SDK’s & Emulator’sJ2ME Sun Java Wireless SDK - http://www.oracle.com/technetwork/java/download-135801.html (Emulation/debugging environment) NetBeans SDK – IDE with OEM SDK’s plug in integration for pre-device testing and debugging Mobile Junit by SEMC Jbenchmark (http://www.jbenchmark.com/tools.jsp) LG - http://developer.lgmobile.com/lge.mdn.tnd.RetrieveDocAndTools.dev?object Type=T#1 Samsung - http://innovator.samsungmobile.com/down/cnts/toolSDK.detail.view.do?platf ormId=1&cntsId=9460

Mobile Application Testing – SDK’s & Emulator’sSymbian Symbian signed pre-certification tools (http://developer.symbian.org/wiki/index.php/Symbian_Signed_Tools) Windows Mobile (I assume it will be EOL’d in the near future due to WindowsPhone 7 platform) Test Complete (Allows connection from device to Microsoft ‘ActiveSync’/’Windows Mobile device center Synchronization’ SW). http://www.automatedqa.com/products/testcomplete/testing-pda-apps/ Windows mobile emulators - http://www.microsoft.com/downloads/en/details.aspx?FamilyID=38c46aa8- 1dd7-426f-a913-4f370a65a582&DisplayLang=en

Real Life Experience overviewDriver and Firmware for WLAN/BT/GPS testing on mobile platform runningAndroid/Linux (IOP testing)Android SDK – Use Android SDK to pre-test your application on a variety ofEmulators (Uses the latest open source file system) Google CTS ADB Shell

Mobile Trends and ForecastMobile Devices Begin to Replace Credit Cards2011 looks like the year when, in some countries at least, usingyour phone as a credit card for lower value purchases willbecome a reality. Google recently announced that NFC (NearField Communication) technology will be supported in the nextrelease of Android – 2.3 or “Gingerbread”; HTC Flyer phone announced to support NFC within a yearSocial networking based devices (Recently HTC announced thatthey launched social networking specific devices - ChaCha andSalsa)

Mobile Trends and Forecast - ContinueSmartphones invading the enterprise (Security threats will be big factors indetermining which companies aggressively deploy smartphones)Mobile OS battles will continue (Android will become the leading Smartphoneplatform for at least the next four years, according to the latest data from IDC. Theanalyst firm estimates that Android will account for 45 per cent of the globalSmartphone market by 2015, up from under 40 per cent today (30-Mar-11). RIM andiOS will see more modest growth and a slight fall in overall market share to around 15per centSmartphone and tablet convergence (growing number of Smartphone manufacturerswill feel pressured to introduce tablet computers, and will likely do so this year.Offering one but not the other will become an intensifying threat to their businesses)

Mobile Trends and Forecast - ContinueCloud in Mobile – Apple set to launch cloud-based music serviceahead of Google (In this model, people can upload and store alltheir music to Apple servers, and launch it from any device orshare it with other users (Using an iTunes account)). Apple cuts Amazon & Google by signing the music companies

Applications Store Ranking

Appendix A – Sample certificationsSymbian Signed certification – Mandatory for any S60 V3 application which isintended for commercial launch https://www.symbiansigned.com/app/page Java Verified Certification – Sun’s guidelines for Java/Java ME compliantapplications http://javaverified.com Windows Mobile application signing - http://msdn.microsoft.com/en-us/windowsmobile/dd569132.aspx & Windows Phone 7 Certification JSR TCK tests – Specific JSR compatibility tests which are mandatory to any JSROEM implementer - https://jtharness.dev.java.net/ (open source test harness) – For acomplete list of JSR’s seehttp://www.jcp.org/en/jsr/platform?listBy=1&listByType=platform Java Device test framework – specific JSR test suite with functional tests (Providedto reduce fragmentation and increase quality -http://java.sun.com/products/javadevice/index.jsp & https://jdtf.dev.java.net)

052-2424314ryonish@tact.co.il

http://10.150.200.102	12
http://10.150.200.76	8
http://www.techgig.com	2

Qa extreme 2011 eran kinsbruner

by Eran Kinsbrunner

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 22

Statistics

Qa extreme 2011 eran kinsbruner Presentation Transcript