Your SlideShare is downloading. ×
Advanced Mobile Testing - Ravindran Antonysamy
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Advanced Mobile Testing - Ravindran Antonysamy

4,150
views

Published on

Ever wonder why some mobile applications loved by millions ? Ever wonder why few mobile applications stand on top of the list of million apps ? Its the quality and usability of the product that drives …

Ever wonder why some mobile applications loved by millions ? Ever wonder why few mobile applications stand on top of the list of million apps ? Its the quality and usability of the product that drives them to the top of the -already-crowded- marketplace. do they stand on top just because the makers do app store marketing and SEO ? or Its just the genre of the application that is need of the hour ?

How can you deliver an application that is useful and also usable ? How can you deliver an application that won't crash on most of the phones and run flawlessly ? How can you provide the maximum test coverage of your mobile app, that is practically impossible to achieve ?

Do I have to perform expensive usability testing ? do I have to buy mobile clouds ? do I have to perform emulator testing along with real device testing ? Why should I have a social media channel that engage real users ? Why should I crowd source my mobile application testing ? upto what extend I can crowd source ? how much lab testing is sufficient and upto what % of defects can be captured using lab testing ? do I have to consider the power consumed by my mobile app ? what level of mobile application testing automation I shall be able to perform ? what are all the tools I shall be using ?

This webinar will try to address most of the questions.
Mobile testing OS / Market Coverage / Challenges in Mobile testing World
Available Testing techniques (Lab testing, Beta user testing (Internal and external) , Usability testing, Field testing, Functional Automation testing)
Emulator testing Vs Real device testing
Practical Usability testing
Mobile Testing Functional Automation (Open Source Tools)
Is it possible to provide 100% coverage in Mobile testing ?


Who should attend: Mobile Application testers, Mobile Handheld testers, Mobile Usability testers, Product Managers, Test and Tech Leads

Published in: Education, Technology, Business

1 Comment
5 Likes
Statistics
Notes
No Downloads
Views
Total Views
4,150
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
260
Comments
1
Likes
5
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Advanced Mobile Testing !! 1 Ravindran Antonysamy, QA Head, Nimbuzz !!! http://www.slideshare.net/ravindranantonysamy/advanced-mobile-testing Youtube Video of the Webinar : http://www.youtube.com/watch?feature=player_embedded&v=-LYtG45HXb8 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 2. 2 Who Am I Target Audience Mobile Handset / Application ECO System / Smartphone market OEM / Handset testing Vs Application testing Native Vs Hybrid Vs Web Apps Mobile App Test Strategy - Emulator Testing Vs Real Device - Ambassador testing - Crowdsourcing - Social Media channels - Mobile Clouds - Usability testing Test Cases MOS in Mobile Application testing Power Consumption testing Mobile Application testing Automation Tools – Calaba.sh MobileQAZone.com Links Ultimate Cheat sheet for Mobile Testing Nimbuzz !! 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 3. 3 Who Am I & What I love 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 4. 4 Who Am I & What I love 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 5. Disc:$ I may be wrong ! I may be missing things ! Feel free to disagree with me and argue with me ! 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com 5
  • 6. 6 Target Audience Fresh College Pass Outs Software Testers and ‘Wanna’ be software / mobile testers Automation Testers Test Leads, Tech Leads Product and Project Managers Friends and Family. 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 7. 7 Some Mobile Facts Total Number of Mobile Phones in the World : 6 billion Mobile Phone internet usage is surpassed PC internet users – 2013 Total Mobile apps in Android Play store 900,000, Apple store 800,000 Smartphone users spend at least 30 hours in their phones in a month 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 8. 8 Mobile Handset / Application ECO System – P1 OEM’s: Samsung, Apple, Nokia, ZTE, Blackberry, Huawei, HTC, Motorola, Sony, Toshiba, Lenovo, Sagem, JCB, Network Equipment : Ericsson, NSN, Huawei, Juniper Chipset / Board Vendors : ST-Ericsson, Qualcomm, MTK, M- Star, Broadcom, Atheros, NVIDIA, Intel, TI, Freescale, Service Provides : Three, Vodafone, Airtel, MTN, Singtel, Telstra, AT&T Mobile Operating Systems : Android, iOS, Windows Phone, Java - MIDP, Blackberry, Symbian, Bada (deprecated), Web OS (Probably deprecated), Jolla, Ubuntu Edge, Firefox OS App Makers, Developers : Whatsapp, Nimbuzz, Real Racing (EA Games), Runtastic, Running with Friends (Zynga) Ad Networks : AdMob, MadsOne, Smaato, AdNear, Innteractive, Ripple, iAds, Enpocket, MobYD, L eadBolt, AdInch, AdFonic, Amobee 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 9. 9 Mobile Handset / Application ECO System – P2 Appstores : Google Play, Apple App Store, Windows Phone Marketplace, Blackberry App World, OVI Store, getjar, Mobango, Amazon app store Mobile App test crowdsourcing : Mob4hire.com, utest.com, Testcovery.co, 99tests.com Hybrid App Making Tools : Titanium (Appcelerator), Phone Gap, Sencha Touch, RHO Mobile Authorities / Forums / Bodies : 3GPP, 3GPP2, OMA, OHA, ITU-T, GSMA, Wi-Fi Alliance, Bluetooth SIG, NFC Forum, GCF, PTCRB, FCS, IEEE Test Automation Tools (Open source / Paid / Native / Cross Platform) : Robotium, Calaba.sh, Appium.io, SeeTest, MonkeyTalk, Telerik, Android Monkey, UI Automator for Android, Frank, KIF, Anteater, TouchTest, UI Automation for iOS, Zucchini, Test Plant (egg plant), ZAP-Fix, Silk Test for Mobile, UFT Mobile, Selenium Web driver, Other : LIMO, Linaro, MobileIRON (Enterprise Mobility), Mobile Antivirus, Mob VAS, Flurry, Localytics9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 10. 10 OEM Handset testing Vs Application Testing OEM Handset testing performed by OEM’s. Mobile Application Testing (Native / Hybrid / Web) Performed by Application Vendor / Service Companies or Developers. 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 11. OEM Testing Types – Drop Testing 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com 11
  • 12. OEM Testing Types – Keypad stress test 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com 12
  • 13. OEM Testing Types – Bending test 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com 13 Upto 100 nuwtons applied
  • 14. OEM Testing Types – Drop water test 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com 14
  • 15. OEM Testing Types – Dust box test 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com 15
  • 16. OEM Testing Types – Twisting test 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com 16
  • 17. OEM Testing Types – Wearing test 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com 17
  • 18. Field Testing (with tools like TEMS) Call Performance Testing (with tools like Optis Innowireless) Battery testing (Test of the battery itself) Native Application testing (SMS, PB, Native apps) Power consumption testing (Power consumed by various apps / features / calling / media playback – Using Agilent / Anritsu signaling testers) Bluetooth, Wireless LAN, NFC / RF Integration Testing Video / Call Quality testing (with Tools like SwissQual) 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com 18 OEM Testing Types – Other Tests
  • 19. 19 OEM / Handset testing Vs Application testing - Only around the OS / Device - No need to test the Native Apps (like SMS, Camera) - Only around the App - Functional testing, Integration testing, Performance testing, Security testing, Usability testing, UAT 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 20. 20 Native Vs Hybrid Vs Web Apps - Native Apps : Made by Native SDK’s (Android SDK, Xcode, WP SDK, BB SDK) - Hybrid Apps : Made by Hybrid tools (Phone Gap, Appcelerater Titanium, RHOMobile) - Web Apps (HTML 5 / CSS) 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 21. 21 Mobile App Test Strategy - How many devices I should buy ? - My DUT’s should have what OS Version ? - Do I have to test in Emulator or Device or both ? - Do I have to use Ambassadors or Social Media ? - Do I have to crowd source my app testing ? - How can I share my test app to users ? (test flight / appbundle / knappsack / Hockeyapp) - What to do with the comments I receive in app stores ? - What to do with the crash reports from users on Dev dashboard ? (http://postimg.org/image/yom95zfmp/) - Do I have to use automation ? Upto what extend ? What tools to use ? - Performance testing ? Benchmarking ? - Security testing ? Encryption ? - Do I have to perform Usability testing ? Is that expensive ? - Do I have to use device Clouds ? How much It would cost me ? - How many SIM Cards I should have for testing ? - Will social media helps me in testing my mobile app ? 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 22. 22 Test Cases and Test Preparation What kind of test cases I should write ? What is MMI (Man Machine Interface) Why I should have a Pre-Condition ? What is Interoperability testing ? (OMA Test Fest) SIM Cards, Data Packs Interrupts Considering Mobile QoS Test Data / Screen Sizes (http://www.quirksmode.org/m/tests/widthtest.html) 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 23. 23 MOS in Mobile Application testing 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com Parameter Score User Interface 5 Performance / Sluggishness 4 Functionality 4.5 Connectivity 4 Usability 4 Call Quality 4 Battery Consumption 4 Circulate this kind of sheet within the organization, friends, family before releasing the app. Make sure everyone feels as great as your developers. you are good to go !
  • 24. 24 Power Consumption testing Old Method - Start the timer - Charge the mobile 100% - Perform different scenarios - Note down the battery bars - Once battery dead - End the timer New Method - Start the timer - Charge the mobile 100% - Perform different scenarios - Check the settings for battery % in each scenario - Once battery dead, - End the timer Fool Proof Method Use Agilent 8960 or Anritsu 8475A to setup the data acquisition unit / dummy battery / power supply setup !! 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com Tools Power Tutor GSAM Battery Life Pro EZ Battery Life
  • 25. 25 Practical Usability Testing - What is sample size and Why Its important ? - What is test protocol in Usability testing ? - Who will be the users ? - Who will assess the results ? - What is Feedback forms ? 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 26. 26 Mobile Application testing Automation - Is it possible to automate all the test cases in a mobile app ? - Is there reliable open source tools available ? - Hardware / software setup ? - Do I need to know Java to be a mobile application automation tester ? 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 27. 27 Tools – Calaba.sh - Easy to setup - you can write test scripts in readable english - Easy to update - Easy to maintain the scripts - Open source community support Watch this : http://www.youtube.com/watch?v=mvzGAs9aD20 – Mancef http://www.youtube.com/watch?v=9FAjxMLyTco – Karl Krukow 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 28. 28 Mobile Browser Testing Tools - iPhony - W3C Mobile OK Checker - http://validator.w3.org/mobile/) - iPADPeek - Modify Headers Add On in Firefox - Adobe Device Central CS5 - Google Mobilizer - http://www.google.com/gwt/n - Opera Mini http://www.opera.com/developer/opera-mini-simulator 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 29. 29 MobileQAZone.com A Community dedicated for mobile testers. Discuss new tools. Chat with other mobile testers. Solve problems. Contribute. 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 30. 30 Some Links www.gsmarena.com www.gsma.com www.openmobilealliance.org www.quirksmode.org http://www.mobileqazone.com/ www.3gpp.org http://calaba.sh http://appium.io http://phonegap.com https://sellanapp.com/ http://www.google.com 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 31. 31 Ultimate Cheat Sheet for Mobile Testing How to start learning mobile testing ? Start by pointing your browser to android.com, download the SDK and Eclipse. Do read the documentation (Its very comprehensive) – Create your first AVD (Android Virtual device) and run your first mobile application in the emulator. You are all set. How to test iPhone applications ? You must have an iPhone. Or else, a MAC & XCode (IDE for making iPhone Apps). Do I have to buy a tool for mobile testing automation ? The Answer is No. you can manage with Open source tools. (I am not working for a mobile test automation tool making company yet, so this is how I will answer) 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 32. 32 Ultimate Cheat Sheet for Mobile Testing How can I take my application logs when I am performing android manual testing ? Install eclipse (download it from android.com where you can get the full SDK pack.) Keep USB debugging ON in your mobile, go to DDMS view in your eclipse, you will be able to see the logs. You can also use an app called alogcat (search it in play.google.com), It will also show all the logs. How to run adb from anywhere in the command prompt? From Android platform location, Add platform-tools path (where ADB is located) to your environment variables – path. Now see the magic. (go to command prompt and type adb) How can I stress test my android mobile application ? Use Android Monkey. In your command prompt say adb shell monkey –p package name –v verbos (Example : adb shell monkey –p com.nimbuzz –v 50000) 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 33. 33 Ultimate Cheat Sheet for Mobile Testing I don’t have a MAC. How can I take logs for my iPhone App ? Use iPhone Configuration Utility for Windows. (google It I say) How to Sceen cast my Android application from my device to my PC ? Use Androidscreencast. Do I need developers support for testing iPhone applications ? Yes, they need to add your UUID into their build system / certificate. Only then you will be able to test the iPhone application coming from the developer. 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 34. 34 Ultimate Cheat Sheet for Mobile Testing What is the test setup for Windows Phones ? In-order to Install the .xap (WP executable), you need Zune + XAP Installer. Also you need to have the developer account. Without Windows Developer account, you will not be able to install the XAP. How to test the BlackBerry .cod file.? Forget it. BB is dead. Even Symbian in that context. Android, iOS, Windows Phone is what we will work in the future. Jolla, Ubuntu Edge, Firefox OS’s will run android apps. Define mobile testing strategy? You cover different networks, different OS, different screen sizes, different processor speeds, different OEM vendors, different ways of approaching your functionality. 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 35. 35 Ultimate Cheat Sheet for Mobile Testing Do I have to worry about NDK ? No. Developers will take care of that. Do I need to know Java to perform Mobile test automation ? Knowing that is good. But Its not required. Calabash scripts can be written in readable English. You read English right ? Tell me about Telerik for iOS ? Great record and playback tool. Free too. Worth trying for Web apps. But for native apps, calabash is best ! 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 36. 36 Ultimate Cheat Sheet for Mobile Testing What is the Smart phone OS Market Coverage ? 80% of the smart phones running in Android. 15% iPhones. 4% Windows Phones. Rest of them are running (what the heck is to run ?) Never heard about ZTE. Go to hell. They are the 4th biggest phone makers in the world. Can I hire you ? Send me an Offer ! 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 37. 37 Nimbuzz !! Download Nimbuzz from www.nimbuzz.com (If you are in a PC) Or if you have a smart phone, use your mobile browser to point http://get.nimbuzz.com (Nimbuzz will automatically find your phone and give you the software) I want you to try this out and send your feedbacks to me !! Join Nimbuzz Beta Testers Facebook Group and post your views… https://www.facebook.com/groups/nimbuzz.beta/ 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com
  • 38. 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com 38 Ha Ha Ha !! I just gave a Medal to the Medal.
  • 39. Divert your feedbacks to: http://twitter.com/senthazalravi ra2012vi@gmail.com 39 9/20/2013 Go get Nimbuzz from http://get.nimbuzz.com

×