Testing in the New Internet Era

1. Testing in the New Internet Era Tony Chung QA Manager

2. A Trip down Memory Lane...

3. A Trip down Memory Lane...

4. The Internet of Yesterday

11. The Internet of Today

19. What Will we Cover Today?

20. What Will we Cover Today?• My Top 4 Internet Themes• Why, Mozilla?• Testing Approaches• Ways to Get involved!

21. Mozilla Mission

22. http://www.mozilla.org/about/mission.html

25. Top 4 Internet Themes

26. #4: Desktop

27. #4: Desktop Browsing

28. #4: Desktop Browsing• Despite slowing growth on PC usage, still the top way to access internet• Users still averaging ~7.7 hours a day usage* • In Southeast Asia, average 14-25 hours a week (source: Nielsen Wire, Nov 2011)*• Users still spend 1/3 of time online using Email, Search, Social, Gaming, and video• Security and Privacy even more important than ever

29. #4: Desktop Browsing • Despite slowing growth on PC usage, still the top way to access internet • Users still averaging ~7.7 hours a day usage* • In Southeast Asia, average 14-25 hours a week (source: Nielsen Wire, Nov 2011)* • Users still spend 1/3 of time online using Email, Search, Social, Gaming, and video • Security and Privacy even more important than ever* Source: http://blog.nielsen.com/nielsenwire/global/surging-internet-usage-in-southeast-asia-reshaping-the-media-landscape/

30. #4: Why, Mozilla?

31. #4: Why, Mozilla?• Firefox is still our Flagship gateway to the Internet• Always pushing for Open Standards• 18-week release support (Nightly, Aurora, Beta)• Accelerated deployments has introduced new headaches, so still lots of work to improve on

32. #4: Testing Approach

33. #4: Testing Approach• Over 30,000 automated nightly tests ran, multiple platforms• Test Execution every week against Nightly, Aurora, and Beta channels• Automated regression tests via Mozmill Test Framework• Keeping up with new features and Testplans• Cross platform coverage, Localization, Security releases, Support• Web Compatibility and Exploration through Crowdsourcing

34. #3: Cloud

35. #3: Cloud

36. #3: Cloud• Online Storage: Access your data from Any device• Online computing: Relying less power on native apps• Apple, Google, Amazon, Netﬂix, etc.. investing hundreds of millions into Cloud storage and infrastructure • Storing music, video, apps, photos, docs• Broadband and Virtualization brings speed up, costs down

38. #3: Why, Mozilla?• Cloud Services are important building blocks to provide better User experiences• Built out 3 co-locations to support 1.6M Sync users• Secure: Ease of setup, yet Encrypted Desktop to Phone data syncing• building reusable libraries into a framework for Developers using REST apis

39. #3: Testing Approach

40. #3: Testing Approach• Weekly test coverage on Client & Server deployment trains• Functional feature coverage on Dev and Staging environments while generating balanced load• New coverage on upcoming services products like BrowserID and Notiﬁcations• Better ways to support new Service Apps, RESTful APIs

41. #2: Web Apps

42. #2: Web Apps

43. #2: Web Apps• Applications moving to Cloud, less relying on native OS• Mobile applications is redeﬁning browsing and usage• Light storage, rapid application, Ubiquitous• Distribution model through Web stores and marketplaces

45. #2: Why, Mozilla?• Web Developers want an Appstore model for the web to showcase their work without proprietary silos• An OpenWebApp store to promote Apps written in HTML5, CSS, and Javascript• Uphold high security guidelines while respecting user privacy

46. Preview of OWA

47. Preview of OWA Click

48. #2: Test Approach

49. #2: Test Approach• Cross Functional teams, lots of collaboration• Manual and automated testing approach• Project management challenge: Team dependencies, Mix of Code repositories, Independent deadlines, mixture of tools and bug tracking systems

50. #1: Mobile

51. #1: Mobile

52. #1: Mobile• Mobile is Everywhere! Now world’s most widely distributed computing system• Smartphones and Tablets are ﬂooding the market • iOS: 27.3%, Android: 43.7%, Others: 30%* • Has led to explosion of mobile Applications• Affordable and Plentiful • “The mobile revolution is moving from Ear to Hand” - Jussi Hinkkanen, Head of Policy, Nokia Africa

53. #1: Mobile • Mobile is Everywhere! Now world’s most widely distributed computing system • Smartphones and Tablets are ﬂooding the market • iOS: 27.3%, Android: 43.7%, Others: 30%* • Has led to explosion of mobile Applications • Affordable and Plentiful • “The mobile revolution is moving from Ear to Hand” - Jussi Hinkkanen, Head of Policy, Nokia Africa* comScore, Inc, US household, Aug 2011

54. * Source: Mobile Usage Statistics, Stats & Facts 2011 | Digital Buzz Blog

56. #1: Why, Mozilla?• We need to go where the internet is going, and uphold our mission doing it.• Firefox on Android. from XUL to Native• Web APIs support to mobile platform• Large investment in Boot 2 Gecko OS• Apps need to be on a common platform, not walled-garden approaches

57. B2G and WebAPI

58. B2G and WebAPI

59. #1: Test Approach

60. #1: Test Approach• Lots of challenges: regional, hardware limitations, distribution channels• Firefox Android follows 18-week progression cycle• Manual and Automated testing of features and behavior • Execute litmus tests, building automated test harness• Web and Device compatibility coverage, More Crowdsourcing!

61. Who can help with Testing?

62. You Can!

63. You Can!• Do you have a computer or Android phone? Yes!• Do you want to produce quality products? Yes!• Do you support the Open Web? Yes!• Here’s how you can help...

64. Desktop Testing

65. Desktop Testing• Download a Firefox build on Nightly, Aurora, or Beta• Run Litmus tests on a localized build• File bugs in bugzilla!• Join our Firefox Testdays for real-time interaction with the community

66. Cloud Services Testing

67. Cloud Services Testing• Set Up Sync on Firefox!• Sync your bookmarks and history• Visit or Create a site using Browser ID and report bugs• File bugs in bugzilla!• Join our Sync or BrowserID Testdays for real-time interaction with the community

68. Open Web Apps Testing

69. Open Web Apps Testing• Download a Web App through the App Marketplace (coming soon)• Create a Web App and upload one!• File bugs in bugzilla!• Join our WebApp Testdays for real-time interaction with the community

70. Mobile Testing

71. Mobile Testing• Android phone users -- install Firefox for Android• Visit your favorite websites on different devices• Develop automated java tests in Robotium against Firefox Android• File bugs in bugzilla!• Join our Mobile Testdays for real-time interaction with the community

72. • Get started by visiting • http://quality.mozilla.org• Chat with us real-time on • irc.mozilla.org, channel #qa• File and Verify Bugs!

73. Join Us!

74. Join Us!

Testing in the New Internet Era

Anthony Chung

Testing in the New Internet Era