Your SlideShare is downloading. ×
0
The SoLoMo RevolutionAnd The Resulting Tectonic Shift In Testing                                                          ...
The Challenge   SoLoMo   Huh?                |                    2
SoLoMo Breakdown So Lo Mo       Social                                     Local                             Mobile       ...
SoLoMo Coming Fast In the past six months…                                                                                ...
QA Leaders Better Sit Down  • Media & analysts aren’t the only ones talking about it       –   Marketing       –   Product...
The Challenge   Trends In   Social Apps                 |                     6
Not Just For B2C Anymore• The social revolution isn’t coming… it’s here    – Nothing is disconnected    – No industry is i...
Goes Soooo Much Further                                                                                    |  In-The-Wild ...
The Challenge   Trends In   Local Apps                |                    9
Not Just For Marketing• Businesses use location for more than just marketing• Growing trend toward location-based intellig...
Increasing Granularity  Good news for users (and marketers)… good luck to testers.                                        ...
The Challenge   Trends In   Mobile Apps                 |                     12
Mobile Apps Boom Market• ABI: mobile revenues will hit $133 billion in 2014• B2B mobility no longer just for email    –   ...
Mobile’s Just Getting Started                                                                                    |  In-The...
Did I Mention…                                                                                    |  In-The-Wild Testing f...
Psst… It’s Not Just Mobile  • Smartphones & tablets make QA immeasurably tougher  • Now imagine a whole world of apps:    ...
The Challenge   SoLoMo Impact   On Testing                   |                       17
Brave New World   • Impact on design, dev & IT well-recognized & resourced   • Sadly, but not at all surprisingly, not tru...
Social: Impact On Testing• Social presents functional test challenges    – Testing code & integrations that aren’t yours… ...
Social: Impact On Testing• New challenges include how will you:    – Regression test multiple social integrations for web ...
Local: Impact On Testing                                              • Not just a mobile problem                         ...
Local: Impact On Testing                                              • New challenges include, how will you:             ...
Mobile: Impact On Testing• New challenges include how will you:   – Native apps: Test across ever-changing OS, devices and...
Mobile: Impact On Testing• Mobile web vs. native apps   –   2009: 5-horse race   –   2010: 4-horse race   –   2011: 2-hors...
F|R|A|G|M|E|N|T|E|D• Android alone (and that’s only by OS version)                                                        ...
Mobile Testing Landscape• Form factors matter again   – Phones (feature & smart)   – Tablets   – Hybrids• Prohibitive HW &...
Multiple Launch Schedules  Multiple products, roadmaps & launch schedules                                                 ...
Test CoverageWeb Testing Matrix Is Tough Features                               OS & Browser                              ...
Test CoverageMobile Testing Matrix Is Insane Features                                          Handset Makers             ...
Test CoverageMobile Testing Matrix Is Insane Features                                                         Wireless    ...
Test CoverageMobile Testing Matrix Is Insane Features                                                                     ...
Increasingly Complex Testing• The laughably simplistic notion of a “test matrix”                                          ...
The Challenge   Ummmm…   So What Do We Do?                       |                           33
Sufficiently Freaked Out?!?• Don’t be… the good news is that testing ain’t dead!   – Chance for QA leaders to step up…   –...
…And The Bad News…• You should be scared… need to completely re-think testing   – Can’t automate our way out of this   – C...
Lab-Based Testing Innovation                          Manual vs.                          Automation                      ...
Users Work & Live In The Wild  I nThe                                                        Wi l  In-The-Wild Testing for...
To Make Matters Worse• Users are less tolerant of quality failures• Social media & app stores give everyone a megaphone• A...
The Challenge   A Call To Action For   Test Leaders (Two, Actually)                            |                          ...
1. Look Beyond The Lab• Yes, keep testing in-the lab   –   Keep training in-house   –   Keep outsourcing   –   Keep automa...
2. Be The Voice For QA  • SoLoMo is coming – with or without QA’s buy-in  • Want QA to be heard? Want quality to be built ...
The Challenge   SoLoMoWoSto   A Few Stories (if there’s time)                              |                              ...
The Challenge   Questions?   Answers.   Matt Johnston | CMO @ uTest   mattj@utest.com | @matjohnston                      ...
Upcoming SlideShare
Loading in...5
×

uTest STPcon spring 2012 presentation

867

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
867
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
26
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "uTest STPcon spring 2012 presentation"

  1. 1. The SoLoMo RevolutionAnd The Resulting Tectonic Shift In Testing |Matt Johnston | @matjohnston | CMO @ uTest | March 2012
  2. 2. The Challenge SoLoMo Huh? | 2
  3. 3. SoLoMo Breakdown So Lo Mo Social Local Mobile | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 3
  4. 4. SoLoMo Coming Fast In the past six months… | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 4
  5. 5. QA Leaders Better Sit Down • Media & analysts aren’t the only ones talking about it – Marketing – Product – Engineering – IT – Sales – C-suite • Want a seat at the table? – Get informed – Get involved – Develop an opinion – Advocate strongly | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 5
  6. 6. The Challenge Trends In Social Apps | 6
  7. 7. Not Just For B2C Anymore• The social revolution isn’t coming… it’s here – Nothing is disconnected – No industry is immune• The stats on social media adoption in Fortune 500 firms: – 77% integrate Twitter – 84% incorporate LinkedIn – 93% utilize Facebook – 57% publish on YouTube• Gartner: By 2016, social integrated w/ nearly all apps – Websites & apps – Mobile apps & mobile web – Desktop clients | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 7
  8. 8. Goes Soooo Much Further | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 8
  9. 9. The Challenge Trends In Local Apps | 9
  10. 10. Not Just For Marketing• Businesses use location for more than just marketing• Growing trend toward location-based intelligence in apps – Asset tracking – Maps & directions – Location finders – Check-in services – Geo-based personalization – Location-based ads | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 10
  11. 11. Increasing Granularity Good news for users (and marketers)… good luck to testers. | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 11
  12. 12. The Challenge Trends In Mobile Apps | 12
  13. 13. Mobile Apps Boom Market• ABI: mobile revenues will hit $133 billion in 2014• B2B mobility no longer just for email – Business apps: CRM, ERP, HR systems – Productivity apps: docs, spreadsheets, presentations – Collaboration apps: email, IM, publishing – Medical apps: health records, patient education• B2C mobility growth even steeper – Retail: location-intelligent m-commerce – Media: magazines & newspapers going purely digital – Travel: mobile bookings, check-ins, maps, deals – Education: tablets in every classroom – Healthcare: patient records, physician notes – Social: no explanation required – Gaming: ditto | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 13
  14. 14. Mobile’s Just Getting Started | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 14
  15. 15. Did I Mention… | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 15
  16. 16. Psst… It’s Not Just Mobile • Smartphones & tablets make QA immeasurably tougher • Now imagine a whole world of apps: E-Readers Cars Household Appliances Smart Homes Gaming Consoles Connected TVs | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 16
  17. 17. The Challenge SoLoMo Impact On Testing | 17
  18. 18. Brave New World • Impact on design, dev & IT well-recognized & resourced • Sadly, but not at all surprisingly, not true for testing – Want proof? Specialized designers & devs… but who tests? | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 18
  19. 19. Social: Impact On Testing• Social presents functional test challenges – Testing code & integrations that aren’t yours… – That constantly changes… – And you get the blame• As well as security & privacy pitfalls – Internally, a channel for spam, malware and data breaches – Confidential data may be posted and trigger regulatory penalties – Externally, increasingly used for user authentication on your products | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 19
  20. 20. Social: Impact On Testing• New challenges include how will you: – Regression test multiple social integrations for web & mobile apps? – Write (and execute) test cases with zero visibility into social’s roadmaps? – Automate anything? – Test sharing/liking in a QA environment without publishing via social? – Test your social footprint? - FB page (and apps) - LinkedIn page - Twitter page – Ensure security/privacy if you enable users to use social sign-on? | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 20
  21. 21. Local: Impact On Testing • Not just a mobile problem – Localization testing – Geo-based personalization • But it is a mobile problem too – Apps used outside the confines of the QA lab, under in-the-wild conditions • Impossible to test effectively from only inside the lab | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 21
  22. 22. Local: Impact On Testing • New challenges include, how will you: – Replicate location (answer: you don’t) – Test in NYC, SF, LA, DFW, CHI? – Test abroad - SIM cards - Carriers – Test GPS-based intelligence? – Test IP-based localization? | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 22
  23. 23. Mobile: Impact On Testing• New challenges include how will you: – Native apps: Test across ever-changing OS, devices and carriers? – Mobile web: Test across wide range of OS and mobile browsers? – Either way, how does your testing replicate real-world conditions - Specific and varied locales - Tower-to-tower hopping - Roaming - Low battery - Touch screen actions (pinch, swipe, pan, etc) - Portrait vs. landscape - Display definition | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 23
  24. 24. Mobile: Impact On Testing• Mobile web vs. native apps – 2009: 5-horse race – 2010: 4-horse race – 2011: 2-horse race (depending upon your industry) – 2012: 3 horses (including HTML5-driven mobile web) | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 24
  25. 25. F|R|A|G|M|E|N|T|E|D• Android alone (and that’s only by OS version) | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 25
  26. 26. Mobile Testing Landscape• Form factors matter again – Phones (feature & smart) – Tablets – Hybrids• Prohibitive HW & SW costs• Here we go again – Late-90s web, part deux - Lack of mature tools - Lack of usability standards - Lack of security standards - Extremely fluid landscape | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 26
  27. 27. Multiple Launch Schedules Multiple products, roadmaps & launch schedules | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 27
  28. 28. Test CoverageWeb Testing Matrix Is Tough Features OS & Browser | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 28
  29. 29. Test CoverageMobile Testing Matrix Is Insane Features Handset Makers OS & Browser & Models | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 29
  30. 30. Test CoverageMobile Testing Matrix Is Insane Features Wireless Handset Makers Carriers OS & Browser & Models | 30 In-The-Wild Testing for Functional + Security + Load + Usability + Localization
  31. 31. Test CoverageMobile Testing Matrix Is Insane Features Location Wireless Handset Makers Carriers OS & Browser & Models | 31 In-The-Wild Testing for Functional + Security + Load + Usability + Localization
  32. 32. Increasingly Complex Testing• The laughably simplistic notion of a “test matrix” | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 32
  33. 33. The Challenge Ummmm… So What Do We Do? | 33
  34. 34. Sufficiently Freaked Out?!?• Don’t be… the good news is that testing ain’t dead! – Chance for QA leaders to step up… – To lead the charge… – To defend users and app quality | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 34
  35. 35. …And The Bad News…• You should be scared… need to completely re-think testing – Can’t automate our way out of this – Can’t hire or train our way out – Can’t solve this problem from soley within the lab• More distributed problem than ever before – Need a new testing mindset – Need to bridge the growing gap - Between testing & users - Between artificial and real - Between in-the-lab & in-the-wild testing | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 35
  36. 36. Lab-Based Testing Innovation Manual vs. Automation Managing vs. Leading Teams In-House vs. Outsourced Agile vs. Waterfall Exploratory vs. Testers vs. Test Cases Engineers Offshore vs. Nearshore | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 36
  37. 37. Users Work & Live In The Wild I nThe Wi l In-The-Wild Testing for Functional + Security + Load + Usability + Localization | 37
  38. 38. To Make Matters Worse• Users are less tolerant of quality failures• Social media & app stores give everyone a megaphone• And their problem… is your problem | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 38
  39. 39. The Challenge A Call To Action For Test Leaders (Two, Actually) | 39
  40. 40. 1. Look Beyond The Lab• Yes, keep testing in-the lab – Keep training in-house – Keep outsourcing – Keep automating – Keep innovating• Recognize it will never again be sufficient on its own• Build your company’s plan for testing in-the-wild – Real-world conditions: Live testers, real devices, imperfect connectivity – Mirror your user base - Technologically: OS, browser, anti-virus, device, carrier - Geographically: Continent, country, city, language - Demographically: Age, gender, education, employment, industry, hobby• In-house QA leadership is vitally important – You have the business and product context – You know what your firm cares about – You need to be the orchestrator who leads this symphony - What to test in the wild, how to test in the wild, holistic test planning | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 40
  41. 41. 2. Be The Voice For QA • SoLoMo is coming – with or without QA’s buy-in • Want QA to be heard? Want quality to be built in? – Don’t wait: No one else is coming to the rescue – Get informed: Develop an thoughtful opinion – Get involved: Find out when/where decisions are being made – Be proactive: Build a plan for quality from the ground up – Be protective: You’re a protector of the user… of the brand | In-The-Wild Testing for Functional + Security + Load + Usability + Localization 41
  42. 42. The Challenge SoLoMoWoSto A Few Stories (if there’s time) | 42
  43. 43. The Challenge Questions? Answers. Matt Johnston | CMO @ uTest mattj@utest.com | @matjohnston | 43
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×