STPCon fall 2012: The Testing Renaissance Has Arrived

552 views
459 views

Published on

This session shares how you can catapult your career with cloud-based test automation and mobile testing techniques that are as exciting to learn and use as they are impactful to your end results and personal success!

1) Become a cloud testing expert
2) Build and articulate a distributed mobile testing strategy
3) Champion a new approach to realistic, repeatable web and mobile performance testing
4) Establish yourself as an agile testing expert with Continuous Testing
5) Defend “Test” in a world heading toward “DevOps”

Cloud Computing is a proven advantage for testers and mobile app testing is a veritable testing green field for those willing to charge ahead.

Learn the new “arts” - embrace this new world – and become a Renaissance Tester!

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
552
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
12
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

STPCon fall 2012: The Testing Renaissance Has Arrived

  1. 1. © 2012 SOASTA. All rights reserved. 1
  2. 2. o  Brad Johnson - @bradjohnsonsv •  SOASTA (“09-Present) •  Borland (“06-”09) •  Mercury Interactive (“02-”05) •  Compuware (“99-”02) o  I’ve seen a few testing market trends •  Client-server to Web •  Web to BIG Web •  Custom to packaged and back to custom apps •  Waterfall to agile to continuous •  Cloud computing •  Mobile © 2012 SOASTA. All rights reserved. May 22, 2012 2
  3. 3. o  Mobile testing is a vacuumo  Cloud testing is in demand (and a no brainer)o  Testing in production is an accepted practiceo  Agile testing has come of age… with Continuous Integrationo  DevOps is a movement, just add the testing and succeed © 2012 SOASTA. All rights reserved. May 22, 2012 3
  4. 4. What’s Driving the Renaissance? o  Explosion in mobile applications o  The immensity of the web & everything web o  Rapid introduction of new technologies o  Incredible pace of development and delivery
  5. 5. o  This is the greatest opportunity since Webo  Every company has a mobile strategyo  Very few have mobile testerso  Fewer have mobile testing strategieso  Global outsourcers are seeking help © 2012 SOASTA. All rights reserved. May 22, 2012 5
  6. 6. Keyboards Mouse Gestures Voice Bendable Phones Kinetics 3D Navigation Hovering Gestures © 2012 SOASTA. All rights reserved. May 22, 2012 6
  7. 7. o  Shear Number of Devices (953M Smartphones) o  Different Operating Systemso  Scale of Global Customers (6B)o  Dynamic Content (Video, Annimation)o  Emerging People to Machine InterfacesManual Testing Can Not Deliver Adequate Coverage © 2012 SOASTA. All rights reserved. May 22, 2012 7
  8. 8. o  Nearly 80% of all mobile apps are tested manually (IDC) Take Actiono  Learn and implement effective automation o  Focus on user experience o  Consider the impact of precision and performance on functionality and satisfaction o  Prioritize and streamline to keep pace with the rate of change o  Communicate the complexity © 2012 SOASTA. All rights reserved. May 22, 2012 8
  9. 9. o  Read everything you cano  Understand how mobile apps are built and how they are consumedo  Use mobile as an opportunity to “fix” software developmento  Join SOASTA’s Early Access Team (see me) © 2012 SOASTA. All rights reserved. May 22, 2012 9
  10. 10. o  Testing with cloud, testing in the cloud, testing for the cloud…o  Covered by all major (and minor) analysts: Gartner, Forrester, Aberdeen, IDC, Ovum…o  Cost and scale benefits are indubitableo  Expertise is an enterprise QA requirement •  IBM testers required skills © 2012 SOASTA. All rights reserved. May 22, 2012 10
  11. 11. What Makes a Prime Cloud Testing Project?o  Critical web launch – mobile support, new website o  Tight timeframes - no room for experimentationo  Web-scale expectations – 100s to millions of userso  Quick start, many test iterations – adapt to flexible dev scheduleo  High cost of tools and infrastructure - artificial ceiling increases risk © 2012 SOASTA. All rights reserved. May 22, 2012 11
  12. 12. o  Read everything you cano  Open a cloud computing accounto  Watch webinarso  Try a cloud testing toolo  Run a cloud test © 2012 SOASTA. All rights reserved. May 22, 2012 12
  13. 13. CodeFrom the wall of a very traditional retailer’s datacenter © 2012 SOASTA. All rights reserved. May 22, 2012 13
  14. 14. Latency between systems Network Network configuration Capacity limits & Ops Network bandwidth CDN file placement Conflict with other apps DNS routingStage / Team Load Balancer configuration Firewall max capacity Auto-scaling failures Unbalanced web servers Max sockets exceeded s Release Global latency variance & Deploy Slow third-party plug-ins Security bottlenecks Caching issues and benefits Search technology limits Default configuration settings Inadequate server resources Garbage collection Memory leaks Memory leaks Database thread counts Dev Slow pages & Test Inefficient database queries Code bugs Method-level tuning Test Lab Staging Production (100% +++) Scale and Time © 2012 SOASTA. All rights reserved. May 22, 2012 14
  15. 15. o  Read some good whitepaperso  Watch/download Seth Eliot’s session and slideso  Attend a webinar on Oct. 30 (Seth & SOASTA) © 2012 SOASTA. All rights reserved. May 22, 2012 15
  16. 16. o  Still can’t find a good online diagram, though! © 2012 SOASTA. All rights reserved. May 22, 2012 16
  17. 17. Continuous Testing is Progressive Dev & Test Lab Production Environment Include Hit expected Collect realStart small, Recheck functional scale in user data toearly to root Test in regularly for automation & staging and feed back out code- Production regression performance maintenancelevel issues and capacity (RUM) in CI windows Keys to agile testing? SPEED – SCALE – COST © 2012 SOASTA. All rights reserved. May 22, 2012 17
  18. 18. Many approaches are process and pre-test heavy Traditional Approaches TEST BUILDING HW Setup TEST ANALYZE FIX RETEST REPEAT? TIME BUILD TEST TEST TEST TEST TEST Extra ANALYZE ANALYZE ANALYZE ANALYZE ANALYZE FIX time! FIX FIX FIX FIX Eliminate constraints and TEST MORE, at SCALE, in LESS TIME Agile approaches deliver results fast © 2012 SOASTA. All rights reserved. May 22, 2012 18
  19. 19. Test Types Test Objective SPRINT 1 Load Test Determine Limits 10 hours Refinement of Measure User Experience Performance Test (Benchmark) Expectations SPRINT 2 Test Specific Silos Targeted Infrastructure Test 8 hours SPRINT 3 Verify Redundancy Failover Test 3 hours SPRINT 4 Stress Test Exceed Break PointsHours for 5 hoursexample only SPRINT 5 Capacity Test Plan for future 8 hours SPRINT 6 Soak Test Reliability over time 20 hours Test Deploy Implement Assess Design Define Delivery Cycle © 2012 SOASTA. All rights reserved. May 22, 2012 19
  20. 20. Production Devs Check in Or QA Test Results ✓ ☐ Pass Source Code Repository (Git, Subversion, Atlassian Jira ☐ Fail Microsoft VS Team Foundation Server) Check out Test Automation Run Tests Continuous Integration server (unit, component, functional) automates builds, launches tests, manages results Dev or Test Lab (Jenkins, Bamboo, VSTFS, Anthill,…) © 2012 SOASTA. All rights reserved. May 22, 2012 20
  21. 21. Beta Users Devs Check in(ad-hoc server) Test Results ✓ ☐ Pass Source Code Repository ☐ Fail Check out Test Automation Run Tests Integration Server i.e. Jenkins Slave (Mac) Execute on Devices Push to Device Real Devices © 2012 SOASTA. All rights reserved. May 22, 2012 21
  22. 22. o  ALM meets APMo  What’s old is new…just FASTERo  This is Process Compression Test Monitor Build Deliver Deploy Design Define Manage © 2012 SOASTA. All rights reserved. May 22, 2012 22
  23. 23. o  ALM meets APMo  What’s old is new…just FASTERo  This is Process Compression DevOps! Dev Ops © 2012 SOASTA. All rights reserved. May 22, 2012 23
  24. 24. o  ALM meets APMo  What’s old is new…just FASTERo  This is Process Compression Dev Psst… DevOps! (TEST) Ahhhh. you forgot something Ops! © 2012 SOASTA. All rights reserved. May 22, 2012 24
  25. 25. o  Read everything you can – ALM and APMo  Seek visibility solutions – reporting, collaborationo  Expand your sphere of contact and influenceo  Join or run a production performance test © 2012 SOASTA. All rights reserved. May 22, 2012 25
  26. 26. o  Champion a mobile testing team or project?o  Run a cloud test?o  Be the Continuous Integration agent-of- change?o  Put the “test” in DevOps? THANKS! Questions? (Please turn in the eval.) © 2012 SOASTA. All rights reserved. May 22, 2012 26

×