Testing a SaaS Platform

About me - Joel Montvelisky PractiTest – Co Founder & Prod. Architect QA Instructor & Consultant Mercury Interactive - QA Manager (retired ) TD, QC, WR, QTP, etc ITCB (IL) Advisory Board QABlog.practitest.com © 2010 -

Today’s Agenda   SaaS - Software as a Service   Things we Can Stop Testing   Non-trivial Stuff to Think About & TEST   Leveraging Live Feedback into the Testing Lifecycle   Open Season for Questions… © 2010 -

Software as a Service   Hosted Platforms – Total Control – No Maintenance   No Client Installations   Customers buys each month – keep them happy  Users don’t care it is SaaS, it needs to be better than the alternative © 2010 -

SaaS QA Management   End-to-End QA Management (Reqs – Tests – Bugs)   Enterprise Level SaaS Solution - Methodology - Customizations & Flexibility   Global platform - Communication - Languages - Available 24x7 Worldwide   Intuitive & Simple   Agile Dev, RoR, on Amazon EC2 © 2010 -

Things we can STOP Testing ✘  Client or Server installations ✘  Multiplatform backend support ✘  Multiple version upgrades “Savings” of approx. 1/3 ✘  Backwards compatibility of the Traditional Enterprise Application Testing Cycle © 2010 -

The regular testing scenarios   Functionality - Manual scripts - Exploratory sessions Agile Sprints & Updates - Checklists of 3 – 4 weeks - End-2-End Scenarios - Selenium Each sprint’s cycle is - Cucumber & RSpec based on its content   Load & Stress   Multi-platform & Multi-browser support   I18N English, Swedish, German, Chinese Simplified, Hebrew © 2010 -

Non-trivial stuff to think about & TEST 1. Remote accessibility & usage   Staging Servers in USA - Amazon EC2 Development & Testing teams in Israel (& WW collaborators)   Globally distributed monitoring services (www.pingdom.com) © 2010 -

Non-trivial stuff to think about & TEST 2. Security Application-related: Infrastructure-based:   Cross-site scripting   Secured communication   SQL injections   Backups & storage policies   HTTP header injections   Controlled Access to the site & servers etc… © 2010 -

Non-trivial stuff to think about & TEST 3. Live updates & deployments (3 to 4 weeks)   No concrete separation between Dev & IT   Minimal service disruption   Dressed rehearsals (including rollbacks!)   Automated sanity & manual verification   Patches & hot-fixes (extraordinary occasions )   Prove of Concepts & Beta Releases QA & Testing as a facilitator for Balance: Stable & Professional, yet Flexible Environment © 2010 -

Non-trivial stuff to think about & TEST 4. Disaster recovery procedures Two Main scenarios: (a) System down to be brought up quickly   Configure machines   Install & deploy software   Restore data Define a policy & schedule regular drills to ensure you can achieve it! (b) Rollback to last known stable data (with or without data restore) © 2010 -

Levering live feedback into the Testing Process (1) Internally developed monitoring tools - Check usage patterns No need to - Validate need & usability assumptions continue guessing!!! - Analyze patterns and create realistic & prioritized testing scenarios © 2010 -

Levering live feedback into the Testing Process (2) Site Monitoring systems LiveAdmin (www.liveadmin.net)   Real-time usage to schedule updates   Allow users to contact the company with questions or issues Pingdom (www.pingdom.com)   Up to date response times for real projects and data.   Proactive alerts when things start slowing down © 2010 -

Levering live feedback into the Testing Process (3) Proactive mail notifications on issues (www.hoptoad.com)   Real-time notifications on issues on the system.   Information about the issue including system parameters that allow you to do an initial trouble-shoot of the issue © 2010 -

Levering live feedback into the Testing Process (4) Ask users for Real Projects   No better way to test than using real data   Most users will trust you not to abuse their information in exchange for assurance of working software   Since the data sits in your system it is easier to copy projects to your testing environment  Need to be careful with who has access to the data; making sure your testing environment is as secured as your production environment © 2010 -

Wrapping up   SaaS is only the delivery model, customers will not compromise on Functionality, Stability or Response Time   Since the system is hosted you can save all tests related to backend installation and support   You need to pay more attention to stuff like accessibility, security, deployments & DRPs   The lines between the R&D and IT organizations in SaaS are blurry   SaaS application allows better access to real data to be leveraged to improve the effectiveness of our testing operations. © 2010 -

Testing a SaaS Platform

by Anne-Marie Charrett on Apr 22, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 79

Statistics

Testing a SaaS Platform — Presentation Transcript

http://mavericktester.com	74
http://www.slideshare.net	5