SaaS Testing Overview - Foundation
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

SaaS Testing Overview - Foundation

on

  • 13,954 views

SaaS Testing Overview - Foundation is an introduction to SaaS and SaaS Testing. In subsequent presenation, I will cover SaaS Testing Overview - Expert

SaaS Testing Overview - Foundation is an introduction to SaaS and SaaS Testing. In subsequent presenation, I will cover SaaS Testing Overview - Expert

Statistics

Views

Total Views
13,954
Views on SlideShare
13,703
Embed Views
251

Actions

Likes
10
Downloads
788
Comments
1

10 Embeds 251

http://askcap.ru 165
http://www.sp-assurance.com 49
http://localhost 21
http://www.techgig.com 6
http://www.linkedin.com 4
http://dev.askcap.ru 2
https://si0.twimg.com 1
https://www.linkedin.com 1
http://www.blogger.com 1
https://twitter.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

SaaS Testing Overview - Foundation Presentation Transcript

  • 1. SaaS Testing Overview Ram Garg CEO Software Performance Assurance (SPA) Software Performance Assurance Confidential
  • 2. Outline  SaaS Introduction  What is it?  Why SaaS?  SaaS Challenges  Key Challenges  SaaS Testing  How to approach SaaS testing?  How SaaS Testing is different?  SaaS Test Tools Software Performance Assurance Confidential
  • 3. What is SaaS? SaaS Introduction Software Performance Assurance Confidential
  • 4. SaaS: What is it? Software-as-a-Service (SaaS) is the application layer of the cloud computing model. Application Software as a Service (SaaS) Platform Platform as a Service (PaaS) Infrastructure Infrastructure as a Service (IaaS) Software Performance Assurance Confidential
  • 5. SaaS Definition “Software as a service (SaaS), sometimes referred to as "software on demand," is software that is 100% deployed over the internet … . With SaaS, a provider licenses an application to customers as a service on demand, through a subscription or a "pay-as-you-go" model.” – Wikipedia definition Software Performance Assurance Confidential
  • 6. Architecture: Three Key Differentiators Multi-tenant Efficient Scalable Configurable Key Architecture Differentiators Software Performance Assurance Confidential
  • 7. Why SaaS? SaaS Introduction Software Performance Assurance Confidential
  • 8. The Software Business Model Changes … non -SaaS SaaS Software Ownership Customer Provider Infrastructure Customer Provider management Responsibility Installation Client - Server Central Upgrades Larger, less frequent Shorter, frequent Platform Multi-version Single-version Software Performance Assurance Confidential
  • 9. Changing the Business Model cont … non -SaaS SaaS Configurability Low High, Critical Requirement Multi-tenancy No Yes Backward compatibility No Yes Maintenance Customer Provider Software Performance Assurance Confidential
  • 10. Changing the Business Model cont … non -SaaS SaaS Cost, Allocation High, CapEx Low, OpEx Close the deal, Provide value, Sales focus / Success License Revenue Lack of Churn Profits Initial sale + maintenance Ongoing Subscription Software Performance Assurance Confidential
  • 11. Ecosystem SaaS (Software)  Starting point to every service within cloud computing ecosystem PaaS (Platform)  Highest on the maturity level and growth curve amongst XaaS IaaS  Highest revenue generating service (Infrastructure) within cloud model for next several years
  • 12. SaaS Catalysts Intrigue of access to unlimited Computing and storage resources, made available and paid for only as needed (Pay as you go) Decreasing cost of bandwidth Customer frustration with traditional software buying cycle (license, maintenance contract, upgrades) Perceived increase in control over the vendor relationship Early SaaS success stories – Salesforce.com, WebEx, etc. Demonstrate broader value proposition You focus on your application and not on infrastructure Software Performance Assurance Confidential
  • 13. SaaS Early Adoption SaaS Introduction Software Performance Assurance Confidential
  • 14. SaaS Adoption: Verticals AMI-Partners„ report - "Software-as-Service Hot Adoption rate of Spots” : Education, SaaS in SMB is Healthcare, and Financial" higher than large enterprises All research points out that In-Stat research on SaaS has been adopted vertical market adoption shows the healthcare heavily in : market is currently most  HR applications (ERP, satisfied with hosted Salesforce automation, applications, with eRecruiting) education identified as an  Online Collaboration (Web important potential target conferencing) market also.  CRM  Healthcare Software Performance Assurance Confidential
  • 15. SaaS Adoption Factors (source: Forrester) Availability & Reliability (71%) Fast Deployment (45%) Reduced dependencies on Flexible consumption (35%) internal IT (24%) Simplified pricing model (32%) Software Performance Assurance Confidential
  • 16. The SaaS Market is and will Continue to Grow… Software Performance Assurance Confidential
  • 17. Today and tomorrow … Software Performance Assurance Confidential
  • 18. Today and tomorrow … Software Performance Assurance Confidential
  • 19. Today and tomorrow … Software Performance Assurance Confidential
  • 20. Challenges SaaS Challenges Software Performance Assurance Confidential
  • 21. Primary Concerns with the SaaS Model Security concerns Concerns over compliance with standards Concerns over integration with legacy applications Concerns over reliability and robustness of service Not comfortable with putting mission-critical application on shared infrastructure and have access over internet SaaS can not be configured to meet my corporation‟s all specific needs Software Performance Assurance Confidential
  • 22. Numerous Challenges Security Performance Availability SLA Compliance Data Integrity Privacy with Regulatory Standards Lack of Integration Multi-tenancy confidence Software Performance Assurance Confidential
  • 23. SaaS Testing Software Performance Assurance Confidential
  • 24. Why SaaS Testing? SaaS Testing Software Performance Assurance Confidential
  • 25. The average enterprise IT Why SaaS Testing? department devotes up to 50 percent of its entire technology infrastructure to development A delay in website page and test, with up to 90 percent loading by even one second of the available test can impact conversions by infrastructure remaining idle. 7% and customer satisfaction IBM Research by 16%. Aberdeen Group Cloud Testing Those using cloud facilities to run applications, 49.8% said they were doing so The average online experimentally or for business had 8 days of prototyping; 28.6% for non- downtime a year due to critical business applications performance issues and 21.6% for business leading to lost revenues critical applications Aberdeen Group Evans Data Survey Software Performance Assurance Confidential
  • 26. Trend leads to trend … Overall growth in Cloud Computing (investment, revenue, ROI, market cap, etc.) is leading to a marked increase in adoption of SaaS Testing
  • 27. What is SaaS Testing? SaaS Testing Software Performance Assurance Confidential
  • 28. Classifying SaaS Testing “Testing The Cloud” SaaS” Testing applications which are in cloud or migrating to cloud. Testing Test the performance, reliability & the SaaS security of applications Cloud / SaaS “Testing SaaS Using Cloud” “Testing Using Cloud” Testing applications using the Testing cloud-based hardware, software, Testing infrastructure and computing resources. SaaS Using Tests performed like performance, Cloud load & stress, security and compatibility Software Performance Assurance Confidential
  • 29. How SaaS Testing is Different Easier Challenges • No client-server • Frequent live upgrades installation • Too many pages • No multiplatform support • Multiple privileges • No multiple version upgrades • Immature SaaS-specific • No backward compatibility testing tools • Access partitions critical Software Performance Assurance Confidential
  • 30. Approach SaaS Testing Software Performance Assurance Confidential
  • 31. SaaS Testing Preparation Prepare Assess Understand Prepare Test Plan Prepare Testing Usage Test and Test Test Data Needs Patterns Environment Suite Software Performance Assurance Confidential
  • 32. SaaS Testing Areas Infrastructure Testing Network Application testing testing Secured SaaS Application Ensure SaaS testing does not damage the shared infrastructure Software Performance Assurance Confidential
  • 33. SaaS Functional Testing • Perform Manual and Exploratory test (similar to traditional applications) • Check Browser compatibility • Perform regression • SaaS regression must cover significant more functionality as single defect will impact all the customers • Functional and regression test automation. • This is important to comprehensive testing cost-effectively as above point indicates that, in case of SaaS, regression should cover more functionality. • Test SaaS on cloud-based staging environment • Testing on in-house environment and then directly deploying on cloud- based production environment may result in chaos • Test in a multi-user environment where different users are performing different activities. Software Performance Assurance Confidential
  • 34. SaaS Security Testing • Test application security • Cross Site scripting, SQL Injection, HTTP header injection • Test network security • Secured communication, Backup and storage policies, Controlled access to sites and servers • Test for possible attacks • Test roles and privileges in multi-tenant environment • Many roles and privilege levels • Test all the upgrades to detect broken security, Ensure security patches are not missed • Test data security / integrity among multiple tenants Software Performance Assurance Confidential
  • 35. SaaS Security Testing cont … • Identify scenarios where application might be vulnerable • Test PCI compliance • Test cookie setting, SSL configuration, information leak, etc. • Log errors, warnings and suspicious requests • Tests tampering with the licensing mechanism; gaining unauthorized access to application data, editing registry entities or configuration files Software Performance Assurance Confidential
  • 36. SaaS Performance Testing • Test performance extensively first time and for each upgrade • SaaS is revenue generating and business impacting applications. Single performance issue will impact all your customers • Test performance extensively as SaaS need to handle large number of users and we don‟t have luxury of re-booting or going down every once in a while • Test load characteristics, ramp up rate, protocols, load origination, • Establish ability to test performance of your SaaS isolated fashion so that network related issues can be identified and passed to network infrastructure and internet providers • Test key metrics and SLAs • Test performance for mobile users Software Performance Assurance Confidential
  • 37. In-Service Upgrade Testing • Ability to upgrade (full release, patches) system with zero- to-minimum down time is essential • Global users make this complicated as somewhere in the world someone might be accessing your application • Thoroughly test in staging environment before perform upgrade ass this is one of the most complex features Remote Accessibility and Usage • Global access with acceptable response time • 24 hours availability Software Performance Assurance Confidential
  • 38. Disaster Recovery • Shutdown to bring up application quickly: server configuration; install and deploy software ; restore data • Roll back to last known stable data Localization • Make sure it supports international characters as SaaS application can be accessed from anywhere in the world • Optimize by supporting target market language Software Performance Assurance Confidential
  • 39. Test Tools SaaS Testing Software Performance Assurance Confidential
  • 40. Traditional and Cloud-based Test Tools Cloud-based Test Traditional Test Tools Tools QTP, WinRunner Rational Robot Rational Functional Tester Functional SilkTest Gomez PushToTest, Selenium, STAFF, Fitness, Watir SOASTA CloudTesting LoadRunner, Rational Robot, uTest Rational Performance Tester Many others Silk Performer Performance Jmeter, Grinder, WebLoad, OpenSTA Software Performance Assurance Confidential
  • 41. Traditional Vs. Cloud-based Tools Testing SaaS Using Testing SaaS Using Traditional Tools Cloud-based Tools Many providers who provide A few providers who provide SaaS testing using traditional SaaS testing using their own testing tools cloud-based test tools Many open source matured Limited open source test tools tools options Many options for professional Captive professional service by services by independent test tools provider which is not service providers even core for their business Professional service options Captive profession service is allow cheaper cost sold at very high rate Software Performance Assurance Confidential
  • 42. Conclusion Software Performance Assurance Confidential
  • 43. Conclusion Customers buy each month – keep them happy SaaS is only a delivering model, customers will not compromise on functionality, stability, security, and performance. They will not switch to SaaS it if not better than traditional systems. Save time on support, maintenance, and upgrade, etc. You need to pay more attention to stuff like accessibility, security, deployments. The way you earn reputation Don‟t buy cloud-based test solution just because you are testing SaaS. In most cases, traditional test solutions work just fine and save your 30% – 50% cost. Software Performance Assurance Confidential