TF
Half-day Tutorials
5/6/2014 8:30:00 AM
Testing Cloud Services:
SaaS, PaaS, and IaaS
Presented by:
Martin Pol
Jeroen Men...
Martin Pol
Polteq
Martin Pol has played a significant role in helping to raise the awareness and improve the
performance o...
10-4-2014
© Polteq 1
Testing Cloud Services: SaaS, PaaS and IaaS
Martin Pol
Jeroen Mengerink
Agenda
• Introduction Cloud c...
10-4-2014
© Polteq 2
ISBN 978-1-937538-38-5
In the cloud?
10-4-2014
© Polteq 3
searching, recording, accounting, paying, writing,
reviewing, tracking, calculating, developing, list...
10-4-2014
© Polteq 4
storage claim
80% unused
redundancy limitations
environmentally unfriendly
management overheadcosts f...
10-4-2014
© Polteq 5
US: National Institute of Standards and Technology
http://www.nist.gov
Essential characteristics
On-d...
10-4-2014
© Polteq 6
US: National Institute of Standards and Technology
http://www.nist.gov
Essential characteristics
On-d...
10-4-2014
© Polteq 7
US: National Institute of Standards and Technology
http://www.nist.gov
Essential characteristics
On-d...
10-4-2014
© Polteq 8
Service models
• Nocloud
• Infrastructure as a Service
• Platform as a Service
• Software as a Servic...
10-4-2014
© Polteq 9
What is “done” in the cloud?
>500
Private
Hybride
Community
IaaS, PaaS, DaaS, SaaS
Taas
*aaS
Data Cen...
10-4-2014
© Polteq 10
Continuity
Privacy
Multi platform
Legislation
Cyber crime
Impact organisation
StandardsPerformancePe...
10-4-2014
© Polteq 11
PerformancePerformance
SecuritySecurity
Availability & ContinuityAvailability & Continuity
Functiona...
10-4-2014
© Polteq 12
PerformancePerformance
SecuritySecurity
Availability & ContinuityAvailability & Continuity
Functiona...
10-4-2014
© Polteq 13
PerformancePerformance
SecuritySecurity
Availability & ContinuityAvailability & Continuity
Functiona...
10-4-2014
© Polteq 14
PerformancePerformance
SecuritySecurity
Availability & ContinuityAvailability & Continuity
Functiona...
10-4-2014
© Polteq 15
PerformancePerformance
SecuritySecurity
Availability & ContinuityAvailability & Continuity
Functiona...
10-4-2014
© Polteq 16
PerformancePerformance
SecuritySecurity
Availability & ContinuityAvailability & Continuity
Functiona...
10-4-2014
© Polteq 17
Backup and recovery
Taken care of.Taken care of.
Who will support
me?
Who will support
me?
Performan...
10-4-2014
© Polteq 18
Updates, patches, fixes,
Planned and
controlled
Planned and
controlled
Do I have a
choice?
Do I have...
10-4-2014
© Polteq 19
Where is my data?
And is that OK?
In house.In house.
SomewhereSomewhere
PerformancePerformance
Secur...
10-4-2014
© Polteq 20
PerformancePerformance
SecuritySecurity
Availability & ContinuityAvailability & Continuity
Functiona...
10-4-2014
© Polteq 21
PerformancePerformance
SecuritySecurity
Availability & ContinuityAvailability & Continuity
Functiona...
10-4-2014
© Polteq 22
Testing!
Check
Review
Monitor
Interview
Proof of concept
TestenProef
Intake
InterviewProof of concep...
10-4-2014
© Polteq 23
PerformancePerformance
SecuritySecurity
Availability & ContinuityAvailability & Continuity
Functiona...
10-4-2014
© Polteq 24
PerformancePerformance
SecuritySecurity
Availability & ContinuityAvailability & Continuity
Functiona...
10-4-2014
© Polteq 25
Selection
Implementation
Production
Performance TestingPerformance Testing
Security TestingSecurity ...
10-4-2014
© Polteq 26
Performance TestingPerformance Testing
Security TestingSecurity Testing
Manageability TestingManagea...
10-4-2014
© Polteq 27
“Inspiration List”
CRITERION PRIO
Functional
Do the service and the specific business processes alig...
10-4-2014
© Polteq 28
“Inspiration List”
CRITERION PRIO
Support
Are changes in the service announced beforehand?
Are suffi...
10-4-2014
© Polteq 29
“Inspiration List”
CRITERION PRIO
Law and regulations
Does the data location comply to all legal req...
10-4-2014
© Polteq 30
Proof of Concept
Performance TestingPerformance Testing
Security TestingSecurity Testing
Manageabili...
10-4-2014
© Polteq 31
Known measures
tuned and tweaked
New measures developed
Performance TestingPerformance Testing
Secur...
10-4-2014
© Polteq 32
Operational profile
Performance testing
• Test cases aimed at specific bottlenecks
• Including cloud...
10-4-2014
© Polteq 33
Stress Testing
Yes, you can!Yes, you can!
Definitely NOT!Definitely NOT!
Performance TestingPerforma...
10-4-2014
© Polteq 34
load
load test – ‘up’
extend?
200
charged
100
charged
no
yes
path test
99
100
101
boundary values
‘u...
10-4-2014
© Polteq 35
ISO 27001 aspects:
• Confidentiality of the data and
the accompanying risk that
unauthorized people ...
10-4-2014
© Polteq 36
• Security at:
– Network
– Supplier
– User
Performance TestingPerformance Testing
Security TestingSe...
10-4-2014
© Polteq 37
• Security at:
– Network
– Supplier
– User
• Encryption
• Authentication and
authorisation
• Test lo...
10-4-2014
© Polteq 38
Manageablity of test environments
• Everything in the cloud
Manageablity of test environments
• Link...
10-4-2014
© Polteq 39
Manageablity of test environments
• Link Production to the real service
• Link other environments to...
10-4-2014
© Polteq 40
Defect Management
Incident
Supplier resolves it
Client resolves it
Incident not resolved
Test
Change...
10-4-2014
© Polteq 41
• Role of system architecture
• Monitoring and Logging
• Guarantees and SLA’s
• Test fail-over mecha...
10-4-2014
© Polteq 42
Fail-over testing
A: disrupted
B: active
A: active
B: inactive
A is disrupted
B takes over service
A...
10-4-2014
© Polteq 43
Online – Offline
Use case testing.
Global testing.
Use case testing.
Global testing.
Performance Tes...
10-4-2014
© Polteq 44
Online – Offline
Use case testing.
Global testing.
Use case testing.
Global testing.
Performance Tes...
10-4-2014
© Polteq 45
Functional test objectives
• Does the service fit the business processes and vv?
• Is the service qu...
10-4-2014
© Polteq 46
Any device – any platform
Multiplatform
testing.
Multiplatform
testing.
Multiplatform
testing.
Multi...
10-4-2014
© Polteq 47
Internet Explorer 6
Internet Explorer 7
Internet Explorer 8
Firefox 3.5
Firefox 3.6
Firefox 4
Safari...
10-4-2014
© Polteq 48
Any device – any platform
Multiplatform
testing.
Multiplatform
testing.
Multiplatform
testing.
Multi...
10-4-2014
© Polteq 49
Performance TestingPerformance Testing
Security TestingSecurity Testing
Manageability TestingManagea...
10-4-2014
© Polteq 50
Other aspects
• Cleaning data defects
– solved before migration
– no problems during migration
• Tes...
10-4-2014
© Polteq 51
Legislation + Regulations
=
Test basis
Incidental testing.Incidental testing.
Compliancy testing.Com...
10-4-2014
© Polteq 52
Checking for legislation and regulations
• List where data that is stored in the cloud
• Find the re...
10-4-2014
© Polteq 53
Example: Dropbox
Compliance with Laws and Law Enforcement Requests; Protection of
Dropbox's Rights.
...
10-4-2014
© Polteq 54
Performance TestingPerformance Testing
Security TestingSecurity Testing
Manageability TestingManagea...
10-4-2014
© Polteq 55
Performance TestingPerformance Testing
Security TestingSecurity Testing
Manageability TestingManagea...
Upcoming SlideShare
Loading in...5
×

Testing Cloud Services: SaaS, PaaS, and IaaS

1,010

Published on

Cloud computing has changed the environment of testing. Its use is increasing for hosting business applications (SaaS) and testing (TaaS). Martin Pol and Jeroen Mengerink focus on SaaS, describing the relevant infrastructure and platform services (IaaS and PaaS). How do we test performance of the cloud itself? How do we make sure that the continuity of services is guaranteed? How do we cope with elasticity and the philosophy of bring-your-own-device (BYOD)? Martin and Jeroen discuss the risks that arise when implementing cloud computing―some traditional, but others completely new. Learn how to mitigate these risks with current, modified, and new test techniques. As testers, we must be involved earlier in the cloud selection process. Testers should help to create and evaluate selection criteria to minimize risk. In addition, testers should be involved in the project longer as testing in production is needed to determine if the Service Level Agreements are being met.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,010
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
168
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Testing Cloud Services: SaaS, PaaS, and IaaS"

  1. 1. TF Half-day Tutorials 5/6/2014 8:30:00 AM Testing Cloud Services: SaaS, PaaS, and IaaS Presented by: Martin Pol Jeroen Mengerink Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  2. 2. Martin Pol Polteq Martin Pol has played a significant role in helping to raise the awareness and improve the performance of testing worldwide. Martin provides international testing consulting services through POLTEQ Test Services BV. He’s gained experience by managing testing processes and implementing and improving structured testing in many organizations around the world. A co-author of Test Process Improvement, a classic text on models for improving testing, Martin has developed approaches to successfully manage test outsourcing services. In 2010, Martin received the Knight in the Order of Orange-Nassau award from The Netherlands for his lifetime contributions to the IT and software industries. Jeroen Mengerink Polteq As a test consultant for the Netherlands-based Polteq Test Services B.V. Jeroen Mengerink has performed multiple TPI assessments worldwide. His technical skills allow him to team with developers in testing websites, APIs, and web services. Jeroen performs both functional testing and performance testing. In addition to his work for clients, he is involved within various test innovations in the area of agile. Jeroen teaches the Certified Agile Tester course and several test courses on agile, SOA, and cloud; coauthored Testing Cloud Services; and blogs at jmengerink.wordpress.com. Follow him on Twitter @AngusVB.
  3. 3. 10-4-2014 © Polteq 1 Testing Cloud Services: SaaS, PaaS and IaaS Martin Pol Jeroen Mengerink Agenda • Introduction Cloud computing • Challenges Risks • Solutions Test measures
  4. 4. 10-4-2014 © Polteq 2 ISBN 978-1-937538-38-5 In the cloud?
  5. 5. 10-4-2014 © Polteq 3 searching, recording, accounting, paying, writing, reviewing, tracking, calculating, developing, listening, analyzing, transmitting, learning, controlling, purchasing, testing, alarming, changing, updating, deleting, accessing, rejecting, correcting, studying, booking, receiving, tracing, protecting, deciding, managing, teaching, facilitating, identifying, copying, removing, demonstrating, checking, showing, selecting, subscribing, unsubscribing, sharing, mailing, communicating, reading, playing, working, meeting, gambling, shopping, storing, cross checking, retrieving, configuring, sketching, saving, accelerating, enhancing, creating, growing, checking in, checking out, finding out, reaching, denying, talking, designing, making, verifying, measuring Email Surf Transfer Develop and Test Operate and Manage Store
  6. 6. 10-4-2014 © Polteq 4 storage claim 80% unused redundancy limitations environmentally unfriendly management overheadcosts for innovation standard software bandwidth internet technologySOA virtualization
  7. 7. 10-4-2014 © Polteq 5 US: National Institute of Standards and Technology http://www.nist.gov Essential characteristics On-demand service Self service provisioning, pay-per-use No human interaction US: National Institute of Standards and Technology http://www.nist.gov Essential characteristics On-demand service Broad network access Standard mechanisms over networks “Any” client
  8. 8. 10-4-2014 © Polteq 6 US: National Institute of Standards and Technology http://www.nist.gov Essential characteristics On-demand service Broad network access Resource pooling Multi-tenant Storage, processing, memory, virtual machines, … Location independent US: National Institute of Standards and Technology http://www.nist.gov Essential characteristics On-demand service Broad network access Resource pooling Rapid elasticity Rapid scale in and out “Any quantity” at any time
  9. 9. 10-4-2014 © Polteq 7 US: National Institute of Standards and Technology http://www.nist.gov Essential characteristics On-demand service Broad network access Resource pooling Rapid elasticity Measured service Controlled resource use Transparency, pay-per-use US: National Institute of Standards and Technology http://www.nist.gov Essential characteristics On-demand service Broad network access Resource pooling Rapid elasticity Measured service Deployment models – private cloud – community cloud – public cloud – hybrid cloud Service Models Software as a Service Platform as a Service Infrastructure as a Service
  10. 10. 10-4-2014 © Polteq 8 Service models • Nocloud • Infrastructure as a Service • Platform as a Service • Software as a Service Application Platform Virtualization Hardware CloudInternal Implementation models • Public • Private • Community • Hybrid
  11. 11. 10-4-2014 © Polteq 9 What is “done” in the cloud? >500 Private Hybride Community IaaS, PaaS, DaaS, SaaS Taas *aaS Data Centre Data Management Business processes Consumer Public SaaS Surf and mail Apps Social media Dropbox Google services Spotify Picasa Games …………… <500 employees Public *aaS Mail Storage Infrastructure CRM Finance Business processes Continuity Privacy Multi platform Legislation Cyber crime Impact organisation Standards 143143
  12. 12. 10-4-2014 © Polteq 10 Continuity Privacy Multi platform Legislation Cyber crime Impact organisation StandardsPerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks
  13. 13. 10-4-2014 © Polteq 11 PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks Other customers YOUR Operational Profile YOUR Operational Profile YOUR Operational Profile PLUS YOUR Operational Profile PLUS PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks
  14. 14. 10-4-2014 © Polteq 12 PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks Everything over the web The idea: “it’s safe” The idea: “it’s safe” Home ground for hackers Home ground for hackers PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality ManageabilityManageability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks
  15. 15. 10-4-2014 © Polteq 13 PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks Bring Your Own Device No free choice of device. No free choice of device. Endless possibilities. Endless possibilities. PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks
  16. 16. 10-4-2014 © Polteq 14 PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks Internet connection lost @ supplier @ user @ other systems ‘Off line” does not work Information is lost
  17. 17. 10-4-2014 © Polteq 15 PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks
  18. 18. 10-4-2014 © Polteq 16 PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks Mismatch service <> business process Functionality is changed Insufficient usability PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks
  19. 19. 10-4-2014 © Polteq 17 Backup and recovery Taken care of.Taken care of. Who will support me? Who will support me? PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks
  20. 20. 10-4-2014 © Polteq 18 Updates, patches, fixes, Planned and controlled Planned and controlled Do I have a choice? Do I have a choice? PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality ManageabilityManageability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & regulationsLegislation & regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks
  21. 21. 10-4-2014 © Polteq 19 Where is my data? And is that OK? In house.In house. SomewhereSomewhere PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & regulationsLegislation & regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & regulationsLegislation & regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks
  22. 22. 10-4-2014 © Polteq 20 PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & regulationsLegislation & regulations Suppliers & outsourcingSuppliers & outsourcing RisksRisks PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & regulationsLegislation & regulations Suppliers & outsourcingSuppliers & outsourcing RisksRisks
  23. 23. 10-4-2014 © Polteq 21 PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & regulationsLegislation & regulations Suppliers & outsourcingSuppliers & outsourcing RisksRisks Vendor lock in No agreements Supplier of the supplier of the supplier Supplier is taken over Testing? Check Review Monitor Interview Proof of concept
  24. 24. 10-4-2014 © Polteq 22 Testing! Check Review Monitor Interview Proof of concept TestenProef Intake InterviewProof of concept Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures TestenProef Intake InterviewProof of concept
  25. 25. 10-4-2014 © Polteq 23 PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures
  26. 26. 10-4-2014 © Polteq 24 PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasuresPerformance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks Architecture From “individual” risks to “individual” test measures Architecture From “individual” risks to “individual” test measures
  27. 27. 10-4-2014 © Polteq 25 Selection Implementation Production Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures
  28. 28. 10-4-2014 © Polteq 26 Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks Selection Criteria Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Completeness Controllable For service For supplier Spec’s and terms References
  29. 29. 10-4-2014 © Polteq 27 “Inspiration List” CRITERION PRIO Functional Do the service and the specific business processes align? Does the service fit well in the E2E business process? Is the service sufficiently adaptable to specific requirements? Are many adjustments needed? Is customization possible Is (a lot of) customization needed? Are the required platforms supported? Are “het nieuwe werken” and BYOD supported sufficiently? Is it possible to connect / integrate the service with the other systems? Are sufficient manuals and/or courses available? Implementation Is the impact on current activities acceptable? Is a feasible route for migration towards the service available?
  30. 30. 10-4-2014 © Polteq 28 “Inspiration List” CRITERION PRIO Support Are changes in the service announced beforehand? Are sufficient test facilities available around the service (test environment, test tooling, testware, access to infrastructure, …)? Are there sufficient support facilities? Is it clear how incidents can be reported? Are incidents resolved fast enough? Performance Are response times low enough? Is the number of possible simultaneous users high enough? Is bandwidth sufficient? Is sufficient potential for growth available? Is the actual use charged correctly? “Inspiration List” CRITERION PRIO Security Are adequate authorization and authentication possibilities in place? Is the physical security of the service locations sufficient? Is the support access security of the service sufficient? Is mutual access security between customers sufficient? Are data changes traceable? Is data storage for the service reliable? Is deleting data in the service reliable? Is security of the connection to the service sufficient? Are security options for the customer sufficient? Does the supplier have security certificates? (for example SAS 70 type II)? Availability Is the level of availability for the service sufficient? Are back-up / fail-over / disaster-recovery provisions sufficient?
  31. 31. 10-4-2014 © Polteq 29 “Inspiration List” CRITERION PRIO Law and regulations Does the data location comply to all legal requirements? Does the data processing comply to all legal requirements? Do the terms contain parts that are conflicting to the duties of the customer? Supplier Is clear what happens when the contract ends, or in case of bankruptcy or conflict? Is a good helpdesk available? Does the supplier have experience in: - Offering this particular service? - Offering services in general? - Developing services? - The customer’s field? - Developing, testing and supporting services (know how)? Do methods used by supplier align with those of the customer (if relevant)? “Inspiration List” CRITERION PRIO Supplier Is quality assurance arranged? Is the supplier ahead in its field? Is the size of the supplier in accordance with the expectations of the customer? Does the supplier have a good reputation (are there references)? Is providing services the core business of the supplier? Does the supplier have opportunities for future expansion? Does the supplier speak the same language? Is transparency and flexibility of the supplier sufficient?
  32. 32. 10-4-2014 © Polteq 30 Proof of Concept Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Dynamic testing More suppliers Time boxing Representative Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks
  33. 33. 10-4-2014 © Polteq 31 Known measures tuned and tweaked New measures developed Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Load Testing YOUR Operational Profile YOUR Operational Profile YOUR Operational Profile PLUS ACTUAL MOMENT YOUR Operational Profile PLUS ACTUAL MOMENT Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures
  34. 34. 10-4-2014 © Polteq 32 Operational profile Performance testing • Test cases aimed at specific bottlenecks • Including cloud aspects in test cases • Test setup for a performance test • Representative?
  35. 35. 10-4-2014 © Polteq 33 Stress Testing Yes, you can!Yes, you can! Definitely NOT!Definitely NOT! Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Elasticity Load and stress.Load and stress. Load and elasticity.Load and elasticity. Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures
  36. 36. 10-4-2014 © Polteq 34 load load test – ‘up’ extend? 200 charged 100 charged no yes path test 99 100 101 boundary values ‘up’ tc 1: use=99, pay 100 tc 2: use=100, pay 100 tc 3: use=101, pay 200 ‘down’ tc1: use=101, pay 200 tc2: use=100, pay 100 tc3: use=99, pay 100 boundary values load test – ‘down’ load load test – ‘up’ extend? 200 charged 100 charged no yes path test 99 100 101 boundary values ‘up’ tc 1: use=99, pay 100 tc 2: use=100, pay 100 tc 3: use=101, pay 200 ‘down’ tc1: use=101, pay 200 tc2: use=100, pay 100 tc3: use=99, pay 100 boundary values load test – ‘down’ • (Automatic) scaling up or down does not perform as required • At scaling moments functional problems emerge • Insight in use based costs is not sufficient
  37. 37. 10-4-2014 © Polteq 35 ISO 27001 aspects: • Confidentiality of the data and the accompanying risk that unauthorized people can view the data • Integrity of data and the accompanying risk that data is altered or lost unintentionally • Availability of data and the accompanying risk that data (and services) is not available when it is required Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures ISO 27001 aspects: • Confidentiality of the data and the accompanying risk that unauthorized people can view the data • Integrity of data and the accompanying risk that data is altered or lost unintentionally • Availability of data and the accompanying risk that data (and services) is not available when it is required Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures • Who has access to the data? • Can the user trust that the data is correct? • Can the user gain access to the data at all times?
  38. 38. 10-4-2014 © Polteq 36 • Security at: – Network – Supplier – User Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Testing security robustness against Internet attacks - Directory traversal. Read and/or write in directories other than those allowed. - XML external entity attack. Include extra (bad) data in an XML file. - SQL injection. Request and/or change data by manipulating SQL queries. - Cross-site scripting (XSS). Transfer data to other websites without the user knowing. - Session manipulation. Skip steps or validation in a session. • Security at: – Network – Supplier – User • Encryption • Authentication and authorisation Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures IDaaS
  39. 39. 10-4-2014 © Polteq 37 • Security at: – Network – Supplier – User • Encryption • Authentication and authorisation • Test logs and audit trails • Security Audits Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures IDaaS Experts Security patch routines • Completeness and correctness of specifications and manuals – Supplier – User • Availability of test environments Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Interface specifications Supported platforms Business process specs User manuals
  40. 40. 10-4-2014 © Polteq 38 Manageablity of test environments • Everything in the cloud Manageablity of test environments • Link all current environments to the service
  41. 41. 10-4-2014 © Polteq 39 Manageablity of test environments • Link Production to the real service • Link other environments to a MOCK SERVICE (or another instance of the service) • Completeness and correctness of specifications and manuals – Supplier – User • Availability of test environments • Management of: – Defects – Changes Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures
  42. 42. 10-4-2014 © Polteq 40 Defect Management Incident Supplier resolves it Client resolves it Incident not resolved Test Change work process Change configuration Custom solution Service not selected Terminate use of service Workaround work instruction Test Test Test Test Migrate and test • Completeness and correctness of specifications and manuals – Supplier – User • Availability of test environments • Management of: – Defects – Changes • Maintainability of the software Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures
  43. 43. 10-4-2014 © Polteq 41 • Role of system architecture • Monitoring and Logging • Guarantees and SLA’s • Test fail-over mechanism • Test online/offline Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Fail-over testing A: disrupted B: active A: active B: inactive A is disrupted B takes over service A: inactive B: active disruptioninAended nochange Aisdisrupted nochange A: active B: disrupted B is disrupted A takes over service Bisdisrupted nochange disruptioninBended nochange A: disrupted B: disrupted
  44. 44. 10-4-2014 © Polteq 42 Fail-over testing A: disrupted B: active A: active B: inactive A is disrupted B takes over service A: inactive B: active disruptioninAended nochange Aisdisrupted nochange A: active B: disrupted B is disrupted A takes over service Bisdisrupted nochange disruptioninBended nochange A: disrupted B: disrupted • Has the configuration been disturbed? • Is the failure even noticed? • Does the automatic failover start to work? • Are there any transactions lost? • Is there any data lost (counts, checksums)? • If there is an audit trail, does it function properly? • Is performance back to normal? • Are there any incidents from the functional regression test (perhaps a limited set, for instance aimed at the fifty most used or most vital functions)? Fail-over testing A: disrupted B: active A: active B: inactive A is disrupted B takes over service A: inactive B: active disruptioninAended nochange Aisdisrupted nochange A: active B: disrupted B is disrupted A takes over service Bisdisrupted nochange disruptioninBended nochange A: disrupted B: disrupted Test management aspects • Sufficient technical support • Sufficient functional knowledge of the E2E processes • All planned service tests completed • The right authorizations in the services • A supplier willing to cooperate.
  45. 45. 10-4-2014 © Polteq 43 Online – Offline Use case testing. Global testing. Use case testing. Global testing. Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Online – Offline Use case testing. Global testing. Use case testing. Global testing. Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Off line tests focussed on problems: • Work continues, based on out-of-date information, and this information could be changed in the cloud during the offline period. • The users are not aware that they are working (partly) online (and are lead to believe differently).* • Synchronization conflicts arise because data is changed locally as well as in the cloud.
  46. 46. 10-4-2014 © Polteq 44 Online – Offline Use case testing. Global testing. Use case testing. Global testing. Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Off line test cases: • End the connection and check whether the users can see that they are working offline. • Disrupt the connection (for instance, a port or a certain type of IP traffic) and check whether problems arise. • Check whether changes that are made offline find their way to the cloud when online status is regained. • Check whether conflicts between offline and cloud data are handled robustly (which is in fact a functional requirement). Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing caused by Legislation & Regulations Testing caused by Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures
  47. 47. 10-4-2014 © Polteq 45 Functional test objectives • Does the service fit the business processes and vv? • Is the service quality sufficient (number of bugs)? • Is the service sufficiently user friendly? • Is the service configuration done correctly? • Does supplier customization function properly? • Does customer customization function properly? • Do interfaces work properly? • Are platforms properly supported? • Does everything work after changes (is there no regression)? Functional test objectives • Does the service fit the business processes and vv? • Is the service quality sufficient (number of bugs)? • Is the service sufficiently user friendly? • Is the service configuration done correctly? • Does supplier customization function properly? • Does customer customization function properly? • Do interfaces work properly? • Are platforms properly supported? • Does everything work after changes (is there no regression)? PCT UCT E2E ET User documentation Technique – syntax – semantics – non functional
  48. 48. 10-4-2014 © Polteq 46 Any device – any platform Multiplatform testing. Multiplatform testing. Multiplatform testing. Multiplatform testing. Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing caused by Legislation & Regulations Testing caused by Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures 3997 distinct Android devices http://opensignal.com/reports/fragmentation.php
  49. 49. 10-4-2014 © Polteq 47 Internet Explorer 6 Internet Explorer 7 Internet Explorer 8 Firefox 3.5 Firefox 3.6 Firefox 4 Safari 4 Safari 5 Chrome11 Opera11 Windows XP Windows Vista Windows 7 Windows 2003 Server Windows 8 Windows CE Linux Unix Mac OS Lion Mac OS Snow Leopard iOS Android Operating systems Browsers Multi-platform testing Devices Computer Mobile phones Tablet PC Macintosh SUN NOKIA Samsung Windows Mobile iPhone ... MOTOROLA Blackberry ASUS ... Internet Explorer 6 Internet Explorer 7 Internet Explorer 8 Firefox 3.5 Firefox 3.6 Firefox 4 Safari 4 Safari 5 Chrome11 Opera11 Windows XP Windows Vista Windows 7 Windows 2003 Server Windows 8 Windows CE Linux Unix Mac OS Lion Mac OS Snow Leopard iOS Android Operating systems Browsers Multi-platform testing Devices Computer Mobile phones Tablet PC Macintosh SUN NOKIA Samsung Windows Mobile iPhone ... MOTOROLA Blackberry ASUS ...
  50. 50. 10-4-2014 © Polteq 48 Any device – any platform Multiplatform testing. Multiplatform testing. Multiplatform testing. Multiplatform testing. Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing caused by Legislation & Regulations Testing caused by Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Off line Apps Web services Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing caused by Legislation & Regulations Testing caused by Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Testing in SOA environments Testing mobile apps
  51. 51. 10-4-2014 © Polteq 49 Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Scenarios • Transfer into the cloud, applications remain the same – data moved to another location • Transfer to SaaS – data migrated to new service • Transfer from one to another SaaS – similar • Transfer out of the cloud. – similar Data conversion • Testing conversion rules • Testing conversion on input data • Testing if any data is lost • Testing ongoing transactions Existing systems Extraction Conversion Import Conversion software Service • Rounding (totals incorrect) • Field lengths (truncation) • Totals (information lost) • Date and time conversions what means 08-09-11? • Audit trail, check sums • E2E business scenario’s
  52. 52. 10-4-2014 © Polteq 50 Other aspects • Cleaning data defects – solved before migration – no problems during migration • Testing security aspects – during and after migration – not TOO much data migrated • Testing performance – speed (how long does it take?) – volume (capacity sufficient?) – stability at full volume Example: email to the cloud • Tools migrate existing emails to the cloud • Low risk: – migrating one or some mailboxes and executing a limited testing – if successful: implementation for all mail boxes • High risk: – no emails lost in migration? – formatting of the emails still correct? – all attachments still there? – all attributes migrated (priorities, timestamps, flags, …)? Legal importance of email reading, forwarding, replying, check on contents
  53. 53. 10-4-2014 © Polteq 51 Legislation + Regulations = Test basis Incidental testing.Incidental testing. Compliancy testing.Compliancy testing. Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Sarbanes Oxley Where is my data stored? – nothing, or hardly anything, to be found on this subject – service stores data outside the borders of permitted countries additional measures? – service stores data within the borders of permitted counties okay data owner is responsible for ensuring that the protection of personal data is at the required level wherever it is held
  54. 54. 10-4-2014 © Polteq 52 Checking for legislation and regulations • List where data that is stored in the cloud • Find the requirements that are applicable to this data • Check supplier terms with customer’s requirements • Perform (external) audit for high risk • Test manager provides advice, management decides Legal support needed for high risk Example. A supplier of a storage service claims to be the owner of the intellectual capital of all data stored at their facilities. It is highly unlikely that this is compatible with the interests of the organization that is the actual owner of the data. Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Legal issues – threats
  55. 55. 10-4-2014 © Polteq 53 Example: Dropbox Compliance with Laws and Law Enforcement Requests; Protection of Dropbox's Rights. • We may disclose to parties outside Dropbox files stored in your Dropbox and information about you that we collect when we have a good faith belief that disclosure is reasonably necessary to (a) comply with a law, regulation or compulsory legal request; (b) protect the safety of any person from death or serious bodily injury; (c) prevent fraud or abuse of Dropbox or its users; or (d) to protect Dropbox’s property rights. If we provide your Dropbox files to a law enforcement agency as set forth above, we will remove Dropbox’s encryption from the files before providing them to law enforcement. However, Dropbox will not be able to decrypt any files that you encrypted prior to storing them on Dropbox. Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks
  56. 56. 10-4-2014 © Polteq 54 Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures Continuous End-to-End Testing Continuous Change Continuity Privacy Multi platform Legislation Cyber crime Impact organisation Standards Check Intake Monitor Interview Proof of concept
  57. 57. 10-4-2014 © Polteq 55 Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality MaintainabilityMaintainability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks Testing starts early: in selection Scope of testing is widened Testing continues in production Testing starts early: in selection Scope of testing is widened Testing continues in production Performance TestingPerformance Testing Security TestingSecurity Testing Manageability TestingManageability Testing Availability & Continuity Testing Availability & Continuity Testing Functional TestingFunctional Testing Migration TestingMigration Testing Testing due to Legislation & Regulations Testing due to Legislation & Regulations Testing in ProductionTesting in Production Testing during SelectionTesting during Selection TestMeasuresTestMeasures PerformancePerformance SecuritySecurity Availability & ContinuityAvailability & Continuity FunctionalityFunctionality ManageabilityManageability Legislation & RegulationsLegislation & Regulations Suppliers & OutsourcingSuppliers & Outsourcing RisksRisks Thank you!Thank you!

×