Testing of Cloud ServicesThe Approach: From Risks to Test Measures    Kees Blokland and Jeroen Mengerink, Polteq          ...
Testing of Cloud ServicesThe Approach: From Risks to Test            Measures              Kees Blokland           Jeroen ...
searching, recording, accounting, paying, writing,reviewing, tracking, calculating, developing, listening,analyzing, trans...
Develop and Test     Email                         Surf                         Transfer    Operate and Manage     Store5
Develop and Test                                     Email                                                         Surf   ...
Develop and Test                                 Email                                                     Surf           ...
searching, recording, accounting, paying, writing,reviewing, tracking, calculating, developing, listening,analyzing, Test ...
Essential characteristics On-demand service    Self service provisioning, pay-per-use    No human interactionUS: Nation...
Essential characteristics On-demand service Broad network access    Standard mechanisms over networks    “Any” clientU...
Essential characteristics On-demand service Broad network access Resource pooling    Multi-tenant    Storage, process...
Essential characteristics On-demand service Broad network access Resource pooling Rapid elasticity    Rapid scale in ...
Essential characteristics On-demand service Broad network access Resource pooling Rapid elasticity Measured service  ...
Essential characteristics On-demand service Broad network access                                      Deployment models...
Standards                      Cyber crime                      Continuity                        Privacy                 ...
Performance                  Standards                           Security Cyber crime                                     ...
Performance                                                   YOUR                                              Operationa...
Performance                                                     The idea:                                                 ...
Performance                                                 No free choice of                                             ...
Performance                                                 Taken care of.                         Security         Availa...
Performance                                                     Planned and                                               ...
Performance                                                 In house.                         Security         Availabilit...
Performance                         Security         Availability & Continuity                    Functionality           ...
Testing?           Interview           Check             Trial                                Proof of concept            ...
Testing!                         Interview                         Check                           Trial                  ...
Te s t M e a s u r e s                         Testing during Selection                         Performance Testing       ...
Te s t M e a s u r e s                    Performance      Testing during Selection                                     Pe...
Te s t M e a s u r e s                    Performance      Testing during Selection                                     Pe...
Te s t M e a s u r e s                    Performance      Testing during Selection                                     Pe...
Te s t M e a s u r e s                    Performance            Testing during Selection                                 ...
Te s t M e a s u r e s                      Testing during Selection                      Performance Testing       Select...
Te s t M e a s u r e s                    Performance      Testing during Selection                                     Pe...
Te s t M e a s u r e s                          Testing during Selection                          Performance Testing     ...
Te s t M e a s u r e s                        Testing during Selection                        Performance Testing         ...
Te s t M e a s u r e s                    Performance      Testing during Selection                                     Pe...
Te s t M e a s u r e s                              Testing during Selection                              Performance Test...
Te s t M e a s u r e s                           Testing during Selection      YOUR Operational Profile       Performance ...
Te s t M e a s u r e s                               Testing during Selection                               Performance Te...
Te s t M e a s u r e s                                 Testing during Selection      Multiplatform        testing.        ...
Windows XP                                                                      Internet Explorer 6                       ...
Te s t M e a s u r e s                                 Testing during Selection      Multiplatform        testing.        ...
Te s t M e a s u r e s                                    Testing during Selection     Incidental testing.            Perf...
European Commissioner Kroes, Sept 27• Strategy document to promote cloud computing• 2,5 million jobs, 160 billion Euros• M...
Te s t M e a s u r e s                                    Testing during Selection     Incidental testing.            Perf...
Te s t M e a s u r e s                    Performance      Testing during Selection                                     Pe...
Te s t M e a s u r e s                       Testing during Selection                       Performance Testing       Cont...
Standards                        Cyber crime                        Check                        Continuity    Interview  ...
Standards                        Cyber crime                        Check                        Continuity    Interview  ...
Te s t M e a s u r e s                    Performance              Testing during Selection                               ...
52
Te s t M e a s u r e s                    Performance        Testing during Selection                                     ...
Te s t M e a s u r e s                    Performance         Testing during Selection                                    ...
Testing cloud services - EuroSTAR
Testing cloud services - EuroSTAR
Upcoming SlideShare
Loading in …5
×

Testing cloud services - EuroSTAR

255 views
195 views

Published on

Presentation at EuroSTAR on how to test when cloud computing is involved.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
255
On SlideShare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Testing cloud services - EuroSTAR

  1. 1. Testing of Cloud ServicesThe Approach: From Risks to Test Measures Kees Blokland and Jeroen Mengerink, Polteq @esconfs www.eurostarconferences.com #esconfs
  2. 2. Testing of Cloud ServicesThe Approach: From Risks to Test Measures Kees Blokland Jeroen Mengerink Polteq Test Services BV The Netherlands
  3. 3. searching, recording, accounting, paying, writing,reviewing, tracking, calculating, developing, listening,analyzing, transmitting, learning, controlling,purchasing, testing, alarming, changing, updating,deleting, accessing, rejecting, correcting, studying,booking, receiving, tracing, protecting, deciding,managing, teaching, facilitating, identifying, copying,removing, demonstrating, checking, showing,selecting, subscribing, unsubscribing, sharing,mailing, communicating, reading, playing, working,meeting, gambling, shopping, storing, crosschecking, retrieving, configuring, sketching, saving,accelerating, enhancing, creating, growing, checkingin, checking out, finding out, reaching, denying,talking, designing, making, verifying, measuring
  4. 4. Develop and Test Email Surf Transfer Operate and Manage Store5
  5. 5. Develop and Test Email Surf Transfer redundancy limitations 80% unused storage claim environmentally unfriendly Operate and Manage Store6
  6. 6. Develop and Test Email Surf Transfer SOA internet technology virtualization standard software bandwidth Operate and Manage Store7
  7. 7. searching, recording, accounting, paying, writing,reviewing, tracking, calculating, developing, listening,analyzing, Test Develop and transmitting, learning, controlling, Email Surfpurchasing, testing, alarming, changing, updating, Transferdeleting, accessing, rejecting, correcting, studying,booking, receiving, tracing, protecting, deciding,managing, teaching, facilitating, identifying, copying,removing, demonstrating, checking, showing,selecting, subscribing, unsubscribing, sharing,mailing, communicating, reading, playing, working,meeting, gambling, shopping, storing, crosschecking, retrieving, configuring, sketching, saving,accelerating, enhancing, creating, growing,Store Operate and Manage checkingin, checking out, finding out, reaching, denying,talking, designing, making, verifying, measuring8
  8. 8. Essential characteristics On-demand service  Self service provisioning, pay-per-use  No human interactionUS: National Institute of Standards and Technology http://www.nist.gov
  9. 9. Essential characteristics On-demand service Broad network access  Standard mechanisms over networks  “Any” clientUS: National Institute of Standards and Technology http://www.nist.gov
  10. 10. Essential characteristics On-demand service Broad network access Resource pooling  Multi-tenant  Storage, processing, memory, virtual machines, …  Location independentUS: National Institute of Standards and Technology http://www.nist.gov
  11. 11. Essential characteristics On-demand service Broad network access Resource pooling Rapid elasticity  Rapid scale in and out  “Any quantity” at any timeUS: National Institute of Standards and Technology http://www.nist.gov
  12. 12. Essential characteristics On-demand service Broad network access Resource pooling Rapid elasticity Measured service  Controlled resource use  Transparency, pay-per-useUS: National Institute of Standards and Technology http://www.nist.gov
  13. 13. Essential characteristics On-demand service Broad network access Deployment models Resource pooling – private cloud – community cloud Rapid elasticity – public cloud Measured service – hybrid cloud Service Models Software as a Service Platform as a Service Infrastructure as a ServiceUS: National Institute of Standards and Technology http://www.nist.gov
  14. 14. Standards Cyber crime Continuity Privacy LegislationMulti platform Impact organisation 143
  15. 15. Performance Standards Security Cyber crime Continuïty Availability & Continuity Functionality Privacy Legislation Manageability Multi platform Legislation & RegulationsRisks Impact organisation Suppliers & Outsourcing
  16. 16. Performance YOUR Operational Profile Security Availability & Continuity Functionality Other customers Manageability Legislation & RegulationsRisks YOUR Operational Profile Suppliers & Outsourcing PLUS19
  17. 17. Performance The idea: “it’s safe” Security Availability & Continuity Functionality Everything over the web Manageability Legislation & RegulationsRisks Home ground for hackers20 Suppliers & Outsourcing
  18. 18. Performance No free choice of device. Security Availability & Continuity Functionality Bring Your Own Device Manageability Legislation & RegulationsRisks Endless possibilities.21 Suppliers & Outsourcing
  19. 19. Performance Taken care of. Security Availability & Continuity Functionality Backup and recovery Manageability Legislation & RegulationsRisks Who will support me?22 Suppliers & Outsourcing
  20. 20. Performance Planned and controlled Security Availability & Continuity Functionality Updates, patches, fixes, … Manageability Legislation & RegulationsRisks Do I have a choice?23 Suppliers & Outsourcing
  21. 21. Performance In house. Security Availability & Continuity Where is my data? Functionality Is it OK? Manageability Legislation & RegulationsRisks Somewhere…24 Suppliers & Outsourcing
  22. 22. Performance Security Availability & Continuity Functionality Manageability Legislation & RegulationsRisks25 Suppliers & Outsourcing
  23. 23. Testing? Interview Check Trial Proof of concept Intake26
  24. 24. Testing! Interview Check Trial Proof of concept Proof ofInterview Proef concept Testen Intake Intake27
  25. 25. Te s t M e a s u r e s Testing during Selection Performance Testing Security Testing Manageability Testing Availability & Continuity Testing Proof ofInterview Proef concept Testen Functional Testing Intake Migration Testing Testing caused by Legislation & Regulations28 Testing in Production
  26. 26. Te s t M e a s u r e s Performance Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Availability & Continuity Functionality Testing Functional Testing Manageability Migration Testing Legislation & Regulations Testing caused byRisks Legislation & Regulations29 Suppliers & Outsourcing Testing in Production
  27. 27. Te s t M e a s u r e s Performance Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Availability & Continuity Functionality Testing Functional Testing Manageability Migration Testing Legislation & Regulations Testing caused byRisks Legislation & Regulations30 Suppliers & Outsourcing Testing in Production
  28. 28. Te s t M e a s u r e s Performance Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Availability & Continuity Functionality Testing Functional Testing Manageability Migration Testing Legislation & Regulations Testing caused byRisks Legislation & Regulations31 Suppliers & Outsourcing Testing in Production
  29. 29. Te s t M e a s u r e s Performance Testing during Selection Performance Testing Security Security Testing Availability & Continuity Architecture Manageability Testing Availability & Continuity Functionality Testing From “individual” risks Functional Testing Manageability to “individual” test measures Migration Testing Legislation & Regulations Testing caused byRisks Legislation & Regulations32 Suppliers & Outsourcing Testing in Production
  30. 30. Te s t M e a s u r e s Testing during Selection Performance Testing Selection Security Testing Manageability Testing Availability & Continuity Implementation Testing Functional Testing Migration Testing Production Testing caused by Legislation & Regulations33 Testing in Production
  31. 31. Te s t M e a s u r e s Performance Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Availability & Continuity Functionality Testing Functional Testing Manageability Migration Testing Legislation & Regulations Testing caused byRisks Legislation & Regulations34 Suppliers & Outsourcing Testing in Production
  32. 32. Te s t M e a s u r e s Testing during Selection Performance Testing Security Testing Manageability Testing Availability & Continuity Selection Criteria Testing Functional Testing Migration Testing Testing caused by Legislation & Regulations35 Testing in Production
  33. 33. Te s t M e a s u r e s Testing during Selection Performance Testing Security Testing Manageability Testing Availability & Continuity Proof of Concept Testing Functional Testing Migration Testing Testing caused by Legislation & Regulations36 Testing in Production
  34. 34. Te s t M e a s u r e s Performance Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Availability & Continuity Functionality Testing Functional Testing Manageability Migration Testing Legislation & Regulations Testing caused byRisks Legislation & Regulations37 Suppliers & Outsourcing Testing in Production
  35. 35. Te s t M e a s u r e s Testing during Selection Performance Testing Security Testing Known measures Manageability Testing tuned and tweaked Availability & Continuity Testing Functional Testing New measures developed Migration Testing Testing caused by Legislation & Regulations38 Testing in Production
  36. 36. Te s t M e a s u r e s Testing during Selection YOUR Operational Profile Performance Testing Security Testing Manageability Testing Availability & Continuity Load Testing Testing Functional Testing Migration Testing YOUR Testing caused by Operational Profile Legislation & Regulations PLUS39 ACTUAL MOMENT Testing in Production
  37. 37. Te s t M e a s u r e s Testing during Selection Performance Testing Security Testing Manageability Testing Availability & Continuity Online – Offline Testing Functional Testing Migration Testing Testing caused by Use case testing. Legislation & Regulations Global testing.40 Testing in Production
  38. 38. Te s t M e a s u r e s Testing during Selection Multiplatform testing. Performance Testing Security Testing Manageability Testing Availability & Continuity Any device – any platform Testing Functional Testing Migration Testing Testing caused by Multiplatform Legislation & Regulations testing.41 Testing in Production
  39. 39. Windows XP Internet Explorer 6 Internet Explorer 7 Windows Vista Internet Explorer 8 Windows 7 Firefox 3.5 Windows 2003 server Firefox 3.6 Windows 8 Browsers Firefox 4 Safari 4 Windows CE Safari 5 Linux Operating Systems Chrome11 Unix Opera11 Mac OS Lion Multiplatform PC Mac OS Snowleopard SUN iOS Computer Macintosh Android iPhone .. Windows Mobile Samsung … Devices Mobile NOKIA … Xxx … ASUS.. Blackberry… Tablet MOTOROLA… Xxx …42
  40. 40. Te s t M e a s u r e s Testing during Selection Multiplatform testing. Performance Testing Security Testing Manageability Testing Availability & Continuity Any device – any platform Testing Functional Testing Migration Testing Testing caused by Multiplatform Legislation & Regulations testing.43 Testing in Production
  41. 41. Te s t M e a s u r e s Testing during Selection Incidental testing. Performance Testing Security Testing Manageability Testing Legislation + Regulations Availability & Continuity = Testing Test basis Functional Testing Migration Testing Testing caused by Legislation & Regulations Compliancy testing.44 Testing in Production
  42. 42. European Commissioner Kroes, Sept 27• Strategy document to promote cloud computing• 2,5 million jobs, 160 billion Euros• Major barriers: – Many different standards – Contract issues – Many different rules and policies• Privacy legislation: differences in 27(!) EU-countries45
  43. 43. Te s t M e a s u r e s Testing during Selection Incidental testing. Performance Testing Security Testing Manageability Testing Legislation + Regulations Availability & Continuity = Testing Test basis Functional Testing Migration Testing Testing caused by Legislation & Regulations Compliancy testing.46 Testing in Production
  44. 44. Te s t M e a s u r e s Performance Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Availability & Continuity Functionality Testing Functional Testing Manageability Migration Testing Legislation & Regulations Testing caused byRisks Legislation & Regulations47 Suppliers & Outsourcing Testing in Production
  45. 45. Te s t M e a s u r e s Testing during Selection Performance Testing Continuous Security Testing End-to-End Test Manageability Testing Availability & Continuity Testing Functionals Functional Testing and non-functionals Migration Testing Testing caused by Legislation & Regulations48 Testing in Production
  46. 46. Standards Cyber crime Check Continuity Interview Privacy LegislationTrial Proof of concept Multi platform Intake Impact organisation
  47. 47. Standards Cyber crime Check Continuity Interview Privacy LegislationTrial Proof of concept Multi platform Intake Impact organisation
  48. 48. Te s t M e a s u r e s Performance Testing during Selection Performance Testing Security Security Testing Availability & Continuity Architecture Manageability Testing Functionality starts earlier Test Availability & Continuity Testing From “individual” risks Test scope is widened Functional Testing Test will never stop Manageability to “individual” test measures Migration Testing Legislation & Regulations Testing caused byRisks Legislation & Regulations51 Suppliers & Outsourcing Testing in Production
  49. 49. 52
  50. 50. Te s t M e a s u r e s Performance Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Availability & Continuity Functionality Testing Questions? Functional Testing Manageability Migration Testing Legislation & Regulations Testing caused byRisks Legislation & Regulations53 Suppliers & Outsourcing Testing in Production
  51. 51. Te s t M e a s u r e s Performance Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Availability & Continuity Functionality Testing Thank you! Functional Testing Manageability Migration Testing Legislation & Regulations Testing caused byRisks Legislation & Regulations54 Suppliers & Outsourcing Testing in Production

×