Testing in a Multi-CloudEnvironmentSeema Jethani
Who am I?Seema Jethani @seemajSenior Product Manager, Dell( formerly Enstratius / enStratus )Blog★http://seemaj.wordpress....
CLOUDS ARE LIKEBABIES
Bringing joy
But also presenting...UNIQUE CHALLENGES
Speaking their own language
Capability TerminologyAccountAWS = AccountTerremark = OrganizationRegionAWS = RegionTerremark = EnvironmentData centerAWS ...
And are sometimes hard towork with
Examples of challenging cloud behavior1. Requiring paperwork / manual processing2. Requiring calling support to create pri...
Each is able to do differentthings in different ways
Partial Cloud Feature MatrixFeature AWS Rackspace Terremark JoyentLaunch Server S S S SPause Server S NS S SMake Image S S...
Examples of the cloud nuances1.Make ImageCloudSigma requires server to be stopped before making an image2.Attaching a volu...
Cloud requires us torethink our testingapproach
We need to learn “cloud speak”Understand differences in behaviorBut also importantly
Plan for failure
How is this different?Customers expect cloud failurebut alsoExpect recovery from cloud behavior to be transparent. Theydon...
How is this different?Customers expect cloud failureBut don’t expect it to impact their businessTest for cloud failure and...
Enstratius testing journey
Enstratius Cloud Management Platform
The release process
buildTodayArtifacts Installer API testsGithubArtifactoryVagrantAWS
JenkinsReportNG ReportAutomatedTestFramework(TestNG)GitHubEnstratiusAPI ServerCloud ACloud ZestrovantAPI Test Framework
Cloud accountaccess keys, testconfigsHeaderConfig.xmlCloud specificresource/methodexclusionsJSON/XML Inputs forREST Calls ...
API test workflow<?xml&version="1.0"&encoding="UTF98"?>!<cloudService!name="AWS">!<excluded>!<resource!name="kvdb"!/>!<reso...
The extra steps1. Not all functions are supported by all clouds. An exclusion listneeds to be maintained.2. Cloud specific ...
BuildNear-term updatesArtifacts LaunchAPI testsGithubArtifactoryAWSJenkinsInstallerCheckout Teardown
BuildFutureArtifacts LaunchAPI testsGithubArtifactoryAWSJenkinsInstaller1. Checkout 6. TeardownSelenium
Other tools on the radar
Visualizing performancehttp://twitter.github.io/zipkin/
Vulnerability scans
Upcoming SlideShare
Loading in...5
×

Multi-Cloud testing

956

Published on

Testing in a multi-cloud environment presents unique challenges. This presentation reviews some of these challenges and includes some examples of how they can be addressed

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
956
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
36
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Multi-Cloud testing

  1. 1. Testing in a Multi-CloudEnvironmentSeema Jethani
  2. 2. Who am I?Seema Jethani @seemajSenior Product Manager, Dell( formerly Enstratius / enStratus )Blog★http://seemaj.wordpress.com★http://cloudfieldnotes.com/author/seema/Slidesharehttp://www.slideshare.net/svjethani/presentations
  3. 3. CLOUDS ARE LIKEBABIES
  4. 4. Bringing joy
  5. 5. But also presenting...UNIQUE CHALLENGES
  6. 6. Speaking their own language
  7. 7. Capability TerminologyAccountAWS = AccountTerremark = OrganizationRegionAWS = RegionTerremark = EnvironmentData centerAWS = Availability ZoneTerremark = Compute PoolMachine ImageAWS = Machine ImageVMware = Template vAppJoyent = DatasetsOpenStack = ImageCloudStack, Terremark = TemplateFirewallAWS, OpenStack, CloudStack = Security GroupTerremark = FirewallVolumeAWS = Elastic Block StorageVMware, IBM, Terremark = DiskOpenStack = Cinder or Nova VolumePartialList
  8. 8. And are sometimes hard towork with
  9. 9. Examples of challenging cloud behavior1. Requiring paperwork / manual processing2. Requiring calling support to create private images3. Requiring a new account for new API versions, thus forgettingbackward compatibility4. Using same API version, but continuing to make changes5. Taking hours to launch a server or launching server requestsserially6. Having a ridiculously low cut-off for throttling API requests e.g.dozens of API calls per second7. Not providing a way for the VM to discover its own ID
  10. 10. Each is able to do differentthings in different ways
  11. 11. Partial Cloud Feature MatrixFeature AWS Rackspace Terremark JoyentLaunch Server S S S SPause Server S NS S SMake Image S S S NSCreate Snapshot S S NS NSCreate Volume S S NS NSCreate Loadbalancer S S NS NSCreate/ Reserve IP S NS S NS
  12. 12. Examples of the cloud nuances1.Make ImageCloudSigma requires server to be stopped before making an image2.Attaching a volumee.g. Terremark requires the server to be stopped for for volume tobe attached3.Creating a snapshote.g. Make a snapshot from the volume fails in CloudCentral if theVolume is not attached to a server4.Connecting via sshCloudSigma requires users to access a centos server via vnc firstto enable ssh
  13. 13. Cloud requires us torethink our testingapproach
  14. 14. We need to learn “cloud speak”Understand differences in behaviorBut also importantly
  15. 15. Plan for failure
  16. 16. How is this different?Customers expect cloud failurebut alsoExpect recovery from cloud behavior to be transparent. Theydon’t care about root cause infrastructure issuesTest for failure , auto-recovery , MTTR
  17. 17. How is this different?Customers expect cloud failureBut don’t expect it to impact their businessTest for cloud failure and its impact on customer applicationperformance.
  18. 18. Enstratius testing journey
  19. 19. Enstratius Cloud Management Platform
  20. 20. The release process
  21. 21. buildTodayArtifacts Installer API testsGithubArtifactoryVagrantAWS
  22. 22. JenkinsReportNG ReportAutomatedTestFramework(TestNG)GitHubEnstratiusAPI ServerCloud ACloud ZestrovantAPI Test Framework
  23. 23. Cloud accountaccess keys, testconfigsHeaderConfig.xmlCloud specificresource/methodexclusionsJSON/XML Inputs forREST Calls commonto all cloudsSetup Test bed on Cloud under testTear Down Test bed on Cloud under testUpdateAdd Delete ListAtomic TestsCloud Specific Inputsfor REST Calls (if any)estrov
  24. 24. API test workflow<?xml&version="1.0"&encoding="UTF98"?>!<cloudService!name="AWS">!<excluded>!<resource!name="kvdb"!/>!<resource!name="customer">!<method!name="add"!/>!<method!name="changeCurrency"!/>!<method!name="revertCurrency"!/>!<method!name="setTimeZone"!/>!<method!name="revertTimeZone"!/>!</resource>!</excluded>!</cloudService>{"addLoadBalancer":![!{!"region":{"regionId":REGION_ID},!"budget":BILLINGCODE_ID,!"description":"TestNG_Trial_LB",!"name":DYNAMIC,!"label":"red",!"customer":{"customerId":CUSTOMER_ID},!"dataCenters":[{"dataCenterId":DATACENTER_ID}],!"owningGroups":[{"groupId":GROUP_ID}],!"listeners":![{"protocol":"HTTP","publicPort":80,"privatePort":80}]!}!]!}startRead the cloudaccess keys & testconfigsIterate through thetest resourcesExcludeResource?Run atomic testsGenerate reportHeaderconfig.xmlAWS.xmlRead resource/method inputs<?xml&version="1.0"&encoding="UTF98"&standalone="no"?>!<CONFIG>!<BILLINGCODE_ID>403</BILLINGCODE_ID>!<USER_ID>50950</USER_ID>!!<DUPLICATE_BILLING_ID>402</DUPLICATE_BILLING_ID>!<DUPLICATE_GRP_ID>501</DUPLICATE_GRP_ID>!<ROLE_ID>55501</ROLE_ID>!<GROUP_ID>500</GROUP_ID>!!<ACCOUNT_ID>200</ACCOUNT_ID>!<CLOUD_ID>1</CLOUD_ID>!<CUSTOMER_ID>200</CUSTOMER_ID>!</CONFIG>GenerateDependencyresources<?xml&version="1.0"&encoding="UTF98"&standalone="yes"?>!<header>!<csp>AWS</csp>!<resource>all</resource>!<accessKey>aws_accesskey</accessKey>!<secretKey>aws_secretkey</secretKey>!<apiServer>enstratius_apiserver_ip</apiServer>!<apiServerPort>enstratius_apiserver_port</apiServerPort>!<version>enstratius_apiserver_version</version>!<testForFailure>false</testForFailure>!</header>stopLoadbalancer/add.jsonConfig_200.xmlFinished iteratingYesNo
  25. 25. The extra steps1. Not all functions are supported by all clouds. An exclusion listneeds to be maintained.2. Cloud specific behavior needs to be codede.g. Terremark requires the server to be stopped for for volume to beattached3. Sanity check - Issues found must be verified directly using cloudconsole / cloud API
  26. 26. BuildNear-term updatesArtifacts LaunchAPI testsGithubArtifactoryAWSJenkinsInstallerCheckout Teardown
  27. 27. BuildFutureArtifacts LaunchAPI testsGithubArtifactoryAWSJenkinsInstaller1. Checkout 6. TeardownSelenium
  28. 28. Other tools on the radar
  29. 29. Visualizing performancehttp://twitter.github.io/zipkin/
  30. 30. Vulnerability scans
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×