Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Multi-Cloud testing


Published on

Testing in a multi-cloud environment presents unique challenges. This presentation reviews some of these challenges and includes some examples of how they can be addressed

Published in: Technology
  • Be the first to comment

Multi-Cloud testing

  1. 1. Testing in a Multi-CloudEnvironmentSeema Jethani
  2. 2. Who am I?Seema Jethani @seemajSenior Product Manager, Dell( formerly Enstratius / enStratus )Blog★★
  4. 4. Bringing joy
  5. 5. But also presenting...UNIQUE CHALLENGES
  6. 6. Speaking their own language
  7. 7. Capability TerminologyAccountAWS = AccountTerremark = OrganizationRegionAWS = RegionTerremark = EnvironmentData centerAWS = Availability ZoneTerremark = Compute PoolMachine ImageAWS = Machine ImageVMware = Template vAppJoyent = DatasetsOpenStack = ImageCloudStack, Terremark = TemplateFirewallAWS, OpenStack, CloudStack = Security GroupTerremark = FirewallVolumeAWS = Elastic Block StorageVMware, IBM, Terremark = DiskOpenStack = Cinder or Nova VolumePartialList
  8. 8. And are sometimes hard towork with
  9. 9. Examples of challenging cloud behavior1. Requiring paperwork / manual processing2. Requiring calling support to create private images3. Requiring a new account for new API versions, thus forgettingbackward compatibility4. Using same API version, but continuing to make changes5. Taking hours to launch a server or launching server requestsserially6. Having a ridiculously low cut-off for throttling API requests e.g.dozens of API calls per second7. Not providing a way for the VM to discover its own ID
  10. 10. Each is able to do differentthings in different ways
  11. 11. Partial Cloud Feature MatrixFeature AWS Rackspace Terremark JoyentLaunch Server S S S SPause Server S NS S SMake Image S S S NSCreate Snapshot S S NS NSCreate Volume S S NS NSCreate Loadbalancer S S NS NSCreate/ Reserve IP S NS S NS
  12. 12. Examples of the cloud nuances1.Make ImageCloudSigma requires server to be stopped before making an image2.Attaching a volumee.g. Terremark requires the server to be stopped for for volume tobe attached3.Creating a snapshote.g. Make a snapshot from the volume fails in CloudCentral if theVolume is not attached to a server4.Connecting via sshCloudSigma requires users to access a centos server via vnc firstto enable ssh
  13. 13. Cloud requires us torethink our testingapproach
  14. 14. We need to learn “cloud speak”Understand differences in behaviorBut also importantly
  15. 15. Plan for failure
  16. 16. How is this different?Customers expect cloud failurebut alsoExpect recovery from cloud behavior to be transparent. Theydon’t care about root cause infrastructure issuesTest for failure , auto-recovery , MTTR
  17. 17. How is this different?Customers expect cloud failureBut don’t expect it to impact their businessTest for cloud failure and its impact on customer applicationperformance.
  18. 18. Enstratius testing journey
  19. 19. Enstratius Cloud Management Platform
  20. 20. The release process
  21. 21. buildTodayArtifacts Installer API testsGithubArtifactoryVagrantAWS
  22. 22. JenkinsReportNG ReportAutomatedTestFramework(TestNG)GitHubEnstratiusAPI ServerCloud ACloud ZestrovantAPI Test Framework
  23. 23. Cloud accountaccess keys, testconfigsHeaderConfig.xmlCloud specificresource/methodexclusionsJSON/XML Inputs forREST Calls commonto all cloudsSetup Test bed on Cloud under testTear Down Test bed on Cloud under testUpdateAdd Delete ListAtomic TestsCloud Specific Inputsfor REST Calls (if any)estrov
  24. 24. API test workflow<?xml&version="1.0"&encoding="UTF98"?>!<cloudService!name="AWS">!<excluded>!<resource!name="kvdb"!/>!<resource!name="customer">!<method!name="add"!/>!<method!name="changeCurrency"!/>!<method!name="revertCurrency"!/>!<method!name="setTimeZone"!/>!<method!name="revertTimeZone"!/>!</resource>!</excluded>!</cloudService>{"addLoadBalancer":![!{!"region":{"regionId":REGION_ID},!"budget":BILLINGCODE_ID,!"description":"TestNG_Trial_LB",!"name":DYNAMIC,!"label":"red",!"customer":{"customerId":CUSTOMER_ID},!"dataCenters":[{"dataCenterId":DATACENTER_ID}],!"owningGroups":[{"groupId":GROUP_ID}],!"listeners":![{"protocol":"HTTP","publicPort":80,"privatePort":80}]!}!]!}startRead the cloudaccess keys & testconfigsIterate through thetest resourcesExcludeResource?Run atomic testsGenerate reportHeaderconfig.xmlAWS.xmlRead resource/method inputs<?xml&version="1.0"&encoding="UTF98"&standalone="no"?>!<CONFIG>!<BILLINGCODE_ID>403</BILLINGCODE_ID>!<USER_ID>50950</USER_ID>!!<DUPLICATE_BILLING_ID>402</DUPLICATE_BILLING_ID>!<DUPLICATE_GRP_ID>501</DUPLICATE_GRP_ID>!<ROLE_ID>55501</ROLE_ID>!<GROUP_ID>500</GROUP_ID>!!<ACCOUNT_ID>200</ACCOUNT_ID>!<CLOUD_ID>1</CLOUD_ID>!<CUSTOMER_ID>200</CUSTOMER_ID>!</CONFIG>GenerateDependencyresources<?xml&version="1.0"&encoding="UTF98"&standalone="yes"?>!<header>!<csp>AWS</csp>!<resource>all</resource>!<accessKey>aws_accesskey</accessKey>!<secretKey>aws_secretkey</secretKey>!<apiServer>enstratius_apiserver_ip</apiServer>!<apiServerPort>enstratius_apiserver_port</apiServerPort>!<version>enstratius_apiserver_version</version>!<testForFailure>false</testForFailure>!</header>stopLoadbalancer/add.jsonConfig_200.xmlFinished iteratingYesNo
  25. 25. The extra steps1. Not all functions are supported by all clouds. An exclusion listneeds to be maintained.2. Cloud specific behavior needs to be codede.g. Terremark requires the server to be stopped for for volume to beattached3. Sanity check - Issues found must be verified directly using cloudconsole / cloud API
  26. 26. BuildNear-term updatesArtifacts LaunchAPI testsGithubArtifactoryAWSJenkinsInstallerCheckout Teardown
  27. 27. BuildFutureArtifacts LaunchAPI testsGithubArtifactoryAWSJenkinsInstaller1. Checkout 6. TeardownSelenium
  28. 28. Other tools on the radar
  29. 29. Visualizing performance
  30. 30. Vulnerability scans