Multi-Cloud testing
Upcoming SlideShare
Loading in...5
×
 

Multi-Cloud testing

on

  • 1,149 views

Testing in a multi-cloud environment presents unique challenges. This presentation reviews some of these challenges and includes some examples of how they can be addressed

Testing in a multi-cloud environment presents unique challenges. This presentation reviews some of these challenges and includes some examples of how they can be addressed

Statistics

Views

Total Views
1,149
Views on SlideShare
1,124
Embed Views
25

Actions

Likes
1
Downloads
29
Comments
0

2 Embeds 25

https://twitter.com 23
http://moderation.local 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Multi-Cloud testing Multi-Cloud testing Presentation Transcript

  • Testing in a Multi-CloudEnvironmentSeema Jethani
  • Who am I?Seema Jethani @seemajSenior Product Manager, Dell( formerly Enstratius / enStratus )Blog★http://seemaj.wordpress.com★http://cloudfieldnotes.com/author/seema/Slidesharehttp://www.slideshare.net/svjethani/presentations
  • CLOUDS ARE LIKEBABIES
  • Bringing joy
  • But also presenting...UNIQUE CHALLENGES
  • Speaking their own language
  • Capability TerminologyAccountAWS = AccountTerremark = OrganizationRegionAWS = RegionTerremark = EnvironmentData centerAWS = Availability ZoneTerremark = Compute PoolMachine ImageAWS = Machine ImageVMware = Template vAppJoyent = DatasetsOpenStack = ImageCloudStack, Terremark = TemplateFirewallAWS, OpenStack, CloudStack = Security GroupTerremark = FirewallVolumeAWS = Elastic Block StorageVMware, IBM, Terremark = DiskOpenStack = Cinder or Nova VolumePartialList
  • And are sometimes hard towork with
  • Examples of challenging cloud behavior1. Requiring paperwork / manual processing2. Requiring calling support to create private images3. Requiring a new account for new API versions, thus forgettingbackward compatibility4. Using same API version, but continuing to make changes5. Taking hours to launch a server or launching server requestsserially6. Having a ridiculously low cut-off for throttling API requests e.g.dozens of API calls per second7. Not providing a way for the VM to discover its own ID
  • Each is able to do differentthings in different ways
  • Partial Cloud Feature MatrixFeature AWS Rackspace Terremark JoyentLaunch Server S S S SPause Server S NS S SMake Image S S S NSCreate Snapshot S S NS NSCreate Volume S S NS NSCreate Loadbalancer S S NS NSCreate/ Reserve IP S NS S NS
  • Examples of the cloud nuances1.Make ImageCloudSigma requires server to be stopped before making an image2.Attaching a volumee.g. Terremark requires the server to be stopped for for volume tobe attached3.Creating a snapshote.g. Make a snapshot from the volume fails in CloudCentral if theVolume is not attached to a server4.Connecting via sshCloudSigma requires users to access a centos server via vnc firstto enable ssh
  • Cloud requires us torethink our testingapproach
  • We need to learn “cloud speak”Understand differences in behaviorBut also importantly
  • Plan for failure
  • How is this different?Customers expect cloud failurebut alsoExpect recovery from cloud behavior to be transparent. Theydon’t care about root cause infrastructure issuesTest for failure , auto-recovery , MTTR
  • How is this different?Customers expect cloud failureBut don’t expect it to impact their businessTest for cloud failure and its impact on customer applicationperformance.
  • Enstratius testing journey
  • Enstratius Cloud Management Platform
  • The release process
  • buildTodayArtifacts Installer API testsGithubArtifactoryVagrantAWS
  • JenkinsReportNG ReportAutomatedTestFramework(TestNG)GitHubEnstratiusAPI ServerCloud ACloud ZestrovantAPI Test Framework
  • Cloud accountaccess keys, testconfigsHeaderConfig.xmlCloud specificresource/methodexclusionsJSON/XML Inputs forREST Calls commonto all cloudsSetup Test bed on Cloud under testTear Down Test bed on Cloud under testUpdateAdd Delete ListAtomic TestsCloud Specific Inputsfor REST Calls (if any)estrov
  • API test workflow<?xml&version="1.0"&encoding="UTF98"?>!<cloudService!name="AWS">!<excluded>!<resource!name="kvdb"!/>!<resource!name="customer">!<method!name="add"!/>!<method!name="changeCurrency"!/>!<method!name="revertCurrency"!/>!<method!name="setTimeZone"!/>!<method!name="revertTimeZone"!/>!</resource>!</excluded>!</cloudService>{"addLoadBalancer":![!{!"region":{"regionId":REGION_ID},!"budget":BILLINGCODE_ID,!"description":"TestNG_Trial_LB",!"name":DYNAMIC,!"label":"red",!"customer":{"customerId":CUSTOMER_ID},!"dataCenters":[{"dataCenterId":DATACENTER_ID}],!"owningGroups":[{"groupId":GROUP_ID}],!"listeners":![{"protocol":"HTTP","publicPort":80,"privatePort":80}]!}!]!}startRead the cloudaccess keys & testconfigsIterate through thetest resourcesExcludeResource?Run atomic testsGenerate reportHeaderconfig.xmlAWS.xmlRead resource/method inputs<?xml&version="1.0"&encoding="UTF98"&standalone="no"?>!<CONFIG>!<BILLINGCODE_ID>403</BILLINGCODE_ID>!<USER_ID>50950</USER_ID>!!<DUPLICATE_BILLING_ID>402</DUPLICATE_BILLING_ID>!<DUPLICATE_GRP_ID>501</DUPLICATE_GRP_ID>!<ROLE_ID>55501</ROLE_ID>!<GROUP_ID>500</GROUP_ID>!!<ACCOUNT_ID>200</ACCOUNT_ID>!<CLOUD_ID>1</CLOUD_ID>!<CUSTOMER_ID>200</CUSTOMER_ID>!</CONFIG>GenerateDependencyresources<?xml&version="1.0"&encoding="UTF98"&standalone="yes"?>!<header>!<csp>AWS</csp>!<resource>all</resource>!<accessKey>aws_accesskey</accessKey>!<secretKey>aws_secretkey</secretKey>!<apiServer>enstratius_apiserver_ip</apiServer>!<apiServerPort>enstratius_apiserver_port</apiServerPort>!<version>enstratius_apiserver_version</version>!<testForFailure>false</testForFailure>!</header>stopLoadbalancer/add.jsonConfig_200.xmlFinished iteratingYesNo
  • The extra steps1. Not all functions are supported by all clouds. An exclusion listneeds to be maintained.2. Cloud specific behavior needs to be codede.g. Terremark requires the server to be stopped for for volume to beattached3. Sanity check - Issues found must be verified directly using cloudconsole / cloud API
  • BuildNear-term updatesArtifacts LaunchAPI testsGithubArtifactoryAWSJenkinsInstallerCheckout Teardown
  • BuildFutureArtifacts LaunchAPI testsGithubArtifactoryAWSJenkinsInstaller1. Checkout 6. TeardownSelenium
  • Other tools on the radar
  • Visualizing performancehttp://twitter.github.io/zipkin/
  • Vulnerability scans