CloudStack, jclouds, Jenkins and CloudCat

  • 2,879 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,879
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
57
Comments
0
Likes
8

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. CloudStack and jclouds Andrew Bayer @abayerSaturday, December 1, 2012
  • 2. What is jclouds? • Portable Java API for many cloud APIs and providers • Compute and Blobstore • Lower-level Java API for cloud-specific APIs as well • Usable with any JVM languageSaturday, December 1, 2012
  • 3. CloudStack and jclouds • CloudStack is supported by jclouds • 2.x and 3.x tested regularly • 4.x hasn’t been tested much yet, but minimal API changes from 3.xSaturday, December 1, 2012
  • 4. Standard jclouds API and CloudStack • CloudStack support for jcloud standard API: • Instance creation/management • Listing available “hardware” (service offerings), “images” (templates) • Security groupsSaturday, December 1, 2012
  • 5. Lower-level CloudStack API with jclouds • Pretty much everything the CloudStack API can do is implemented in jclouds • Administration tasks • Template creation • User management • Much, much moreSaturday, December 1, 2012
  • 6. What can you do with jclouds and CloudStack?Saturday, December 1, 2012
  • 7. A lot. Here are some examples.Saturday, December 1, 2012
  • 8. jclouds Jenkins plugin • Dynamically provision Jenkins build slaves from CloudStack (or any other supported cloud) • Much like the Jenkins ec2 plugin, but more features and obviously more cloudsSaturday, December 1, 2012
  • 9. How the Jenkins plugin works • Uses standard jclouds API for instance creation and destruction • Allows you to choose “image” or OS family/version, and “hardware” (from list of available) or minimum CPU/RAM • Uses underlying clouds’ APIs to find the right image/hardware for the given inputSaturday, December 1, 2012
  • 10. Saturday, December 1, 2012
  • 11. Saturday, December 1, 2012
  • 12. Saturday, December 1, 2012
  • 13. Saturday, December 1, 2012
  • 14. Saturday, December 1, 2012
  • 15. Single use slave • The single use slave option lets you use a slave once and only once • Slave is marked offline after job completes, guaranteeing the next time that job runs, it gets a fresh slaveSaturday, December 1, 2012
  • 16. Jenkins Instance Creation • Create instances at beginning of build, tear down instances at end of build • Use clouds and templates available as slaves • Exposes IPs of instances to build steps • Useful for testing clusters, etcSaturday, December 1, 2012
  • 17. Building an app using CloudStack and jcloudsSaturday, December 1, 2012
  • 18. CloudCat • Open source, Grails-based reporting and provisioning application for CloudStack • github.com/abayer/cloudcatSaturday, December 1, 2012
  • 19. Using Lower-Level API • CloudStack admin information (accounts, hosts, etc) not available through jclouds standard API • So we’ve got to use the CloudStack API itself - which we can do through jclouds! • Same jclouds connection whether using standard API or CloudStack-specificSaturday, December 1, 2012
  • 20. Saturday, December 1, 2012
  • 21. Mapping CloudStack Data • jclouds has objects for data from CloudStack API • VirtualMachine, Template, ServiceOffering, etc • In CloudCat, we map those objects to Grails domain classesSaturday, December 1, 2012
  • 22. Saturday, December 1, 2012
  • 23. Importing CloudStack data into Grails • Run API commands as admin • Persist results into database once an hour, rather than querying API live • Pull credentials and endpoint from config domain object • Use jclouds’ *Clients to get lists of hosts, templates, service offerings, instancesSaturday, December 1, 2012
  • 24. Saturday, December 1, 2012
  • 25. Associating templates, instances, etc • We want to know how instances, templates, service offerings and hosts are related • We want to keep historical information on instances, and know what instances were still existent the last time we got reported dataSaturday, December 1, 2012
  • 26. Saturday, December 1, 2012
  • 27. Alert Notification • Email notification when CloudStack has problems - not available in CloudStack itself, so far as I can tell. • Events and alerts are available from the API - check periodically for new alerts, send email when new ones are seen.Saturday, December 1, 2012
  • 28. Saturday, December 1, 2012
  • 29. Alert Email ContentSaturday, December 1, 2012
  • 30. Provisioning through CloudCat • CloudStack UI for instance creation doesn’t make it easy to create a bunch of otherwise identical instances • Hence, CloudCat provisioned instance groupsSaturday, December 1, 2012
  • 31. jclouds standard API or CloudStack-specific? • jclouds provides instance creation in a standard way across all clouds, including CloudStack • But the standard instance creation makes a lot of largely redundant API calls, especially to guarantee instance name uniquenessSaturday, December 1, 2012
  • 32. jclouds standard API or CloudStack-specific? • We can guarantee uniqueness within CloudCat • Using lower level CloudStack API (still through jclouds) improves performance considerably • Especially since we’re doing API calls as admin userSaturday, December 1, 2012
  • 33. Speaking of accounts... • How do we create instances for a logged-in user while making API calls as admin? Two options: • Use account and domainid options for deployVirtualMachine • Fetch account’s keys and make API call using those keysSaturday, December 1, 2012
  • 34. Speaking of accounts... • After experimentation, I went with using the logged-in user’s account’s keys • Less API traffic when run as logged-in userSaturday, December 1, 2012
  • 35. Grails • Groovy • Rails-like • I’m lazy and bad with UIs, so I like getting sortable tables, MVC, and much more for free, with even more available from plugins!Saturday, December 1, 2012
  • 36. So how does this all look?Saturday, December 1, 2012
  • 37. Logged OutSaturday, December 1, 2012
  • 38. Logged inSaturday, December 1, 2012
  • 39. Instance ListingSaturday, December 1, 2012
  • 40. Filtering InstancesSaturday, December 1, 2012
  • 41. HostsSaturday, December 1, 2012
  • 42. TemplatesSaturday, December 1, 2012
  • 43. Create instancesSaturday, December 1, 2012
  • 44. Instance StatusSaturday, December 1, 2012
  • 45. Creation errorsSaturday, December 1, 2012
  • 46. Creation successSaturday, December 1, 2012
  • 47. Instance group listSaturday, December 1, 2012
  • 48. Activity loggingSaturday, December 1, 2012
  • 49. Known Issues • I’m not a very good programmer! • Only really tested with basic networking • Init script requires hardcoded credentials on instances • Needs work on packaging/installation • Only one user so far!Saturday, December 1, 2012
  • 50. More Information • jclouds: http://jclouds.org • jclouds Jenkins plugin: https://wiki.jenkins- ci.org/display/JENKINS/JClouds+Plugin • CloudCat • source: http://github.com/abayer/cloudcat • demo: http://cloudcat.andrewbayer.comSaturday, December 1, 2012
  • 51. Thank you!Saturday, December 1, 2012