CloudStack, jclouds, Jenkins and CloudCat

CloudStack and jclouds Andrew Bayer @abayerSaturday, December 1, 2012

What is jclouds? • Portable Java API for many cloud APIs and providers • Compute and Blobstore • Lower-level Java API for cloud-speciﬁc APIs as well • Usable with any JVM languageSaturday, December 1, 2012

CloudStack and jclouds • CloudStack is supported by jclouds • 2.x and 3.x tested regularly • 4.x hasn’t been tested much yet, but minimal API changes from 3.xSaturday, December 1, 2012

Standard jclouds API and CloudStack • CloudStack support for jcloud standard API: • Instance creation/management • Listing available “hardware” (service offerings), “images” (templates) • Security groupsSaturday, December 1, 2012

Lower-level CloudStack API with jclouds • Pretty much everything the CloudStack API can do is implemented in jclouds • Administration tasks • Template creation • User management • Much, much moreSaturday, December 1, 2012

What can you do with jclouds and CloudStack?Saturday, December 1, 2012

A lot. Here are some examples.Saturday, December 1, 2012

jclouds Jenkins plugin • Dynamically provision Jenkins build slaves from CloudStack (or any other supported cloud) • Much like the Jenkins ec2 plugin, but more features and obviously more cloudsSaturday, December 1, 2012

How the Jenkins plugin works • Uses standard jclouds API for instance creation and destruction • Allows you to choose “image” or OS family/version, and “hardware” (from list of available) or minimum CPU/RAM • Uses underlying clouds’ APIs to ﬁnd the right image/hardware for the given inputSaturday, December 1, 2012

Saturday, December 1, 2012

Single use slave • The single use slave option lets you use a slave once and only once • Slave is marked ofﬂine after job completes, guaranteeing the next time that job runs, it gets a fresh slaveSaturday, December 1, 2012

Jenkins Instance Creation • Create instances at beginning of build, tear down instances at end of build • Use clouds and templates available as slaves • Exposes IPs of instances to build steps • Useful for testing clusters, etcSaturday, December 1, 2012

Building an app using CloudStack and jcloudsSaturday, December 1, 2012

CloudCat • Open source, Grails-based reporting and provisioning application for CloudStack • github.com/abayer/cloudcatSaturday, December 1, 2012

Using Lower-Level API • CloudStack admin information (accounts, hosts, etc) not available through jclouds standard API • So we’ve got to use the CloudStack API itself - which we can do through jclouds! • Same jclouds connection whether using standard API or CloudStack-speciﬁcSaturday, December 1, 2012

Mapping CloudStack Data • jclouds has objects for data from CloudStack API • VirtualMachine, Template, ServiceOffering, etc • In CloudCat, we map those objects to Grails domain classesSaturday, December 1, 2012

Importing CloudStack data into Grails • Run API commands as admin • Persist results into database once an hour, rather than querying API live • Pull credentials and endpoint from conﬁg domain object • Use jclouds’ *Clients to get lists of hosts, templates, service offerings, instancesSaturday, December 1, 2012

Associating templates, instances, etc • We want to know how instances, templates, service offerings and hosts are related • We want to keep historical information on instances, and know what instances were still existent the last time we got reported dataSaturday, December 1, 2012

Alert Notiﬁcation • Email notiﬁcation when CloudStack has problems - not available in CloudStack itself, so far as I can tell. • Events and alerts are available from the API - check periodically for new alerts, send email when new ones are seen.Saturday, December 1, 2012

Alert Email ContentSaturday, December 1, 2012

Provisioning through CloudCat • CloudStack UI for instance creation doesn’t make it easy to create a bunch of otherwise identical instances • Hence, CloudCat provisioned instance groupsSaturday, December 1, 2012

jclouds standard API or CloudStack-speciﬁc? • jclouds provides instance creation in a standard way across all clouds, including CloudStack • But the standard instance creation makes a lot of largely redundant API calls, especially to guarantee instance name uniquenessSaturday, December 1, 2012

jclouds standard API or CloudStack-speciﬁc? • We can guarantee uniqueness within CloudCat • Using lower level CloudStack API (still through jclouds) improves performance considerably • Especially since we’re doing API calls as admin userSaturday, December 1, 2012

Speaking of accounts... • How do we create instances for a logged-in user while making API calls as admin? Two options: • Use account and domainid options for deployVirtualMachine • Fetch account’s keys and make API call using those keysSaturday, December 1, 2012

Speaking of accounts... • After experimentation, I went with using the logged-in user’s account’s keys • Less API trafﬁc when run as logged-in userSaturday, December 1, 2012

Grails • Groovy • Rails-like • I’m lazy and bad with UIs, so I like getting sortable tables, MVC, and much more for free, with even more available from plugins!Saturday, December 1, 2012

So how does this all look?Saturday, December 1, 2012

Logged OutSaturday, December 1, 2012

Logged inSaturday, December 1, 2012

Instance ListingSaturday, December 1, 2012

Filtering InstancesSaturday, December 1, 2012

HostsSaturday, December 1, 2012

TemplatesSaturday, December 1, 2012

Create instancesSaturday, December 1, 2012

Instance StatusSaturday, December 1, 2012

Creation errorsSaturday, December 1, 2012

Creation successSaturday, December 1, 2012

Instance group listSaturday, December 1, 2012

Activity loggingSaturday, December 1, 2012

Known Issues • I’m not a very good programmer! • Only really tested with basic networking • Init script requires hardcoded credentials on instances • Needs work on packaging/installation • Only one user so far!Saturday, December 1, 2012

More Information • jclouds: http://jclouds.org • jclouds Jenkins plugin: https://wiki.jenkins- ci.org/display/JENKINS/JClouds+Plugin • CloudCat • source: http://github.com/abayer/cloudcat • demo: http://cloudcat.andrewbayer.comSaturday, December 1, 2012

Thank you!Saturday, December 1, 2012

CloudStack, jclouds, Jenkins and CloudCat

by Andrew Bayer on Dec 01, 2012

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 11

Statistics

CloudStack, jclouds, Jenkins and CloudCat Presentation Transcript