SlideShare a Scribd company logo
CloudStack and jclouds
                             Andrew Bayer
                               @abayer




Saturday, December 1, 2012
What is jclouds?
                    • Portable Java API for many cloud APIs and
                             providers
                    • Compute and Blobstore
                    • Lower-level Java API for cloud-specific APIs
                             as well
                    • Usable with any JVM language

Saturday, December 1, 2012
CloudStack and jclouds

                    • CloudStack is supported by jclouds
                    • 2.x and 3.x tested regularly
                     • 4.x hasn’t been tested much yet, but
                             minimal API changes from 3.x




Saturday, December 1, 2012
Standard jclouds API
                               and CloudStack
                    • CloudStack support for jcloud standard
                             API:
                             • Instance creation/management
                             • Listing available “hardware” (service
                               offerings), “images” (templates)
                             • Security groups

Saturday, December 1, 2012
Lower-level CloudStack
                    API with jclouds
                    • Pretty much everything the CloudStack API
                             can do is implemented in jclouds
                             • Administration tasks
                             • Template creation
                             • User management
                             • Much, much more
Saturday, December 1, 2012
What can you do with
                        jclouds and
                       CloudStack?


Saturday, December 1, 2012
A lot.
                             Here are some
                               examples.


Saturday, December 1, 2012
jclouds Jenkins plugin

                    • Dynamically provision Jenkins build slaves
                             from CloudStack (or any other supported
                             cloud)
                    • Much like the Jenkins ec2 plugin, but more
                             features and obviously more clouds




Saturday, December 1, 2012
How the Jenkins plugin
                          works
                    • Uses standard jclouds API for instance
                             creation and destruction
                    • Allows you to choose “image” or OS
                             family/version, and “hardware” (from list of
                             available) or minimum CPU/RAM
                             • Uses underlying clouds’ APIs to find the
                               right image/hardware for the given input


Saturday, December 1, 2012
Saturday, December 1, 2012
Saturday, December 1, 2012
Saturday, December 1, 2012
Saturday, December 1, 2012
Saturday, December 1, 2012
Single use slave

                    • The single use slave option lets you use a
                             slave once and only once
                    • Slave is marked offline after job completes,
                             guaranteeing the next time that job runs, it
                             gets a fresh slave




Saturday, December 1, 2012
Jenkins Instance
                                    Creation
                    • Create instances at beginning of build, tear
                             down instances at end of build
                    • Use clouds and templates available as slaves
                    • Exposes IPs of instances to build steps
                    • Useful for testing clusters, etc

Saturday, December 1, 2012
Building an app using
                  CloudStack and jclouds


Saturday, December 1, 2012
CloudCat


                    • Open source, Grails-based reporting and
                             provisioning application for CloudStack
                    • github.com/abayer/cloudcat


Saturday, December 1, 2012
Using Lower-Level API
                    • CloudStack admin information (accounts,
                             hosts, etc) not available through jclouds
                             standard API
                    • So we’ve got to use the CloudStack API
                             itself - which we can do through jclouds!
                    • Same jclouds connection whether using
                             standard API or CloudStack-specific


Saturday, December 1, 2012
Saturday, December 1, 2012
Mapping CloudStack
                                    Data
                    • jclouds has objects for data from
                             CloudStack API
                    • VirtualMachine, Template, ServiceOffering,
                             etc
                    • In CloudCat, we map those objects to
                             Grails domain classes



Saturday, December 1, 2012
Saturday, December 1, 2012
Importing CloudStack
                         data into Grails
                    • Run API commands as admin
                    • Persist results into database once an hour,
                             rather than querying API live
                    • Pull credentials and endpoint from config
                             domain object
                    • Use jclouds’ *Clients to get lists of hosts,
                             templates, service offerings, instances


Saturday, December 1, 2012
Saturday, December 1, 2012
Associating templates,
                          instances, etc
                    • We want to know how instances,
                             templates, service offerings and hosts are
                             related
                    • We want to keep historical information on
                             instances, and know what instances were
                             still existent the last time we got reported
                             data


Saturday, December 1, 2012
Saturday, December 1, 2012
Alert Notification

                    • Email notification when CloudStack has
                             problems - not available in CloudStack
                             itself, so far as I can tell.
                    • Events and alerts are available from the API
                             - check periodically for new alerts, send
                             email when new ones are seen.



Saturday, December 1, 2012
Saturday, December 1, 2012
Alert Email Content
Saturday, December 1, 2012
Provisioning through
                              CloudCat
                    • CloudStack UI for instance creation doesn’t
                             make it easy to create a bunch of
                             otherwise identical instances
                    • Hence, CloudCat provisioned instance
                             groups




Saturday, December 1, 2012
jclouds standard API or
                    CloudStack-specific?
                    • jclouds provides instance creation in a
                             standard way across all clouds, including
                             CloudStack
                    • But the standard instance creation makes a
                             lot of largely redundant API calls, especially
                             to guarantee instance name uniqueness



Saturday, December 1, 2012
jclouds standard API or
                    CloudStack-specific?
                    • We can guarantee uniqueness within
                             CloudCat
                    • Using lower level CloudStack API (still
                             through jclouds) improves performance
                             considerably
                             • Especially since we’re doing API calls as
                               admin user


Saturday, December 1, 2012
Speaking of accounts...
                    • How do we create instances for a logged-in
                             user while making API calls as admin? Two
                             options:
                             • Use account and domainid options for
                               deployVirtualMachine
                             • Fetch account’s keys and make API call
                               using those keys


Saturday, December 1, 2012
Speaking of accounts...


                    • After experimentation, I went with using
                             the logged-in user’s account’s keys
                    • Less API traffic when run as logged-in user


Saturday, December 1, 2012
Grails

                    • Groovy
                    • Rails-like
                    • I’m lazy and bad with UIs, so I like getting
                             sortable tables, MVC, and much more for
                             free, with even more available from plugins!



Saturday, December 1, 2012
So how does this all
                                   look?


Saturday, December 1, 2012
Logged Out
Saturday, December 1, 2012
Logged in
Saturday, December 1, 2012
Instance Listing
Saturday, December 1, 2012
Filtering Instances
Saturday, December 1, 2012
Hosts
Saturday, December 1, 2012
Templates
Saturday, December 1, 2012
Create instances
Saturday, December 1, 2012
Instance Status
Saturday, December 1, 2012
Creation errors
Saturday, December 1, 2012
Creation success
Saturday, December 1, 2012
Instance group list
Saturday, December 1, 2012
Activity logging
Saturday, December 1, 2012
Known Issues
                    • I’m not a very good programmer!
                    • Only really tested with basic networking
                    • Init script requires hardcoded credentials
                             on instances
                    • Needs work on packaging/installation
                    • Only one user so far!
Saturday, December 1, 2012
More Information
                    • jclouds: http://jclouds.org
                    • jclouds Jenkins plugin: https://wiki.jenkins-
                             ci.org/display/JENKINS/JClouds+Plugin
                    • CloudCat
                     • source: http://github.com/abayer/cloudcat
                     • demo: http://cloudcat.andrewbayer.com
Saturday, December 1, 2012
Thank you!



Saturday, December 1, 2012

More Related Content

What's hot

Openstack Study Nova 1
Openstack Study Nova 1Openstack Study Nova 1
Openstack Study Nova 1
Jinho Shin
 
OpenStack Cloud Tutorial | What is OpenStack | OpenStack Tutorial | OpenStack...
OpenStack Cloud Tutorial | What is OpenStack | OpenStack Tutorial | OpenStack...OpenStack Cloud Tutorial | What is OpenStack | OpenStack Tutorial | OpenStack...
OpenStack Cloud Tutorial | What is OpenStack | OpenStack Tutorial | OpenStack...
Edureka!
 
Kubernetes - Using Persistent Disks with WordPress and MySQL
Kubernetes - Using Persistent Disks with WordPress and MySQLKubernetes - Using Persistent Disks with WordPress and MySQL
Kubernetes - Using Persistent Disks with WordPress and MySQL
pratik rathod
 
Openstack - An introduction/Installation - Presented at Dr Dobb's conference...
 Openstack - An introduction/Installation - Presented at Dr Dobb's conference... Openstack - An introduction/Installation - Presented at Dr Dobb's conference...
Openstack - An introduction/Installation - Presented at Dr Dobb's conference...
Rahul Krishna Upadhyaya
 
Comparing IaaS: VMware vs OpenStack vs Google’s Ganeti
Comparing IaaS: VMware vs OpenStack vs Google’s GanetiComparing IaaS: VMware vs OpenStack vs Google’s Ganeti
Comparing IaaS: VMware vs OpenStack vs Google’s Ganeti
Giuseppe Paterno'
 
DevCloud - Setup and Demo on Apache CloudStack
DevCloud - Setup and Demo on Apache CloudStack DevCloud - Setup and Demo on Apache CloudStack
DevCloud - Setup and Demo on Apache CloudStack
buildacloud
 
OpenStack Glance Project Update
OpenStack Glance Project UpdateOpenStack Glance Project Update
OpenStack Glance Project Update
Brian Rosmaita
 
2 Day Bootcamp for OpenStack--Cloud Training by Mirantis (Preview)
2 Day Bootcamp for OpenStack--Cloud Training by Mirantis (Preview)2 Day Bootcamp for OpenStack--Cloud Training by Mirantis (Preview)
2 Day Bootcamp for OpenStack--Cloud Training by Mirantis (Preview)
Mirantis
 
Masterless Puppet Using AWS S3 Buckets and IAM Roles
Masterless Puppet Using AWS S3 Buckets and IAM RolesMasterless Puppet Using AWS S3 Buckets and IAM Roles
Masterless Puppet Using AWS S3 Buckets and IAM Roles
Malcolm Duncanson, CISSP
 
OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...
OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...
OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...
Mark Voelker
 
OpenStack for VMware Administrators
OpenStack for VMware AdministratorsOpenStack for VMware Administrators
OpenStack for VMware Administrators
Trevor Roberts Jr.
 
Dev cloud
Dev cloudDev cloud
Dev cloud
Rajesh Battala
 
What is Docker Architecture | Edureka
What is Docker Architecture | EdurekaWhat is Docker Architecture | Edureka
What is Docker Architecture | Edureka
Edureka!
 
Cloudstack vs Openstack
Cloudstack vs OpenstackCloudstack vs Openstack
Cloudstack vs Openstack
Huzefa Husain
 
Introduction to Apache jclouds at NYJavaSIG
Introduction to Apache jclouds at NYJavaSIGIntroduction to Apache jclouds at NYJavaSIG
Introduction to Apache jclouds at NYJavaSIG
Everett Toews
 
OpenStack 101 - All Things Open 2015
OpenStack 101 - All Things Open 2015OpenStack 101 - All Things Open 2015
OpenStack 101 - All Things Open 2015
Mark Voelker
 
JClouds at San Francisco Java User Group
JClouds at San Francisco Java User GroupJClouds at San Francisco Java User Group
JClouds at San Francisco Java User Group
Marakana Inc.
 
Hypervisor Selection in Apache CloudStack 4.4
Hypervisor Selection in Apache CloudStack 4.4Hypervisor Selection in Apache CloudStack 4.4
Hypervisor Selection in Apache CloudStack 4.4
Tim Mackey
 
jclouds High Level Overview by Adrian Cole
jclouds High Level Overview by Adrian Colejclouds High Level Overview by Adrian Cole
jclouds High Level Overview by Adrian Cole
Everett Toews
 
Comparing open source private cloud platforms
Comparing open source private cloud platformsComparing open source private cloud platforms
Comparing open source private cloud platforms
OSCON Byrum
 

What's hot (20)

Openstack Study Nova 1
Openstack Study Nova 1Openstack Study Nova 1
Openstack Study Nova 1
 
OpenStack Cloud Tutorial | What is OpenStack | OpenStack Tutorial | OpenStack...
OpenStack Cloud Tutorial | What is OpenStack | OpenStack Tutorial | OpenStack...OpenStack Cloud Tutorial | What is OpenStack | OpenStack Tutorial | OpenStack...
OpenStack Cloud Tutorial | What is OpenStack | OpenStack Tutorial | OpenStack...
 
Kubernetes - Using Persistent Disks with WordPress and MySQL
Kubernetes - Using Persistent Disks with WordPress and MySQLKubernetes - Using Persistent Disks with WordPress and MySQL
Kubernetes - Using Persistent Disks with WordPress and MySQL
 
Openstack - An introduction/Installation - Presented at Dr Dobb's conference...
 Openstack - An introduction/Installation - Presented at Dr Dobb's conference... Openstack - An introduction/Installation - Presented at Dr Dobb's conference...
Openstack - An introduction/Installation - Presented at Dr Dobb's conference...
 
Comparing IaaS: VMware vs OpenStack vs Google’s Ganeti
Comparing IaaS: VMware vs OpenStack vs Google’s GanetiComparing IaaS: VMware vs OpenStack vs Google’s Ganeti
Comparing IaaS: VMware vs OpenStack vs Google’s Ganeti
 
DevCloud - Setup and Demo on Apache CloudStack
DevCloud - Setup and Demo on Apache CloudStack DevCloud - Setup and Demo on Apache CloudStack
DevCloud - Setup and Demo on Apache CloudStack
 
OpenStack Glance Project Update
OpenStack Glance Project UpdateOpenStack Glance Project Update
OpenStack Glance Project Update
 
2 Day Bootcamp for OpenStack--Cloud Training by Mirantis (Preview)
2 Day Bootcamp for OpenStack--Cloud Training by Mirantis (Preview)2 Day Bootcamp for OpenStack--Cloud Training by Mirantis (Preview)
2 Day Bootcamp for OpenStack--Cloud Training by Mirantis (Preview)
 
Masterless Puppet Using AWS S3 Buckets and IAM Roles
Masterless Puppet Using AWS S3 Buckets and IAM RolesMasterless Puppet Using AWS S3 Buckets and IAM Roles
Masterless Puppet Using AWS S3 Buckets and IAM Roles
 
OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...
OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...
OpenStack + VMware: Deploy, Upgrade, & Operate a Powerful Production OpenStac...
 
OpenStack for VMware Administrators
OpenStack for VMware AdministratorsOpenStack for VMware Administrators
OpenStack for VMware Administrators
 
Dev cloud
Dev cloudDev cloud
Dev cloud
 
What is Docker Architecture | Edureka
What is Docker Architecture | EdurekaWhat is Docker Architecture | Edureka
What is Docker Architecture | Edureka
 
Cloudstack vs Openstack
Cloudstack vs OpenstackCloudstack vs Openstack
Cloudstack vs Openstack
 
Introduction to Apache jclouds at NYJavaSIG
Introduction to Apache jclouds at NYJavaSIGIntroduction to Apache jclouds at NYJavaSIG
Introduction to Apache jclouds at NYJavaSIG
 
OpenStack 101 - All Things Open 2015
OpenStack 101 - All Things Open 2015OpenStack 101 - All Things Open 2015
OpenStack 101 - All Things Open 2015
 
JClouds at San Francisco Java User Group
JClouds at San Francisco Java User GroupJClouds at San Francisco Java User Group
JClouds at San Francisco Java User Group
 
Hypervisor Selection in Apache CloudStack 4.4
Hypervisor Selection in Apache CloudStack 4.4Hypervisor Selection in Apache CloudStack 4.4
Hypervisor Selection in Apache CloudStack 4.4
 
jclouds High Level Overview by Adrian Cole
jclouds High Level Overview by Adrian Colejclouds High Level Overview by Adrian Cole
jclouds High Level Overview by Adrian Cole
 
Comparing open source private cloud platforms
Comparing open source private cloud platformsComparing open source private cloud platforms
Comparing open source private cloud platforms
 

Similar to CloudStack, jclouds, Jenkins and CloudCat

Cloud Camp Chicago Dec 2012 Slides
Cloud Camp Chicago Dec 2012 SlidesCloud Camp Chicago Dec 2012 Slides
Cloud Camp Chicago Dec 2012 Slides
Ryan Koop
 
Cloud Camp Chicago Dec 2012 - All presentations
Cloud Camp Chicago Dec 2012 - All presentationsCloud Camp Chicago Dec 2012 - All presentations
Cloud Camp Chicago Dec 2012 - All presentations
CloudCamp Chicago
 
Azure fundamentals 03
Azure fundamentals 03Azure fundamentals 03
Azure fundamentals 03
Thi Nguyen Dinh
 
Lessons Learned from Dockerizing Spark Workloads: Spark Summit East talk by T...
Lessons Learned from Dockerizing Spark Workloads: Spark Summit East talk by T...Lessons Learned from Dockerizing Spark Workloads: Spark Summit East talk by T...
Lessons Learned from Dockerizing Spark Workloads: Spark Summit East talk by T...
Spark Summit
 
Cloud computing
Cloud computingCloud computing
Containers docker-docker hub-azureacr-azure aci
Containers docker-docker hub-azureacr-azure aciContainers docker-docker hub-azureacr-azure aci
Containers docker-docker hub-azureacr-azure aci
Rajesh Kolla
 
Lessons Learned from Dockerizing Spark Workloads
Lessons Learned from Dockerizing Spark WorkloadsLessons Learned from Dockerizing Spark Workloads
Lessons Learned from Dockerizing Spark Workloads
BlueData, Inc.
 
Docker + jenkins in the enterprise (3)
Docker + jenkins in the enterprise (3)Docker + jenkins in the enterprise (3)
Docker + jenkins in the enterprise (3)
Kurt Madel
 
Lessons Learned Running Hadoop and Spark in Docker Containers
Lessons Learned Running Hadoop and Spark in Docker ContainersLessons Learned Running Hadoop and Spark in Docker Containers
Lessons Learned Running Hadoop and Spark in Docker Containers
BlueData, Inc.
 
Docker Networking with Project Calico
Docker Networking with Project CalicoDocker Networking with Project Calico
Docker Networking with Project Calico
Andrew Kennedy
 
Cloud Computing Essentials
Cloud Computing EssentialsCloud Computing Essentials
Cloud Computing Essentials
Velocity Technology Solutions
 
Cloud Computing from Academic Perspective
Cloud Computing from Academic PerspectiveCloud Computing from Academic Perspective
Cloud Computing from Academic Perspective
Dedy Hariyadi
 
Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...
Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...
Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...
IndicThreads
 
J1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. Nielsen
J1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. NielsenJ1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. Nielsen
J1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. Nielsen
MS Cloud Summit
 
Java dans Windows Azure Cloud Services, big data avec hd insight et les nouve...
Java dans Windows Azure Cloud Services, big data avec hd insight et les nouve...Java dans Windows Azure Cloud Services, big data avec hd insight et les nouve...
Java dans Windows Azure Cloud Services, big data avec hd insight et les nouve...
Microsoft
 
State of Containers in OpenStack
State of Containers in OpenStackState of Containers in OpenStack
State of Containers in OpenStack
openstackindia
 
State of Containers in Openstack
State of Containers in OpenstackState of Containers in Openstack
State of Containers in Openstack
Madhuri Kumari
 
MicroServices on Azure
MicroServices on AzureMicroServices on Azure
MicroServices on Azure
Sergey Seletsky
 
Understanding Docker and IBM Bluemix Container Service
Understanding Docker and IBM Bluemix Container ServiceUnderstanding Docker and IBM Bluemix Container Service
Understanding Docker and IBM Bluemix Container Service
Andrew Ferrier
 
jclouds at HKJUG
jclouds at HKJUGjclouds at HKJUG
jclouds at HKJUG
Everett Toews
 

Similar to CloudStack, jclouds, Jenkins and CloudCat (20)

Cloud Camp Chicago Dec 2012 Slides
Cloud Camp Chicago Dec 2012 SlidesCloud Camp Chicago Dec 2012 Slides
Cloud Camp Chicago Dec 2012 Slides
 
Cloud Camp Chicago Dec 2012 - All presentations
Cloud Camp Chicago Dec 2012 - All presentationsCloud Camp Chicago Dec 2012 - All presentations
Cloud Camp Chicago Dec 2012 - All presentations
 
Azure fundamentals 03
Azure fundamentals 03Azure fundamentals 03
Azure fundamentals 03
 
Lessons Learned from Dockerizing Spark Workloads: Spark Summit East talk by T...
Lessons Learned from Dockerizing Spark Workloads: Spark Summit East talk by T...Lessons Learned from Dockerizing Spark Workloads: Spark Summit East talk by T...
Lessons Learned from Dockerizing Spark Workloads: Spark Summit East talk by T...
 
Cloud computing
Cloud computingCloud computing
Cloud computing
 
Containers docker-docker hub-azureacr-azure aci
Containers docker-docker hub-azureacr-azure aciContainers docker-docker hub-azureacr-azure aci
Containers docker-docker hub-azureacr-azure aci
 
Lessons Learned from Dockerizing Spark Workloads
Lessons Learned from Dockerizing Spark WorkloadsLessons Learned from Dockerizing Spark Workloads
Lessons Learned from Dockerizing Spark Workloads
 
Docker + jenkins in the enterprise (3)
Docker + jenkins in the enterprise (3)Docker + jenkins in the enterprise (3)
Docker + jenkins in the enterprise (3)
 
Lessons Learned Running Hadoop and Spark in Docker Containers
Lessons Learned Running Hadoop and Spark in Docker ContainersLessons Learned Running Hadoop and Spark in Docker Containers
Lessons Learned Running Hadoop and Spark in Docker Containers
 
Docker Networking with Project Calico
Docker Networking with Project CalicoDocker Networking with Project Calico
Docker Networking with Project Calico
 
Cloud Computing Essentials
Cloud Computing EssentialsCloud Computing Essentials
Cloud Computing Essentials
 
Cloud Computing from Academic Perspective
Cloud Computing from Academic PerspectiveCloud Computing from Academic Perspective
Cloud Computing from Academic Perspective
 
Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...
Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...
Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...
 
J1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. Nielsen
J1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. NielsenJ1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. Nielsen
J1 T1 3 - Azure Data Lake store & analytics 101 - Kenneth M. Nielsen
 
Java dans Windows Azure Cloud Services, big data avec hd insight et les nouve...
Java dans Windows Azure Cloud Services, big data avec hd insight et les nouve...Java dans Windows Azure Cloud Services, big data avec hd insight et les nouve...
Java dans Windows Azure Cloud Services, big data avec hd insight et les nouve...
 
State of Containers in OpenStack
State of Containers in OpenStackState of Containers in OpenStack
State of Containers in OpenStack
 
State of Containers in Openstack
State of Containers in OpenstackState of Containers in Openstack
State of Containers in Openstack
 
MicroServices on Azure
MicroServices on AzureMicroServices on Azure
MicroServices on Azure
 
Understanding Docker and IBM Bluemix Container Service
Understanding Docker and IBM Bluemix Container ServiceUnderstanding Docker and IBM Bluemix Container Service
Understanding Docker and IBM Bluemix Container Service
 
jclouds at HKJUG
jclouds at HKJUGjclouds at HKJUG
jclouds at HKJUG
 

Recently uploaded

20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Neo4j
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
James Anderson
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024
Pierluigi Pugliese
 
GridMate - End to end testing is a critical piece to ensure quality and avoid...
GridMate - End to end testing is a critical piece to ensure quality and avoid...GridMate - End to end testing is a critical piece to ensure quality and avoid...
GridMate - End to end testing is a critical piece to ensure quality and avoid...
ThomasParaiso2
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial ApplicationsLarge Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
Rohit Gautam
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Adtran
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Nexer Digital
 

Recently uploaded (20)

20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024
 
GridMate - End to end testing is a critical piece to ensure quality and avoid...
GridMate - End to end testing is a critical piece to ensure quality and avoid...GridMate - End to end testing is a critical piece to ensure quality and avoid...
GridMate - End to end testing is a critical piece to ensure quality and avoid...
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial ApplicationsLarge Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
 

CloudStack, jclouds, Jenkins and CloudCat

  • 1. CloudStack and jclouds Andrew Bayer @abayer Saturday, December 1, 2012
  • 2. What is jclouds? • Portable Java API for many cloud APIs and providers • Compute and Blobstore • Lower-level Java API for cloud-specific APIs as well • Usable with any JVM language Saturday, December 1, 2012
  • 3. CloudStack and jclouds • CloudStack is supported by jclouds • 2.x and 3.x tested regularly • 4.x hasn’t been tested much yet, but minimal API changes from 3.x Saturday, December 1, 2012
  • 4. Standard jclouds API and CloudStack • CloudStack support for jcloud standard API: • Instance creation/management • Listing available “hardware” (service offerings), “images” (templates) • Security groups Saturday, December 1, 2012
  • 5. Lower-level CloudStack API with jclouds • Pretty much everything the CloudStack API can do is implemented in jclouds • Administration tasks • Template creation • User management • Much, much more Saturday, December 1, 2012
  • 6. What can you do with jclouds and CloudStack? Saturday, December 1, 2012
  • 7. A lot. Here are some examples. Saturday, December 1, 2012
  • 8. jclouds Jenkins plugin • Dynamically provision Jenkins build slaves from CloudStack (or any other supported cloud) • Much like the Jenkins ec2 plugin, but more features and obviously more clouds Saturday, December 1, 2012
  • 9. How the Jenkins plugin works • Uses standard jclouds API for instance creation and destruction • Allows you to choose “image” or OS family/version, and “hardware” (from list of available) or minimum CPU/RAM • Uses underlying clouds’ APIs to find the right image/hardware for the given input Saturday, December 1, 2012
  • 15. Single use slave • The single use slave option lets you use a slave once and only once • Slave is marked offline after job completes, guaranteeing the next time that job runs, it gets a fresh slave Saturday, December 1, 2012
  • 16. Jenkins Instance Creation • Create instances at beginning of build, tear down instances at end of build • Use clouds and templates available as slaves • Exposes IPs of instances to build steps • Useful for testing clusters, etc Saturday, December 1, 2012
  • 17. Building an app using CloudStack and jclouds Saturday, December 1, 2012
  • 18. CloudCat • Open source, Grails-based reporting and provisioning application for CloudStack • github.com/abayer/cloudcat Saturday, December 1, 2012
  • 19. Using Lower-Level API • CloudStack admin information (accounts, hosts, etc) not available through jclouds standard API • So we’ve got to use the CloudStack API itself - which we can do through jclouds! • Same jclouds connection whether using standard API or CloudStack-specific Saturday, December 1, 2012
  • 21. Mapping CloudStack Data • jclouds has objects for data from CloudStack API • VirtualMachine, Template, ServiceOffering, etc • In CloudCat, we map those objects to Grails domain classes Saturday, December 1, 2012
  • 23. Importing CloudStack data into Grails • Run API commands as admin • Persist results into database once an hour, rather than querying API live • Pull credentials and endpoint from config domain object • Use jclouds’ *Clients to get lists of hosts, templates, service offerings, instances Saturday, December 1, 2012
  • 25. Associating templates, instances, etc • We want to know how instances, templates, service offerings and hosts are related • We want to keep historical information on instances, and know what instances were still existent the last time we got reported data Saturday, December 1, 2012
  • 27. Alert Notification • Email notification when CloudStack has problems - not available in CloudStack itself, so far as I can tell. • Events and alerts are available from the API - check periodically for new alerts, send email when new ones are seen. Saturday, December 1, 2012
  • 29. Alert Email Content Saturday, December 1, 2012
  • 30. Provisioning through CloudCat • CloudStack UI for instance creation doesn’t make it easy to create a bunch of otherwise identical instances • Hence, CloudCat provisioned instance groups Saturday, December 1, 2012
  • 31. jclouds standard API or CloudStack-specific? • jclouds provides instance creation in a standard way across all clouds, including CloudStack • But the standard instance creation makes a lot of largely redundant API calls, especially to guarantee instance name uniqueness Saturday, December 1, 2012
  • 32. jclouds standard API or CloudStack-specific? • We can guarantee uniqueness within CloudCat • Using lower level CloudStack API (still through jclouds) improves performance considerably • Especially since we’re doing API calls as admin user Saturday, December 1, 2012
  • 33. Speaking of accounts... • How do we create instances for a logged-in user while making API calls as admin? Two options: • Use account and domainid options for deployVirtualMachine • Fetch account’s keys and make API call using those keys Saturday, December 1, 2012
  • 34. Speaking of accounts... • After experimentation, I went with using the logged-in user’s account’s keys • Less API traffic when run as logged-in user Saturday, December 1, 2012
  • 35. Grails • Groovy • Rails-like • I’m lazy and bad with UIs, so I like getting sortable tables, MVC, and much more for free, with even more available from plugins! Saturday, December 1, 2012
  • 36. So how does this all look? Saturday, December 1, 2012
  • 47. Instance group list Saturday, December 1, 2012
  • 49. Known Issues • I’m not a very good programmer! • Only really tested with basic networking • Init script requires hardcoded credentials on instances • Needs work on packaging/installation • Only one user so far! Saturday, December 1, 2012
  • 50. More Information • jclouds: http://jclouds.org • jclouds Jenkins plugin: https://wiki.jenkins- ci.org/display/JENKINS/JClouds+Plugin • CloudCat • source: http://github.com/abayer/cloudcat • demo: http://cloudcat.andrewbayer.com Saturday, December 1, 2012