The document discusses Apache CloudStack's plugin model and how it allows for innovation through adaption. It provides an example of adding Hyper-V support through a plugin. The plugin model balances openness with structure by having plugins serve two masters - the CloudStack server and resource components. Following CloudStack's process for new features and using simpler steps can make it easier to learn and contribute plugins. Repurposing existing code rather than rewriting also helps.
EWD 3 Training Course Part 44: Creating MicroServices with QEWD.jsRob Tweed
This is part 44 of the EWD 3 Training Course. In this presentation we'll examine how multiple QEWD.js systems can be configured to break down your applications into highly scalable, high-performance MicroServices
EWD 3 Training Course Part 45: Using QEWD's Advanced MicroService FunctionalityRob Tweed
This is Part 45 of the EWD Training Course. In this presentation we'll examine in detail the advanced MicroService capabilities of QEWD, eg dynamic routes, templated paths, custom response handling and chained MicroServices
WebSphere Application Server Liberty Profile and DockerDavid Currie
Latest incarnation of my Liberty and Docker presentation including coverage of the Java EE7 images and the auto-scaling beta. As presented to the UK WebSphere User Group on 1 October 2015.
Big Data & Cloud | Cloud Storage Simplified | Adrian ColeJAX London
2011-11-01 | 04:20 PM - 05:10 PM
Key/value stores are the most common storage offerings in the cloud today. We'll introduce storage concepts and differences between cloud storage providers including Amazon S3, Rackspace Cloud Files & Microsoft Azure Blob Service. We'll demonstrate how jclouds BlobStore helps Java and Clojure developers avoid lock-in and increase testability without restricting access to provider-specific features
EWD 3 Training Course Part 44: Creating MicroServices with QEWD.jsRob Tweed
This is part 44 of the EWD 3 Training Course. In this presentation we'll examine how multiple QEWD.js systems can be configured to break down your applications into highly scalable, high-performance MicroServices
EWD 3 Training Course Part 45: Using QEWD's Advanced MicroService FunctionalityRob Tweed
This is Part 45 of the EWD Training Course. In this presentation we'll examine in detail the advanced MicroService capabilities of QEWD, eg dynamic routes, templated paths, custom response handling and chained MicroServices
WebSphere Application Server Liberty Profile and DockerDavid Currie
Latest incarnation of my Liberty and Docker presentation including coverage of the Java EE7 images and the auto-scaling beta. As presented to the UK WebSphere User Group on 1 October 2015.
Big Data & Cloud | Cloud Storage Simplified | Adrian ColeJAX London
2011-11-01 | 04:20 PM - 05:10 PM
Key/value stores are the most common storage offerings in the cloud today. We'll introduce storage concepts and differences between cloud storage providers including Amazon S3, Rackspace Cloud Files & Microsoft Azure Blob Service. We'll demonstrate how jclouds BlobStore helps Java and Clojure developers avoid lock-in and increase testability without restricting access to provider-specific features
Scaling and Embracing Failure: Clustering Docker with MesosRob Gulewich
My talk at the Docker-YVR meetup, Jan 20, 2016. In case it's not clear from the slides - we are happy overall with Mesos. I just wanted to give a balanced account of what it's like to run it in production.
Human API has an interesting problem: building a dynamic, heavily-utilized system that processes terabytes of health data every day. In this talk, Rob will discuss how Human API has scaled out an elastic Docker ecosystem using Mesos: the motivations, challenges, and war stories.
How Can Hypervisors Leverage Advanced Storage Features ? - VMFS(x) on the storage attached to the ESX/ESXi hosts works perfectly fine, but the network usage(IP/FC/etc) goes up significantly when the storage is coming from NAS or SAN.The goal is to offload the file operations to the NAS/SAN based Arrays and leverage maximum benefits to increase I/O performance,storage utilization and reduced network usage.
Presentation on OSGi Cloud Ecosystems as presented during EclipseCon Europe 2012 (http://www.eclipsecon.org/europe2012/sessions/osgi-and-cloud-computing)
Java Tech & Tools | Grails in the Java Enterprise | Peter LedbrookJAX London
2011-11-01 | 03:00 PM - 03:50 PM
With all the buzz around rapid web application development frameworks, are enterprise developers left looking on enviously? Not at all. Grails brings the same benefits to Java developers while providing many options for enterprise integration. This talk shows you how to build Grails projects with Ant and Maven; what's involved in talking to legacy databases; and how to talk to Java components.
Spring Boot Revisited with KoFu and JaFuVMware Tanzu
SpringOne 2020
Spring Boot Revisited with KoFu and JaFu
Audrey Neveu, Software Engineer on Reactor at VMware
Sébastien Deleuze, Spring Framework Committer at VMware
This was a quick (15 minutes!) tour of Cloud Foundry that I gave at JFokus 2012 introducing Cloud Foundry as the answer to the question, "I've got a working web application and Spring made it easy, but where do I host it?"
EWD 3 Training Course Part 16: QEWD ServicesRob Tweed
This presentation is Part 16 of the EWD 3 Training Course. It describes and explains QEWD Services and shows you how to use them to create re-usable back-end message handlers.
A Walking Tour of (almost) all of Springdom Joshua Long
this is the deck for my 3+ hour walking tour talk that I give as a workshop at various conferences. This talk introduces practically everything in Spring -- come into the talk unaware of the concepts or frameworks and leave with a working knowledge of all the frameworks, and of all the applications for the technologies.
Using Puppet - Real World Configuration ManagementJames Turnbull
Configuration management is the oft-misunderstood (and possibly black) art of managing your IT environment, infrastructure, and costs. Done well it can reduce operational errors and outages, simplify your environment, and help maintain the sanity of your IT staff.
Puppet is part of the bright future of configuration management for heterogeneous Unix systems. It combines automation, a powerful abstraction language, and uses a client-server model that can scale to suit enterprise-size environments. Puppet is written in Ruby and authored by recovering system administrator-turned-developer Luke Kanies.
This session explains why configuration management is important, the benefits configuration management will deliver, and how all of this can be achieved using Puppet. The session also explains emerging best practices in configuration management and addresses:
* What is configuration management? Or why am I here?
* Benefits, risks, and challenges: build fire resistant infrastructure rather than fight fires
* Best practice: how do we do this configuration management magic right?
* Where does Puppet fit in and why should management pay for its implementation?
* Why using Puppet will save you money and help staff retention (although is unlikely to stop world hunger)
* Real world configuration management using Puppet: code, examples, explanations, and using Puppet in anger
* Measuring the results and pocketing the returns
* Where to from here: some ideas about the future (may include wild-arse guesses)
IBM Power - System software
Med IBM Power System Software kan du optimere og forenkle driften af din IT. Hør bl.a. om vores effektive nye software modul, IBM PowerSC (Security and Compliance).
Jan Kristian Nielsen, Client Architect, IBM
Scaling and Embracing Failure: Clustering Docker with MesosRob Gulewich
My talk at the Docker-YVR meetup, Jan 20, 2016. In case it's not clear from the slides - we are happy overall with Mesos. I just wanted to give a balanced account of what it's like to run it in production.
Human API has an interesting problem: building a dynamic, heavily-utilized system that processes terabytes of health data every day. In this talk, Rob will discuss how Human API has scaled out an elastic Docker ecosystem using Mesos: the motivations, challenges, and war stories.
How Can Hypervisors Leverage Advanced Storage Features ? - VMFS(x) on the storage attached to the ESX/ESXi hosts works perfectly fine, but the network usage(IP/FC/etc) goes up significantly when the storage is coming from NAS or SAN.The goal is to offload the file operations to the NAS/SAN based Arrays and leverage maximum benefits to increase I/O performance,storage utilization and reduced network usage.
Presentation on OSGi Cloud Ecosystems as presented during EclipseCon Europe 2012 (http://www.eclipsecon.org/europe2012/sessions/osgi-and-cloud-computing)
Java Tech & Tools | Grails in the Java Enterprise | Peter LedbrookJAX London
2011-11-01 | 03:00 PM - 03:50 PM
With all the buzz around rapid web application development frameworks, are enterprise developers left looking on enviously? Not at all. Grails brings the same benefits to Java developers while providing many options for enterprise integration. This talk shows you how to build Grails projects with Ant and Maven; what's involved in talking to legacy databases; and how to talk to Java components.
Spring Boot Revisited with KoFu and JaFuVMware Tanzu
SpringOne 2020
Spring Boot Revisited with KoFu and JaFu
Audrey Neveu, Software Engineer on Reactor at VMware
Sébastien Deleuze, Spring Framework Committer at VMware
This was a quick (15 minutes!) tour of Cloud Foundry that I gave at JFokus 2012 introducing Cloud Foundry as the answer to the question, "I've got a working web application and Spring made it easy, but where do I host it?"
EWD 3 Training Course Part 16: QEWD ServicesRob Tweed
This presentation is Part 16 of the EWD 3 Training Course. It describes and explains QEWD Services and shows you how to use them to create re-usable back-end message handlers.
A Walking Tour of (almost) all of Springdom Joshua Long
this is the deck for my 3+ hour walking tour talk that I give as a workshop at various conferences. This talk introduces practically everything in Spring -- come into the talk unaware of the concepts or frameworks and leave with a working knowledge of all the frameworks, and of all the applications for the technologies.
Using Puppet - Real World Configuration ManagementJames Turnbull
Configuration management is the oft-misunderstood (and possibly black) art of managing your IT environment, infrastructure, and costs. Done well it can reduce operational errors and outages, simplify your environment, and help maintain the sanity of your IT staff.
Puppet is part of the bright future of configuration management for heterogeneous Unix systems. It combines automation, a powerful abstraction language, and uses a client-server model that can scale to suit enterprise-size environments. Puppet is written in Ruby and authored by recovering system administrator-turned-developer Luke Kanies.
This session explains why configuration management is important, the benefits configuration management will deliver, and how all of this can be achieved using Puppet. The session also explains emerging best practices in configuration management and addresses:
* What is configuration management? Or why am I here?
* Benefits, risks, and challenges: build fire resistant infrastructure rather than fight fires
* Best practice: how do we do this configuration management magic right?
* Where does Puppet fit in and why should management pay for its implementation?
* Why using Puppet will save you money and help staff retention (although is unlikely to stop world hunger)
* Real world configuration management using Puppet: code, examples, explanations, and using Puppet in anger
* Measuring the results and pocketing the returns
* Where to from here: some ideas about the future (may include wild-arse guesses)
IBM Power - System software
Med IBM Power System Software kan du optimere og forenkle driften af din IT. Hør bl.a. om vores effektive nye software modul, IBM PowerSC (Security and Compliance).
Jan Kristian Nielsen, Client Architect, IBM
Enhanced introduction to CloudStack for the Geneva Java User Group. Includes Apache processes, DevCloud use cases, opportunities for Java developers and 4.0 release testing procedure.
Balconies, Patios, Terraces, and Bridges. Architectural approaches for moving...mfrancis
OSGi is a great platform for building new applications, but what if you have 250.000 lines of legacy Java code that uses custom classloaders, dynamic invocation, and complex resource loading techniques? There are many approaches to moving such a product to OSGi. This talk will explore the approaches Software AG evaluated while moving their flagship integration platform from plain old Java to OSGi as well as challenges encountered as part of the move.
3. The Call For Submissions sets the context
• “Open Source Community Leadership Drives Enterprise-Grade Innovation”
ᵒCloudStack‟s plugin model permits enterprise-grade adaptions
• “Apache initiatives play a key role in powering today's Cloud”
ᵒPlugin model allows cloud to adapt to compute loads (not the other way around)
• “particular focus on those [talks] demonstrating real-world experience of
solving specific problems.”
ᵒCase study of adding Hyper-V support as a newcomer
4. Innovators Need the System to be Disaggregated
CloudStack WebServices API
OAM&P API End User API AWS API Pluggable Service API Engine
Business Logic Provisioning
Resource
Capacity
Accounts
Mgr
Update
Mgr
Rules
Mgr
Mgr
Mgr
HA
Security Mgr
Events Manager
Adapters
XenServer
Orchestration
VMWare
Usage Manager
OVM
KVM
Network Guru
Template Mgr
Network Mgr
VM Manager
Storage Mgr
Domain Manager
Snapshot
Manager
Network Element
Deployment
Account Mgr Planner
Limits Manager Hypervisor Guru
Framework
Agent Manager Cluster Manager Data Access Layer
6. Disaggregation Started with Hardware Management
CloudStack Orchestration Adapters CloudStack Provisioning
Plugins
Network Guru
Snapshot Manager
Template Manager
Network Manager
Storage Manager
VM Manager
XenServer
HypervisorGuru
VMWare
OVM
KVM
Etc…
Framework
Agent Manager Cluster Manager Data Access Layer
7. Understand that the Plugin serves two masters
• Server Component:
Rest API ᵒJava
ᵒAdapter APIs
Plugin API
Implementation ᵒDAO
ᵒRESTful API
Data Access Layer
• ServerResource:
ServerResource
- Optional. Required if Plugin needs to be co-
ᵒAgent Proxy, e.g. KVM
located with the resource • „Message Bus‟ of JSON over TCP
- Implements translation layer to talk to resource
- Communicates with server component via JSON ᵒDirect connect, e.g. XenServer
9. Follow the process for new features
• https://cwiki.apache.org/confluence/display/CLOUDSTACK/Adding+new+features+and+design+documents
• Announce over mailing list
- Attempt to get consensus: awareness & acceptance
• Publish Functional Spec & Design
• JIRA ticket for feature
• Setup a Dev Environment
• Branch on github, use your own (public) branch
• Submit changes to Review Board
- post-review for large packages of changes.
• Decide on the wiki you want
• Incubator wiki cleaner, simpler view
- http://incubator.apache.org/cloudstack/develop/developer-faq.html
• CloudStack wiki for in depth development
- https://cwiki.apache.org/confluence/display/CLOUDSTACK/Home
• Avoid the pre-Apache wiki (http://wiki.cloudstack.org/dashboard.action)
10. Simpler Steps Make it Easier to Learn CloudStack
CloudStack Manager CloudStack Manager
Hyper-V Types Hyper-V Types
Phase 1: Phase 2:
Plugin Server Component Plugin Server Component
Proxy ServerResource ServerResource
Message Bus WS-Management
Connected Agent
WMI (or PowerShell) WMI
Hyper-V Server 2012 Hyper-V Server 2012
XenServer Cluster Hyper-V based
(System VMs) System VMs
11. Reuse and repurpose rather than rewrite
Phase 1
Remote WMI via
Python
Agent
Server Resource (KVM)
Agent Message Bus
AgentShell
O/S
12. ServerResource commands are easier to log and replay
public interface HypervisorResource extends ServerResource {
StartAnswer execute(StartCommand cmd);
StopAnswer execute(StopCommand cmd);
RebootAnswer execute(RebootCommand cmd); }
@Test
public void TestCreateCommand() {
String sample = "{"volId":10,"pool":{"id":201,"uuid":""+testLocalStoreUUID+"","h
","path":"+testLocalStorePathJSON+","port":0,"type":"Filesystem"},"diskCharact
""tags":[],"type":"ROOT","name":"ROOT-9","useLocalStorage":true,"recreatab
""volumeId":10,"hyperType":"Hyperv"},"templateUrl":"+testSampleTemplateURLJ
…
13. CloudStack is evolving, it may fix your problem
• Storage disaggregated
• SystemVM creation broadened & simplified
XenServer Cluster Hyper-V based
(System VMs) System VMs
14. Make advance preparations for IP clearance
# Copyright (c) 2010 Cloud.com, Inc
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
• http://www.apache.org/licenses/
ᵒhttp://www.apache.org/legal/src-headers.html
ᵒhttp://apache.org/licenses/LICENSE-2.0.html#apply
15. Bonus Tips – Read at your leisure
• SystemVMs have logs
ᵒConnect form hypervisor console, user:root, password: 6m1ll10n.
• Avoids SSH over 3392 with management server RSA keys
• Import Maven projects into Eclipse using m2e
• Maven debuggable task paused waiting for Eclipse to attach:
export MAVEN_OPTS="-Xdebug -Xnoagent -Djava.compiler=NONE -
Xrunjdwp:transport=dt_socket,address=4000,server=y,suspend=y
- See http://www.mojavelinux.com/blog/archives/2007/03/remote_debugging_with_jetty/
• Expand Log4j logging
ᵒChange Threshold from INFO to TRACE
• :%s/INFO/TRACE/g
16. Summary
• Innovators Need the System to be Disaggregated
• Disaggregation Started with Hardware Management
• Understand that the Plugin serves two masters
• Follow the process for new features
• Simpler Steps Make it Easier to Learn CloudStack
• Repurpose rather than rewrite
• ServerResource commands are easier to log and replay
• Keep an eye out for evolving solutions
• Make advance preparations for IP clearance
17. “It is not the strongest of the species that
survives, nor the most intelligent that
survives. It is the one that is the most
adaptable to change.”
– Charles Darwin
17
18. References
• Theory behind plugins http://www.youtube.com/watch?v=FMM-YgK1jmg
• Disaggregating CloudStack
ᵒhttp://www.slideshare.net/buildacloud/cloudstack-collaboration-conference-12-
refactoring-cloud-stack
ᵒhttp://www.youtube.com/watch?v=iGk3s68Meh0
• Hyper-V Plugin Wiki
ᵒhttps://cwiki.apache.org/CLOUDSTACK/hyper-v-2012-30-support.html