Modularity in the cloud a case study

1,006 views

Published on

Deploying applications in the cloud mean a lot of new possibilities but also challenges. Modular applications bring even greater deployment flexibility. In this session, you will learn how to use basic infrastructure as a service (IaaS) to enable autoscaling and automated failover and recovery and how to deal with hot, zero-downtime updates. The speakers have been doing this in a large project for the past two years. In the presentation, they share their experiences in using an open source OSGi-based development and runtime stack.

As presented at JavaOne 2013 by Paul Bakker and Marcel Offermans: https://oracleus.activeevents.com/2013/connect/sessionDetail.ww?SESSION_ID=2775

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,006
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
12
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Modularity in the cloud a case study

  1. 1. Modularity in the cloud A case study
  2. 2. Paul Bakker Architect at Luminis Technologies @pbakker Marcel Offermans Director at Luminis Technologies Member at Apache @m4rr5
  3. 3. The case Educational system focussed on personalized learning Used in high schools in The Netherlands Expand to other countries in the near future
  4. 4. Cross device HTML 5 + JS RESTful web services OSGi services Mongo Apache Felix A m d a t u S3
  5. 5. Amdatu: OSGi Cloud components JAX-RS MONGO DB BlobStores Multi-tenancy Search OpenSocial ... Apache Licensed open source project
  6. 6. Some numbers 190 bundles in a deployment 120 PulseOn bundles
  7. 7. School B Load Deployment Load Balancer PulseOn node PulseOn node PulseOn node Load Balancer PulseOn node PulseOn node PulseOn node School A Mongo Mongo Mongo Mongo Mongo Mongo
  8. 8. Availability zones Load Balancer PulseOn node PulseOn node PulseOn node PulseOn node PulseOn node PulseOn node Availability Zone 1 Mongo Mongo Mongo Mongo Mongo Mongo Availability Zone 2
  9. 9. Horizontal scalability Horizontal scaling requires stateless nodes HTML5 clients need less server side state Any state should go to some kind of store
  10. 10. Auto scaling Considerable higher loads during school hours Enough capacity Without paying for idle servers at night...
  11. 11. Load Balancer small node Cluster per school Always use a load balancer because we don’t want downtime during scaling
  12. 12. Load Balancer small node large node large node Early morning...
  13. 13. End of the day... Load Balancer small node large node large node
  14. 14. But how do we install our software on a node?
  15. 15. Provisioning servers
  16. 16. ace screenshot User Interface REST API Shell scripting
  17. 17. Node AWS Auto Scaling 1. Start Load Balancer Apache ACE 2. register 3. register 4. provision deployment package Provisioning servers
  18. 18. ./as-­‐create-­‐launch-­‐config  demo            -­‐-­‐image-­‐id  ami-­‐0ee8e07a            -­‐-­‐instance-­‐type  m1.small            -­‐-­‐region  eu-­‐west-­‐1            -­‐-­‐group  sg-­‐ce1420ba            -­‐-­‐user-­‐data-­‐file  userdata.txt ./as-­‐create-­‐auto-­‐scaling-­‐group  demo          -­‐-­‐launch-­‐configuration  demo          -­‐-­‐min-­‐size  1          -­‐-­‐max-­‐size  1          -­‐-­‐availability-­‐zones  eu-­‐west-­‐1a          -­‐-­‐load-­‐balancers  demo          -­‐-­‐tag  "k=Name,v=demo,p=true"
  19. 19. Build releases CI server Release OBR -runbundles: org.apache.felix.http.jetty;version=2.2.0, org.apache.felix.gogo.command;version=0.10.0, org.apache.felix.gogo.runtime;version=0.10.0, org.apache.felix.gogo.shell;version=0.10.0, osgi.cmpn;version='(4.2.0,4.2.1]', org.amdatu.security.tokenprovider.api;version=latest, org.amdatu.security.tokenprovider.impl;version=latest, org.amdatu.web.rest.doc, com.lopexs.arl.opensocial.config;version=latest, jackson-core-asl, Apache ACE 2. Baseline 3. Publish 4. Distribution Targets 1. GIT tag 5. Provision
  20. 20. How do we know which bundles to update?
  21. 21. Semantic versioning is usually about package versions Export-Package: org.apache.felix.dm;version="3.1.0"
  22. 22. Semantic versioning demystified major.minor.micro(.qualifier) Major: Backward incompatible change Minor: Backward compatible change Micro: Implementation change Qualifier: Label, e.g. build number 1.0.0.x
  23. 23. Semantic versioning on bundles Bundle-Version: 1.0.0 Important for deployment!
  24. 24. Baselining Bytecode analysis Compare build with latest release Check if version numbers should be bumped
  25. 25. Continuous deployment CI server Release OBR -runbundles: org.apache.felix.http.jetty;version=2.2.0, org.apache.felix.gogo.command;version=0.10.0, org.apache.felix.gogo.runtime;version=0.10.0, org.apache.felix.gogo.shell;version=0.10.0, osgi.cmpn;version='(4.2.0,4.2.1]', org.amdatu.security.tokenprovider.api;version=latest, org.amdatu.security.tokenprovider.impl;version=latest, org.amdatu.web.rest.doc, com.lopexs.arl.opensocial.config;version=latest, jackson-core-asl, Apache ACE 2. Baseline 3. Publish 4. Distribution CD server 1. GIT tag 5. Provision Snapshot OBR
  26. 26. ACE scripting with GoGo targetrepourl = "$DEPLOYMENT_TARGETOBR_URL" releaserepourl = "$DEPLOYMENT_RELEASEOBR_URL" autoconfurl = "$DEPLOYMENT_AUTOCONF_URL" sourceindex = (repo:index /tmp/ace-cdeploy) sourcerepo = (repo:repo R5 $sourceindex) targetrepo = (repo:repo OBR $targetrepourl) releaserepo = (repo:repo OBR $releaserepourl) echo "Deploying release resources" deployed = repo:cd $releaserepo $sourcerepo $targetrepo echo "Opening client workspace" workspace = (ace:cw) echo "Removing old artifacts" artifacts = ($workspace la) each $artifacts { $workspace da $it } echo "Creating new artifacts" each $deployed { identity = $it getIdentity version = $it getVersion name = "$identity - $version" url = $it getUrl mimetype = $it getMimetype
  27. 27. ACE scripting with GoGo echo "Creating new artifacts" each $deployed { identity = $it getIdentity version = $it getVersion name = "$identity - $version" url = $it getUrl mimetype = $it getMimetype if { $mimetype equals "application/xml:osgi-autoconf" } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" filename="$name" processorPid="org.osgi.deployment.rp.autoconf" ] [ cdartifact="true" ] } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" Bundle-SymbolicName="$identity" Bundle-Version="$version" ] [ cdartifact="true" ] } } echo "Create artifact to feature association" if { (coll:first ($workspace la2f "(rightEndpoint=*name=cdfeature*)")) } {
  28. 28. ACE scripting with GoGo echo "Create artifact to feature association" if { (coll:first ($workspace la2f "(rightEndpoint=*name=cdfeature*)")) } { } { $workspace ca2f "(cdartifact=true)" "(name=cdfeature)" "N" "1" } echo "Create feature" if { (coll:first ($workspace lf "(name=cdfeature)")) } { } { $workspace cf "cdfeature" } echo "Create feature to distribution association" if { (coll:first ($workspace lf2d "(leftEndpoint=*name=cdfeature*)")) } { } { $workspace cf2d "(name=cdfeature)" "(name=cddistribution)" "1" "1" } echo "Create distribution" if { (coll:first ($workspace ld "(name=cddistribution)")) } { } { $workspace cd "cddistribution" } echo "Create distribution to target association" if { (coll:first ($workspace ld2t "(leftEndpoint=*name=cddistribution*)")) } { } { $workspace cd2t "(name=cddistribution)" "(id=*)" "1" "N" } echo "Create target" target = (coll:first ($workspace lt "(id=cdtarget)"))
  29. 29. ACE scripting with GoGo echo "Create target" target = (coll:first ($workspace lt "(id=cdtarget)")) if { $target } { } { target = ($workspace ct "cdtarget") } if { $target isRegistered } { } { $target register } echo "Configure target properties" $target addTag "ip-address" "127.0.0.1" $target addTag "schoolid" "someschool" $target addTag "timezone" "Europe/Amsterdam" echo "Committing workspace" $workspace commit
  30. 30. Cluster OSGi Target Load Balancer MongoDB MongoDB MongoDB OSGi Target Infrastructure Management Component Provisioning Cluster Management System Monitoring Automatic Scaling Starts/stops nodes Deploys and updates software components MonitorsQualityofService aspects Adds/rem oves nodes from the cluster Instantiates one or more template based clusters Cluster TemplateCluster TemplateCluster Template Artifacts, Features & Deployments 5 Cluster Configuration 1 Requests nodes based on provided requirements 2 3Assigns softw are distributions to targets 4 Gets configured with artifacts for deployments Provides usage data for scaling algorithms 6 7 89
  31. 31. Tooling and frameworks are now mature enough How does this fit other applications? The architecture fits most modern web applications Works in the cloud (but not a requirement)
  32. 32. amdatu.org
  33. 33. Recommended reading ;) Available NOW! Book signing Tue 14.30 @ O’Reilly Booth (JavaOne Pavilion) Free books for the early birds...
  34. 34. There is more... Building Modular Cloud Applications in Java: Lessons Learned Tue 10am - 11am Modular JavaScript Wed 3pm - 4pm
  35. 35. Cloud provisioning http://ace.apache.org/ Cloud OSGi services http://www.amdatu.org/ Eclipse OSGi plugin http://bndtools.org/ That’s us http://luminis.eu/ There is more...
  36. 36. Takk Grazie Thank you Obrigado Mahalo DankeDank U Merci Gracias

×