Lessons learned from a large scale OSGii web app - P Bakker & J de Vreede

1,615 views

Published on

Building a large scale, cloud hosted, multi device product that changes the way students are taught in high schools, turns out to be not entirely trivial. We have been building PulseOn for the last two years. We faced many technical challenges, and have tried many different technologies. Some ideas turned out to be extremely helpful, some other ideas that looked promising turned out to be bad practices. In this talk we will discuss what we have learned about building modular, scalable web applications. We will go into different areas of the technology stack, from storage and (nosql) databases to a modular RESTful backend and a multi device HTML5 based frontend. We will also go into the cloud, and discuss topics such as auto scaling and failover. We will discuss what works and what doesn’t, and hopefully help other to make the right decisions.

Some topics that we will touch upon:
- Modular architecture with OSGi
- Failing at linked data and semantic databases
- Using MongoDB from OSGi
- Release process and semantic versioning
- Continuous deployment
- JavaScript frameworks

Bios:
Paul Bakker
Paul is a software architect for Luminis Technologies and the author of “Building Modular Cloud Apps With OSGi”. He believes that modularity and the cloud are the two main challenges we have to deal with to bring technology to the next level, and is working on making this possible for mainstream software development. Today he is working on educational software focussed on personalised learning for high school students in the Netherlands. Paul is an active contributor on open source projects such as Amdatu, Apache ACE and BndTools.

He has a background as a trainer on Java related technology and is a regular speaker on conferences such as JavaOne, Devoxx and JFokus.

Jago de Vreede
Jago is a software engineer at Luminis Technologies, as a software engineer he has seen a broad-spectrum of projects and he has been working on a large OSGi project for the last year. His work is not exclusive to java development but also does front-end development, and the integration between these. Performance tuning and optimizations are also part of his work.

Published in: Technology

Lessons learned from a large scale OSGii web app - P Bakker & J de Vreede

  1. 1. Lessons learned from a large scale OSGi web app Jago de Vreede Paul Bakker
  2. 2. Paul Bakker @pbakker
  3. 3. Jago deVreede
  4. 4. Agenda ! High level architecture Modular architecture Modularity with OSGi Cloud deployments with Apache ACE Frontend frameworks
  5. 5. Ask whenever you want!
  6. 6. The case PulseOn ! Educational system focussed on personalized learning ! Used in high schools in the Netherlands ! Expand to other countries in the near future
  7. 7. Cloud provisioning http://ace.apache.org/ bndtools Eclipse OSGi plugin http://bndtools.org/ Eclipse http://eclipse.org/ Source control Tools Stash, Jira, Bamboo http://atlassian.com/ WebStorm http://jetbrains.com/ webstorm/
  8. 8. HTML 5 + JavaScript RESTful services OSGi services Mongo Apache Felix S3 High level architecture Requirements: ! Agile and modular Modern web app UI mostly offloaded to clients or devices Integration via REST API Horizontally scalable A m d a t u
  9. 9. Auth Blob stores MongoDB Multi-tenancy OpenSocial Search Remote Services REST Template Web ... Components
  10. 10. Load Load Balancer PulseOn PulseOn PulseOn School B Mongo Mongo Mongo Deployment Load Load Balancer PulseOn PulseOn PulseOn School A Mongo Mongo Mongo Load Load Balancer DAMS DAMS DAMS Content backend Mongo Mongo Mongo
  11. 11. Profiles Rest Profiles API Profiles Service MongoDB Progress Rest Progress API Progress Service MongoDB Curriculum API Curriculum Service MongoDB ... Rest ... API ... Service services all the way down
  12. 12. A service should only do onesingle thing
  13. 13. Services are the (reusable) building blocks of our application
  14. 14. Benefits of a services based architecture ! Small services are easy to maintain Small services are composable Services are easily testable
  15. 15. Packaging services Services are packaged in small bundles Related bundles may be generated from a single Bndtools project
  16. 16. Data Storage
  17. 17. Semantic data store (triple store) MySQL (with JPA) MongoDB NowBeginning
  18. 18. Relational databases in a modular system A part of data should be owned by 1 service How to set boundaries in a relational world!?
  19. 19. Relational databases in the modern web ! ORM is REALLY difficult to get right Relational databases don’t scale well
  20. 20. MongoDB: our database of choice ! Object Mapping trivial Easy to scale Powerful aggregation/map-reduce framework
  21. 21. Code example: MongoDB with Amdatu Amdatu Mongo Service Setup Object Mapper Execute query
  22. 22. Release & Deployment
  23. 23. Binaries Release to release repo Release to release OBR
  24. 24. JavaScript minification closure compiler deterministic? => fixed March 3, 2014 Baseline OBR Release OBR CI Build Server
  25. 25. Deployment Software distribution framework Manages the installation and upgrade of bundles, configuration, etc. to heterogenous targets
  26. 26. Load Load Balancer PulseOn PulseOn PulseOn School B Mongo Mongo Mongo Deployment Load Load Balancer PulseOn PulseOn PulseOn School A Mongo Mongo Mongo Load Load Balancer DAMS DAMS DAMS Content backend Mongo Mongo Mongo
  27. 27. Deployment Load Load Balancer PulseOn PulseOn PulseOn School Load Load Balancer DAMS DAMS DAMS Content backend
  28. 28. Deployment Load Load Balancer Target Target Target School Load Load Balancer Target Target Target Content backend Release OBR Apache ACE Baseline OBR CI Build Server
  29. 29. Continuous Deployment Load Load Balancer Target School Load Load Balancer Target Content backend Snapshot OBR Apache ACE CI Build Server Like to know more? Continuous Automated Deployment with Apache ACE youtube.com/watch?v=4S_zvgG_MLw
  30. 30. Unit testing OSGi code is trivial But what if we want more?
  31. 31. Integration testing turns out to be trivial as well! Automated dependency injection of service Setup Config Admin Junit style asserts
  32. 32. Frontend frameworks
  33. 33. jQuery Pros Can do a lot with not much code Lot of helpful plugins available Easy to get started
  34. 34. jQuery Hard to maintain Cons Hard to test
  35. 35. Backbone.js Pros Designed towards consuming REST data Well structured
  36. 36. Backbone.js Cons Hard to maintain A LOT OF CODE!! Testable
  37. 37. AngularJS
  38. 38. AngularJS Pros Designed towards consuming REST data Well structured No guessing in htmlTestable
  39. 39. TypeScript Pros Well structured Code completion compile time checks
  40. 40. TypeScript Cons Well structured Slow-ish Need to have a definition to work
  41. 41. ! A modular architecture gives us:! Maintainability Extensibility Freedom to change Wrap up
  42. 42. But what if I want Spring EJB Hibernate? Just don’t… You really don’t need to.
  43. 43. Cloud provisioning http://ace.apache.org/ Eclipse OSGi plugin http://bndtools.org/ That’s us http://luminis- technologies.com Cloud OSGi services http://www.amdatu.org/ Amdatu Paul Bakker paul.bakker@luminis.eu 
 @pbakker Jago de Vreede jago.devreede@luminis.eu 

  44. 44. Learn more? ! Friday 10.20 - 12.50 or 14.45 - 17.15 ! Tutorial “Developing modular cloud applications with OSGi”

×