A Web Widget Platform

586 views

Published on

A presentation about a distributed Erlang web platform focused on scalability and feature isolation.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
586
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

A Web Widget Platform

  1. 1. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsA Web Widget PlatformEnrique PazSenior Backend Developer13/06/20131/33
  2. 2. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsAbout Me• Passionate Erlang developer• Testing enthusiast• Love beautiful code!2/33
  3. 3. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsSpilgames• Gaming Platform• Serving data to 190+ countries world-wide• 180+ million unique users per month• Multiple Platforms: Desktop, Mobile Native & Web• 300+ employees• Offices in The Netherlands & China• Revenue: Advertising & EUM3/33
  4. 4. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsOld Portals Landscape4/33
  5. 5. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsBrands and Sites• 5 Brands/Channels for target audiences• 1-15 Sites for different geographical regions• Strong domainswww.spel.se for teens in Swedenwww.ojogos.pt for families in Portugal and Brazil. . .5/33
  6. 6. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsGaming Portals6/33
  7. 7. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsArchitectural Overview• Monolithic portals perbrand• Static pages with lots ofAJAX calls• Architectural layersdeveloped independently7/33
  8. 8. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsThe Good• Ability to develop per brand and site• Acceptable performance• Clear layer separation• Fast growth8/33
  9. 9. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsThe Bad• Inability to share features between brand codebases• Aggressive multi-level caching needed to perform decently• Lots of logic on the client• Developers can’t switch brands easily• Long time to market9/33
  10. 10. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsWish List10/33
  11. 11. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsIsolation & Performance• Isolated requests, features and crashes• Feature toggling on runtime• Decent number of requests/s with linearscalability• Technology aligned application and storagelayers11/33
  12. 12. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsIncrease Development Speed• Only one modular codebase• Small features, easy to deploy and rollback• Changes testable in isolation12/33
  13. 13. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsWhat’s out there• MVC Frameworks (Chicago Boss, Zotonic. . . )(+) Nice layer separation(+) Mature solutions(+) Easy to use13/33
  14. 14. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsWhat’s out there• MVC Frameworks (Chicago Boss, Zotonic. . . )(+) Nice layer separation(+) Mature solutions(+) Easy to use• But. . .(-) Feature isolation not key in their design(-) A VC framework fits better (abstracted model)13/33
  15. 15. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsA Widget Platform14/33
  16. 16. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsA Widget Platform15/33
  17. 17. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsWidgets• Isolated• Independent Erlangapplications• Use ErlyDTL for the view• Specification in the .app.src• Can only use what theyspecify16/33
  18. 18. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsWidgets: declaring everything• Children• Public properties• Private properties• Themes• JS decoration• CSS decoration{ env , [{ children , [ wdg_game_list_breadcrumb ] } ,{ interface , [{ page , "1" } ,{ pageSize , " 20 " }] } ,{ private_interface , [{ template , wdg_game_list } ,{ css , [ " wdg_game_list " ] } ,{ js , [ " i n i t . j s " ] } ,{games , [ ] } ,{ pagination_bar , [ ] }] } ,{ themes , [{ " teens " , [{ template , wdg_game_list_teens } ,{ css , [ " teens / wdg_game_list_teens " ] } ,{ js , [ " teens / i n i t . j s " ] }] }] } ,{ js_modules , [ ] } ,{ js_event_config , [{ l i s t e n , [ ’ system . user . search . request ’ ] } ,{ emit , [ ] }] }] }17/33
  19. 19. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsWidget Libraries• Widget type definitions• Widget behaviour• Wrappers for common actions• Connection to the applicationlayer• ErlyDTL custom tags18/33
  20. 20. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsWidget Platform• Widget dependencies• Path to widget mapping• Erlang web server• Request Flow• Global error pages19/33
  21. 21. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsNGINX Virtual Host declaration• Domain mapping• Widget static files• Static error pages• URL rewriting20/33
  22. 22. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsRequest Flow21/33
  23. 23. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsSets Of Widgets In One Platform• Allows the cration of small deltas• Different prearranged sets can be requested• Different sets can be built per request22/33
  24. 24. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsMaster & Slaves In 1 VM• The platform master node runs:A web serverA management interfaceA specific set of widgets23/33
  25. 25. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsMaster & Slaves In 1 VM• The platform master node runs:A web serverA management interfaceA specific set of widgets• Each slave node runs:Its own specific set of widgets23/33
  26. 26. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionswdg1/?origins=w1,preview;w5,preview• Widgets can be called in anynode• Widgets can fail• Widgets can be disabled24/33
  27. 27. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsManaging Widgets25/33
  28. 28. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsThe Need For A Widget Manager• To manage widgets and versions in the platform nodes• To keep the clustered platforms in sync• To make changes in the cluster in an easy way26/33
  29. 29. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsHow does it work• SWIFT as a central widgetrepository• HTTP interface with simpleGUI• Uses the platformmanagement interface:List nodesList widgets in a nodeEnable/disable widget in anodeChange widget version in anode• Enforces configuration to newcluster members27/33
  30. 30. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsWhat’s in the Future?28/33
  31. 31. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsLessons Learned• Reinforce the concepts adoption often• Review, review, review• Measure performance from earlystages29/33
  32. 32. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsComing in H2• Native widget to backend connectivity• Per widget render timeout• In req caching of the BE calls• Performance analysis and improvements• Router to be open sourced30/33
  33. 33. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsWhat We’ve Used• ErlyDTL• Cowboy• Lager• Rebar(*) semantic versioning, i.e. [">=1.3.1", "<2.0.0"](*) shared dependencies(*) xref fixes• Lhttpc• Estatsd31/33
  34. 34. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsQuestions?32/33
  35. 35. IntroductionSpilgames & MeOld PortalsLandscapeOverviewEvaluationWish ListTech WishesTaskforce WishesAvailableFrameworksA WidgetPlatformComponentsThe FlowTesting &DeployingManagingWidgetsHolding ControlNext StepsSummaryContributionsSee You Around!Thanks!33/33

×