From 1 To 30How To DisassembleOne Monster App Into An Ecosystem Of 30Jonathan Palley, CTO/COOGuo Lei, Chief Architect© 2010 Idapted, Ltd.
AnExperience
A Tale of Two Buildings
 ShanghaiLotus Riverside Community1909 BeijingForbidden City
130
What is one?
The entire web application/system/platform runs as one single rails application(We are talking about really large systems.  Multiple different types of clients/functions)
ProblemsConfused new staffHard to test/extend/scale
What is 30?
A ecosystem of applications
Independent
Linked and Seamless
Basic features of each appSeparate databaseRuns independently (complete story) Lightweight (single developer)Tight internal cohesion and loose external couplingAdvantagesIndependent Development Cycle
Developer autonomy
Technology (im)maturity safetyAPPEAL TO DEVELOPER LAZINESS
What’s the mystery of the forbidden city?
Consistent UIShared CSS/JS/StyleguideCommon Helpers in Shared GemSafely try new things
interfaceAll applications use the same base CSS/JSKeep all the application the same style<%=idp_include_js_css%># =><script src ="/assets/javascripts/frame.js" type="text/javascript"></script><link href="/assets/stylesheets/frame.css" media="screen" rel="stylesheet" type="text/css" />
interfaceCSS Framework
interfaceAbstract Common Helpers to GemSearch function for models
interfaceCommon Helpers: Combo search (cont)View:<%= search_form_for(HistoryRecord, :interaction_id, :released,[:rating, {:collection=>assess_ratings}],[:mark_spot_num,{:range=>true}], [:created_at, {:ampm=>true}]) %>Controller:@history_records= HistoryRecord.combo_search(params)
interfaceCommon Helpers: List tablewell formattedwith pagination sortablecustomizable
interfaceCommon Helpers: List table (cont)<%=idp_table_for(@history_records,:sortable=>true,:customize => "history_records") do |item, col|col.add:id, link_to(item.id, admin_history_record_path(item)),:order=>:idcol.build:duration, :waiting_time, :review_timecol.add:scenario, item.scenario_title, :order => :scenario_titlecol.add:mark_spot_num  end%>
interfaceDevelopment LifecycleImplement new View code/plugin in a second applicationAbstract intoplugin using existing “idp” helpersPut it into main view gem
interfacedata
dataHow do applications share data?	(remember: each app has its own data)“Read Only” Database Connections
 Services
 AJAX Loaded View SegmentsBusiness exampledatauserpurchasecourselearning process
Purchase AppdataRequirement: List course packages for user to select to purchasebutThe course package data is stored in the “course” application
Solutiondatareadonly db connectioncourse
CodedataModel:class CoursePackage< ActiveRecord::Baseacts_as_readonly:courseendView:<ul><%CoursePackage.all.eachdo |package| %><li><%= package.title %> <%= package.price %></li><%end %></ul>
dataWhy doesn’t this break the rule of loose coupling?
dataacts_as_readonly in Depthdef acts_as_readonly(name, options = {})config = CoreService.app(name).databaseestablish_connectionconfig[Rails.env]set_table_name(self.connection.current_database + (options[:table_name]||table_name).to_s)end
dataacts_as_readonly in Depthdef acts_as_readonly(name, options = {})config = CoreService.app(name).databaseestablish_connectionconfig[Rails.env]set_table_name(self.connection.current_database + (options[:table_name]||table_name).to_s)end
dataCore serviceclassCoreService < ActiveResource::Baseself.site = :userdefself.app(app_name)CoreService.find(app_name)endend
dataCentralized configurationHow does Core know all the configurations?
dataEach app posts its configuration to core when it is started
dataconfig/site_config.ymlapp: courseapi:course_list: package/coursesconfig/initializers/idp_initializer.rbCoreService.reset_config
datacore_service.rb in idp_libAPP_CONFIG = YAML.load(Rails.root.join(“config/site_config.yml”))defreset_configself.post(:reset_config, :app => {          :name => APP_CONFIG["app"], :settings => APP_CONFIG,          :database => YAML.load_file(Rails.root.join("config/database.yml"))})end
data
dataAgain, implemented in gemconfig/environment.rbconfig.gem ‘idp_helpers’config.gem ‘idp_lib’
datagems
dataWeb services for “write” interactionsclass CoursePackageService< ActiveSupport::Baseself.site = :courseend
exampledataRoadmap needs to be generated after learner pays.
codesdataCourse: app/controllers/roadmap_services_controller.rbdef  createRoadmap.generate(params[:user_id], params[:course_id])endPurchase: app/models/roadmap_service.rbclass RoadmapService < ActiveSupport::Baseself.site = :courseendPurchase: app/models/order.rbdefactivate_roadmapRoadmapService.create(self.user_id, self.course_id)end
dataAJAX Loaded Composite View<div><%=ajax_load(url_of(:course, :course_list)) %></div>Ecosystem url_forFetched from different applications
interfacedatauser
Features of User ServiceuserRegistration/loginProfile managementRole Based Access Control
Access ControluserEach Controller is one Node* Posted to user service when app starts
Access Controluserbefore_filter:check_access_rightdefcheck_access_rightunlessxml_request?orinner_request?access_deniedunlesshas_page_right?(params[:controller])endend* Design your apps so access control can be by controller!
userHow to share?
Step 1: User Authuser
Step 1: User Authuserconfig/initializers/idp_initializer.rbActionController::Base.session_store = :active_record_storeActiveRecord::SessionStore::Session.acts_as_remote:user,	:readonly=> false
Step 2: Access ControluserTell core its controllers structureCoreService. reset_rightsdefself.reset_rightsdata = load_controller_structureself.post(:reset_rights, :data =>data)end
Step 2: Access Controluserbefore_filter:check_access_rightdefcheck_access_rightunlessxml_request?orinner_request?access_deniedunlesshas_page_right?(params[:controller])endend
Step 2: Access Controluserhas_page_right?Readonly dbconn again
Step 2: Access Controluserclass IdpRoleRight < ActiveRecord::Baseacts_as_readonly:user, :table_name=> "role_rights"enddef has_page_right?(page)roles = current_user.rolesroles_of_page = IdpRoleRight.all(:conditions => ["path = ?", page]).map(&:role_id)(roles - (roles - roles_of_page)).size > 0end
userAgain, gems!config/environment.rbconfig.gem ‘idp_helpers’config.gem ‘idp_lib’config.gem ‘idp_core’
interfaceservicedatauser
serviceSupport applicationsFileMailComet service
Fileclass Article < ActiveRecord::Basehas_filesend Specify Class that Has FilesUpload File in Background to FileServiceIdp_file_formStore with app_name, model_name, model_idUse readonly magic to easily display@article.files.first.url
serviceCometclassChatRoom < ActiveRecord::Baseacts_as_realtimeend <%= realtime_for(@chat_room, current_user.login) %><%= realtime_data(dom_id, :add, :top) %>@chat_room.realtime_channel.broadcast(“hi world", current_user.login)
servicemailMail servicesMailService.send(“test@idapted.com”, :welcome, :user => “test”)
Host all in one domainLoad each rails app into a subdir, we use Unicorn  unicorn_rails --path /user  unicorn_rails --path /studycenter  unicorn_rails --path /scenario
Host all in one domainuse Nginx as a reverse proxylocation /user {    proxy_pass http://rails_app_user; } location /studycenter {    proxy_pass http://rails_app_studycenter; }
Host all in one domainAll you see is a uniform URL   www.eqenglish.com/user   www.eqenglish.com/studycenter   www.eqenglish.com/scenario
Pair-deploy
How to split one into many?
By Story		Each App is one group of similar features.By Data		Each App writes to the same data
ExampleUser ManagementCourse packagePurchaseLearning process…
Iteration
Be adventurous at the beginning.Split one into as many as you think is sensitive
Then you may findSome applications interact with each other frequently.Lots of messy and low efficiency code to deal with interacting.
Merge them into one.
MeasurementCritical and core task of single app should not call services of others.One doesn’t need to know much about others’ business to do one task  (or develop).Independent Stories

RailsConf 2010: From 1 to 30 - How to refactor one monolithic application into an application ecosystem

Editor's Notes

  • #9 New staff “Agile web development with rails”
  • #14 Make it easier to do something than not.
  • #27 Services are too slow.Svn:externals
  • #28 Use webservice. SLOW.Facebook stuff.Thrift
  • #32 Discovery. HUGE PAIN: Configuring each app to know about each other app.
  • #43 Interacting with Erlang
  • #59 Easy.BackupLike “one application”. But shared between many.Polymorphic Connection!MySQL constraints!
  • #62 Resources – Memory consumption. One app used a lot. Others not much. Memory is cheap.