As your business grows bigger, you just can’t stop adding new models/controllers to your original rails application – resulting in a messy, unmaintainable and difficult to deploy monolithic application. Its time to refactor. This talk will share our experience, results and best practices in splitting a single rails “application-system” into 30 independently maintainable yet interconnected applications. After two and a half years of development (starting in pre-Rails 1.0 days!), our live-trainer English learning system now supported multiple roles (learner/trainer/trainer supervisor/sales/materials creation/support/etc) and an exhaustive list of features to support our complex business processes. We set ourselves a year-long goal of splitting this monolithic system into small cooperating applications that could be developed independently by individual developers. At the same time, we could not lose the usability cohesiveness and data-interdependence that defined the power of our system. Through numerous iterations, many mistakes and a bit of pure-luck we developed an optimized process for the refactor and best practices for making 30 independent rails apps behave as one. The results: lower development time, greater stability and scalability and much higher developer happiness. We’ll talk about specific code, measurements, pitfalls, plugins, process and best practices to answer questions such as: How to know where to split single applications into many. How to measure the result. How the applications should interact with each other. How to reduce administration and DRY configuration applications. How to share data among applications. How to DRY for common logic. How to make a consistent user experience. How to interact with non-Ruby technology; in our case Erlang, FreeSWITCH (VoIP) and Flex

1. From 1 To 30 How To Disassemble One Monster App Into An Ecosystem Of 30 Jonathan Palley, CTO/COO Guo Lei, Chief Architect © 2010 Idapted, Ltd.

2. An Experience

3. A Tale of Two Buildings

4. Shanghai Lotus Riverside Community 1909 Beijing Forbidden City

5. 1 30

6. What is one?

7. The entire web application/system/platform runs as one single rails application (We are talking about really large systems. Multiple different types of clients/functions)

8. Problems Confused new staff Hard to test/extend/scale

9. What is 30?

10. A ecosystem of applications

11. Independent

12. Linked and Seamless

14. Developer autonomy

15. Technology (im)maturity safetyAPPEAL TO DEVELOPER LAZINESS

16. What’s the mystery of the forbidden city?

17. Consistent UI Shared CSS/JS/Styleguide Common Helpers in Shared Gem Safely try new things

18. interface All applications use the same base CSS/JS Keep all the application the same style <%=idp_include_js_css%> # => <script src ="/assets/javascripts/frame.js" type="text/javascript"></script> <link href="/assets/stylesheets/frame.css" media="screen" rel="stylesheet" type="text/css" />

19. interface CSS Framework

20. interface Abstract Common Helpers to Gem Search function for models

21. interface Common Helpers: Combo search (cont) View: <%= search_form_for(HistoryRecord, :interaction_id, :released,[:rating, {:collection=>assess_ratings}],[:mark_spot_num,{:range=>true}], [:created_at, {:ampm=>true}]) %> Controller: @history_records= HistoryRecord.combo_search(params)

22. interface Common Helpers: List table well formatted with pagination sortable customizable

23. interface Common Helpers: List table (cont) <%= idp_table_for(@history_records,:sortable=>true,:customize => "history_records") do |item, col| col.add:id, link_to(item.id, admin_history_record_path(item)),:order=>:id col.build:duration, :waiting_time, :review_time col.add:scenario, item.scenario_title, :order => :scenario_title col.add:mark_spot_num end %>

24. interface Development Lifecycle Implement new View code/plugin in a second application Abstract intoplugin using existing “idp” helpers Put it into main view gem

25. interface data

27. Services

29. Purchase App data Requirement: List course packages for user to select to purchase but The course package data is stored in the “course” application

30. Solution data readonly db connection course

31. Code data Model: class CoursePackage< ActiveRecord::Base acts_as_readonly:course end View: <ul> <%CoursePackage.all.eachdo |package| %> <li><%= package.title %> <%= package.price %></li> <%end %> </ul>

32. data Why doesn’t this break the rule of loose coupling?

33. data acts_as_readonly in Depth def acts_as_readonly(name, options = {}) config = CoreService.app(name).database establish_connectionconfig[Rails.env] set_table_name(self.connection.current_database + (options[:table_name]||table_name).to_s) end

34. data acts_as_readonly in Depth def acts_as_readonly(name, options = {}) config = CoreService.app(name).database establish_connectionconfig[Rails.env] set_table_name(self.connection.current_database + (options[:table_name]||table_name).to_s) end

35. data Core service classCoreService < ActiveResource::Base self.site = :user defself.app(app_name) CoreService.find(app_name) end end

36. data Centralized configuration How does Core know all the configurations?

37. data Each app posts its configuration to core when it is started

38. data config/site_config.yml app: course api: course_list: package/courses config/initializers/idp_initializer.rb CoreService.reset_config

39. data core_service.rb in idp_lib APP_CONFIG = YAML.load(Rails.root.join(“config/site_config.yml”)) defreset_config self.post(:reset_config, :app => { :name => APP_CONFIG["app"], :settings => APP_CONFIG, :database => YAML.load_file( Rails.root.join("config/database.yml"))}) end

40. data

41. data Again, implemented in gem config/environment.rb config.gem ‘idp_helpers’ config.gem ‘idp_lib’

42. data gems

43. data Web services for “write” interactions class CoursePackageService< ActiveSupport::Base self.site = :course end

44. example data Roadmap needs to be generated after learner pays.

45. codes data Course: app/controllers/roadmap_services_controller.rb def create Roadmap.generate(params[:user_id], params[:course_id]) end Purchase: app/models/roadmap_service.rb class RoadmapService < ActiveSupport::Base self.site = :course end Purchase: app/models/order.rb defactivate_roadmap RoadmapService.create(self.user_id, self.course_id) end

46. data AJAX Loaded Composite View <div> <%=ajax_load(url_of(:course, :course_list)) %> </div> Ecosystem url_for Fetched from different applications

47. interface data user

48. Features of User Service user Registration/login Profile management Role Based Access Control

49. Access Control user Each Controller is one Node * Posted to user service when app starts

50. Access Control user before_filter:check_access_right defcheck_access_right unlessxml_request?orinner_request? access_deniedunlesshas_page_right?(params[:controller]) end end * Design your apps so access control can be by controller!

51. user How to share?

52. Step 1: User Auth user

53. Step 1: User Auth user config/initializers/idp_initializer.rb ActionController::Base.session_store = :active_record_store ActiveRecord::SessionStore::Session.acts_as_remote:user, :readonly=> false

54. Step 2: Access Control user Tell core its controllers structure CoreService. reset_rights defself.reset_rights data = load_controller_structure self.post(:reset_rights, :data =>data) end

55. Step 2: Access Control user before_filter:check_access_right defcheck_access_right unlessxml_request?orinner_request? access_deniedunlesshas_page_right?(params[:controller]) end end

56. Step 2: Access Control user has_page_right? Readonly dbconn again

57. Step 2: Access Control user class IdpRoleRight < ActiveRecord::Base acts_as_readonly:user, :table_name=> "role_rights" end def has_page_right?(page) roles = current_user.roles roles_of_page = IdpRoleRight.all(:conditions => ["path = ?", page]).map(&:role_id) (roles - (roles - roles_of_page)).size > 0 end

58. user Again, gems! config/environment.rb config.gem ‘idp_helpers’ config.gem ‘idp_lib’ config.gem ‘idp_core’

59. interface service data user

60. service Support applications File Mail Comet service

61. File class Article < ActiveRecord::Base has_files end Specify Class that Has Files Upload File in Background to FileService Idp_file_form Store with app_name, model_name, model_id Use readonly magic to easily display @article.files.first.url

62. service Comet classChatRoom < ActiveRecord::Base acts_as_realtime end <%= realtime_for(@chat_room, current_user.login) %> <%= realtime_data(dom_id, :add, :top) %> @chat_room.realtime_channel.broadcast(“hi world", current_user.login)

63. service mail Mail services MailService.send(“test@idapted.com”, :welcome, :user => “test”)

64. Host all in one domain Load each rails app into a subdir, we use Unicorn unicorn_rails --path /user unicorn_rails --path /studycenter unicorn_rails --path /scenario

65. Host all in one domain use Nginx as a reverse proxylocation /user { proxy_pass http://rails_app_user; } location /studycenter { proxy_pass http://rails_app_studycenter; }

66. Host all in one domain All you see is a uniform URL www.eqenglish.com/user www.eqenglish.com/studycenter www.eqenglish.com/scenario

68. Pair-deploy

69. How to split one into many?

70. By Story Each App is one group of similar features. By Data Each App writes to the same data

71. Example User Management Course package Purchase Learning process …

72. Iteration

73. Be adventurous at the beginning. Split one into as many as you think is sensitive

74. Then you may find Some applications interact with each other frequently. Lots of messy and low efficiency code to deal with interacting.

75. Merge them into one.

76. Measurement Critical and core task of single app should not call services of others. One doesn’t need to know much about others’ business to do one task (or develop). Independent Stories

77. Pitfalls Applications need to be on the same intranet. No “right place” for certain cases.

79. More developer autonomy

80. Safer

81. Scalable

82. Easier to “jump in”