From 1 To 30
         How To Disassemble
One Monster App Into An Ecosystem Of 30

            Beta 技术沙龙
              http...
An
Experience
A Tale of Two Buildings
2009 Shanghai
                 Lotus Riverside Community




1909 Beijing
Forbidden City
1


30
What is one?
The entire web
   application/system/platform
     runs as one single rails
            application

(We are talking about...
Problems


Confused new staff




                     Hard to test/extend/scale
What is 30?
A ecosystem of applications
Independent
Linked and Seamless
Basic features of each app
•   Separate database
•   Runs independently (complete story)
•   Lightweight (single developer...
What’s the mystery of the forbidden city?
Consistent UI
• Shared CSS/JS/Styleguide
• Common Helpers in Shared Gem
• Safely try new things
interface
      All applications use the same base CSS/JS


   Keep all the application the same style
   <%= idp_include_...
interface

            CSS Framework
interface

   Abstract Common Helpers to Gem
   Search function for models
interface
        Common Helpers: Combo search
                  (cont)
   View:
   <%=
   search_form_for(HistoryRecord, ...
interface

            Common Helpers: List table




well formatted        sortable
with pagination       customizable
interface

   Common Helpers: List table (cont)
   <%=
    idp_table_for(@history_records,:sortable=>true,:customize =>
  ...
interface

            Development Lifecycle
  1. Implement new View code/plugin in a
     second application
  2. Abstrac...
interface   data
data


How do applications share data?
 (remember: each app has its own data)

 -“Read Only” Database Connections
 - Servi...
data
         Business example

                     user

                             purchase
       course

          ...
data
                Purchase App

   Requirement: List course packages for
         user to select to purchase

         ...
data
                    Solution

       readonly db connection




           course
data
                               Code

 Model:
 class CoursePackage < ActiveRecord::Base
    acts_as_readonly :course
 ...
data

    Why doesn’t this break the rule of
           loose coupling?
 Model:
 class CoursePackage < ActiveRecord::Base
...
data

       acts_as_readonly in Depth

 def acts_as_readonly(name, options = {})
   config = CoreService.app(name).databa...
data

        acts_as_readonly in Depth

 def acts_as_readonly(name, options = {})

   config = CoreService.app(name).data...
data

                    Core service

 class CoreService < ActiveResource::Base
     self.site = :user
     def self.app...
data

       Centralized configuration

How does Core know all the configurations?
data


Each app posts its configuration to core
  when it is started
data

 config/site_config.yml
       app: course
       api:
         course_list: package/courses

 config/initializers/i...
data


 core_service.rb in idp_lib
       APP_CONFIG =
         YAML.load(Rails.root.join(“config/site_config.yml”))

    ...
data




       Model in Purchase:
       class CoursePackage < ActiveRecord::Base
          acts_as_readonly :course
    ...
data



       Again, implemented in gem

   config/environment.rb

   config.gem ‘idp_helpers’
   config.gem ‘idp_lib’
data




       gems
data




• Web services for “write” interactions

 class CoursePackageService < ActiveSupport::Base
    self.site = :cours...
data
                 example

       Roadmap needs to be generated
        after learner pays.
data
                           codes
 Course: app/controllers/roadmap_services_controller.rb
 def create
    Roadmap.gene...
data
                AJAX Loaded Composite View
<div>
<%= ajax_load(url_of(:course, :course_list)) %>
</div>
             ...
data

              Open Source



   http://github.com/idapted/eco_apps
interface   data   user
user
       Features of User Service

 • Registration/login
 • Profile management
 • Role Based Access Control
user
                 Access Control
 Each Controller is one Node




       * Posted to user service when app starts
user
                  Access Control
before_filter :check_access_right

def check_access_right
     unless xml_request? o...
user




       How to share?
user
       Step 1: User Auth

              SSO

             Shared
             Session


           Same Session
     ...
user
                  Step 1: User Auth
config/initializers/idp_initializer.rb

ActionController::Base.session_store = :a...
user
             Step 2: Access Control
Tell core its controllers structure
CoreService. reset_rights

def self.reset_rig...
user
            Step 2: Access Control
before_filter :check_access_right

def check_access_right
     unless xml_request?...
user
       Step 2: Access Control


         has_page_right?

       Readonly db conn again
user
             Step 2: Access Control
class IdpRoleRight < ActiveRecord::Base
    acts_as_readonly :user, :table_name =...
user
          Again, gems!

  config/environment.rb

  config.gem ‘idp_helpers’
  config.gem ‘idp_lib’
  config.gem ‘idp_...
interface   data   user   service
service

            Support applications

          • File
          • Mail
          • Comet service
File
Specify Class that
                     class Article < ActiveRecord::Base
    Has Files         has_files
          ...
service

                               Comet
  class ChatRoom < ActiveRecord::Base
   acts_as_realtime
  end


          ...
service

                        mail

  Mail services
  MailService.send(“test@idapted.com”, :welcome,
  :user => “test”)
Host all in one domain
Load each rails app into a subdir, we
  use Unicorn
  unicorn_rails --path /user
  unicorn_rails --...
Host all in one domain
use Nginx as a reverse proxy
 location /user {
    proxy_pass http://rails_app_user;
  }

  locatio...
Host all in one domain
All you see is a uniform URL

   www.eqenglish.com/user
   www.eqenglish.com/studycenter
   www.eqe...
Pair-deploy
How to split one into many?
By Story
   Each App is one group of similar features.

By Data
   Each App writes to the same data
Example

•   User Management
•   Course package
•   Purchase
•   Learning process
•   …
Iteration
Be adventurous at the beginning.
Split one into as many as you think
  is sensitive
Then you may find

• Some applications interact with each other
  frequently.
• Lots of messy and low efficiency code to d...
Merge them into one.
Measurement

• Critical and core task of single app should
  not call services of others.
• One doesn’t need to know much ...
Pitfalls

• Applications need to be on the
  same intranet.
• No “right place” for certain cases.
Results: Higher Productivity


-Faster build to deploy
-More developer autonomy
-Safer
- Scalable
-Easier to “jump in”
- G...
Support Tech
• FreeSWITCH
    • VoIP
    • http://www.freeswitch.org.cn
• Flex
• Erlang
    • concurrent tasks
• Titanium
...
Full Stack of Us



CTO          Rails/Flex


 VoIP     UI     UE       System
Full Stack of Us
Develop environment:
  • 4 mbp + 4 black apples + 1 linux
  • textmate & netbeans

Servers:
  • test: 2 p...
Full Stack of Us
Web Server:
  • nginx + unicorn


Rails:
  • Rails version 2.3.4
  • Ruby version: 1.8.7
Full Stack of Us
Plugins:
  •   exception_notification
  •   will_paginate
  •   in_place_editor_plus
  •   acts_as_list
 ...
Full Stack of Us
Gems:
  • rspec
  • factory_girl
  • thinking-sphinx
DB:
  • mysql
Cache:
  • memcache
Full Stack of Us
Test:
  • rspec + factory_girl
  • Autospec


Deploy:
  • webistrano
Full Stack of Us
Team Build:
  •   board
  •   code review
  •   tea break
  •   retreat
Join Us
If you are:
  •   smart
  •   creative
  •   willing to do great things with great people
  •   happen to know JS/...
Thank you!

          Q&A
http://developer.idapted.com

jpalley@idapted.com (@jpalley)
guolei@idapted.com (@fiyuer)

     ...
How to disassemble one monster app into an ecosystem of 30
Upcoming SlideShare
Loading in …5
×

How to disassemble one monster app into an ecosystem of 30

1,457 views

Published on

Published in: Technology
  • Be the first to comment

How to disassemble one monster app into an ecosystem of 30

  1. 1. From 1 To 30 How To Disassemble One Monster App Into An Ecosystem Of 30 Beta 技术沙龙 http://club.blogbeta.com 官方twitter : @betasalon Groups : http:// groups.google.com/group/betasalon Jonathan Palley, CTO/COO Guo Lei, Chief Architect © 2010 Idapted, Ltd.
  2. 2. An Experience
  3. 3. A Tale of Two Buildings
  4. 4. 2009 Shanghai Lotus Riverside Community 1909 Beijing Forbidden City
  5. 5. 1 30
  6. 6. What is one?
  7. 7. The entire web application/system/platform runs as one single rails application (We are talking about really large systems. Multiple different types of clients/functions)
  8. 8. Problems Confused new staff Hard to test/extend/scale
  9. 9. What is 30?
  10. 10. A ecosystem of applications
  11. 11. Independent
  12. 12. Linked and Seamless
  13. 13. Basic features of each app • Separate database • Runs independently (complete story) • Lightweight (single developer) • Tight internal cohesion and loose external coupling Advantages • Independent Development Cycle • Developer autonomy • Technology (im)maturity safety APPEAL TO DEVELOPER LAZINESS
  14. 14. What’s the mystery of the forbidden city?
  15. 15. Consistent UI • Shared CSS/JS/Styleguide • Common Helpers in Shared Gem • Safely try new things
  16. 16. interface All applications use the same base CSS/JS Keep all the application the same style <%= idp_include_js_css %> # => <script src ="/assets/javascripts/frame.js" type="text/javascript"></script> <link href="/assets/stylesheets/frame.css" media="screen" rel="stylesheet" type="text/css" />
  17. 17. interface CSS Framework
  18. 18. interface Abstract Common Helpers to Gem Search function for models
  19. 19. interface Common Helpers: Combo search (cont) View: <%= search_form_for(HistoryRecord, :interaction_id, :released,[:rating, {:collection=>assess_ratings}],[:mark_spot_num,{:range=>true}], [:created_at, {:ampm=>true}]) %> Controller: @history_records = HistoryRecord.combo_search(params)
  20. 20. interface Common Helpers: List table well formatted sortable with pagination customizable
  21. 21. interface Common Helpers: List table (cont) <%= idp_table_for(@history_records,:sortable=>true,:customize => "history_records") do |item, col| col.add :id, link_to(item.id, admin_history_record_path(item)),:order=>:id col.build :duration, :waiting_time, :review_time col.add :scenario, item.scenario_title, :order => :scenario_title col.add :mark_spot_num end %>
  22. 22. interface Development Lifecycle 1. Implement new View code/plugin in a second application 2. Abstract into plugin using existing “idp” helpers 3. Put it into main view gem
  23. 23. interface data
  24. 24. data How do applications share data? (remember: each app has its own data) -“Read Only” Database Connections - Services - AJAX Loaded View Segments
  25. 25. data Business example user purchase course learning process
  26. 26. data Purchase App Requirement: List course packages for user to select to purchase but The course package data is stored in the “course” application
  27. 27. data Solution readonly db connection course
  28. 28. data Code Model: class CoursePackage < ActiveRecord::Base acts_as_readonly :course end View: <ul> <% CoursePackage.all.each do |package| %> <li><%= package.title %> <%= package.price %></li> <% end %> </ul>
  29. 29. data Why doesn’t this break the rule of loose coupling? Model: class CoursePackage < ActiveRecord::Base acts_as_readonly :course end View: <ul> <% CoursePackage.all.each do |package| %> <li><%= package.title %> <%= package.price %></li> <% end %> </ul>
  30. 30. data acts_as_readonly in Depth def acts_as_readonly(name, options = {}) config = CoreService.app(name).database establish_connection config[Rails.env] set_table_name(self.connection.current_database + (options[:table_name]||table_name).to_s) end
  31. 31. data acts_as_readonly in Depth def acts_as_readonly(name, options = {}) config = CoreService.app(name).database establish_connection config[Rails.env] set_table_name(self.connection.current_database + (options[:table_name]||table_name).to_s) end
  32. 32. data Core service class CoreService < ActiveResource::Base self.site = :user def self.app(app_name) CoreService.find(app_name) end end
  33. 33. data Centralized configuration How does Core know all the configurations?
  34. 34. data Each app posts its configuration to core when it is started
  35. 35. data config/site_config.yml app: course api: course_list: package/courses config/initializers/idp_initializer.rb CoreService.reset_config
  36. 36. data core_service.rb in idp_lib APP_CONFIG = YAML.load(Rails.root.join(“config/site_config.yml”)) def reset_config self.post(:reset_config, :app => { :name => APP_CONFIG["app"], :settings => APP_CONFIG, :database => YAML.load_file( Rails.root.join("config/database.yml"))}) end
  37. 37. data Model in Purchase: class CoursePackage < ActiveRecord::Base acts_as_readonly :course end
  38. 38. data Again, implemented in gem config/environment.rb config.gem ‘idp_helpers’ config.gem ‘idp_lib’
  39. 39. data gems
  40. 40. data • Web services for “write” interactions class CoursePackageService < ActiveSupport::Base self.site = :course end
  41. 41. data example Roadmap needs to be generated after learner pays.
  42. 42. data codes Course: app/controllers/roadmap_services_controller.rb def create Roadmap.generate(params[:user_id], params[:course_id]) end Purchase: app/models/roadmap_service.rb class RoadmapService < ActiveSupport::Base self.site = :course end Purchase: app/models/order.rb def activate_roadmap RoadmapService.create(self.user_id, self.course_id) end
  43. 43. data AJAX Loaded Composite View <div> <%= ajax_load(url_of(:course, :course_list)) %> </div> Ecosystem url_for Fetched from different applications
  44. 44. data Open Source http://github.com/idapted/eco_apps
  45. 45. interface data user
  46. 46. user Features of User Service • Registration/login • Profile management • Role Based Access Control
  47. 47. user Access Control Each Controller is one Node * Posted to user service when app starts
  48. 48. user Access Control before_filter :check_access_right def check_access_right unless xml_request? or inner_request? access_denied unless has_page_right?(params[:controller]) end end * Design your apps so access control can be by controller!
  49. 49. user How to share?
  50. 50. user Step 1: User Auth SSO Shared Session Same Session Store
  51. 51. user Step 1: User Auth config/initializers/idp_initializer.rb ActionController::Base.session_store = :active_record_store ActiveRecord::SessionStore::Session.acts_as_remote :user, :readonly => false
  52. 52. user Step 2: Access Control Tell core its controllers structure CoreService. reset_rights def self.reset_rights data = load_controller_structure self.post(:reset_rights, :data => data) end
  53. 53. user Step 2: Access Control before_filter :check_access_right def check_access_right unless xml_request? or inner_request? access_denied unless has_page_right?(params[:controller]) end end
  54. 54. user Step 2: Access Control has_page_right? Readonly db conn again
  55. 55. user Step 2: Access Control class IdpRoleRight < ActiveRecord::Base acts_as_readonly :user, :table_name => "role_rights" end def has_page_right?(page) roles = current_user.roles roles_of_page = IdpRoleRight.all(:conditions => ["path = ?", page]).map(&:role_id) (roles - (roles - roles_of_page)).size > 0 end
  56. 56. user Again, gems! config/environment.rb config.gem ‘idp_helpers’ config.gem ‘idp_lib’ config.gem ‘idp_core’
  57. 57. interface data user service
  58. 58. service Support applications • File • Mail • Comet service
  59. 59. File Specify Class that class Article < ActiveRecord::Base Has Files has_files end Upload File in Background to Idp_file_form FileService Store with app_name, model_name, model_id Use readonly magic to easily @article.files.first.url display
  60. 60. service Comet class ChatRoom < ActiveRecord::Base acts_as_realtime end <%= realtime_for(@chat_room, current_user.login) %> <%= realtime_data(dom_id, :add, :top) %> @chat_room.realtime_channel.broadcast (“hi world", current_user.login)
  61. 61. service mail Mail services MailService.send(“test@idapted.com”, :welcome, :user => “test”)
  62. 62. Host all in one domain Load each rails app into a subdir, we use Unicorn unicorn_rails --path /user unicorn_rails --path /studycenter unicorn_rails --path /scenario
  63. 63. Host all in one domain use Nginx as a reverse proxy location /user { proxy_pass http://rails_app_user; } location /studycenter { proxy_pass http://rails_app_studycenter; }
  64. 64. Host all in one domain All you see is a uniform URL www.eqenglish.com/user www.eqenglish.com/studycenter www.eqenglish.com/scenario
  65. 65. Pair-deploy
  66. 66. How to split one into many?
  67. 67. By Story Each App is one group of similar features. By Data Each App writes to the same data
  68. 68. Example • User Management • Course package • Purchase • Learning process • …
  69. 69. Iteration
  70. 70. Be adventurous at the beginning. Split one into as many as you think is sensitive
  71. 71. Then you may find • Some applications interact with each other frequently. • Lots of messy and low efficiency code to deal with interacting.
  72. 72. Merge them into one.
  73. 73. Measurement • Critical and core task of single app should not call services of others. • One doesn’t need to know much about others’ business to do one task (or develop). • Independent Stories
  74. 74. Pitfalls • Applications need to be on the same intranet. • No “right place” for certain cases.
  75. 75. Results: Higher Productivity -Faster build to deploy -More developer autonomy -Safer - Scalable -Easier to “jump in” - Greater Happiness
  76. 76. Support Tech • FreeSWITCH • VoIP • http://www.freeswitch.org.cn • Flex • Erlang • concurrent tasks • Titanium • mobile
  77. 77. Full Stack of Us CTO Rails/Flex VoIP UI UE System
  78. 78. Full Stack of Us Develop environment: • 4 mbp + 4 black apples + 1 linux • textmate & netbeans Servers: • test: 2 physical pc with Xen serving 10 virtual servers • production: 2 (China) + 5(US)
  79. 79. Full Stack of Us Web Server: • nginx + unicorn Rails: • Rails version 2.3.4 • Ruby version: 1.8.7
  80. 80. Full Stack of Us Plugins: • exception_notification • will_paginate • in_place_editor_plus • acts_as_list • open_flash_chart • Spawn + workling
  81. 81. Full Stack of Us Gems: • rspec • factory_girl • thinking-sphinx DB: • mysql Cache: • memcache
  82. 82. Full Stack of Us Test: • rspec + factory_girl • Autospec Deploy: • webistrano
  83. 83. Full Stack of Us Team Build: • board • code review • tea break • retreat
  84. 84. Join Us If you are: • smart • creative • willing to do great things with great people • happen to know JS/Flex/Rails Send whatever can prove your ability to: tech-team-jobs@idapted.com
  85. 85. Thank you! Q&A http://developer.idapted.com jpalley@idapted.com (@jpalley) guolei@idapted.com (@fiyuer) © 2010 Idapted, Ltd.

×