• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
RailsConf 2010: From 1 to 30 - How to refactor one monolithic application into an application ecosystem
 

RailsConf 2010: From 1 to 30 - How to refactor one monolithic application into an application ecosystem

on

  • 4,649 views

As your business grows bigger, you just can’t stop adding new models/controllers to your original rails application – resulting in a messy, unmaintainable and difficult to deploy monolithic ...

As your business grows bigger, you just can’t stop adding new models/controllers to your original rails application – resulting in a messy, unmaintainable and difficult to deploy monolithic application. Its time to refactor. This talk will share our experience, results and best practices in splitting a single rails “application-system” into 30 independently maintainable yet interconnected applications.

After two and a half years of development (starting in pre-Rails 1.0 days!), our live-trainer English learning system now supported multiple roles (learner/trainer/trainer supervisor/sales/materials creation/support/etc) and an exhaustive list of features to support our complex business processes. We set ourselves a year-long goal of splitting this monolithic system into small cooperating applications that could be developed independently by individual developers. At the same time, we could not lose the usability cohesiveness and data-interdependence that defined the power of our system.

Through numerous iterations, many mistakes and a bit of pure-luck we developed an optimized process for the refactor and best practices for making 30 independent rails apps behave as one. The results: lower development time, greater stability and scalability and much higher developer happiness.

We’ll talk about specific code, measurements, pitfalls, plugins, process and best practices to answer questions such as:

How to know where to split single applications into many. How to measure the result.
How the applications should interact with each other. How to reduce administration and DRY configuration applications.
How to share data among applications.
How to DRY for common logic.
How to make a consistent user experience.
How to interact with non-Ruby technology; in our case Erlang, FreeSWITCH (VoIP) and Flex

Statistics

Views

Total Views
4,649
Views on SlideShare
4,276
Embed Views
373

Actions

Likes
10
Downloads
46
Comments
1

3 Embeds 373

http://developer.idapted.com 310
http://www.slideshare.net 62
https://duckduckgo.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • New staff “Agile web development with rails”
  • Make it easier to do something than not.
  • Services are too slow.Svn:externals
  • Use webservice. SLOW.Facebook stuff.Thrift
  • Discovery. HUGE PAIN: Configuring each app to know about each other app.
  • Interacting with Erlang
  • Easy.BackupLike “one application”. But shared between many.Polymorphic Connection!MySQL constraints!
  • Resources – Memory consumption. One app used a lot. Others not much. Memory is cheap.

RailsConf 2010: From 1 to 30 - How to refactor one monolithic application into an application ecosystem RailsConf 2010: From 1 to 30 - How to refactor one monolithic application into an application ecosystem Presentation Transcript

  • From 1 To 30
    How To Disassemble
    One Monster App Into An Ecosystem Of 30
    Jonathan Palley, CTO/COO
    Guo Lei, Chief Architect
    © 2010 Idapted, Ltd.
  • An
    Experience
  • A Tale of Two Buildings
  • Shanghai
    Lotus Riverside Community
    1909 Beijing
    Forbidden City
  • 1
    30
  • What is one?
  • The entire web application/system/platform runs as one single rails application
    (We are talking about really large systems.
    Multiple different types of clients/functions)
  • Problems
    Confused new staff
    Hard to test/extend/scale
  • What is 30?
  • A ecosystem of applications
  • Independent
  • Linked and Seamless
  • Basic features of each app
    Separate database
    Runs independently (complete story)
    Lightweight (single developer)
    Tight internal cohesion and loose external coupling
    Advantages
    • Independent Development Cycle
    • Developer autonomy
    • Technology (im)maturity safety
    APPEAL TO DEVELOPER LAZINESS
  • What’s the mystery of the forbidden city?
  • Consistent UI
    Shared CSS/JS/Styleguide
    Common Helpers in Shared Gem
    Safely try new things
  • interface
    All applications use the same base CSS/JS
    Keep all the application the same style
    <%=idp_include_js_css%>
    # =>
    <script src ="/assets/javascripts/frame.js" type="text/javascript"></script>
    <link href="/assets/stylesheets/frame.css" media="screen" rel="stylesheet" type="text/css" />
  • interface
    CSS Framework
  • interface
    Abstract Common Helpers to Gem
    Search function for models
  • interface
    Common Helpers: Combo search (cont)
    View:
    <%= search_form_for(HistoryRecord, :interaction_id, :released,[:rating, {:collection=>assess_ratings}],[:mark_spot_num,{:range=>true}], [:created_at, {:ampm=>true}])
    %>
    Controller:
    @history_records= HistoryRecord.combo_search(params)
  • interface
    Common Helpers: List table
    well formatted
    with pagination
    sortable
    customizable
  • interface
    Common Helpers: List table (cont)
    <%=
    idp_table_for(@history_records,:sortable=>true,:customize => "history_records") do |item, col|
    col.add:id, link_to(item.id, admin_history_record_path(item)),:order=>:id
    col.build:duration, :waiting_time, :review_time
    col.add:scenario, item.scenario_title, :order => :scenario_title
    col.add:mark_spot_num
    end
    %>
  • interface
    Development Lifecycle
    Implement new View code/plugin in a second application
    Abstract intoplugin using existing “idp” helpers
    Put it into main view gem
  • interface
    data
  • data
    How do applications share data?
    (remember: each app has its own data)
    • “Read Only” Database Connections
    • Services
    • AJAX Loaded View Segments
  • Business example
    data
    user
    purchase
    course
    learning process
  • Purchase App
    data
    Requirement: List course packages for user to select to purchase
    but
    The course package data is stored in the “course” application
  • Solution
    data
    readonly db connection
    course
  • Code
    data
    Model:
    class CoursePackage< ActiveRecord::Base
    acts_as_readonly:course
    end
    View:
    <ul>
    <%CoursePackage.all.eachdo |package| %>
    <li><%= package.title %> <%= package.price %></li>
    <%end %>
    </ul>
  • data
    Why doesn’t this break the rule of loose coupling?
  • data
    acts_as_readonly in Depth
    def acts_as_readonly(name, options = {})
    config = CoreService.app(name).database
    establish_connectionconfig[Rails.env]
    set_table_name(self.connection.current_database + (options[:table_name]||table_name).to_s)
    end
  • data
    acts_as_readonly in Depth
    def acts_as_readonly(name, options = {})
    config = CoreService.app(name).database
    establish_connectionconfig[Rails.env]
    set_table_name(self.connection.current_database + (options[:table_name]||table_name).to_s)
    end
  • data
    Core service
    classCoreService < ActiveResource::Base
    self.site = :user
    defself.app(app_name)
    CoreService.find(app_name)
    end
    end
  • data
    Centralized configuration
    How does Core know all the configurations?
  • data
    Each app posts its configuration to core when it is started
  • data
    config/site_config.yml
    app: course
    api:
    course_list: package/courses
    config/initializers/idp_initializer.rb
    CoreService.reset_config
  • data
    core_service.rb in idp_lib
    APP_CONFIG = YAML.load(Rails.root.join(“config/site_config.yml”))
    defreset_config
    self.post(:reset_config, :app => {
    :name => APP_CONFIG["app"],
    :settings => APP_CONFIG,
    :database => YAML.load_file(
    Rails.root.join("config/database.yml"))})
    end
  • data
  • data
    Again, implemented in gem
    config/environment.rb
    config.gem ‘idp_helpers’
    config.gem ‘idp_lib’
  • data
    gems
  • data
    Web services for “write” interactions
    class CoursePackageService< ActiveSupport::Base
    self.site = :course
    end
  • example
    data
    Roadmap needs to be generated after learner pays.
  • codes
    data
    Course: app/controllers/roadmap_services_controller.rb
    def create
    Roadmap.generate(params[:user_id], params[:course_id])
    end
    Purchase: app/models/roadmap_service.rb
    class RoadmapService < ActiveSupport::Base
    self.site = :course
    end
    Purchase: app/models/order.rb
    defactivate_roadmap
    RoadmapService.create(self.user_id, self.course_id)
    end
  • data
    AJAX Loaded Composite View
    <div>
    <%=ajax_load(url_of(:course, :course_list)) %>
    </div>
    Ecosystem url_for
    Fetched from different applications
  • interface
    data
    user
  • Features of User Service
    user
    Registration/login
    Profile management
    Role Based Access Control
  • Access Control
    user
    Each Controller is one Node
    * Posted to user service when app starts
  • Access Control
    user
    before_filter:check_access_right
    defcheck_access_right
    unlessxml_request?orinner_request?
    access_deniedunlesshas_page_right?(params[:controller])
    end
    end
    * Design your apps so access control can be by controller!
  • user
    How to share?
  • Step 1: User Auth
    user
  • Step 1: User Auth
    user
    config/initializers/idp_initializer.rb
    ActionController::Base.session_store = :active_record_store
    ActiveRecord::SessionStore::Session.acts_as_remote:user,
    :readonly=> false
  • Step 2: Access Control
    user
    Tell core its controllers structure
    CoreService. reset_rights
    defself.reset_rights
    data = load_controller_structure
    self.post(:reset_rights, :data =>data)
    end
  • Step 2: Access Control
    user
    before_filter:check_access_right
    defcheck_access_right
    unlessxml_request?orinner_request?
    access_deniedunlesshas_page_right?(params[:controller])
    end
    end
  • Step 2: Access Control
    user
    has_page_right?
    Readonly dbconn again
  • Step 2: Access Control
    user
    class IdpRoleRight < ActiveRecord::Base
    acts_as_readonly:user, :table_name=> "role_rights"
    end
    def has_page_right?(page)
    roles = current_user.roles
    roles_of_page = IdpRoleRight.all(:conditions => ["path = ?", page]).map(&:role_id)
    (roles - (roles - roles_of_page)).size > 0
    end
  • user
    Again, gems!
    config/environment.rb
    config.gem ‘idp_helpers’
    config.gem ‘idp_lib’
    config.gem ‘idp_core’
  • interface
    service
    data
    user
  • service
    Support applications
    File
    Mail
    Comet service
  • File
    class Article < ActiveRecord::Base
    has_files
    end
    Specify Class that Has Files
    Upload File in Background to FileService
    Idp_file_form
    Store with app_name, model_name, model_id
    Use readonly magic to easily display
    @article.files.first.url
  • service
    Comet
    classChatRoom < ActiveRecord::Base
    acts_as_realtime
    end
    <%= realtime_for(@chat_room, current_user.login) %>
    <%= realtime_data(dom_id, :add, :top) %>
    @chat_room.realtime_channel.broadcast(“hi world", current_user.login)
  • service
    mail
    Mail services
    MailService.send(“test@idapted.com”, :welcome,
    :user => “test”)
  • Host all in one domain
    Load each rails app into a subdir, we use Unicorn  unicorn_rails --path /user  unicorn_rails --path /studycenter  unicorn_rails --path /scenario
  • Host all in one domain
    use Nginx as a reverse proxylocation /user {    proxy_pass http://rails_app_user; } location /studycenter {    proxy_pass http://rails_app_studycenter; }
  • Host all in one domain
    All you see is a uniform URL   www.eqenglish.com/user   www.eqenglish.com/studycenter   www.eqenglish.com/scenario
  • Pair-deploy
  • How to split one into many?
  • By Story
    Each App is one group of similar features.
    By Data
    Each App writes to the same data
  • Example
    User Management
    Course package
    Purchase
    Learning process

  • Iteration
  • Be adventurous at the beginning.
    Split one into as many as you think is sensitive
  • Then you may find
    Some applications interact with each other frequently.
    Lots of messy and low efficiency code to deal with interacting.
  • Merge them into one.
  • Measurement
    Critical and core task of single app should not call services of others.
    One doesn’t need to know much about others’ business to do one task (or develop).
    Independent Stories
  • Pitfalls
    Applications need to be on the same intranet.
    No “right place” for certain cases.
  • Results: Higher Productivity
    • Faster build to deploy
    • More developer autonomy
    • Safer
    • Scalable
    • Easier to “jump in”
    • Greater Happiness
  • Thank you!
    Q&A
    http://www.idapted.com
    http://developer.idapted.com
    http://t.sina.com.cn/eqenglishhst
    jpalley@idapted.com (@jpalley)
    guolei@idapted.com (@fiyuer)
    © 2010 Idapted, Ltd.