• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Reliably Deploying Web Apps with Database Migrations
 

Reliably Deploying Web Apps with Database Migrations

on

  • 107 views

 

Statistics

Views

Total Views
107
Views on SlideShare
107
Embed Views
0

Actions

Likes
1
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Reliably Deploying Web Apps with Database Migrations Reliably Deploying Web Apps with Database Migrations Presentation Transcript

    • Reliably Deploying Web Apps with Database Migrations Pratik Vyas
    • To deploy new code with a database migration that takes 10 seconds to run per site The Problem
    • in batches of five sites at a time Then the 600th site would be running new code with unpatched database for 20 minutes! If we have to migrate 600 sites run migration for 600th site code deployed danger zone
    • Deployments need to be handled more carefully
    • Create a new environment with the new code Migrate sites one at a time by moving them to the new environment An Intuitive Solution
    • build x build x+1 site c site d site e site b site a site f site g site h site i site j ● move to build x+1 ● run migrations maintenance area (No HTTP requests) site c The Two Environments
    • The Solution - Part 1
    • Building an Environment Use virtualenv and pip? pip install would also need to compile C extensions eg. lxml takes a lot of cpu time and memory Makes creating new environments EXPENSIVE
    • Wheels are the new standard of python distribution (PEP 427) Unzip the wheel in site-packages and the package is installed … like magic! Build wheels on a different machine and watch a new environment get deployed in a few seconds Python Wheels
    • Cost of creating virtualenv is mostly unzipping No compiler required on production server PyPI availability is not a factor during deployments Benefits of Build Step
    • buildfile.tar.gz ├── scripts │ ├── virtualenv.py │ └── virtualenv_support │ ├── pip-1.5.2-py2.py3-none-any.whl │ └── setuptools-2.1-py2.py3-none-any.whl ├── sites │ └── statics │ ├── css │ └── *.css │ ├── img │ └── *.png │ └── js │ └── *.js └── wheels └── *.whl
    • The Solution - Part 2
    • Running Processes The new code will require us to start a new wsgi process + workers use Supervisor
    • Supervisor Use Jinja and generate process list per virtualenv Protip: For adding/removing a group of processes, use reread and update instead of reload. It prevents non-related processes from restarting.
    • The Solution - Part 3
    • Serving sites, both old and new Serve unmigrated sites using old code, migrated sites using new code. nginx to the rescue
    • Configuring nginx awesomeness Use Jinja to generate configuration and reload nginx Fallback to old environment based on error condition For performance, 404s should be cheap
    • upstream upstream-1 { server 127.0.0.1:8000 fail_timeout=0; } upstream upstream-2 { server 127.0.0.1:8001 fail_timeout=0; } server { . . . . location / { error_page 404 = @upstream2handler; proxy_redirect off; proxy_intercept_errors on; proxy_pass http://upstream-1; } location @upstream2handler { proxy_redirect off; proxy_intercept_errors on; proxy_pass http://upstream-2; } }
    • Background Workers? Stop executing scheduled jobs when in handover state Preferably, use a fresh broker for every build Track pending jobs per site (we prepend sitename to queue name for this)
    • The Final Picture build x build x+1 site c site d site e site b site a site f site g site h site i site j ● Delay enqueuing scheduled tasks. ● Continue if no pending jobs, else retry after 60 seconds. ● Run migrations. ● Move to build x+1. site c maintenance area (No HTTP requests)
    • ध यवाद @pdvyas, @frappe_io