Envato Dev Ops - Alt.Net Melbourne

Dev Ops, Ruby, Rails,
and Envato
A team of 8 deploy heaps of times a day to a Ruby on
Rails app that has scaled up to around 20 million
requests a week without an ops team.

lolwut?

Thursday, 30 June 2011

John Barton

@johnbarton

john@envato.com


Envato?

http://envato.com


Stock Marketplaces
&
Tutorial Network


Marketplaces


Like iStockPhoto but for
other creative niches
or
eBay for digital goods


Tuts+ Network


Big Blog Network for
education in creative ﬁelds


The Marketplace


August 2006


One Marketplace


FlashDen


Rails 0.13b
No Users
No Trafﬁc


June 2011


9 Marketplaces


ActiveDen (nee FlashDen)
AudioJungle
ThemeForest
VideoHive
GraphicRiver
CodeCanyon
3D Ocean
Tuts+ Marketplace
PhotoDune


Rails 2.3.11
830,462 Users
22 Million Requests Weekly


One Codebase and One
Production Environment


What’s interesting about
our deployments?


deploy frequently, high
scale, no ops team


What do you need to do
the same?


Some technical
infrastructure, some
business/cultural


Cultural Bits


The “Dev Ops” Culture


“Culture of respect &
trust, good attitude toward
failure...”
Ted Dzubia


“How about "culture of
stop fucking up"?”

Ted Dzubia
http://teddziuba.com/2011/03/devops-scam.html


A problem of competing
priorities


Our Solution:
Don’t have Ops


Outsource commodity
platform bits:
virtualisation/cloud, have
rackspace take care of db/
mailserver


Ensure the dev team has
the skills to take care of the
rest


Take advantage of an
individual's talents, but
don’t rely upon them

ie. don’t accidentally create an ops guy


OK to have specialists... for
special work.

Keeping the app running
isn’t “special”


Within the Business


Preserve The “Golden Age”


3 Developers
30 minute feedback cycle
deploy, discuss on forums,
deploy again


Keep doing everything good
about the the startup days
for as long as we can


Avoid Corporate
Antipatterns


We’ve all been doing dev
long enough to see this
stuff screwed up over and
over again


Decision 3:
Don’t do all that stuff


Ultimate Responsibility


"The fault, dear
Brutus, is not in our
QA or Ops,
But in ourselves."

Test Driven Development
vs.
QA Team


Test Driven Infrastructure
vs.
Ops Team


Both as a team and as
individuals we own our
work from when we are
asked to do it...


... until is is demonstrably
error free and performant
in production


Everyone is in the (paid) on
call roster


Everyone takes a turn at
Level 2 Customer Support


Want those jobs to be
easier?

Stop fucking up.


Process


LEAN / TPS Principles

...without the process


You cannot write code any
faster than you can deploy
it to production


Long running projects?


A. B. C.

Always Be Cmerging

(the c is silent)


I would not want to try this
with Subversion, or Visual
Source Safe... and I never
had enough money to try
“Team Foundation Server”


Use GIT

http://git-scm.com/


Dark Launch
Feature Flags
Private Beta


User Community


We do trip up running this
fast


But through years of
openness with our users
via our forums and owning
up to our mistakes


... we’ve ended up with a
(relatively) sympathetic
community


Time Zones both help and
hurt


Trafﬁc peaks during US day
means that if things go
wrong we’re usually asleep


But it makes it very easy to
deploy during our business
hours


Open Source


No point you solving the
same problem as the guy
down the street


No point the guy down the
street solving the same
problem you did last week


Technology


Commodity Hosting is a
prerequisite


Notice I’m not saying
“Cloud Computing”


You can't
trust “the
cloud” but
you can
trust “the
cloud to
be “the
cloud”.

Make a conservative
platform choice so we
don’t have to sweat the
details
ie. Ubuntu 10.4 LTS


I guess you guys have to
choose like Windows
Ultimate 2009 Edition?


Seriously though, Ruby guys
love shiny new things, and
we’re spoiled for choice
_even_ when it harms us.


So we make an effort to
keep the stack as Vanilla as
possible


Virtualised servers in our
own sandbox.

Cloud Flexibility - Cloud
Shit-ness = WIN


Automate Conﬁguration
Management


Babushka

http://babushka.me/


Test drive your
architecture...


...or borrow “deps” built by
other people


Lets look at some code


Babushka is designed to be
run manually on the
remote machine


Use Dollhouse
https://github.com/geelen/
dollhouse

To drive babushka remotely


moar code


A lot of people I take
seriously, take Chef
seriously.

http://wiki.opscode.com/


Automate Deployment


Capistrano

https://github.com/
capistrano/capistrano


Codes?


The un-sung hero


rake db:migrate


I’m so used to having it I
can’t even remember how I
ever deployed database
changes in the past


Performance & Scaling


Not as big a deal as
everyone thinks


.NET runtime is faster... but
it’s really not that big a deal


Shared-nothing load
balanced app servers + out
of request queue workers

not rocket surgery


Measure, deploy, measure
again and then tweak or
rollback

New Relic FTW


There is no code faster
than no code.


Why I’m happier than I was
as a .NET programmer


Mainly because of
System.Threading.Timer


No Viewstate

(Yes I heard you’ve got
MVC now instead)


A culture of people who ﬁx
things instead of waiting for
them to be ﬁxed


People who care about
expressiveness of code and
ﬁnding the right
abstractions


Question?

Envato Dev Ops - Alt.Net Melbourne

Recommended

Recommended

More Related Content

What's hot

What's hot (11)

Viewers also liked

Viewers also liked (17)

Similar to Envato Dev Ops - Alt.Net Melbourne

Similar to Envato Dev Ops - Alt.Net Melbourne (20)

More from John Barton

More from John Barton (10)

Recently uploaded

Recently uploaded (20)

Envato Dev Ops - Alt.Net Melbourne