Scaling :
What Went Wrong,
What Went Right
Ross Snyder
ross@etsy.com
@beamrider9 Sept. 30, 2011
1
Etsy is the world’s handmade marketplace.
(vintage and supplies, too)
2
Etsy was founded in mid-2005
and is constantly growing.
Gross Merchandise Sales ($MM)
3
From humble beginnings...
June Four employees, one web*,
2005: one db, founder’s apartment
* until getting slashdotted by a link
from Boing Boing in Aug. 2005
4
... to today’s handmade juggernaut.
Sept. 250+ employees, multiple
2011: offices, billions of pageviews
(NYC Mayor Mike Bloomberg visited
Etsy in June 2011)
5
Etsy Architecture: 2007
“Let’s write some middleware!”
(runners up: “Let’s rewrite the site in Java!”
and “Let’s rewrite the site in Python!”)
20
Conway’s Law:
“Any organization that designs
a system (defined broadly) will
produce a design whose
structure is a copy of the
organization's communication
structure.”
- Melvin Conway, 1968
21
Sprouter: Not-As-Good Parts
Web Sprouter DB
(PHP) (Python) (Postgres)
Homegrown daemon + dependencies
for Ops to maintain
45
Sprouter: Not-As-Good Parts
Web Sprouter DB
(PHP) (Python) (Postgres)
Lack of community support / provability
46
Sprouter: Not-As-Good Parts
Web Sprouter DB
(PHP) (Python) (Postgres)
Complex synchronization required to deploy
(due to tight coupling with Postgres)
47
Sprouter: Not-As-Good Parts
Web Sprouter DB
(PHP) (Python) (Postgres)
Database remains single point of failure
(sharding features never fully formed)
48
Sprouter: Summary
Extra barriers to development
+ Negligible (negative?) effect on site reliability
+ Deploys even more painful
+ Requires extra Ops/Dev resources
52
Sprouter: Summary
Extra barriers to development
+ Negligible (negative?) effect on site reliability
+ Deploys even more painful
+ Requires extra Ops/Dev resources
=
53
The Way Forward: Part 4
Database Sharding
Battle-tested, well-known
89
The Way Forward: Part 4
Database Sharding
Scales horizontally to infinity
(or close enough)
90
The Way Forward: Part 4
Database Sharding
No single points of failure
(master-master replication)
91
The Way Forward: Part 4
Database Sharding
Gradually phase out Sprouter,
phase in ORM / sharded data
92
Sprouter: Timeline
Fall ’07: Idea first discussed
Spring ’08: Alpha version debuts
Fall ’08: Released in production
Spring ’09: Sprouter deprecated
93
Sprouter: Timeline
Fall ’07: Idea first discussed
Spring ’08: Alpha version debuts
Fall ’08: Released in production
Spring ’09: Sprouter deprecated
Spring ’11: Sprouter turned off
94