The document discusses the transition of Gilt, a flash sales business, from a monolithic Ruby application to a distributed microservices architecture using Scala. It highlights the challenges faced during this transition, including deployment difficulties and the need for better service ownership among teams. The presentation also emphasizes the benefits of microservices, such as simplified deployments, improved concurrency, and better team empowerment.

1. SCALING
GILT
From Monolith Ruby App to Distributed
Scala Micro·Services
QCon · Brooklyn · 2014

2. Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations/
scale-gilt
InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month

3. Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. - GiltDirect, Sale Personalization, Loyalty,
SED, Post-purchase, Login/Registration
- MIT eS BS/Meng I Google I IBM I IDF
- Israel I Brooklyn I eoffee I JS/Node I
Arduino I Running I Kite Surfing I Poker

5. The lessons and challenges that we
had/have with
micro·service architecture

7. WHAT IS
GILT?
Flash Sales Business
Founded in 2007
Top 50 Internet-Retailer
~150 Engineers

9. ANOTHER WAY TO LOOK
AT GILT

10. THE CLASSIC
STARTUP STORY

11. THE EARLY
DAYS
2007 - Ruby on Rails
the hottest new thing
The goal was to get to market fast

13. We were able to handle our traffic
pretty well

14. UNTIL LOUBOUTIN CAME
TO GILT

15. TECHNOLOGY PAIN POINTS
·
2009
Spike required to launch 1,000s of ruby
processes
Postgres was overloaded
Routing traffic between ruby processes
sucked
INote to selfI hide from the ruby fans

16. DEY PAIN POINTS
1000 Models/Controllers, 200K LOC,
100s of jobs
Lots of contributors + no ownership
Difficult deployments with long
integration cycles
Hard to identify root causes

17. WE NEEDED TO
SOLVE THE
PROBLEM FAST

18. THREE THINGS HAPPENED
Started the transition to the JVM
M(a/i)cro-Service Era Started
Dedicated data stores

19. WHY JVM?
Widely adopted
Stable
Better support for concurrency
Better GC vs MRI

20. FIRST 10 SERVICES

22. We solved 90% of our arch scaling
problem
But not the Dev points

23. SOLVED PAIN POINTS
Spike required to launch 1,000s of
ruby
processes
Postgres was overloaded
Routing traffic between ruby
processes
sucked

24. STILL OPEN PAIN
POINTS
New services became semi-monolithic
1000Models/Controllers, 200K LOC,
100s of jobs
Lots of contributors + no ownership
Difficult deployments with long
integration cycles

25. WHY WE DOUBLED DOWN
ON MICRO·SERVICES
Empower teams and ownership
Smaller scope
Simpler and Easier deployments and
rollbacks

26. As of last week we have around
400 services in Prod

27. We began the transition to Scala and
Play
LOSA • Lots Of Small (Web)
Apps
Same as micro-services but for web-apps

28. DEMO

36. why the increase?

37. APP BOOTSTRAP
rake bootstrap:admin-web # Bootstrap a ad
rake bootstrap:babylon-docs # Bootstrap
rake bootstrap:client-server-core # Bootstrap
rake bootstrap:jersey-java # Bootstrap
rake bootstrap:jersey-scala # Bootstrap
rake bootstrap:play # Bootstrap
rake bootstrap:play-ui-build # Bootstrap
rake bootstrap:sbt-library # Bootstrap
rake bootstrap:schema # Bootstrap

38. HOW TO DEFINE A
MICROSERVICE?
Functionality scope
Number of devs involved

40. NEW CHALLENGES
Deployments and Testing
(Functional/lntegration)
Dev/lntegration Environments
Who owns this service!?
Monitoring

41. ON DEPLOYMENTS
AND TESTING
"Testing isHARD" -
the dev that sits on your left

42. T HE C H A LLENG ES TH A T
WE FACED:
Hard to execute functional tests between
services
Frustrating to deploy semi-manually
(Capistrano)
Scary to deploy other teams services

43. SBT
Motivation: Scala adaption
Complex Scala syntax
Cool features: ~test, shell, console
Hard to debug

44. GILT·SBT·BUILD
Simple config for all the services
Pulls many plugins:
[nexus, testing,RPMs, run scripts,
Monitoring, SemVer, ...]
Custom commands (e.g 'sbt release')

45. ION·CANNON + SBT
Run tests on dedicated Env
Supports Canary releases
Easy rollbacks
Integrated health checks

47. On Dev/lntegration Environments
The hardware is not strong enough
No one wants to compile 20 services
Service Dependencies

48. EACH TEAM HAS A STAGING
ENV
SERVICE_PORTS=[
4001, #listing-service
8235, #svc-user-set
9420, #svc-free-fall
7895, #svc-Loyalty
8155, #web-loyalty
9410, #web inventory status
7898, #admin-loyalty
7899, #notification
7102, #rouge
9530, #svc-component

49. STAGING DIFFICULTIES:
Hard to keep all the services up to date
Maxed our staging env capacities
Requires to have internet connection for
some of the services (e.g LOSA-apps)

50. Dependency Fun [Demo]

54. THE FUTURE
GO Reactive

55. Docker
An extension to Linux
Containers
(LXC)
Decentralization
SimpleConfigurations
Much lighter than a VM
Immutable
Supports multiple platforms

56. ON OWNERSHIP
"code stays much longer than people" - SB

57. CODE OWNERSHIP

58. CURRENT APPROACH
Code Review!Code Review!Code
Review!
Team owns services, not individual
developers
Ownership transfer

61. DATA OWNERSHIP

62. WE TRANSITIONED TO MICRO·
DBS
Third of the services have their own
MongoDB I Postgres I Voldemort

63. MANAGE
MICRO·RELATIONAL DBS
SCHEMA
EVOLUTION
MANAGER
https://github.com/gilt/schema-evolution-
manager

64. PRINCIPLES OF
SCHEMA
EVOLUTION
MANAGER
Can manage the schema evolutions in a
Git repo
Schema changes are deployed as tar flies
No rollbacks
Schema changes are required to be
incremental

65. ON MONITORING

67. THE TOOLS WE USE
graphite / openTSDB

73. Cheat Sheet
Your organization has > 30 developers
Deployments and integrations are
difficult [You need a team for that]
You can abstractly separate features and
parts of your site
Special hardware or performance needs
for some features

74. MAIN TAKEAWAYS
Simplicity - Do you really need it?
MicroServices promise works for most case
As of 2014 - You will need to invest in Tools
We feel that it was the right choice for us

75. WHAT'S NEXT ?
BUILD YOUR NEXT
FEATURE IN A NEW SERVICE

76. QUESTION TIME
We are hiring...
@yoni_goldberg
jgoldberg@gilt.com
www.yonigoldberg.com

77. SCALA BREAK

78. Why switch to Scala from Java
Object-Functional Programming
Akka
Immutability that leads to easier
concurrency
Great libraries: like Salat, Scalaz
Less boilerplate code - e.g Case classes,
App
Scala's Collections

79. Traits
Cake Pattern
Console
SBT (in scala, release process)
Option

80. Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/scale-gilt