Download as PDF, PPTX
![EACH TEAM HAS A STAGING ENV
SRIEPRS[
EVC_OT=
40,#itn-evc
01 lsigsrie
83,#v-srst
25 scue-e
92,#v-refl
40 scfe-al
79,#v-oat
85 scLyly
85,#e-oat
15 wblyly
91,#e ivnoysau
40 wb netr tts
79,#di-oat
88 amnlyly
79,#oiiain
89 ntfcto
70,#og
12 rue
93,#v-opnn
50 sccmoet
60,#v-atitsbi
82 scwils-umt
46,#v-cinsl
06 scato-ae
..
..
PR_OWR_RSSRIEPRSmp{|ot
OTFRADAG=EVC_OT.a
pr|
[-' "{ot:oahs:{ot"
'L, #pr}lclot#pr}]
}
ee([wsh- - - -} PR_OWR_RS G_OT.l
xc*%{s a C N n, OTFRADAG, WHS]f](https://image.slidesharecdn.com/gilt-frommonolithrubyapptomicroservicescalaservicearchitecture-140115150436-phpapp01/75/Scaling-Gilt-from-monolith-ruby-app-to-micro-service-scala-service-architecture-30-2048.jpg)
![GILT-SBT-BUILD
Simple config for all the services
Pulls many plugins:
[nexus, testing, RPMs, run scripts, Monitoring,
SemVer, ...]
Custom commands (e.g 'sbt release')](https://image.slidesharecdn.com/gilt-frommonolithrubyapptomicroservicescalaservicearchitecture-140115150436-phpapp01/75/Scaling-Gilt-from-monolith-ruby-app-to-micro-service-scala-service-architecture-55-2048.jpg)
Uploaded byGilt Tech Talks
Scaling Gilt: from monolith ruby app to micro service scala service architecture
This document summarizes Yoni Goldberg's talk on Gilt's transition from a monolithic Ruby on Rails application to a distributed system of over 450 Scala microservices. Some of the key challenges they faced included scaling issues as traffic grew, difficulties deploying and testing changes across many interdependent services, and a lack of clear ownership over parts of the codebase. Gilt addressed these by moving to the JVM, embracing microservice architecture, implementing tools like Docker and a custom SBT build to simplify deployments, and establishing clear ownership for each microservice.
More Related Content
Similar to Scaling Gilt: from monolith ruby app to micro service scala service architecture
Scaling Gilt: from monolith ruby app to micro service scala service architecture
- 1. SCALING GILT From MonolithRuby App to Distributed Scala Micro-Services #NYCTECHTALKS Yoni (Jonathan) Goldberg Lead Engineer - GILT Podium => http://bit.ly/podiumapp
- 2. ABOUT ME - Leadingthe Popeye Team - Sale Personalization, Loyalty, SEO Post-purchase, Login/Registration flows - MIT CS BS/Meng | Google | IBM | IDF - Brooklyn | Coffee | Arduino | Running | Kite Surfing | Online Collaboration | Poker Excited to be part of the NYC Tech community
- 3. THE LESSONS ANDCHALLENGES THAT WE HAD/HAVE WITH MICRO-SERVICE ARCHITECTURE
- 5. WHAT IS GILT? FlashSales Business Founded in 2007 Top 50 Internet-Retailer ~150 Engineers
- 7. ANOTHER WAY TOLOOK AT GILT Three day traffic pattern
- 8.
- 9. THE EARLY DAYS 2007- Ruby on Rails the hottest new thing The goal was to get to market fast
- 11. WE WERE ABLETO HANDLE OUR TRAFFIC PRETTY WELL
- 12. UNTIL LOUBOUTIN CAMETO GILT
- 13. TECHNOLOGY PAIN POINTS- 2009 Spike required to launch 1,000s of ruby processes Postgres was overloaded Routing traffic between ruby processes sucked |Note to self| - hide from the ruby fan boys
- 14. DEV PAIN POINTS 1000Models/Controllers, 200K LOC, 100s of jobs Lots of contributors + no ownership Difficult deployments with long integration cycles Hard to identify root causes
- 15. WE NEEDED TOSOLVE THE PROBLEM FAST
- 16. THREE THINGS HAPPENED Startedthe transition to the JVM M(a/i)cro-Service Era Started Dedicated data stores
- 17. WHY JVM? Widely adopted Stable Bettersupport for concurrency Better GC vs MRI
- 18.
- 20. We solved 90%of our arch scaling problem But not the Dev points
- 21. PAIN POINTS Spike requiredto launch 1,000s of ruby processes Postgres was overloaded Routing traffic between ruby processes sucked New services became semi-monolithic 1000 Models/Controllers, 200K LOC, 100s of jobs Lots of contributors + no ownership Difficult deployments with long integration cycles
- 22. WHY WE DOUBLEDDOWN ON MICRO-SERVICES Empower teams and ownership Smaller scope Simpler and Easier deployments and rollbacks
- 23. MICRO SERVICE ARCHITECTURE STARTEDTO GET TRACTION
- 24. AS OF LASTWEEK WE HAVE MORE THAN 450 SERVICES
- 25. APP BOOTSTRAP rk bosrpamnwb aeotta:di-e #Bosrpaamnwbsrie otta di-e evc rk bosrpbblndc ae otta:ayo-os #Bosrpabblndc srie otta ayo-os evc rk bosrpcin-evrcr #Bosrpacin-evrcr sr ae otta:letsre-oe otta letsre-oe ev rk bosrpjre-aa ae otta:esyjv #Bosrpajre-aasrie otta esyjv evc rk bosrpjre-cl ae otta:esysaa #Bosrpajre-cl srie otta esysaa evc rk bosrppa ae otta:ly #Bosrpapa srie otta ly evc rk bosrppa-ibid ae otta:lyu-ul #Bosrpapa-ibidsrie otta lyu-ul evc rk bosrpstlbay ae otta:b-irr #Bosrpastlbaysrie otta b-irr evc rk bosrpshm ae otta:cea #Bosrpashm srie otta cea evc
- 26. WE BEGAN THETRANSITION TO SCALA AND PLAY LOSA - Lots Of Small Apps Same motivation and benefits of Micro-Service Architecture
- 28. NEW CHALLENGES Dev/Integration Environments Whoowns this service!? Monitoring Deployments and Testing (Functional/Integration)
- 29. ON DEV/INTEGRATION ENVIRONMENTS Thehardware is not strong enough No one wants to compile 20 services
- 30. EACH TEAM HASA STAGING ENV SRIEPRS[ EVC_OT= 40,#itn-evc 01 lsigsrie 83,#v-srst 25 scue-e 92,#v-refl 40 scfe-al 79,#v-oat 85 scLyly 85,#e-oat 15 wblyly 91,#e ivnoysau 40 wb netr tts 79,#di-oat 88 amnlyly 79,#oiiain 89 ntfcto 70,#og 12 rue 93,#v-opnn 50 sccmoet 60,#v-atitsbi 82 scwils-umt 46,#v-cinsl 06 scato-ae .. .. PR_OWR_RSSRIEPRSmp{|ot OTFRADAG=EVC_OT.a pr| [-' "{ot:oahs:{ot" 'L, #pr}lclot#pr}] } ee([wsh- - - -} PR_OWR_RS G_OT.l xc*%{s a C N n, OTFRADAG, WHS]f
- 31. STAGING DIFFICULTIES: Hard tokeep all the services up to date Maxed our staging env capacities Requires to have internet connection for some of the services (e.g LOSA-apps)
- 32.
- 33. DOCKER An extension toLinux Containers (LXC) Decentralization Simple Configurations Much lighter than a VM Immutable Supports services and platforms
- 34. ON OWNERSHIP "code staysmuch longer than people" - SB
- 35.
- 36. CURRENT APPROACH Code Review!CodeReview!Code Review! Team owns services, not individual developers Ownership transfer
- 39.
- 40. WE TRANSITIONED TOMICRO-DBS Third of the services have their own MongoDB Postgres Voldemort
- 41. MANAGE MICRO-RELATIONAL DBS SCHEMAEVOLUTION MANAGER https://github.com/gilt/schema-evolution-manager
- 42. PRINCIPLES OF SCHEMA EVOLUTIONMANAGER Can manage the schema evolutions in a Git repo Schema changes are deployed as tar flies No rollbacks Schema changes are required to be incremental
- 43. eh "raetberlae (ditgr">nwsl cocet al eess i nee) e.q smad.nwsl#rae agtcmi e-d /e.q Cetd i omt smds #gnrtstetregshm-o-ann...a. e-it eeae h a . ceaincno002tr #Spadutro yu sre c n na n or evr c shm-o-ann002 d ceaincno-.. smapyhs -lclot-nm incno -ue incn e-pl ot -oahs -ae o-ann -sr o-an
- 44.
- 46. THE TOOLS WEUSE graphite / openTSDB
- 52. ON DEPLOYMENTS AND TESTING (FUNCTIONAL/INTEGRATION) "Testingis HARD" - the dev that sits on your left
- 53. THE CHALLENGES THATWE FACED: Hard to execute functional tests between services Frustrating to deploy semi-manually (Capistrano) Scary to deploy other teams services
- 54. SBT Motivation: Scala adaption ComplexScala syntax Cool features: ~test, shell, console Hard to debug
- 55. GILT-SBT-BUILD Simple config forall the services Pulls many plugins: [nexus, testing, RPMs, run scripts, Monitoring, SemVer, ...] Custom commands (e.g 'sbt release')
- 56. ojc BidetnsCinSreCrPoetwt Dpnece{ bet ul xed letevroerjc ih eednis vlnm ='v-aeatvto' a ae scsl-ciain vlcrDp =.. a oees .. vlsreDp =.. a evres . vlcinDp =.. a letes . oerd vlinannrc =InannFsTak vrie a ocnoTak oCno.atrc }
- 57. ION-CANNON + SBT Runfunctional/Selenium tests on dedicated Env Supports Canary releases Easy rollbacks Integrated health checks
- 59. MAIN TAKEAWAYS Simplicity -Do you really need it? We feel that it was the right choice for us MicroServices promise works for most cases As of 2014 - You will need to invest in Tools!
- 60. PODIUM TIME We arehiring... Keep in touch: jgoldberg@gilt.com www.yonigoldberg.com