Downloaded 19 times
![EACH TEAM HAS A STAGING ENV
SRIEPRS[
EVC_OT=
40,#itn-evc
01 lsigsrie
83,#v-srst
25 scue-e
92,#v-refl
40 scfe-al
79,#v-oat
85 scLyly
85,#e-oat
15 wblyly
91,#e ivnoysau
40 wb netr tts
79,#di-oat
88 amnlyly
79,#oiiain
89 ntfcto
70,#og
12 rue
93,#v-opnn
50 sccmoet
60,#v-atitsbi
82 scwils-umt
46,#v-cinsl
06 scato-ae
..
..
PR_OWR_RSSRIEPRSmp{|ot
OTFRADAG=EVC_OT.a
pr|
[-' "{ot:oahs:{ot"
'L, #pr}lclot#pr}]
}
ee([wsh- - - -} PR_OWR_RS G_OT.l
xc*%{s a C N n, OTFRADAG, WHS]f](https://image.slidesharecdn.com/gilt-frommonolithrubyapptomicroservicescalaservicearchitecture-140115150436-phpapp01-140307125527-phpapp01/75/Gilt-from-monolith-ruby-app-to-microservice-scala-service-architecture-30-2048.jpg)
![GILT-SBT-BUILD
Simple config for all the services
Pulls many plugins:
[nexus, testing, RPMs, run scripts, Monitoring,
SemVer, ...]
Custom commands (e.g 'sbt release')](https://image.slidesharecdn.com/gilt-frommonolithrubyapptomicroservicescalaservicearchitecture-140115150436-phpapp01-140307125527-phpapp01/75/Gilt-from-monolith-ruby-app-to-microservice-scala-service-architecture-55-2048.jpg)
Uploaded byJonathan (Yoni) Goldberg
Gilt from monolith ruby app to microservice scala service architecture
This document summarizes Yoni Goldberg's talk on Gilt's transition from a monolithic Ruby on Rails application to a distributed system of over 450 Scala microservices. Some of the key challenges they faced included scaling issues as traffic grew, difficulties deploying and testing changes across many interdependent services, and a lack of clear ownership over parts of the codebase. Gilt addressed these by moving to the JVM, embracing microservice architecture, using Docker for development environments, implementing automated testing and deployment tools, and emphasizing clear ownership of individual services.
More Related Content
Similar to Gilt from monolith ruby app to microservice scala service architecture
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Gilt from monolith ruby app to microservice scala service architecture
- 1. SCALING GILT From MonolithRuby App to Distributed Scala Micro-Services #NYCTECHTALKS Yoni (Jonathan) Goldberg Lead Engineer - GILT Podium => http://bit.ly/podiumapp
- 2. ABOUT ME - Leadingthe Popeye Team - Sale Personalization, Loyalty, SEO Post-purchase, Login/Registration flows - MIT CS BS/Meng | Google | IBM | IDF - Brooklyn | Coffee | Arduino | Running | Kite Surfing | Online Collaboration | Poker Excited to be part of the NYC Tech community
- 3. THE LESSONS ANDCHALLENGES THAT WE HAD/HAVE WITH MICRO-SERVICE ARCHITECTURE
- 5. WHAT IS GILT? FlashSales Business Founded in 2007 Top 50 Internet-Retailer ~150 Engineers
- 7. ANOTHER WAY TOLOOK AT GILT Three day traffic pattern
- 8.
- 9. THE EARLY DAYS 2007- Ruby on Rails the hottest new thing The goal was to get to market fast
- 11. WE WERE ABLETO HANDLE OUR TRAFFIC PRETTY WELL
- 12. UNTIL LOUBOUTIN CAMETO GILT
- 13. TECHNOLOGY PAIN POINTS- 2009 Spike required to launch 1,000s of ruby processes Postgres was overloaded Routing traffic between ruby processes sucked |Note to self| - hide from the ruby fan boys
- 14. DEV PAIN POINTS 1000Models/Controllers, 200K LOC, 100s of jobs Lots of contributors + no ownership Difficult deployments with long integration cycles Hard to identify root causes
- 15. WE NEEDED TOSOLVE THE PROBLEM FAST
- 16. THREE THINGS HAPPENED Startedthe transition to the JVM M(a/i)cro-Service Era Started Dedicated data stores
- 17. WHY JVM? Widely adopted Stable Bettersupport for concurrency Better GC vs MRI
- 18.
- 20. We solved 90%of our arch scaling problem But not the Dev points
- 21. PAIN POINTS Spike requiredto launch 1,000s of ruby processes Postgres was overloaded Routing traffic between ruby processes sucked New services became semi-monolithic 1000 Models/Controllers, 200K LOC, 100s of jobs Lots of contributors + no ownership Difficult deployments with long integration cycles
- 22. WHY WE DOUBLEDDOWN ON MICRO-SERVICES Empower teams and ownership Smaller scope Simpler and Easier deployments and rollbacks
- 23. MICRO SERVICE ARCHITECTURE STARTEDTO GET TRACTION
- 24. AS OF LASTWEEK WE HAVE MORE THAN 450 SERVICES
- 25. APP BOOTSTRAP rk bosrpamnwb aeotta:di-e #Bosrpaamnwbsrie otta di-e evc rk bosrpbblndc ae otta:ayo-os #Bosrpabblndc srie otta ayo-os evc rk bosrpcin-evrcr #Bosrpacin-evrcr sr ae otta:letsre-oe otta letsre-oe ev rk bosrpjre-aa ae otta:esyjv #Bosrpajre-aasrie otta esyjv evc rk bosrpjre-cl ae otta:esysaa #Bosrpajre-cl srie otta esysaa evc rk bosrppa ae otta:ly #Bosrpapa srie otta ly evc rk bosrppa-ibid ae otta:lyu-ul #Bosrpapa-ibidsrie otta lyu-ul evc rk bosrpstlbay ae otta:b-irr #Bosrpastlbaysrie otta b-irr evc rk bosrpshm ae otta:cea #Bosrpashm srie otta cea evc
- 26. WE BEGAN THETRANSITION TO SCALA AND PLAY LOSA - Lots Of Small Apps Same motivation and benefits of Micro-Service Architecture
- 28. NEW CHALLENGES Dev/Integration Environments Whoowns this service!? Monitoring Deployments and Testing (Functional/Integration)
- 29. ON DEV/INTEGRATION ENVIRONMENTS Thehardware is not strong enough No one wants to compile 20 services
- 30. EACH TEAM HASA STAGING ENV SRIEPRS[ EVC_OT= 40,#itn-evc 01 lsigsrie 83,#v-srst 25 scue-e 92,#v-refl 40 scfe-al 79,#v-oat 85 scLyly 85,#e-oat 15 wblyly 91,#e ivnoysau 40 wb netr tts 79,#di-oat 88 amnlyly 79,#oiiain 89 ntfcto 70,#og 12 rue 93,#v-opnn 50 sccmoet 60,#v-atitsbi 82 scwils-umt 46,#v-cinsl 06 scato-ae .. .. PR_OWR_RSSRIEPRSmp{|ot OTFRADAG=EVC_OT.a pr| [-' "{ot:oahs:{ot" 'L, #pr}lclot#pr}] } ee([wsh- - - -} PR_OWR_RS G_OT.l xc*%{s a C N n, OTFRADAG, WHS]f
- 31. STAGING DIFFICULTIES: Hard tokeep all the services up to date Maxed our staging env capacities Requires to have internet connection for some of the services (e.g LOSA-apps)
- 32.
- 33. DOCKER An extension toLinux Containers (LXC) Decentralization Simple Configurations Much lighter than a VM Immutable Supports services and platforms
- 34. ON OWNERSHIP "code staysmuch longer than people" - SB
- 35.
- 36. CURRENT APPROACH Code Review!CodeReview!Code Review! Team owns services, not individual developers Ownership transfer
- 39.
- 40. WE TRANSITIONED TOMICRO-DBS Third of the services have their own MongoDB Postgres Voldemort
- 41. MANAGE MICRO-RELATIONAL DBS SCHEMAEVOLUTION MANAGER https://github.com/gilt/schema-evolution-manager
- 42. PRINCIPLES OF SCHEMA EVOLUTIONMANAGER Can manage the schema evolutions in a Git repo Schema changes are deployed as tar flies No rollbacks Schema changes are required to be incremental
- 43. eh "raetberlae (ditgr">nwsl cocet al eess i nee) e.q smad.nwsl#rae agtcmi e-d /e.q Cetd i omt smds #gnrtstetregshm-o-ann...a. e-it eeae h a . ceaincno002tr #Spadutro yu sre c n na n or evr c shm-o-ann002 d ceaincno-.. smapyhs -lclot-nm incno -ue incn e-pl ot -oahs -ae o-ann -sr o-an
- 44.
- 46. THE TOOLS WEUSE graphite / openTSDB
- 52. ON DEPLOYMENTS AND TESTING (FUNCTIONAL/INTEGRATION) "Testingis HARD" - the dev that sits on your left
- 53. THE CHALLENGES THATWE FACED: Hard to execute functional tests between services Frustrating to deploy semi-manually (Capistrano) Scary to deploy other teams services
- 54. SBT Motivation: Scala adaption ComplexScala syntax Cool features: ~test, shell, console Hard to debug
- 55. GILT-SBT-BUILD Simple config forall the services Pulls many plugins: [nexus, testing, RPMs, run scripts, Monitoring, SemVer, ...] Custom commands (e.g 'sbt release')
- 56. ojc BidetnsCinSreCrPoetwt Dpnece{ bet ul xed letevroerjc ih eednis vlnm ='v-aeatvto' a ae scsl-ciain vlcrDp =.. a oees .. vlsreDp =.. a evres . vlcinDp =.. a letes . oerd vlinannrc =InannFsTak vrie a ocnoTak oCno.atrc }
- 57. ION-CANNON + SBT Runfunctional/Selenium tests on dedicated Env Supports Canary releases Easy rollbacks Integrated health checks
- 59. MAIN TAKEAWAYS Simplicity -Do you really need it? We feel that it was the right choice for us MicroServices promise works for most cases As of 2014 - You will need to invest in Tools!
- 60. PODIUM TIME We arehiring... Keep in touch: jgoldberg@gilt.com www.yonigoldberg.com