Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mircoservices, dev ops and Engineering best practices at

How migrated from a monolith to microservices architecture, what was the effect on the company structure and how Wix built a top of the line engineering group

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

Mircoservices, dev ops and Engineering best practices at

  1. 1. @aviranm Aviran Mordo Head of Microservices, DevOps & Engineering at @aviranm
  2. 2. @aviranm
  3. 3. @aviranm Wix In Numbers Over 87M users Static storage is >2Pb of data 3 data centers + 2 clouds (Google, Amazon) 2B HTTP requests/day 1200 people work at Wix
  4. 4. @aviranm Over 200 Microservices on Production
  5. 5. @aviranm Microservices - What Does it Take
  6. 6. @aviranm How to Get There? (Wix’s journey)
  7. 7. @aviranm About 6 years ago
  8. 8. @aviranm The Monolithic Giant One monolithic server that handled everything Dependency between features Changes in unrelated areas caused deployment of the whole system Failure in unrelated areas will cause system wide downtime Lighttpd (file serving) MySQL DB Wix (Tomcat)
  9. 9. @aviranm Breaking the System Apart
  10. 10. @aviranm
  11. 11. @aviranm Concerns and SLA Many feature request Lower performance requirement Lower availability requirement Write intensive Edit websites Not many product changes High performance High availability Read intensive View sites, created by Wix editor
  12. 12. @aviranm Mono-Wix Phase 1
  13. 13. @aviranm Extract Public Service Editor service (Mono-Wix) Public service
  14. 14. @aviranm Divide and Conquer Editor service Public service Guideline: No runtime, deployment or data dependency
  15. 15. @aviranm Separation by Product Lifecycle Decouple architecture => Decouple teams Deployment independence Areas with frequent changes Editor service Public service
  16. 16. @aviranm Separation by Service Level Scale independently Use different data store Optimize data per use case (Read vs Write) Run on different datacenters / clouds / zones System resiliency (degradation of service vs. downtime) Faster recovery time Editor service Public service
  17. 17. @aviranm
  18. 18. @aviranm Service Boundary
  19. 19. @aviranm Separation of Databases Copy data between segments Optimize data per use case (read vs. write intensive) Different data stores Public serviceEditor service Copy necessary data
  20. 20. @aviranm Serialization
  21. 21. @aviranm Serialization / Protocol Binary? JSON / XML / Text? HTTP? Public serviceEditor service
  22. 22. @aviranm Serialization / Protocol - Tradeoffs Readability? Performance? Debug? Tools? Monitoring? Dependency? Public serviceEditor service
  23. 23. @aviranm API Transport/Protocol
  24. 24. @aviranm How to Expose an API REST? RPC? SOAP? Public serviceEditor service
  25. 25. @aviranm Wix’s Choices REST HTTP Public serviceEditor service Binary JSON-RPC HTTP
  26. 26. @aviranm API Versioning
  27. 27. @aviranm API Versioning Public serviceEditor service Backward compatibility Maybe here API Schema /v1/v2
  28. 28. @aviranm Service Discovery
  29. 29. @aviranm Service Discovery Public serviceEditor service Configuration (DNS+LB) Zookeeper? Consul? Etcd? Eureka?
  30. 30. @aviranm Resilience
  31. 31. @aviranm What does the Arrow Mean? Public serviceEditor service
  32. 32. @aviranm Failure Points = Network I/O Public serviceEditor service Retry policy Circuit breaker Throttlers Be careful – you may cause downtime Retry only on idempotent operations
  33. 33. @aviranm Degradation of Service Public serviceEditor service Feature killer (Killer feature) Fallbacks Self healing
  34. 34. @aviranm Testing
  35. 35. @aviranm Test a Distributed System (at Wix) Public serviceEditor service Unit Test Integration Test Server E2E Automation Client
  36. 36. @aviranm Distributed Logging
  37. 37. @aviranm Build visibility into service
  38. 38. @aviranm What is the Right Size of a Microservice?
  39. 39. @aviranm The Size of a Microservice is the Size of the Team That is Building it. “Organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations” Conway, Melvin
  40. 40. @aviranm How many Engineers can Work on a Single Project ?
  41. 41. @aviranm Microservices = Small Teams Small Teams = Small Rooms @aviranm
  42. 42. @aviranm Ownership
  43. 43. @aviranm Team Work  Microservice is owned by a team  You build it – you run it  No microservice is left without a clear owner  Microservice is NOT a library – it is a live production system
  44. 44. @aviranm Culture that ROQS R O Q S esponsibility wnership uality haring @aviranm
  45. 45. @aviranm What Is The Common Denominator?  Product manager  Project manager  QA  Operations  DBA Develpers can do these jobs @aviranm
  46. 46. @aviranm Develop er Product QA Manage ment Operatio n BI Dev Centric Culture – Involve The Developer  Product definition (with product)  Development  Testing (with QA engineers)  Deployment / Rollback (with operations)  Monitoring / BI (with BI team)  DevOps – to enable deployment and rollback, fully automated Support Circle @aviranm
  47. 47. What did you Learn from Just 2 Services ● Service boundary ● Monitoring infrastructure ● Serialization format ● Synchronous communication protocol (HTTP/Binary) ● Asynchronous (queuing infra) ● Service SLA ● API definition (REST/ RPC / Versioning) ● Data separation ● Deployment strategy ● Testing infrastructure (integration test, e2e test) ● Compatibility (backwards / forward)
  48. 48. @aviranm Continue to Extract More Microservices
  49. 49. @aviranm HTML Editor Flash Editor MSM Private Media Public Media Editor Segment Public Segment Premium Services List DB App Builder App Store App Market Dashboard Mailer TimeZone Public HTML API Public API (Flash) MSP Public Server HTML Renderer HTML SEO Renderer Flash Renderer Flash SEO Renderer Sitemap Renderer Robots.txt Renderer User Server Template Viewer ContactsHUBActivity Site Members Store Mgr Comments Snapshoter User Pref Feed Me Shout-out Hotels PETRI Site Pref Dist LoggerSlicer eCom Renderer eCom Cart eCom Checkout eCom Catalog eCom Orders Payment Facade Account Info HTML API HTML Embeder BlogMobile Mostly writes 2 Data centers Db active-standby (preferably active-active) Performance < 2s 99% Serves mostly site builders Uptime > 99.9 Mostly reads >2 Data centers Db active-active(-active) Performance < 500ms 99% Serves mostly site viewers Uptime > 99.99
  50. 50. @aviranm When to Extract a New Microservice
  51. 51. @aviranm Microservice or Library?  Do I create deployment dependency?  What is DevOps overhead (managing middleware) ?  Who owns it?  Does it have its own development lifecycle?  Does it fit the scalability / availability concerns?  Can a different team develop it? I need time zone from an IP address
  52. 52. @aviranm Microservice has Ops, Library is Only Computational
  53. 53. @aviranm Which Technology Stack to Use
  54. 54. @aviranm Free to Chose? Microservices gives the freedom to use a different technology stacks. Enables innovation
  55. 55. @aviranm Default to the Stack You Know how to Operate.
  56. 56. @aviranm Innovate on Non Critical Microservices and Take Full Responsibility for its Operation.
  57. 57. @aviranm
  58. 58. @aviranm Why Microservices Scale engineering Development Velocity Scale system
  59. 59. @aviranm Microservices is the First Post DevOps Architecture
  60. 60. @aviranm Every Microservice is a Overhead
  61. 61. @aviranm Wix’s Company Model Company Guild Guild Company Gang GangGang Gang Product Product Cross-Engineering Teams A Guild is a group of people that share expertise, knowledge, tools, code and practice Company focus on large segment Has all the resources it needs to be independent People within the company are aligned with the Guilds
  62. 62. @aviranm Production State Changes Every 5 min
  63. 63. @aviranm @aviranm
  64. 64. @aviranm @aviranm Quality = Better Engineers Better Engineers = Professional Growth Professional Growth = Investment in People
  65. 65. @aviranm Guild Day  Engineers work 4 days with their company  Thursday is Guild day.  Developers conduct quality enhancing activities with the Guild.
  66. 66. @aviranm Guild Day Goals  Builds cross-team relationships  Shares knowledge  Assimilate the culture  Lesson learned  Continuous improvement  Promotes innovation  Professional development @aviranm
  67. 67. @aviranm Guild Day Schedule 10:00-11:00 Open Space 11:00-11:15 Break 11:15-11:40 Project spotlight 11:45-13:00 Tech talk or Workshop @aviranm
  68. 68. @aviranm Open Space
  69. 69. @aviranm
  70. 70. @aviranm Guild Week – Games of Gangs One week each quarter Pair programming with a person from another company Enhancing infrastructure Building tools Helping companies Work on open source Goal #1 – Improve engineering skills and quality
  71. 71. @aviranm Fast Features Better Quality
  72. 72. @aviranm Thank You
  73. 73. @aviranm Q&A @aviranm Aviran Mordo Head of Engineering @WixEng