From Parts to a Whole: Modular Development of a Large-Scale e-Commerce Site


Published on

Video and slides synchronized, mp3 and slide download available at URL

Oliver Wegner, Stefan Tilkov show how OTTO, Germany’s largest online fashion retailer, used a system-of-systems approach to enable modular, parallel development of its ambitious shop relaunch. Filmed at

Oliver Wegner has been working at OTTO E-Commerce since 2006 on establishing technology as a core competence at OTTO. To this end he built up an engineering organization and formed teams for software development, quality assurance and architecture. Stefan Tikov is a decent-frequency blogger, tweets at @stilkov and is a frequent speaker at conferences in Germany and abroad.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

From Parts to a Whole: Modular Development of a Large-Scale e-Commerce Site

  1. 1. @olliwegner @stilkov
  2. 2. News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on! /modular-ecommerce-website
  3. 3. Presented at QCon London Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  4. 4. From Parts to a Whole: Modular Development of a Large-Scale e-Commerce Site QCon, London 06/03/2014 Oliver Wegner • Stefan Tilkov
  5. 5. 1. Reviewing architectures
  6. 6. Generic Architecture Review Results Building features takes too long Technical debt is well-known and not addressed Deployment is way too complicated and slow Replacement would be way too expensive Scalability has reached its limit Architectural quality has degraded “-ility” problems abound
  7. 7., Krissy Mayhew Any architecture’s quality is directly proportional to the number of bottlenecks limiting its evolution, development, and operations
  8. 8. Conway’s Law “Organizations which design systems are constrained to produce systems which are copies of the communication structures of these organizations.” – M.E. Conway Organization → Architecture
  9. 9. Reversal 1 Any particular architecture approach constraints organizational options – i.e. makes some organizational models simple and others hard to implement. Architecture → Organization
  10. 10. Reversal 2 Choosing a particular architecture can be a means of optimizing for a desired organizational structure. Architecture → Organization
  11. 11. 2. Rebuilding
  12. 12. e-Commerce Solutions & Technology Product 5 March 2014 Seite 10 Percentageofturnover For the last 15 years the E-Commerce business has become more important 4.200 Employees > 2.1 Billion € turnover > 2 Million items on 80% turnover online
  13. 13. But why rebuild
  14. 14. Non-functional: Functional: Goals Scalable Simple Personalized Realtime Time To Market Data-Driven Test-Driven Fast Reliable Features
  15. 15. OTTO Backend Infrastructure How green is the green field? Product Information Management Customer Management OTTO E-Commerce Frontend Infrastructure Articles Orders Order Management Customer
  16. 16. Start of the project LHOTSE Technical system architecture Open Source as core technologies One Prototype to define the technology stack Project organization with autonomous teams Scrum as an agile development method
  17. 17. 3. A system-of-systems approach
  18. 18. Macro (technical) architecture Domain architecture
  19. 19. JRuby C# Scala Groovy Java Clojure
  21. 21. RDBMS NoSQL K/V RDBMS RDBMS/ DWH NoSQL DocDB Micro architecture
  22. 22. Persistence Logic UI ModuleA ModuleB ModuleC
  23. 23. System A Persistence Logic UI System B Persistence Logic UI System C Persistence Logic UI
  24. 24. 4. The Otto architecture
  25. 25. Buying Process – as you already know it SearchDiscover Assess Order Check Customer Journey
  26. 26. The E-Commerce Business Architecture – Vertical and Horizontal aspects of the product 1 Website = 1 Product = 1 System = 1 Engineering Team ? Discover Search Assess Order Check Usability Webanalytics and Testing Online and Performance Marketing Platform Engineering
  27. 27. System architecture is vertical Search Product Order User After Sales UI UI UI UI UI …
  28. 28. User Auth UI UI After Sales UI 1 Team à N Systems 1 System à 1 Team Organizational aspects
  29. 29. RESTful Architecture Shared Nothing Vertical Design Data Governance Buy when non core Common Technologies Macro-Architecture Micro-Architecture Architecture rules
  30. 30. The 3 Faces of Product Ownership Technical Lead Project Lead Business Lead Product Owner
  31. 31. Project start with distributed teams Team Search Team Discover Team Order … Team Check But how to deal with frontend integration?
  32. 32. 5. Frontend Integration
  33. 33. Development Deployment Storage Backend call Edge integration Server-side integration options RPC RESTRMI ESI Homegrown (Portal server) Build tools Chef, Puppet, … Asset pipeline Git/SVN submodules Gems Maven artifacts DB replication Feeds
  34. 34. Link Replaced link Client-side integration options Client call Magical integration concept Unobtrusive JS ROCA-style oEmbed SPA-style
  35. 35. – detail view
  36. 36. - Basket
  37. 37. Management of JS and CSS from each team Order Personalization User System View
  38. 38. Separate teams for horizontal aspects Team Discover Team Search Team Order Team Assetserver
  39. 39. Decoupling Versioned storage GitHub Local SCM Asset Server
  40. 40. 6. A/B-Testing
  41. 41. AB-Testing in a distributed environment – What is an AB- Test 99% 1%
  42. 42. Solution with a centralized framework every team has to include in their repository Backoffice UI for Managing Tests Persistence DB R E S T Vertical (e.g. Search) Pull Experiment Data from Backoffice User Request User Response Testing specific and Vertical independent logic DB
  43. 43. Solution with a dedicated Vertical and loose coupling Backoffice UI for Managing Tests Persistence DB R E S T Pull Data from Back- office Testing Vertical Testing Logic Persists all Tests DB Vertical (e.g. Search) Implement and Deliver Alternative A and B DB Request Response R E S T ESI-Includes Frontend- Proxy
  44. 44. 7. Conclusion
  45. 45. Results of the project LHOTSE Finished before schedule: October, 24th à 4 months earlier 2 years in total Scaled to >100 people Finished in budget Finished in quality Minimum Viable Product
  46. 46. Lessons learned in applying a system-of-systems approach Independent, autonomous systems for maximum decoupling Strict macro architecture rules Teams with their own decisions Be skeptical of “easy” solutions Address cross- functional concerns Minimize cross- functional concerns Minimize need for coordination Prefer “pull” to “push” sharing
  47. 47. @olliwegner @stilkov
  48. 48. BACKUP
  49. 49. Plattform Engineering delivers the basic infrastructure for all verticals Team Search Team Discover Team Order … Team Plattform Engineering Logging Deployment Provisioning Infrastructure Components
  50. 50. Microservices? Microservices Approach – Very small – 100s – Does one thing only – Separate client (?) Vertical Systems Approach – Medium-sized – 10s – Small # of related things – Includes UI
  51. 51. Watch the video with slide synchronization on! ecommerce-website