From Parts to a Whole: Modular Development of a Large-Scale e-Commerce Site

Uploaded on

Video and slides synchronized, mp3 and slide download available at URL …

Video and slides synchronized, mp3 and slide download available at URL

Oliver Wegner, Stefan Tilkov show how OTTO, Germany’s largest online fashion retailer, used a system-of-systems approach to enable modular, parallel development of its ambitious shop relaunch. Filmed at

Oliver Wegner has been working at OTTO E-Commerce since 2006 on establishing technology as a core competence at OTTO. To this end he built up an engineering organization and formed teams for software development, quality assurance and architecture. Stefan Tikov is a decent-frequency blogger, tweets at @stilkov and is a frequent speaker at conferences in Germany and abroad.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. @olliwegner @stilkov
  • 2. News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on! /modular-ecommerce-website
  • 3. Presented at QCon London Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  • 4. From Parts to a Whole: Modular Development of a Large-Scale e-Commerce Site QCon, London 06/03/2014 Oliver Wegner • Stefan Tilkov
  • 5. 1. Reviewing architectures
  • 6. Generic Architecture Review Results Building features takes too long Technical debt is well-known and not addressed Deployment is way too complicated and slow Replacement would be way too expensive Scalability has reached its limit Architectural quality has degraded “-ility” problems abound
  • 7., Krissy Mayhew Any architecture’s quality is directly proportional to the number of bottlenecks limiting its evolution, development, and operations
  • 8. Conway’s Law “Organizations which design systems are constrained to produce systems which are copies of the communication structures of these organizations.” – M.E. Conway Organization → Architecture
  • 9. Reversal 1 Any particular architecture approach constraints organizational options – i.e. makes some organizational models simple and others hard to implement. Architecture → Organization
  • 10. Reversal 2 Choosing a particular architecture can be a means of optimizing for a desired organizational structure. Architecture → Organization
  • 11. 2. Rebuilding
  • 12. e-Commerce Solutions & Technology Product 5 March 2014 Seite 10 Percentageofturnover For the last 15 years the E-Commerce business has become more important 4.200 Employees > 2.1 Billion € turnover > 2 Million items on 80% turnover online
  • 13. But why rebuild
  • 14. Non-functional: Functional: Goals Scalable Simple Personalized Realtime Time To Market Data-Driven Test-Driven Fast Reliable Features
  • 15. OTTO Backend Infrastructure How green is the green field? Product Information Management Customer Management OTTO E-Commerce Frontend Infrastructure Articles Orders Order Management Customer
  • 16. Start of the project LHOTSE Technical system architecture Open Source as core technologies One Prototype to define the technology stack Project organization with autonomous teams Scrum as an agile development method
  • 17. 3. A system-of-systems approach
  • 18. Macro (technical) architecture Domain architecture
  • 19. JRuby C# Scala Groovy Java Clojure
  • 21. RDBMS NoSQL K/V RDBMS RDBMS/ DWH NoSQL DocDB Micro architecture
  • 22. Persistence Logic UI ModuleA ModuleB ModuleC
  • 23. System A Persistence Logic UI System B Persistence Logic UI System C Persistence Logic UI
  • 24. 4. The Otto architecture
  • 25. Buying Process – as you already know it SearchDiscover Assess Order Check Customer Journey
  • 26. The E-Commerce Business Architecture – Vertical and Horizontal aspects of the product 1 Website = 1 Product = 1 System = 1 Engineering Team ? Discover Search Assess Order Check Usability Webanalytics and Testing Online and Performance Marketing Platform Engineering
  • 27. System architecture is vertical Search Product Order User After Sales UI UI UI UI UI …
  • 28. User Auth UI UI After Sales UI 1 Team à N Systems 1 System à 1 Team Organizational aspects
  • 29. RESTful Architecture Shared Nothing Vertical Design Data Governance Buy when non core Common Technologies Macro-Architecture Micro-Architecture Architecture rules
  • 30. The 3 Faces of Product Ownership Technical Lead Project Lead Business Lead Product Owner
  • 31. Project start with distributed teams Team Search Team Discover Team Order … Team Check But how to deal with frontend integration?
  • 32. 5. Frontend Integration
  • 33. Development Deployment Storage Backend call Edge integration Server-side integration options RPC RESTRMI ESI Homegrown (Portal server) Build tools Chef, Puppet, … Asset pipeline Git/SVN submodules Gems Maven artifacts DB replication Feeds
  • 34. Link Replaced link Client-side integration options Client call Magical integration concept Unobtrusive JS ROCA-style oEmbed SPA-style
  • 35. – detail view
  • 36. - Basket
  • 37. Management of JS and CSS from each team Order Personalization User System View
  • 38. Separate teams for horizontal aspects Team Discover Team Search Team Order Team Assetserver
  • 39. Decoupling Versioned storage GitHub Local SCM Asset Server
  • 40. 6. A/B-Testing
  • 41. AB-Testing in a distributed environment – What is an AB- Test 99% 1%
  • 42. Solution with a centralized framework every team has to include in their repository Backoffice UI for Managing Tests Persistence DB R E S T Vertical (e.g. Search) Pull Experiment Data from Backoffice User Request User Response Testing specific and Vertical independent logic DB
  • 43. Solution with a dedicated Vertical and loose coupling Backoffice UI for Managing Tests Persistence DB R E S T Pull Data from Back- office Testing Vertical Testing Logic Persists all Tests DB Vertical (e.g. Search) Implement and Deliver Alternative A and B DB Request Response R E S T ESI-Includes Frontend- Proxy
  • 44. 7. Conclusion
  • 45. Results of the project LHOTSE Finished before schedule: October, 24th à 4 months earlier 2 years in total Scaled to >100 people Finished in budget Finished in quality Minimum Viable Product
  • 46. Lessons learned in applying a system-of-systems approach Independent, autonomous systems for maximum decoupling Strict macro architecture rules Teams with their own decisions Be skeptical of “easy” solutions Address cross- functional concerns Minimize cross- functional concerns Minimize need for coordination Prefer “pull” to “push” sharing
  • 47. @olliwegner @stilkov
  • 48. BACKUP
  • 49. Plattform Engineering delivers the basic infrastructure for all verticals Team Search Team Discover Team Order … Team Plattform Engineering Logging Deployment Provisioning Infrastructure Components
  • 50. Microservices? Microservices Approach – Very small – 100s – Does one thing only – Separate client (?) Vertical Systems Approach – Medium-sized – 10s – Small # of related things – Includes UI
  • 51. Watch the video with slide synchronization on! ecommerce-website