Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Big Retail Goes Reactive at Walmart

2,159 views

Published on

Last year, Nurun and Walmart Canada launched the first responsively designed enterprise e-commerce website created for a large Canadian retailer. Built on a new platform with the Play Framework, Scala and Akka at its core, this foundation has proven itself in terms of flexibility, developer productivity, performance and scalability. We’ll share some of the insights we’ve gained in creating a best of breed solution that scales to Walmart’s needs—now and into the future.

Published in: Technology
  • Be the first to comment

Big Retail Goes Reactive at Walmart

  1. 1. Reactive Retail Big Retail Goes Reactive at Walmart NOVEMBER 2014 REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  2. 2. Agenda Introduction Walmart.ca re-architecture Move to Functional Programming Technical Tips & Features Experiences and Impressions Outlook Q&A REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  3. 3. About Nurun AREAS OF EXPERTISE REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART 12 San Francisco Atlanta Nancy Shanghai Quebec Turin Montreal Paris Milan Toronto Chicoutimi Madrid 1,200+ Employees 14 Years In Business Retail Automotive Financial Services Media & Entertainment Fashion & Beauty Utilities Government Telecommunications Travel & Hospitality Health & Pharma 3 We are a global design & technology consultancy with 12 offices worldwide.
  4. 4. Introduction REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  5. 5. Introduction TYPICAL LARGE ECOMMERCE ENVIRONMENTS CDN JSPs E-Commerce Platform RDBMS Product content management REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  6. 6. Introduction TYPICAL LARGE ECOMMERCE ENVIRONMENTS CDN JSPs E-Commerce Platform Search & REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART merchandizing RDBMS Product content management
  7. 7. Introduction INCUMBENT PLATFORM • ATG, Oracle • Fairly monolithic • Rapidly evolving merchandising needs REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  8. 8. Introduction INCUMBENT PLATFORM • ATG, Oracle • Fairly monolithic • Rapidly evolving merchandising needs REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  9. 9. Introduction INCUMBENT PLATFORM • ATG, Oracle • Fairly monolithic • Rapidly evolving merchandising needs • Scalability issues REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  10. 10. Retail priorities EFFICIENT & SCALABLE Need to handle large volumes RESILIENT Downtime is expensive FLEXIBLE & MAINTAINABLE Business requirements change REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  11. 11. Need to go Reactive to be … RESPONSIVE Customer experience, brand promise RESILIENT Handle failure at multiple levels ELASTIC Handle traffic spikes (without breaking the bank) MESSAGE-DRIVEN Enable all of the above REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  12. 12. Walmart.ca Re-architecture REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  13. 13. Walmart.ca re-architecture JSPs ATG Oracle Product content management REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  14. 14. Walmart.ca re-architecture REST API JSPs ATG Oracle REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART REST API Endeca search & merchandizing Product content management
  15. 15. Walmart.ca re-architecture JSPs ATG Oracle REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART REST API Endeca search & merchandizing Akka, Redis Product content management
  16. 16. Walmart.ca re-architecture JSPs ATG Oracle REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART JSON REST API Endeca search & merchandizing Play Akka, Redis Product content management
  17. 17. Walmart.ca re-architecture JSON REST API ATG Oracle REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART JSON REST API Endeca search & merchandizing Play Akka, Redis Product content management
  18. 18. Walmart.ca re-architecture REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART No. 1 +20% Canada’s first responsively designed e-commerce site for a big box retailer +98% Increase in conversion in the first 4 weeks after launch Increase in mobile orders -36% Decrease in average page load time Awarded BEST E-COMMERCE SITE in large retailers by Retail Council of Canada
  19. 19. Walmart.ca re-architecture REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART No. 1 +20% Canada’s first responsively designed e-commerce site for a big box retailer +98% Increase in conversion in the first 4 weeks after launch Increase in mobile orders -36% Decrease in average page load time NO SYSTEM OUTAGES! Awarded BEST E-COMMERCE SITE in large retailers by Retail Council of Canada
  20. 20. Move to Functional Programming REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  21. 21. Move to Functional Programming ● Majority of the team are developers with a Java background ● Play provides an easy entry path to Scala / Reactive / Functional: § Based on familiar MVC patterns § Can start with Java style, mutation, blocking § Incrementally transition to Scala style, immutable, non-blocking REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  22. 22. Move to Functional Programming 1. No nulls (use Option) 2. No vars, mutable state* 3. Use higher-order functions (map, flatMap, filter, for-yield, etc.) 4. Business logic should be side-effect free (testability!) 5. Create and use strong types REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART *unless well justified (e.g. much clearer), and as small a scope as possible
  23. 23. Move to Functional Programming ● Code should be a clear and concise as possible ● Types should be express intent and fit tightly REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  24. 24. Move to Functional Programming: Strong Types sealed trait Language case object English extends Language case object French extends Language def formatPrice(price: Price, lang: Language) = lang match { case English => … case French => … REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART } implicit val binder = new PathBindable[Language] { … }
  25. 25. Technical Tips & Features REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  26. 26. Tips & Features: Caching Futures (on-heap) ● Several concurrent requests for same data ● Whoever requests first initiates the Future (Promise) ● No locking required (narrow window between first request and when Future is cached) REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  27. 27. Tips & Features: Handling JSON P lay Application Makes Heavy Use of JSON • Consumes JSON view advice from Endeca • Proxy & augment JSON data for web service calls REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  28. 28. Tips & Features: Play JSON macro Consuming JSON (e.g. Endeca cartridges) case class Product(id: Long, name: String, …) object Product { implicit val formats = Json.format[Product] REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART } Json.toJson(product) productJson.asOpt[Product]
  29. 29. Tips & Features: ‘Almost Default’ Reads/Writes OPEN CHALLENGE Sometime we want an ‘almost default’ Reads/Writes • e.g. use a default • Read/write a few fields in a particular way • e.g. format two dates in different ways • (Could Shapeless case class generics macro help?) REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  30. 30. Tips & Features: Proxying and Augmenting JSON Proxying JSON (e.g. most web services) • Might want to augment a JSON response • Don’t need to care about knowing the entire structure • Play JSON transformers • play-json-zipper can be helpful (github: mandubian) REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  31. 31. Tips & Features: Dependency Injection CONTROLLER DEPENDENCIES (SERVICES) • Use structural types for DI at the controller level • Create a services module for each request • Want access to request header (logging) REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  32. 32. Tips & Features: Dependency Injection THREADLOCAL • Inject dependencies in Global.getControllerInstance • Initialize services module in Global.onRouteRequest, add to ThreadLocal • Pick up this value in the getControllerInstance and provide it to our controller REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  33. 33. Tips & Features: Monadic Rendering Page Model • Template • Zones • Cartridges REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  34. 34. Rendering View Advice: Templates & Zones The business user selects a template: • Determines the page layout • Each template has ~12 zones REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  35. 35. Rendering View Advice: Cartridges … adds cartridges to the zones: • ~100 cartridges REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART ProductImage ProductDescription ProductPurchase
  36. 36. Rendering View Advice: Whole Page • Endeca fills in data (product info) • Play renders the cartridge as HTML REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  37. 37. Rendering View Advice: JSON to Visual Page • Tmpl1 • Header • Car1 • Car2 • LeftNav • Car3 • MainContent • Car4 • Car5 • Car6 REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  38. 38. Rendering View Advice: JSON to HTML • Tmpl1 • Header • Car1 • Car2 • LeftNav • Car3 • MainContent • Car4 • Car5 • Car6 REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART <body>! <div class=“header”>! ! !…! </div>! <div class=“leftNav”>! ! !…! </div>! !<div class=“main”>! ! !…! </div>! </body>!
  39. 39. Rendering View Advice: Auxiliary Information • Tmpl1 • Header • Car1 • Car2 • LeftNav • Car3 • MainContent • Car4 • Car5 • Car6 REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART <head>! ! </head>! <body>! <div class=“header”>! ! !…! </div>! <div class=“leftNav”>! ! !…! </div>! !<div class=“main”>! ! !…! </div>! ! ! </body>!
  40. 40. Rendering View Advice: Auxiliary Information • Tmpl1 • Header • Car1 • Car2 • LeftNav • Car3 • MainContent • Car4 • Car5 • Car6 REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART <head>! ! </head>! <body>! <div class=“header”>! ! !…! </div>! <div class=“leftNav”>! ! !…! </div>! !<div class=“main”>! ! !…! </div>! ! ! </body>! CSS / JavaScript / RJS modules
  41. 41. Rendering View Advice: Auxiliary Information • Tmpl1 • Header • Car1 • Car2 • LeftNav • Car3 • MainContent • Car4 • Car5 • Car6 REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART <head>! ! </head>! <body>! <div class=“header”>! ! !…! </div>! <div class=“leftNav”>! ! !…! </div>! !<div class=“main”>! ! !…! </div>! ! ! </body>! CSS / JavaScript / RJS modules JavaScript / RJS modules
  42. 42. Rendering View Advice: Auxiliary Information • Tmpl1 • Header • Car1 • Car2 • LeftNav • Car3 • MainContent • Car4 • Car5 • Car6 REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART <head>! ! </head>! <body>! <div class=“header”>! ! !…! </div>! <div class=“leftNav”>! ! !…! </div>! !<div class=“main”>! ! !…! </div>! ! ! </body>! CSS / JavaScript / RJS modules JavaScript / RJS modules Cache control
  43. 43. Monadic Rendering: ValidatedBuilder Monad ValidatedBuilder[X] • “Parse/process” X • ≅ Future[(X, Boolean, Resources)] • Error flag: • Determine if page should be cached • Front-end resources • CSS • RequireJS modules • JavaScript snippets (+ page location requirements) REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  44. 44. Monadic Rendering: ValidatedBuilder Monad ValidatedBuilder[X] • map and flatMap handle combining and propagating validity and front-end resources • Code focuses on the essential task (JSON -> HTML) without clutter • Error prone ‘plumbing’ is handled by the monad abstraction REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  45. 45. Monadic Rendering: ValidatedBuilder Zone v al headerContent: VB[Seq[Cartridge]] = (viewAdvice “header”) .as[Seq[JsValue]] // Seq[JsValue] .map(Cartridge.parse) // Seq[VB[Cartridge]] .vbSeq // VB[Seq[Cartridge]] REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  46. 46. Monadic Rendering: ValidatedBuilder Template val template: VB[Template] = for { header <- headerContent : VB[Seq[Cartridge]] leftNav <- leftNavContent … templateResult <- CayenneTemplate(header, leftNav, …) } yield templateResult REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  47. 47. Monadic Rendering: ValidatedBuilder Page for { content <- template.contents : Future[Template] resources <- template.resources : Future[Resources] isCacheable <- template.isValid : Future[Boolean] yield { val result = template.render(resources) if (isCacheable) result else result.withHeaders(noCache: _*) REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART }
  48. 48. Tips & Features: Dynamic Configuration • Ops can make on the fly configuration changes • Enable/disable services and site features • Change endpoints REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  49. 49. Tips & Features: Dynamic Configuration • Sends configuration delta to Play (JSON object) • Play verifies the change (initialize services module) • Each request sees a snapshot of the configuration • Uses standard Play HOCON API REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  50. 50. Experience and Impressions REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  51. 51. Experience and Impressions ● Never need to say “no” to a client due to technical limitations, ● Heavily concurrency / parallelism, with very little headaches, ● Aggressive refactorings are surprisingly easy (thank you strong types). REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  52. 52. Outlook REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  53. 53. Outlook NEW BREED OF PRODUCTS Reactive core REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  54. 54. Outlook NEW BREED OF PRODUCTS Reactive core Middleware REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  55. 55. Outlook NEW BREED OF PRODUCTS Reactive core Middleware Omnichannel CE REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  56. 56. Outlook NEW BREED OF PRODUCTS Reactive core Middleware Omnichannel CE Streaming big data Machine intelligence REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  57. 57. Outlook NEW BREED OF PRODUCTS Reactive core Middleware Omnichannel CE Streaming big data Machine intelligence CMS REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  58. 58. Outlook NEW BREED OF PRODUCTS Reactive core Middleware Omnichannel CE Streaming big data Machine intelligence CMS BAAS REsponsiveSS REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  59. 59. Outlook NEW BREED OF PRODUCTS Reactive core Middleware Omnichannel CE Streaming big data Machine intelligence CMS BAAS REsponsiveSS In store tech, IoT… REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART
  60. 60. Q & A Ankur Mathur @anchormath Dana Harrington @danagharrington https://github.com/dana-harrington REACTIVE RETAIL BIG RETAIL GOES REACTIVE AT WALMART

×