Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Some lessons learned contributing to #MagentoMSI

236 views

Published on

The slides I presented at Meet Magento Romania 2018, with speaker notes

Published in: Technology
  • Be the first to comment

Some lessons learned contributing to #MagentoMSI

  1. 1. # M M 18 R O @ a le ro n 7 5 Some lessons learned contributing to #MagentoMSI Some lessons learned contributing to #MagentoMSI Photo by Benjamin Davies on Unsplash
  2. 2. # M M 18 R O @ a le ro n 7 5 Hello, my name is Alessandro Ronchi, I come from Italy and I’m COO and Developer at Bitbull. I chose this picture because it represents me: I like new challenges, like running the Big Dam Run being a non-runner and continuously improving. This is the attitude that brought me to become an author, speaker, conference organizer, Magento Master, Contributor, and Maintainer. If you like it, please follow me.
  3. 3. # M M 18 R O @ a le ro n 7 5 Thanks to the Magento Community Engineering team, Magento truly embraced the open source spirit, opening its core to contributions and becoming a real Community-driven platform. Nowadays, it’s a very well known story so I want to focus mainly on what motivates me to contribute to Magento. Photo by Kelly Sikkema on Unsplash
  4. 4. # M M 18 R O @ a le ro n 7 5 Contributing to Magento gives great learning opportunities; we have the privilege to work with Magento engineers and other smart members of its community. That would be enough to motivate us. But a recent episode made me think about an additional side effect that now motivates me at least as well: gratitude. Photo by Raj Eiamworakul on Unsplash
  5. 5. Chris Snedaker @df2002 @aleron75, @philwinkle happen to be a couple amazing people who have helped this situation... and probably don’t realize it. So thanks to you two as well # M M 18 R O @ a le ro n 7 5 Some weeks ago, Chris wrote this message, thanking me for having helped him a lot. I had no clue why he was thanking me. I wasn’t even sure to know him at all. But then I realized that some days before, I gave a simple advice to this guy about how to implement something on MSI Slack channel. I didn’t realize that it was so important for him until his tweet, that made my day. Photo by Cytonn Photography on Unsplash
  6. 6. # M M 18 R O @ a le ro n 7 5 Learn & Give Back Learn & Give Back So this is my first take away: learning is great, but having the privilege of helping others is even greater. Someone even says that it’s the secret of happiness.
  7. 7. # M M 18 R O @ a le ro n 7 5 #MagentoMSI#MagentoMSI I expect that many of you have at least heard about Multi-Source Inventory but let me briefly introduce it. Photo by Samuel Zeller on Unsplash
  8. 8. # M M 18 R O @ a le ro n 7 5 Merchant I want to manage stock in multiple locations so that I can reflect my phisical inventory in Magento without extensions or customization MSI was an internal Magento project whose first requirement was documented on September 2013 but its development was never accomplished. The Community-driven project was kickstarted during the contribution day of MM-DE organized by Firegento in Leipzig on May 2017. I joined in July 2017. Photo by Startaê Team on Unsplash
  9. 9. # M M 18 R O @ a le ro n 7 5 The first set of functionality that lets merchants natively manage multiple stock locations will be officially relased on November 2018, together with Magento 2.3.0 The product backlog is plenty of other features some of which are already developed and will be available in 2.3.1. The rest of them will be relesed later on and will benefit from users feedback. Photo by Vidar Nordli-Mathisen on Unsplash
  10. 10. # M M 18 R O @ a le ro n 7 5 Let me underscore something important: MSI is and will remain a freely available feature of Magento Open Source. Photo by Vidar Nordli-Mathisen on Unsplash
  11. 11. # M M 18 R O @ a le ro n 7 5 MSI is a big project: at date it represents more than 12% of the entire Magento codebase. Almost 70 ppl and 16 agencies contributed over time submitting 700+ PRs so far. The project is a reference for: - best development practices - insight of future Magento architecture - good documentation Photo by Chen Hu on Unsplash
  12. 12. # M M 18 R O @ a le ro n 7 5 It’s time to be more technical. About the lessons learned, I used to talk a lot about Domain-Driven Design in my previous presentations but it’s a very intense topic and I don’t have much time. Thus, I’ll talk about how we applied the concepts from CQRS and Event Sourcing to MSI. Photo by Dina Lydia on Unsplash
  13. 13. # M M 18 R O @ a le ro n 7 5 Before starting, a necessary disclaimer. Once we experience the benefits of new patterns or technologies, we are tempted to use them everywhere. There can be some drawbacks or simply “no additional benefits” with what we will see and we should always take them into account instead of applying them blindly. Photo by Dina Lydia on Unsplash
  14. 14. # M M 18 R O @ a le ro n 7 5 CQRSCQRS First topic is the architectural pattern named Command-Query Responsibility Segregation. Photo by Markus Spiske on Unsplash
  15. 15. # M M 18 R O @ a le ro n 7 5 Bertrand Meyer @Bertrand_Meyer Queries return results, but never change the state. Commands change the state, but never return values. Let’s start from the beginning, that is, from CQS, introduced by Betrand Meyer in the 1990s. CQS stands for Command-Query Separation and it’s an implementation pattern requiring us to avoid assigning both read and write responsibility to the same method of a class. Photo by Markus Spiske on Unsplash
  16. 16. # M M 18 R O @ a le ro n 7 5 A code snippet is worth a thousand words. This is an example of code that doesn’t apply the CQS pattern. I’m pretty sure that every one of us tends to avoid such kind of code but don’t give it always for granted… Photo by Markus Spiske on Unsplash
  17. 17. # M M 18 R O @ a le ro n 7 5 This is real code taken from Magento 2 that doesn’t apply CQS: this method returns a value (so it’s a query) but it’s also a command because changes the state of the system, creating a directory if it doesn’t exist. Photo by Markus Spiske on Unsplash
  18. 18. # M M 18 R O @ a le ro n 7 5 This, instead, is what Meyer means: you can see in this class we have methods that return data (queries) separated from methods that write data (commands) and change the state. Photo by Markus Spiske on Unsplash
  19. 19. # M M 18 R O @ a le ro n 7 5 Greg Young @gregyoung Separate systems that represent the state of the application from those that change it. So what’s CQRS? CQRS stands for Command- Query Responsibility Segregation and despite the similar name, the difference is remarkable: it is an architectural pattern. The responsibility is not only split at method level but at the model level. That’s why we talk about segregation. Photo by Markus Spiske on Unsplash
  20. 20. # M M 18 R O @ a le ro n 7 5 Again, small code examples are worth a thousand words. Here we have two separate classes, one for reading and one for writing. Note that the storages used by these two models can be different even if they don’t necessarily have to be. Photo by Markus Spiske on Unsplash
  21. 21. # M M 18 R O @ a le ro n 7 5 This diagram illustrates the inventory models among which we splitted commands and queries: sources and stocks. Sources are where we store products. We can aggregate multiple sources into Stocks. A Sales Channel reads product availability from its connected stock. To sum up: we change the state of the system updating sources and read the state by querying stocks. Data consistency is guaranteed by an indexing mechanism that aggregates product quantities from different sources into stocks (called projection in CQRS lingo). Photo by Samuel Zeller on Unsplash
  22. 22. # M M 18 R O @ a le ro n 7 5 This diagram shows the consumers of the services exposed by the different models: Frontend users consume queries on the stock to check product availability when placing orders in a sales channel. Backend users issue update commands on the sources when they ship the products. Photo by Samuel Zeller on Unsplash
  23. 23. # M M 18 R O @ a le ro n 7 5 This diagram shows us the process that happens when we place an order. When we add a product in cart, Magento checks for available salable qty of the channel through its related stock. When we place the order, if the product is available, the ordered qty is reserved. The reserved qty is deducted from salable qty. Once shipped, the reserved quantity is deducted from the source and the previous reservation is voided. We will see how reservations are voided in more details when speaking of Event Sourcing. Photo by Samuel Zeller on Unsplash
  24. 24. # M M 18 R O @ a le ro n 7 5 BenefitsBenefits CQRS gives us a better understanding of what is responsible for changing the state of a system. This separation brings to domain simplification: a single model is no more responsible for both operations, thus they can be assigned to two simpler models which are also simpler to adapt to business changes. Furthermore, since we separate the systems that write and read, in situations where, for example, read operations are more frequent than write ones, we can better assign resources and scale our application properly. Photo by Markus Spiske on Unsplash
  25. 25. # M M 18 R O @ a le ro n 7 5 DrawbacksDrawbacks CQRS is not a universal solution. Applying this pattern comes with some drawbacks and we should always take them into account when deciding whether to apply it or not to our application: - more analysis due to the additional complication - duplication handling - be careful when choosing what to make eventual consistent; e.g. avoid in financial transactions Photo by Markus Spiske on Unsplash
  26. 26. # M M 18 R O @ a le ro n 7 5 Simplification over Unification Simplification over Unification Here we come to the second takeaway I want to leave you: even if duplication handling is listed among the things to look-out, duplication also brings simplification or, put differently, it is possible to avoid the coupling given by unification preferring simplification. CQRS and DDD advocate for simplification over unification.
  27. 27. # M M 18 R O @ a le ro n 7 5 Event SourcingEvent Sourcing Second topic is another architectural pattern known as Event Sourcing. Photo by Nathan Anderson on Unsplash
  28. 28. # M M 18 R O @ a le ro n 7 5 Martin Fowler @martinfowler Capture all changes to an application state as a sequence of events. ES is an architectural pattern that can be very useful in some circumstances as we will see. According to Event Sourcing, we store the state of a system by persisting a sequence of events that brought us there. This way not only we can determine the state of the system at any time but we also have an insight of how we got there. Photo by Nathan Anderson on Unsplash
  29. 29. # M M 18 R O @ a le ro n 7 5 The classical example is the list of bank account transactions. The final balance is given by the sum of all the positive and negative transactions performed at a specific date. This way you can exactly know how you got there and who to blame :-) Photo by Alexa Mazzarello on Unsplash
  30. 30. # M M 18 R O @ a le ro n 7 5 I think the best way of understanding the benefits we had with this approach is seeing how it was applied. This is how Checkout works in Magento 2.2 (and 1.x as well). Each time we place an order, the product stocks are updated. That means, stock item records related to the products we are purchasing are locked with a SELECT… FOR UPDATE statement. That means: users purchasing at least one same items must wait. And you know that the worst moment to block users is checkout. Photo by Nathan Anderson on Unsplash
  31. 31. # M M 18 R O @ a le ro n 7 5 This is how checkout works in Magento 2.3 thanks to ES: each time we place an order, some product reservations are appended and that’s it. No lock. No wait. No “conversion killer” code. Photo by Nathan Anderson on Unsplash
  32. 32. # M M 18 R O @ a le ro n 7 5 To be honest, this was the only choice we had. Since determining from which source to ship items could be a time-consuming activity, we couldn’t do that during checkout. We had to move it from placing the order to the fulfilling phase. When an order is placed, we only needed to record the information of the qty to reserve, without updating the inventory in real-time. Photo by Nathan Anderson on Unsplash
  33. 33. # M M 18 R O @ a le ro n 7 5 This table represents our event store, the equivalent of our bank transactions. Product reservations are append-only events that record the information about the product qty that need to be incremented or decremented, depending on the situation. When product qty is updated in source, product reservation voided through compensating reservations. Photo by Nathan Anderson on Unsplash
  34. 34. # M M 18 R O @ a le ro n 7 5 The salable quantity of a product is calculated by taking reservations (and thresholds) into account and keeping product availability consistent and protected from overselling. Photo by Nathan Anderson on Unsplash
  35. 35. # M M 18 R O @ a le ro n 7 5 BenefitsBenefits I think that at this point the benefits of applying a pattern like ES are clear but let’s recap: - thanks to their immutable nature, events are append-only operations - since those records are never updated we can independently optimize reads and writes - since we keep track of all the events related to product quantity, a brand new set of information can be extracted for business analysis Photo by Nathan Anderson on Unsplash
  36. 36. # M M 18 R O @ a le ro n 7 5 This is an example of valuable information we can extract given, for example, we track all our stock changes. The graph on the right is a condition in which, for a given product, we never get under the out-of-stock threshold. This information is valuable for the business: since stocking products requires space and since space costs money, we can reduce excess inventory for given product, thus reducing costs without reducing the quality of service for our customer. This kind on analysis would not be possible if we don’t keep track of all the stock changes. Photo by Nathan Anderson on Unsplash
  37. 37. # M M 18 R O @ a le ro n 7 5 DrawbacksDrawbacks Well, despite it was introduced in 2005 it is still a not very well known pattern. As it happens for CQRS, it requires more analysis, e.g. to find the proper event granularity. In case some data need to be corrected, we can’t count on simple updates because events are immutable. And yes, it requires more disk space, deal with it. Photo by Nathan Anderson on Unsplash
  38. 38. # M M 18 R O @ a le ro n 7 5 Design Software over Write Code Design Software over Write Code And here is the third and last takeaway I want to leave you. As we have seen, some changes in the design of our software can bring huge benefits. Sooner or later, in our development career, we must understand the difference between writing code and designing software. No matter if you are in your 20s or 40s, there is always something new to learn, so never stop improving.
  39. 39. # M M 18 R O @ a le ro n 7 5 Thank you!Thank you! Thank you, for the time you spent listening, watching, or reading this presentation. And big thanks also go to : - The MCE team and all the folks contributing to Magento 2 and MSI – tw + gh - Lori Krell for the beautiful diagrams and for writing the great MSI User Guide - tw - Holger Wolterdorf’s presentation for CQRS and ES inspiration - tw If you have any questions, feel free to reach me on Twitter or Slack. Photo by Benjamin Davies on Unsplash

×