Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Multi-Source Inventory. Imagine. Las Vegas. 2018


Published on

Presentation made by
Mark Brinton - Product Owner of MSI
Igor Miniailo - Architect of MSI
who described the story, product backlog, architecture and community contribution on the Multi-Source Inventory project

Published in: Software
  • Get Paid For Your Opinions! Earn $5-$10 cash on your first survey. ➤➤
    Are you sure you want to  Yes  No
    Your message goes here
  • Earn Up To $316/day! Easy Writing Jobs from the comfort of home! ◆◆◆
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Multi-Source Inventory. Imagine. Las Vegas. 2018

  1. 1. Multi-Source Inventory: The Largest Community-Developed Feature in Magento
  2. 2. Mark Brinton Sr. Manager, Product | Magento Commerce @markabrinton Igor Miniailo Software Engineer | Magento Commerce @iminyaylo
  3. 3. Where did we start? West Central East Requests from merchants and partners • Need Multiple warehouses • Can’t split inventory for a single product Current state • Single stock only • Must customize or extend • Can be complex to manage and upgrade Multiple types needed • Warehouses • Stores • Distribution Centers • Drop Shipping Increasingly common • Even for smaller merchants • Applies to both B2C and B2B
  4. 4. The MSI “Story” Original artwork ©2012 Seth Wolfshornd
  5. 5. Validating the request High interest and common need • Not an edge case • Multiple validation signs • B2C and B2B • SMB and Enterprise Suitable for core development • Impacts many modules or areas of core • Key part of the order process or checkout • Complex to customize properly Base for extension • Drive standard approach • Natural extension points • Creates marketplace opportunity Top 3 in partner voting Most commented issue on GitHub Top 5 on Magento Forums Many merchant types and sizes Inventory is complex and benefits from a consistent architecture Impacts checkout, shipment, returns, etc. API integrations common for inventory Source Selection algorithm logical to customize
  6. 6. Let’s get started! • Originally an internal project • First requirements documented in September 2013 • Assigned to Dragons team • Went through multiple name changes: – Multiple Warehouse / Node / Source Inventory • Final naming decision: Multi-Source Inventory – Source is the most flexible name to handle multiple location types – Consistent with naming approach used in Magento Order Management
  7. 7. Original internal Epic MAGETWO-14308 Progress! Source management Assign products and quantity to sources Priority-based selection algorithm for shipment
  8. 8. Software development is difficult • Didn’t complete all stories for the feature • With only one team assigned, velocity was not as high as we needed to maintain pace • Inventory is complex, with partial or incremental release challenging – Touches many areas of the core and multiple modules – All aspects of merchant inventory process must be supported, from purchasing to orders and through to shipment and returns • Internal constraints on breaking changes while making major changes to inventory created technical debt • Other projects took higher priority
  9. 9. So, now what? • Market research has not changed – the feature is still important • How can we continue without internal teams assigned? • Convenient timing – Community Engineering Team created in February 2017 – Wanted to expand engagement with the community – Interest from community members – Have a documented feature ready, with some work completed (we ended up starting from the beginning)
  10. 10. The community development model Benefits for Magento Improved velocity for new features Localized expertise for inventory management differences by market Direct client perspective More reviewers = faster to find bugs Get feedback while building the feature, not after release Project Goals Showcase the depth and talent of community engineers Improve our working process for complex features with remote developers Increase developer engagement with Magento Reduce time to build without utilizing full team of internal developers
  11. 11. But how can we recruit developers? • All contributors will be volunteers • Can’t assign work to anyone, they must offer to help • Need to make the opportunity compelling for participants • So, how to start? – Join an existing community event to build momentum for the project
  12. 12. Project Kickoff: Meet Magento Germany Thank You!
  13. 13. What did we offer to participants? • Partner with our developers and architects • Learn Magento best practices • Get listed as a contributor in release notes • Intellectual challenge of a complex feature • Chance to collaborate with other developers and build community relationships • Strengthen reputation with merchants • A faster release of MSI for clients • No more customizing inventory for clients • Opportunity to give back to the community Igor Miniailo@RicTempesta
  14. 14. Another essential ingredient: sweets!
  15. 15. Hackathon results
  16. 16. First Pull Requests accepted at MMDE
  17. 17. All events must come to an end Ben Marks Mark Brinton
  18. 18. Remote project management process • Started with – Familiar to developers – Avoided difficult process of granting access to Magento JIRA instance • Multiple features – Code and source control – Wiki for documentation – Individual issue tracking – Comments and collaboration – Pull Requests for contributions • Initially tracked stories in a wiki table
  19. 19. MSI Community: GitHub Our Community Engineering GitHub repository contains all code for the project and is used to manage pull requests, assign stories, and track issues. Any developer can see how the project has developed over time and submit pull requests for review.
  20. 20. MSI Community: Wiki Our Community Engineering wiki contains documentation for the project, including technical vision, architecture, definition for each story, UX, and roadmap. This enables new developers to quickly understand the project and participate.
  21. 21. MSI Community: Slack Our Community Engineering Slack has specific channels for MSI questions and discussion. Developers can get help with the project and collaborate together, regardless of time zone.
  22. 22. Meet Magento Germany Leipzig, Germany Meet Magento Sweden Stockholm, Sweden Magento Live UK London, United Kingdom Mage Unconference Utrecht, Netherlands Magento Hackathon Munich, Germany Meet Magento Romania Cluj, Romania Meet Magento New York New York City, USA Mage Titans Austin Austin, USA MSI Contribution Week Kolbermoor, Germany Meet Magento Spain Madrid, Spain Mage Test Fest Amersfoort, Netherlands Atwix Contribution Day Khmelnytskyi, Ukraine Mage Titans Manchester Manchester, United Kingdom Amasty Contribution Day Minsk, Belarus Mage Conf Kiev Kiev, Ukraine 2017 MSI Community Events
  23. 23. More contributors = more complexity • As more contributors joined the project, it became difficult to manage on the wiki – Many additional stories created – Issues manually linked together – Not possible to report on stories – Prioritization is not easily available • Also, it was not clear where someone new to the project could start participating
  24. 24. MSI Community: ZenHub • Enter Zenhub (demonstrate) – list the specific benefits – prioritzation, stages, epics and stories, estimations in a single easily updated place, reporting. Also given this for free because this was an open source project! We reached an agreement with them
  25. 25. Zenhub Benefits • Track individual stories • Create Epics and relate stories • Assign tasks • Prioritize work • Track estimations • Report on progress
  26. 26. MSI Community: Weekly Demo Demos are held for the MSI project every week on Friday using Zoom. Any developer can attend the demo to see what is happening on the project. All demos are recorded and posted on YouTube.
  27. 27. MSI Community: Standup and Grooming Daily standup and recurring grooming meetings are optional for contributors. In these meetings we discuss issues in more detail and plan stories. All meetings are conducted using Zoom to include remote attendees.
  28. 28. Testing MSI using MFTF* • 1st Community Extension to utilize MFTF • Results to date: – Discovered gaps in MFTF which are being addressed – Finalized the location of Test Folders for Modules: – Discovered gaps in our Jenkins/Bamboo builds which are also being addressed – Created Multiple automated tests (30+) – Discovered several MSI bugs from the tests we created *Magento Functional Testing Framework Old: dev/tests/acceptance/tests/functional/Magento/FunctionalTest/[Module]/Test New: /app/code/Magento/[Module]/Test/Acceptance
  29. 29. Architect Product Manager UX Designer QA Lead Tech Writer Community Engineers How we supported the MSI project
  30. 30. 400 401 15 16 65 66 Submitted Pull Requests Individual Contributors Agencies Represented MSI Project Contributions interjar
  31. 31. Merchant Benefits of MSI • Manage multiple-location inventory from the admin – Familiar interface where they already manage orders, catalog, etc. • Assign products to sources and manage quantity in each source • Create Stocks to define sources eligible to deliver for each website • Control how warehouses are selected/assigned for delivery based on merchant preferences – Initial: simple priority algorithm (MVP) – Later: lowest cost algorithm by distance proxy • Manually control shipment Sources when override is necessary • API coverage for integration with 3rd party inventory systems, warehouses, or custom inventory actions • Inventory reservations for performant checkout and accurate quantity available for sale
  32. 32. Technical Overview @iminyaylo
  33. 33. Technical Overview • MSI is a brand new way to handle inventory – Event Sourcing + CQRS architecture – Scalable Multi-Dimensional indexes for Stocks – Usage of PHP 7.1 features – Highly modular design
  34. 34. Technical Overview • MSI is a brand new way to handle inventory – MSI is much more than just an extension, but a new way to handle inventory and reservations for all merchants – These changes will become part of core in 2.4 – Not just for merchants with >1 source
  35. 35. Size of MSI Project • 25 modules created in the scope of the project
  36. 36. Inventory Reservations in MSI • When an order is placed, a reservation is made to ensure there will be enough quantity available to fulfill the order • Reservations are append only operations to prevent blocking operations and race conditions at checkout • The reservations table is periodically cleaned of reservations that sum to zero Merchant Benefits Performant checkout even with high concurrent sessions Prevents overselling available inventory
  37. 37. Reservation Example – Step 1 France Warehouse SKU-1: 5 qty EU Stock SKU-1: 15 qty Italy Warehouse SKU-1: 10 qty Raw data Index Reservations Salable Qty: 15 (data from index, empty reservations)
  38. 38. Reservation Example – Step 2 Action: Customer buys 5 products on frontend (SKU-1)
  39. 39. Reservation Example – Step 3 France Warehouse SKU-1: 5 qty EU Stock SKU-1: 15 qty Italy Warehouse SKU-1: 10 qty Raw data Index Reservations Salable Qty: 10 (data from index: 15, apply all reservations: -5) Data is NOT changed SKU-1: -5 Order #001 Reservation has been added
  40. 40. Reservation Example – Step 4 Actions: From the original order of 5 products, 3 are returned
  41. 41. Reservation Example – Step 5 France Warehouse SKU-1: 5 qty EU Stock SKU-1: 15 qty Italy Warehouse SKU-1: 10 qty Raw data Index Reservations Salable Qty: 13 (data from index: 15, apply all reservations: -5+3) Data is NOT changed Reservation has been added SKU-1: -5 SKU-1: +3 Order #001 Return #001
  42. 42. Reservation Example – Step 6 Actions: • Admin ships remaining items in order (qty 2) • Reindex on shipment of order
  43. 43. Reservation Example – Step 7 France Warehouse SKU-1: 3 qty EU Stock SKU-1: 13 qty Italy Warehouse SKU-1: 10 qty Raw data Index Reservations Salable Qty: 13 (data from index: 13, apply all reservations: -5+3+2=0) Data is CHANGED SKU-1: -5 SKU-1: +3 SKU-1: +2 Order #001 Return #001 Order #001 Shipment Reservation has been added
  44. 44. Reservation_id Stock_id sku quantity metadata 1 2 SKU-1 -5.000 order_placed:order:1 2 2 SKU-1 3.000 order_cancelled:order:1 3 2 SKU-1 2.000 shipment_created:order:1 Reservation Example – Step 8 Action: • Reservation cleaning • Looping through these reservations we can find reservations which in sum return 0 (Zero) and remove them
  45. 45. Reservation Example – Step 9 (like Step 1) France Warehouse SKU-1: 3 qty EU Stock SKU-1: 13 qty Italy Warehouse SKU-1: 10 qty Raw data Index Reservations Salable Qty: 13 (data from index, empty reservations table) Data is NOT changed Reservations have been removed
  46. 46. Extension points and APIs in MSI • For integration with ERP and PIM systems – Inventory Sales API (Reservations placement) • For extension developers – Source Selection Algorithm APIs to provide own more efficient and business oriented choice of Sources for order fulfillment
  47. 47. Customizing Source Selection Algorithm
  48. 48. Project Summary @markabrinton
  49. 49. Project Challenges Challenge Mitigation Strategies Attracting developers Help developers get started at in-person events Offer incentives (complexity, recognition, collaboration) Retaining developers Groom stories into multiple sizes (S, M, L) to match developer availability Provide detailed feedback and code review Time zones and different working hours Slack channels to answer questions in real time, even when Magento team participants are offline Remote contributors Software tools: Zoom, YouTube, Slack, GitHub Distributed contribution day for virtual participation Estimating velocity (most difficult) Rough sizing based on events and attendees Regular estimates from consistent participants (only works in the short term)
  50. 50. Lessons Learned • Project management methods need to change with contributors and complexity • Experiment with different tools • Start new participants at events, then encourage regular contributions • Use voting to quickly refine priority and approach questions • Demos show progress, build momentum, and help newcomers • Define features partially and take community input
  51. 51. What’s next for Community Projects • Significant investment in Community Engineering • Coming soon: partner internal development teams with community contributors on core features + Austin, Texas 3 Kyiv, Ukraine 6 Frankfurt, Germany 1
  52. 52. Try it out! • MSI development code is already available • Clone on GitHub and follow our progress
  53. 53. Become a Contributor • Want to help with MSI or another community project? – Email Community Engineering to get started – Not an engineer? You can also contribute to documentation! – Other projects • Bulk API • Import / Export enhancements • PHP 7.2 Compatibility • And more!
  54. 54. Q & A