Presentation made by Mark Brinton - Product Owner of MSI Igor Miniailo - Architect of MSI who described the story, product backlog, architecture and community contribution on the Multi-Source Inventory project

2. Multi-Source
Inventory: The Largest
Community-Developed
Feature in Magento

3. Mark Brinton
Sr. Manager, Product | Magento Commerce
@markabrinton
Igor Miniailo
Software Engineer | Magento Commerce
@iminyaylo

4. Where did we start?
West
Central
East
Requests from
merchants and
partners
• Need Multiple
warehouses
• Can’t split
inventory for a
single product
Current state
• Single stock only
• Must customize or
extend
• Can be complex to
manage and
upgrade
Multiple types
needed
• Warehouses
• Stores
• Distribution
Centers
• Drop Shipping
Increasingly
common
• Even for smaller
merchants
• Applies to both
B2C and B2B

5. The MSI “Story”
Original artwork ©2012 Seth Wolfshornd

6. Validating the request
High interest and
common need
• Not an edge case
• Multiple validation signs
• B2C and B2B
• SMB and Enterprise
Suitable for core
development
• Impacts many modules
or areas of core
• Key part of the order
process or checkout
• Complex to customize
properly
Base for extension
• Drive standard approach
• Natural extension points
• Creates marketplace
opportunity
Top 3 in partner voting
Most commented issue
on GitHub
Top 5 on Magento
Forums
Many merchant types
and sizes
Inventory is complex
and benefits from a
consistent architecture
Impacts checkout,
shipment, returns, etc.
API integrations
common for inventory
Source Selection
algorithm logical to
customize

7. Let’s get started!
• Originally an internal project
• First requirements documented in September 2013
• Assigned to Dragons team
• Went through multiple name changes:
– Multiple Warehouse / Node / Source Inventory
• Final naming decision: Multi-Source Inventory
– Source is the most flexible name to handle multiple location types
– Consistent with naming approach used in Magento Order
Management

8. Original internal Epic MAGETWO-14308
Progress!
Source management
Assign products and
quantity to sources
Priority-based selection
algorithm for shipment

9. Software development is difficult
• Didn’t complete all stories for the feature
• With only one team assigned, velocity was not as high as we
needed to maintain pace
• Inventory is complex, with partial or incremental release
challenging
– Touches many areas of the core and multiple modules
– All aspects of merchant inventory process must be supported, from
purchasing to orders and through to shipment and returns
• Internal constraints on breaking changes while making major
changes to inventory created technical debt
• Other projects took higher priority

10. So, now what?
• Market research has not changed – the feature is still important
• How can we continue without internal teams assigned?
• Convenient timing
– Community Engineering Team created in February 2017
– Wanted to expand engagement with the community
– Interest from community members
– Have a documented feature ready, with some work completed (we
ended up starting from the beginning)

11. The community development model
Benefits for Magento
Improved velocity for new features
Localized expertise for inventory
management differences by market
Direct client perspective
More reviewers = faster to find bugs
Get feedback while building the
feature, not after release
Project Goals
Showcase the depth and talent of
community engineers
Improve our working process for
complex features with remote
developers
Increase developer engagement with
Magento
Reduce time to build without utilizing
full team of internal developers

12. But how can we recruit developers?
• All contributors will be volunteers
• Can’t assign work to anyone, they
must offer to help
• Need to make the opportunity
compelling for participants
• So, how to start?
– Join an existing community event
to build momentum for the project

13. Project Kickoff: Meet Magento Germany
Thank You!

14. What did we offer to participants?
• Partner with our developers and architects
• Learn Magento best practices
• Get listed as a contributor in release notes
• Intellectual challenge of a complex feature
• Chance to collaborate with other developers
and build community relationships
• Strengthen reputation with merchants
• A faster release of MSI for clients
• No more customizing inventory for clients
• Opportunity to give back to the community
Igor Miniailo@RicTempesta

15. Another essential ingredient: sweets!

16. Hackathon results

17. First Pull Requests accepted at MMDE

18. All events must come to an end
Ben Marks Mark Brinton

19. Remote project management process
• Started with
– Familiar to developers
– Avoided difficult process of granting
access to Magento JIRA instance
• Multiple features
– Code and source control
– Wiki for documentation
– Individual issue tracking
– Comments and collaboration
– Pull Requests for contributions
• Initially tracked stories in a wiki table

20. MSI Community: GitHub
Our Community
Engineering GitHub
repository contains all
code for the project and is
used to manage pull
requests, assign stories,
and track issues. Any
developer can see how
the project has developed
over time and submit pull
requests for review.
github.com/magento-engcom/msi/

21. MSI Community: Wiki
Our Community
Engineering wiki contains
documentation for the
project, including
technical vision,
architecture, definition for
each story, UX, and
roadmap. This enables
new developers to quickly
understand the project
and participate.
github.com/magento-engcom/msi/wiki

22. MSI Community: Slack
Our Community
Engineering Slack has
specific channels for MSI
questions and discussion.
Developers can get help
with the project and
collaborate together,
regardless of time zone.
magentocommeng.slack.com

23. Meet Magento Germany
Leipzig, Germany
Meet Magento Sweden
Stockholm, Sweden
Magento Live UK
London, United Kingdom
Mage Unconference
Utrecht, Netherlands
Magento Hackathon
Munich, Germany
Meet Magento Romania
Cluj, Romania
Meet Magento New York
New York City, USA
Mage Titans Austin
Austin, USA
MSI Contribution Week
Kolbermoor, Germany
Meet Magento Spain
Madrid, Spain
Mage Test Fest
Amersfoort, Netherlands
Atwix Contribution Day
Khmelnytskyi, Ukraine
Mage Titans Manchester
Manchester, United Kingdom
Amasty Contribution Day
Minsk, Belarus
Mage Conf Kiev
Kiev, Ukraine
2017 MSI Community Events

24. More contributors = more complexity
• As more contributors joined the project, it
became difficult to manage on the wiki
– Many additional stories created
– Issues manually linked together
– Not possible to report on stories
– Prioritization is not easily available
• Also, it was not clear where someone new
to the project could start participating

25. MSI Community: ZenHub
• Enter Zenhub
(demonstrate) – list
the specific benefits –
prioritzation, stages,
epics and stories,
estimations in a
single easily updated
place, reporting. Also
given this for free
because this was an
open source project!
We reached an
agreement with them
github.com/magento-engcom/msi/

26. Zenhub Benefits
• Track individual stories
• Create Epics and relate stories
• Assign tasks
• Prioritize work
• Track estimations
• Report on progress

27. MSI Community: Weekly Demo
Demos are held for the MSI
project every week on
Friday using Zoom. Any
developer can attend the
demo to see what is
happening on the project.
All demos are recorded and
posted on YouTube.
bit.ly/MSIdemo

28. MSI Community: Standup and Grooming
Daily standup and recurring
grooming meetings are
optional for contributors. In
these meetings we discuss
issues in more detail and
plan stories.
All meetings are conducted
using Zoom to include
remote attendees.

29. Testing MSI using MFTF*
• 1st Community Extension to utilize MFTF
• Results to date:
– Discovered gaps in MFTF which are being addressed
– Finalized the location of Test Folders for Modules:
– Discovered gaps in our Jenkins/Bamboo builds which
are also being addressed
– Created Multiple automated tests (30+)
– Discovered several MSI bugs from the tests we created
*Magento Functional Testing Framework
Old: dev/tests/acceptance/tests/functional/Magento/FunctionalTest/[Module]/Test
New: /app/code/Magento/[Module]/Test/Acceptance

30. Architect Product Manager UX Designer
QA Lead Tech Writer Community Engineers
How we supported the MSI project

31. 400
401
15
16
65
66 Submitted
Pull Requests
Individual
Contributors
Agencies
Represented
MSI Project Contributions
interjar

32. Merchant Benefits of MSI
• Manage multiple-location inventory from the admin
– Familiar interface where they already manage orders, catalog, etc.
• Assign products to sources and manage quantity in each source
• Create Stocks to define sources eligible to deliver for each website
• Control how warehouses are selected/assigned for delivery based on
merchant preferences
– Initial: simple priority algorithm (MVP)
– Later: lowest cost algorithm by distance proxy
• Manually control shipment Sources when override is necessary
• API coverage for integration with 3rd party inventory systems,
warehouses, or custom inventory actions
• Inventory reservations for performant checkout and accurate quantity
available for sale

33. Technical Overview
@iminyaylo

34. Technical Overview
• MSI is a brand new way to handle inventory
– Event Sourcing + CQRS architecture
– Scalable Multi-Dimensional indexes for Stocks
– Usage of PHP 7.1 features
– Highly modular design

35. Technical Overview
• MSI is a brand new way to handle inventory
– MSI is much more than just an extension, but a new way to handle
inventory and reservations for all merchants
– These changes will become part of core in 2.4
– Not just for merchants with >1 source

36. Size of MSI Project
• 25 modules created in the scope of the project

37. Inventory Reservations in MSI
• When an order is placed, a reservation is
made to ensure there will be enough
quantity available to fulfill the order
• Reservations are append only operations
to prevent blocking operations and race
conditions at checkout
• The reservations table is periodically
cleaned of reservations that sum to zero
Merchant Benefits
Performant checkout even with high concurrent sessions
Prevents overselling available inventory

38. Reservation Example – Step 1
France Warehouse
SKU-1: 5 qty
EU Stock
SKU-1: 15 qty
Italy Warehouse
SKU-1: 10 qty
Raw data Index Reservations
Salable Qty: 15 (data from index, empty reservations)

39. Reservation Example – Step 2
Action: Customer buys 5 products
on frontend (SKU-1)

40. Reservation Example – Step 3
France Warehouse
SKU-1: 5 qty
EU Stock
SKU-1: 15 qty
Italy Warehouse
SKU-1: 10 qty
Raw data Index Reservations
Salable Qty: 10 (data from index: 15, apply all reservations: -5)
Data is NOT changed
SKU-1: -5 Order #001
Reservation has been added

41. Reservation Example – Step 4
Actions: From the original order of 5 products, 3 are returned

42. Reservation Example – Step 5
France Warehouse
SKU-1: 5 qty
EU Stock
SKU-1: 15 qty
Italy Warehouse
SKU-1: 10 qty
Raw data Index Reservations
Salable Qty: 13 (data from index: 15, apply all reservations: -5+3)
Data is NOT changed Reservation has been added
SKU-1: -5
SKU-1: +3
Order #001
Return #001

43. Reservation Example – Step 6
Actions:
• Admin ships remaining items in order (qty 2)
• Reindex on shipment of order

44. Reservation Example – Step 7
France Warehouse
SKU-1: 3 qty
EU Stock
SKU-1: 13 qty
Italy Warehouse
SKU-1: 10 qty
Raw data Index Reservations
Salable Qty: 13 (data from index: 13, apply all reservations: -5+3+2=0)
Data is CHANGED
SKU-1: -5
SKU-1: +3
SKU-1: +2
Order #001
Return #001
Order #001 Shipment
Reservation has been added

45. Reservation_id Stock_id sku quantity metadata
1 2 SKU-1 -5.000 order_placed:order:1
2 2 SKU-1 3.000 order_cancelled:order:1
3 2 SKU-1 2.000 shipment_created:order:1
Reservation Example – Step 8
Action:
• Reservation cleaning
• Looping through these reservations we can find reservations
which in sum return 0 (Zero) and remove them

46. Reservation Example – Step 9 (like Step 1)
France Warehouse
SKU-1: 3 qty
EU Stock
SKU-1: 13 qty
Italy Warehouse
SKU-1: 10 qty
Raw data Index Reservations
Salable Qty: 13 (data from index, empty reservations table)
Data is NOT changed Reservations have been
removed

47. Extension points and APIs in MSI
• For integration with ERP and PIM systems
– Inventory Sales API (Reservations placement)
• For extension developers
– Source Selection Algorithm APIs to provide own more efficient and
business oriented choice of Sources for order fulfillment

48. Customizing Source Selection Algorithm

49. Project Summary
@markabrinton

50. Project Challenges
Challenge Mitigation Strategies
Attracting developers Help developers get started at in-person events
Offer incentives (complexity, recognition, collaboration)
Retaining developers Groom stories into multiple sizes (S, M, L) to match
developer availability
Provide detailed feedback and code review
Time zones and
different working hours
Slack channels to answer questions in real time, even
when Magento team participants are offline
Remote contributors Software tools: Zoom, YouTube, Slack, GitHub
Distributed contribution day for virtual participation
Estimating velocity
(most difficult)
Rough sizing based on events and attendees
Regular estimates from consistent participants
(only works in the short term)

51. Lessons Learned
• Project management methods
need to change with
contributors and complexity
• Experiment with different tools
• Start new participants at
events, then encourage regular
contributions
• Use voting to quickly refine
priority and approach questions
• Demos show progress, build momentum, and help newcomers
• Define features partially and take community input

52. What’s next for Community Projects
• Significant investment in Community Engineering
• Coming soon: partner internal development teams with
community contributors on core features
+
Austin, Texas
3
Kyiv, Ukraine
6
Frankfurt, Germany
1

53. Try it out!
• MSI development code
is already available
• Clone on GitHub and
follow our progress
github.com/magento-engcom/msi/wiki

54. Become a Contributor
• Want to help with MSI or another community project?
– Email Community Engineering to get started
– Not an engineer? You can also contribute to documentation!
– Other projects
• Bulk API
• Import / Export enhancements
• PHP 7.2 Compatibility
• And more!
engcom@magento.com

55. Q & A