Presentation made by
Mark Brinton - Product Owner of MSI
Igor Miniailo - Architect of MSI
who described the story, product backlog, architecture and community contribution on the Multi-Source Inventory project
Multi-Source Inventory: The Largest Community-Developed Feature in Magento. Brand new inventory management approach which is about to substitute existing CatalogInventory mechanism in Magento2.
Multi Source Inventory (MSI) in Magento 2 Igor Miniailo
Multi Source Inventory (MSI) community project for Magento 2 made on MagentoLive UK 2017
MSI project - https://github.com/magento-engcom/msi
MSI project wiki - https://github.com/magento-engcom/msi/wiki
MSI project board - https://github.com/magento-engcom/msi/projects/1
A long way from Monolith to Service Isolated Architecture #MM19NLIgor Miniailo
Problem:
The application grows in size. Becomes harder to understand
Merchants customization grows in complexity
This pushes Magento for longer release cycles
Harder for merchants to experiment and react to market needs quickly
Solution:
Service Isolation
The long way from Monolith to MicroservicesIgor Miniailo
Agenda:
Magento 1. Big ball of Mud
SOLID and Single Responsibility
Performance vs Scalability
Coupling vs Cohesion
Bounded contexts and their Boundaries
CAP Theorem
ACID vs BASE
Modularity as a first step towards micro-services
Headless Magento
Service Isolation (Code and DB split)
Conway's law
Microservice architecture
Multi-Source Inventory: The Largest Community-Developed Feature in Magento. Brand new inventory management approach which is about to substitute existing CatalogInventory mechanism in Magento2.
Multi Source Inventory (MSI) in Magento 2 Igor Miniailo
Multi Source Inventory (MSI) community project for Magento 2 made on MagentoLive UK 2017
MSI project - https://github.com/magento-engcom/msi
MSI project wiki - https://github.com/magento-engcom/msi/wiki
MSI project board - https://github.com/magento-engcom/msi/projects/1
A long way from Monolith to Service Isolated Architecture #MM19NLIgor Miniailo
Problem:
The application grows in size. Becomes harder to understand
Merchants customization grows in complexity
This pushes Magento for longer release cycles
Harder for merchants to experiment and react to market needs quickly
Solution:
Service Isolation
The long way from Monolith to MicroservicesIgor Miniailo
Agenda:
Magento 1. Big ball of Mud
SOLID and Single Responsibility
Performance vs Scalability
Coupling vs Cohesion
Bounded contexts and their Boundaries
CAP Theorem
ACID vs BASE
Modularity as a first step towards micro-services
Headless Magento
Service Isolation (Code and DB split)
Conway's law
Microservice architecture
The value of a software comes from implementing particular business requirements. Business is interested neither in correct abstractions nor in code reuse. But business is interested in changes and cost of these changes.
The key to writing easy to evolve, extensible and customizable software is strictly defined component dependencies and boundaries.
This presentation considers several best practices that help to write high-quality maintainable Magento 2 modules as well as increase a value of contributed bug fixes and core improvements delivered by Magento Community with minimal code overhead.
Jacopo Nardiello - From CI to Prod: Running Magento at scale with KubernetesMeet Magento Italy
Developing and running web applications has become much more than just writing code. Business and engineers are now required to solve a new set of challenges which range from microservices architectures to set up software delivery pipelines, while releasing constantly new features. In this context, automation is key.
With Container-based architectures and Kubernetes we are on the verge of a copernican revolution in the way we handle operations and design our applications. The so called Cloud-Native stack open doors to unprecedented resilience, scalability and efficiency but the price to pay is complexity: Containers are a fundamental shift which require to re-think how components of your applications and infrastructure are developed and deployed.
In this talk we discussed the Domain specific concepts that power Kubernetes and how they apply to Magento. We immersed ourselves deeply on the architectural choices and challenges of running a production ready Magento deployment on Kubernetes and we will see hands-on how to fully automate the deployment lifecycle, web applications scaling (reducing scaling time from minutes to seconds), failures handling and self-healing.
The speech has been held with Simone Fantini -System Administrator & Magento Specialist – neen
Magento PWA Studio not just for Magento: it’s for making all kinds of Progressive Web Apps. And it’s not just for PWAs: it’s for helping developers to get work done. Though we conceived PWA Studio as the way to create PWAs that belong in the extensible Magento ecosystem, we are building it to be useful for any frontend developer, inside or outside the world of Magento.
Our standard app architecture uses beloved modern frontend ingredients. You may already know them; if not, you can learn them from a large online community. As Magento joins the busy world of modern Web frontend, you can join us there too.
Come see the many ways you can use PWA Studio tools and the many ways you can contribute to them.
Monitoring your cache effectiveness in Magento 2Tony Brown
The Full Page Cache in Magento 2 uses Varnish and is an essential element to empower a performant and scalable solution.
But how effective is this cache on your site? It is not always that easy to understand or see, I will discuss approaches to obtain relevant data, and methods to process this into meaningful and actionable information.
[WSO2 Summit EMEA 2020] Accelerate and Secure Services Integration with WSO2 ...WSO2
API management enabled La Poste BSCC to set up a service oriented architecture and secure access to our information system with standard and custom patterns. The discussion also explores how an API strategy can accelerate new service delivery and integration with partners, facilitating access to new business avenues. Jean-Marc will also provide some tips on issues addressed at La Poste, how the organization worked with WSO2 technology, and details of their platform roadmap.
Watch the session on-demand here: https://wso2.com/library/summit-2020/emea/accelerate-and-secure-services-integration-with-wso2-api-manager/
API-Centric Hybrid Integration Platform for Microservices or ESB Style Archit...WSO2
With the increased usage of SaaS applications, the need for hybrid integration has become a key element in any integration project. Therefore, an integration platform should support both on-premise and hybrid integration capabilities.
This deck will explore the following in detail:
- Hybrid integration platforms and what elements WSO2 Integration solutions provide to cater to modern requirements.
- The latest trends and best practices to follow when designing an API-centric integration platform.
- How hybrid integration can be accommodated in a microservices-based architecture or tried and tested centralized ESB style architecture.
Watch the webinar on-demand here: https://wso2.com/library/webinars/2020/03/api-centric-hybrid-integration-platform-for-microservices-or-esb-style-architecture/
Analysts and leading industry surveys have found more and more banks, even in countries with an absence of open banking regulation, have prioritized implementing open banking to fast-track digital transformation and achieve business goals. This means, to stand out in the crowd in mature open banking ecosystems, and to secure a lasting competitive advantage as an early adopter in new markets, banks should select open banking technology that delivers advanced capabilities and scalability backed by a strong vision and industry-understanding.
In this release webinar, you will learn how WSO2 Open Banking 2.0 improves the way we help your developers and business teams create, quickly deploy, manage and monetize APIs that add real value for your internal teams, partners, and consumers. We will also help you understand how our technology can be best deployed as a part of a successful open banking strategy.
[WSO2 Summit EMEA 2020] Building an Interactive API MarketplaceWSO2
In an API-driven world, consumers want to discover APIs while producers seek to list their APIs and API products in a thriving API ecosystem.
The primary goal of an API marketplace is to create a platform that supports this high-intensive interaction seamlessly while also transforming the technical definition of a consumer and a producer into a natural business experience, which happens between a buyer and a seller.
This session will present the capabilities of an API marketplace, how it can be used for all APIs at different levels in an organization, and how it easily falls into place in an Integrated API Supply Chain.
Watch the session on-demand here: https://wso2.com/library/summit-2020/emea/building-an-interactive-api-marketplace/
Introduction to Versioning (VCS) for DevelopersAmpersand
Introduction to Versioning: SVN, Git
•Manage your code efficiently
•Develop multiple features at the same time
•Allow you to go back if disaster strikes
•Know exactly what was changed, by who and when
•Code reviews
•Continuous integration
•Share code
•Deployments
The value of a software comes from implementing particular business requirements. Business is interested neither in correct abstractions nor in code reuse. But business is interested in changes and cost of these changes.
The key to writing easy to evolve, extensible and customizable software is strictly defined component dependencies and boundaries.
This presentation considers several best practices that help to write high-quality maintainable Magento 2 modules as well as increase a value of contributed bug fixes and core improvements delivered by Magento Community with minimal code overhead.
Jacopo Nardiello - From CI to Prod: Running Magento at scale with KubernetesMeet Magento Italy
Developing and running web applications has become much more than just writing code. Business and engineers are now required to solve a new set of challenges which range from microservices architectures to set up software delivery pipelines, while releasing constantly new features. In this context, automation is key.
With Container-based architectures and Kubernetes we are on the verge of a copernican revolution in the way we handle operations and design our applications. The so called Cloud-Native stack open doors to unprecedented resilience, scalability and efficiency but the price to pay is complexity: Containers are a fundamental shift which require to re-think how components of your applications and infrastructure are developed and deployed.
In this talk we discussed the Domain specific concepts that power Kubernetes and how they apply to Magento. We immersed ourselves deeply on the architectural choices and challenges of running a production ready Magento deployment on Kubernetes and we will see hands-on how to fully automate the deployment lifecycle, web applications scaling (reducing scaling time from minutes to seconds), failures handling and self-healing.
The speech has been held with Simone Fantini -System Administrator & Magento Specialist – neen
Magento PWA Studio not just for Magento: it’s for making all kinds of Progressive Web Apps. And it’s not just for PWAs: it’s for helping developers to get work done. Though we conceived PWA Studio as the way to create PWAs that belong in the extensible Magento ecosystem, we are building it to be useful for any frontend developer, inside or outside the world of Magento.
Our standard app architecture uses beloved modern frontend ingredients. You may already know them; if not, you can learn them from a large online community. As Magento joins the busy world of modern Web frontend, you can join us there too.
Come see the many ways you can use PWA Studio tools and the many ways you can contribute to them.
Monitoring your cache effectiveness in Magento 2Tony Brown
The Full Page Cache in Magento 2 uses Varnish and is an essential element to empower a performant and scalable solution.
But how effective is this cache on your site? It is not always that easy to understand or see, I will discuss approaches to obtain relevant data, and methods to process this into meaningful and actionable information.
[WSO2 Summit EMEA 2020] Accelerate and Secure Services Integration with WSO2 ...WSO2
API management enabled La Poste BSCC to set up a service oriented architecture and secure access to our information system with standard and custom patterns. The discussion also explores how an API strategy can accelerate new service delivery and integration with partners, facilitating access to new business avenues. Jean-Marc will also provide some tips on issues addressed at La Poste, how the organization worked with WSO2 technology, and details of their platform roadmap.
Watch the session on-demand here: https://wso2.com/library/summit-2020/emea/accelerate-and-secure-services-integration-with-wso2-api-manager/
API-Centric Hybrid Integration Platform for Microservices or ESB Style Archit...WSO2
With the increased usage of SaaS applications, the need for hybrid integration has become a key element in any integration project. Therefore, an integration platform should support both on-premise and hybrid integration capabilities.
This deck will explore the following in detail:
- Hybrid integration platforms and what elements WSO2 Integration solutions provide to cater to modern requirements.
- The latest trends and best practices to follow when designing an API-centric integration platform.
- How hybrid integration can be accommodated in a microservices-based architecture or tried and tested centralized ESB style architecture.
Watch the webinar on-demand here: https://wso2.com/library/webinars/2020/03/api-centric-hybrid-integration-platform-for-microservices-or-esb-style-architecture/
Analysts and leading industry surveys have found more and more banks, even in countries with an absence of open banking regulation, have prioritized implementing open banking to fast-track digital transformation and achieve business goals. This means, to stand out in the crowd in mature open banking ecosystems, and to secure a lasting competitive advantage as an early adopter in new markets, banks should select open banking technology that delivers advanced capabilities and scalability backed by a strong vision and industry-understanding.
In this release webinar, you will learn how WSO2 Open Banking 2.0 improves the way we help your developers and business teams create, quickly deploy, manage and monetize APIs that add real value for your internal teams, partners, and consumers. We will also help you understand how our technology can be best deployed as a part of a successful open banking strategy.
[WSO2 Summit EMEA 2020] Building an Interactive API MarketplaceWSO2
In an API-driven world, consumers want to discover APIs while producers seek to list their APIs and API products in a thriving API ecosystem.
The primary goal of an API marketplace is to create a platform that supports this high-intensive interaction seamlessly while also transforming the technical definition of a consumer and a producer into a natural business experience, which happens between a buyer and a seller.
This session will present the capabilities of an API marketplace, how it can be used for all APIs at different levels in an organization, and how it easily falls into place in an Integrated API Supply Chain.
Watch the session on-demand here: https://wso2.com/library/summit-2020/emea/building-an-interactive-api-marketplace/
Introduction to Versioning (VCS) for DevelopersAmpersand
Introduction to Versioning: SVN, Git
•Manage your code efficiently
•Develop multiple features at the same time
•Allow you to go back if disaster strikes
•Know exactly what was changed, by who and when
•Code reviews
•Continuous integration
•Share code
•Deployments
XP teams try to keep systems fully integrated at all times, and shorten the feedback cycle to minutes and hours instead of weeks or months. The sooner you know, the sooner you can adapt.
Watch our record for the webinar "Continuous Integration" to explore how Azure DevOps helps us in achieving continuous feedback using continuous integration.
Git in the Enterprise: How to succeed at DevOps using Git and a monorepoGina Bustos
Join GitLab VP of Product Job van der Voort and Perforce VP of Community Matt Attaway to hear lessons learned from their customers on how to succeed with DevOps using Git and a monorepo.
In this webinar we'll discuss:
- Approaches for managing teams and projects using Git
- Techniques for handling issue tracking across dependent projects
- Methods for automating testing and deployment
- Strategies for bringing enterprise scale, cross-project visibility and security to teams using Git
Git in the Enterprise: How to succeed at DevOps using Git and a monorepo Perforce
Join GitLab VP of Product Job van der Voort and Perforce VP of Community Matt Attaway to hear lessons learned from their customers on how to succeed with DevOps using Git and a monorepo.
In this webinar we'll discuss:
- Approaches for managing teams and projects using Git
- Techniques for handling issue tracking across dependent projects
- Methods for automating testing and deployment
- Strategies for bringing enterprise scale, cross-project visibility and security to teams using Git
This presentation will be useful to those
who would like to get acquainted with lifetime history
of successful monolithic Java application.
It shows architectural and technical evolution of one Java web startup that is beyond daily coding routine and contains a lot of simplifications, Captain Obvious and internet memes.
But this presentation is not intended for monolithic vs. micro services architectures comparison.
A journey of a FinTech application into the cloud, starting as a monolith and growing to MicroServices. Why going functional is so good for finance technology, and how Scala helped us build a better application.
How to overcome challenges in it system evolutionGrupa Unity
This presentation shows the problems and challenges in IT system evolution and the ways to deal with them, from Architect & Team Leader and Project & Portfolio Manager perspective
These are my summarized notes from all the microservices session I attended at QCon 2015. These sessions had tons of learning around how to scale microservices and avoid common pitfalls
Cosa ci aspetta in Visual Studio 2015 e Team Foundation Server 2015? La nuova versione della piattaforma di ALM e' in arrivo, con molte nuove funzionalita' per sviluppatori e non. In questa sessione faremo una panoramica di cio' che ci arrivera'a con la nuova release, con tutte le novita' per sviluppo, metodologie e DevOps.
Big ideas in small packages - How microservices helped us to scale our visionSebastian Schleicher
Verifying Blinkist's product and business visions with a monolithic and later microservice based approach. This talk was held at the Enterprise Architecture Connect Day 2017 at Adidas Global Headquarters.
Out With the Old, in With the Open-source: Brainshark's Complete CMS MigrationAcquia
Choosing the right Content Management System (CMS) for your business requires a lot of thought, research and evaluation. Are you getting enough site monitoring and support? Is the workflow user-friendly? Is the environment secure?
Brainshark, the leader in sales productivity solutions, needed a new CMS to support its business goals. The flexible, supportive framework and user-friendly interface of Drupal, combined with the availability, scalability and security of Acquia made for a platform that correlated directly with their business needs.
In this webinar, you will learn how Brainshark accomplished a successful migration, including topics such as:
-Their evaluation process for a new CMS, and key criteria they could not overlook
-Their migration strategy, execution and lessons learned
-The success they've seen thus far, and the results they're expecting
Architecting for Huper Growth and Great Engineering CultureSARCCOM
Architecting for Huper Growth and Great Engineering Culture
by Ifnu Bima
@SARCCOM MEETUP
http://sarccom.org
https://www.meetup.com/Software-Architect-Indonesia
Similar to Multi-Source Inventory. Imagine. Las Vegas. 2018 (20)
General Overview of Magento Storefront Application efforts
* Magento 2.3 architecture
* Magento Storefront components decomposition
* Standalone GraphQL Server
* Alternative checkout flow
* Price Books
* Bi-directional data flow
* Staging
* Protobuf and language-agnostic data structure and services description
* Out of process extensibility
Magento 2 Automated Testing via examples of Multi-Source Inventory (MSI)Igor Miniailo
- Unit Testing
- Tautological Unit testing
- Integration testing
- Web API
- Functional testing
- Static testing (LiveCoding)
- Examples of test coverage in Magento Multi-Source Inventory
Testing in Magento 2 by examples of Multi-Source Inventory Project (MSI)
- Unit Testing and how to do them right.
- Tautological Test Driven Development
- Integration Testing
- Example of Reservation mechanism in MSI and Integration test coverage for it
- Web API testing (REST, SOAP)
- Magento MSI Service Layer
- Public and Private code
- API vs SPI (Service Provider Interface)
- Backward Compatibility requirements and prohibited code changes
- Refactoring
- Functional programming and functors
- Single-method immutable objects
- Examples of Good Interfaces
- Why Execute but not __invoke
- Magento Repositories
Architecture and workflow of Multi-Source InventoryIgor Miniailo
Magento Multi-Source Inventory (MSI) architecture overview.
- Description of Reservation mechanism
- How MSI will improve Magento checkout performance
- Business Value of MSI Minimal Viable Product
Backward Compatibility Developer's Guide in Magento 2Igor Miniailo
Presentation made on Meet Magento Croatia 2017
- Why Backward Compatibility matters
- Public vs Private code
- Semantic Versioning and Dependency Rules
- APIs vs SPIs (Extension Points) concept
- Prohibited Code changes
- How to make Refactoring complying with Backward Compatibility policy (SuppressWarning Coupling Between Objects)
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Mind IT Systems
Healthcare providers often struggle with the complexities of chronic conditions and remote patient monitoring, as each patient requires personalized care and ongoing monitoring. Off-the-shelf solutions may not meet these diverse needs, leading to inefficiencies and gaps in care. It’s here, custom healthcare software offers a tailored solution, ensuring improved care and effectiveness.
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTier1 app
Even though at surface level ‘java.lang.OutOfMemoryError’ appears as one single error; underlyingly there are 9 types of OutOfMemoryError. Each type of OutOfMemoryError has different causes, diagnosis approaches and solutions. This session equips you with the knowledge, tools, and techniques needed to troubleshoot and conquer OutOfMemoryError in all its forms, ensuring smoother, more efficient Java applications.
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
Software Engineering, Software Consulting, Tech Lead.
Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Security,
Spring Transaction, Spring MVC,
Log4j, REST/SOAP WEB-SERVICES.
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Experience our free, in-depth three-part Tendenci Platform Corporate Membership Management workshop series! In Session 1 on May 14th, 2024, we began with an Introduction and Setup, mastering the configuration of your Corporate Membership Module settings to establish membership types, applications, and more. Then, on May 16th, 2024, in Session 2, we focused on binding individual members to a Corporate Membership and Corporate Reps, teaching you how to add individual members and assign Corporate Representatives to manage dues, renewals, and associated members. Finally, on May 28th, 2024, in Session 3, we covered questions and concerns, addressing any queries or issues you may have.
For more Tendenci AMS events, check out www.tendenci.com/events
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxrickgrimesss22
Discover the essential features to incorporate in your Winzo clone app to boost business growth, enhance user engagement, and drive revenue. Learn how to create a compelling gaming experience that stands out in the competitive market.
top nidhi software solution freedownloadvrstrong314
This presentation emphasizes the importance of data security and legal compliance for Nidhi companies in India. It highlights how online Nidhi software solutions, like Vector Nidhi Software, offer advanced features tailored to these needs. Key aspects include encryption, access controls, and audit trails to ensure data security. The software complies with regulatory guidelines from the MCA and RBI and adheres to Nidhi Rules, 2014. With customizable, user-friendly interfaces and real-time features, these Nidhi software solutions enhance efficiency, support growth, and provide exceptional member services. The presentation concludes with contact information for further inquiries.
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfJay Das
With the advent of artificial intelligence or AI tools, project management processes are undergoing a transformative shift. By using tools like ChatGPT, and Bard organizations can empower their leaders and managers to plan, execute, and monitor projects more effectively.
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Globus
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
3. Mark Brinton
Sr. Manager, Product | Magento Commerce
@markabrinton
Igor Miniailo
Software Engineer | Magento Commerce
@iminyaylo
4. Where did we start?
West
Central
East
Requests from
merchants and
partners
• Need Multiple
warehouses
• Can’t split
inventory for a
single product
Current state
• Single stock only
• Must customize or
extend
• Can be complex to
manage and
upgrade
Multiple types
needed
• Warehouses
• Stores
• Distribution
Centers
• Drop Shipping
Increasingly
common
• Even for smaller
merchants
• Applies to both
B2C and B2B
6. Validating the request
High interest and
common need
• Not an edge case
• Multiple validation signs
• B2C and B2B
• SMB and Enterprise
Suitable for core
development
• Impacts many modules
or areas of core
• Key part of the order
process or checkout
• Complex to customize
properly
Base for extension
• Drive standard approach
• Natural extension points
• Creates marketplace
opportunity
Top 3 in partner voting
Most commented issue
on GitHub
Top 5 on Magento
Forums
Many merchant types
and sizes
Inventory is complex
and benefits from a
consistent architecture
Impacts checkout,
shipment, returns, etc.
API integrations
common for inventory
Source Selection
algorithm logical to
customize
7. Let’s get started!
• Originally an internal project
• First requirements documented in September 2013
• Assigned to Dragons team
• Went through multiple name changes:
– Multiple Warehouse / Node / Source Inventory
• Final naming decision: Multi-Source Inventory
– Source is the most flexible name to handle multiple location types
– Consistent with naming approach used in Magento Order
Management
8. Original internal Epic MAGETWO-14308
Progress!
Source management
Assign products and
quantity to sources
Priority-based selection
algorithm for shipment
9. Software development is difficult
• Didn’t complete all stories for the feature
• With only one team assigned, velocity was not as high as we
needed to maintain pace
• Inventory is complex, with partial or incremental release
challenging
– Touches many areas of the core and multiple modules
– All aspects of merchant inventory process must be supported, from
purchasing to orders and through to shipment and returns
• Internal constraints on breaking changes while making major
changes to inventory created technical debt
• Other projects took higher priority
10. So, now what?
• Market research has not changed – the feature is still important
• How can we continue without internal teams assigned?
• Convenient timing
– Community Engineering Team created in February 2017
– Wanted to expand engagement with the community
– Interest from community members
– Have a documented feature ready, with some work completed (we
ended up starting from the beginning)
11. The community development model
Benefits for Magento
Improved velocity for new features
Localized expertise for inventory
management differences by market
Direct client perspective
More reviewers = faster to find bugs
Get feedback while building the
feature, not after release
Project Goals
Showcase the depth and talent of
community engineers
Improve our working process for
complex features with remote
developers
Increase developer engagement with
Magento
Reduce time to build without utilizing
full team of internal developers
12. But how can we recruit developers?
• All contributors will be volunteers
• Can’t assign work to anyone, they
must offer to help
• Need to make the opportunity
compelling for participants
• So, how to start?
– Join an existing community event
to build momentum for the project
14. What did we offer to participants?
• Partner with our developers and architects
• Learn Magento best practices
• Get listed as a contributor in release notes
• Intellectual challenge of a complex feature
• Chance to collaborate with other developers
and build community relationships
• Strengthen reputation with merchants
• A faster release of MSI for clients
• No more customizing inventory for clients
• Opportunity to give back to the community
Igor Miniailo@RicTempesta
19. Remote project management process
• Started with
– Familiar to developers
– Avoided difficult process of granting
access to Magento JIRA instance
• Multiple features
– Code and source control
– Wiki for documentation
– Individual issue tracking
– Comments and collaboration
– Pull Requests for contributions
• Initially tracked stories in a wiki table
20. MSI Community: GitHub
Our Community
Engineering GitHub
repository contains all
code for the project and is
used to manage pull
requests, assign stories,
and track issues. Any
developer can see how
the project has developed
over time and submit pull
requests for review.
github.com/magento-engcom/msi/
21. MSI Community: Wiki
Our Community
Engineering wiki contains
documentation for the
project, including
technical vision,
architecture, definition for
each story, UX, and
roadmap. This enables
new developers to quickly
understand the project
and participate.
github.com/magento-engcom/msi/wiki
22. MSI Community: Slack
Our Community
Engineering Slack has
specific channels for MSI
questions and discussion.
Developers can get help
with the project and
collaborate together,
regardless of time zone.
magentocommeng.slack.com
23. Meet Magento Germany
Leipzig, Germany
Meet Magento Sweden
Stockholm, Sweden
Magento Live UK
London, United Kingdom
Mage Unconference
Utrecht, Netherlands
Magento Hackathon
Munich, Germany
Meet Magento Romania
Cluj, Romania
Meet Magento New York
New York City, USA
Mage Titans Austin
Austin, USA
MSI Contribution Week
Kolbermoor, Germany
Meet Magento Spain
Madrid, Spain
Mage Test Fest
Amersfoort, Netherlands
Atwix Contribution Day
Khmelnytskyi, Ukraine
Mage Titans Manchester
Manchester, United Kingdom
Amasty Contribution Day
Minsk, Belarus
Mage Conf Kiev
Kiev, Ukraine
2017 MSI Community Events
24. More contributors = more complexity
• As more contributors joined the project, it
became difficult to manage on the wiki
– Many additional stories created
– Issues manually linked together
– Not possible to report on stories
– Prioritization is not easily available
• Also, it was not clear where someone new
to the project could start participating
25. MSI Community: ZenHub
• Enter Zenhub
(demonstrate) – list
the specific benefits –
prioritzation, stages,
epics and stories,
estimations in a
single easily updated
place, reporting. Also
given this for free
because this was an
open source project!
We reached an
agreement with them
github.com/magento-engcom/msi/
26. Zenhub Benefits
• Track individual stories
• Create Epics and relate stories
• Assign tasks
• Prioritize work
• Track estimations
• Report on progress
27. MSI Community: Weekly Demo
Demos are held for the MSI
project every week on
Friday using Zoom. Any
developer can attend the
demo to see what is
happening on the project.
All demos are recorded and
posted on YouTube.
bit.ly/MSIdemo
28. MSI Community: Standup and Grooming
Daily standup and recurring
grooming meetings are
optional for contributors. In
these meetings we discuss
issues in more detail and
plan stories.
All meetings are conducted
using Zoom to include
remote attendees.
29. Testing MSI using MFTF*
• 1st Community Extension to utilize MFTF
• Results to date:
– Discovered gaps in MFTF which are being addressed
– Finalized the location of Test Folders for Modules:
– Discovered gaps in our Jenkins/Bamboo builds which
are also being addressed
– Created Multiple automated tests (30+)
– Discovered several MSI bugs from the tests we created
*Magento Functional Testing Framework
Old: dev/tests/acceptance/tests/functional/Magento/FunctionalTest/[Module]/Test
New: /app/code/Magento/[Module]/Test/Acceptance
30. Architect Product Manager UX Designer
QA Lead Tech Writer Community Engineers
How we supported the MSI project
32. Merchant Benefits of MSI
• Manage multiple-location inventory from the admin
– Familiar interface where they already manage orders, catalog, etc.
• Assign products to sources and manage quantity in each source
• Create Stocks to define sources eligible to deliver for each website
• Control how warehouses are selected/assigned for delivery based on
merchant preferences
– Initial: simple priority algorithm (MVP)
– Later: lowest cost algorithm by distance proxy
• Manually control shipment Sources when override is necessary
• API coverage for integration with 3rd party inventory systems,
warehouses, or custom inventory actions
• Inventory reservations for performant checkout and accurate quantity
available for sale
34. Technical Overview
• MSI is a brand new way to handle inventory
– Event Sourcing + CQRS architecture
– Scalable Multi-Dimensional indexes for Stocks
– Usage of PHP 7.1 features
– Highly modular design
35. Technical Overview
• MSI is a brand new way to handle inventory
– MSI is much more than just an extension, but a new way to handle
inventory and reservations for all merchants
– These changes will become part of core in 2.4
– Not just for merchants with >1 source
36. Size of MSI Project
• 25 modules created in the scope of the project
37. Inventory Reservations in MSI
• When an order is placed, a reservation is
made to ensure there will be enough
quantity available to fulfill the order
• Reservations are append only operations
to prevent blocking operations and race
conditions at checkout
• The reservations table is periodically
cleaned of reservations that sum to zero
Merchant Benefits
Performant checkout even with high concurrent sessions
Prevents overselling available inventory
38. Reservation Example – Step 1
France Warehouse
SKU-1: 5 qty
EU Stock
SKU-1: 15 qty
Italy Warehouse
SKU-1: 10 qty
Raw data Index Reservations
Salable Qty: 15 (data from index, empty reservations)
40. Reservation Example – Step 3
France Warehouse
SKU-1: 5 qty
EU Stock
SKU-1: 15 qty
Italy Warehouse
SKU-1: 10 qty
Raw data Index Reservations
Salable Qty: 10 (data from index: 15, apply all reservations: -5)
Data is NOT changed
SKU-1: -5 Order #001
Reservation has been added
41. Reservation Example – Step 4
Actions: From the original order of 5 products, 3 are returned
42. Reservation Example – Step 5
France Warehouse
SKU-1: 5 qty
EU Stock
SKU-1: 15 qty
Italy Warehouse
SKU-1: 10 qty
Raw data Index Reservations
Salable Qty: 13 (data from index: 15, apply all reservations: -5+3)
Data is NOT changed Reservation has been added
SKU-1: -5
SKU-1: +3
Order #001
Return #001
43. Reservation Example – Step 6
Actions:
• Admin ships remaining items in order (qty 2)
• Reindex on shipment of order
44. Reservation Example – Step 7
France Warehouse
SKU-1: 3 qty
EU Stock
SKU-1: 13 qty
Italy Warehouse
SKU-1: 10 qty
Raw data Index Reservations
Salable Qty: 13 (data from index: 13, apply all reservations: -5+3+2=0)
Data is CHANGED
SKU-1: -5
SKU-1: +3
SKU-1: +2
Order #001
Return #001
Order #001 Shipment
Reservation has been added
45. Reservation_id Stock_id sku quantity metadata
1 2 SKU-1 -5.000 order_placed:order:1
2 2 SKU-1 3.000 order_cancelled:order:1
3 2 SKU-1 2.000 shipment_created:order:1
Reservation Example – Step 8
Action:
• Reservation cleaning
• Looping through these reservations we can find reservations
which in sum return 0 (Zero) and remove them
46. Reservation Example – Step 9 (like Step 1)
France Warehouse
SKU-1: 3 qty
EU Stock
SKU-1: 13 qty
Italy Warehouse
SKU-1: 10 qty
Raw data Index Reservations
Salable Qty: 13 (data from index, empty reservations table)
Data is NOT changed Reservations have been
removed
47. Extension points and APIs in MSI
• For integration with ERP and PIM systems
– Inventory Sales API (Reservations placement)
• For extension developers
– Source Selection Algorithm APIs to provide own more efficient and
business oriented choice of Sources for order fulfillment
50. Project Challenges
Challenge Mitigation Strategies
Attracting developers Help developers get started at in-person events
Offer incentives (complexity, recognition, collaboration)
Retaining developers Groom stories into multiple sizes (S, M, L) to match
developer availability
Provide detailed feedback and code review
Time zones and
different working hours
Slack channels to answer questions in real time, even
when Magento team participants are offline
Remote contributors Software tools: Zoom, YouTube, Slack, GitHub
Distributed contribution day for virtual participation
Estimating velocity
(most difficult)
Rough sizing based on events and attendees
Regular estimates from consistent participants
(only works in the short term)
51. Lessons Learned
• Project management methods
need to change with
contributors and complexity
• Experiment with different tools
• Start new participants at
events, then encourage regular
contributions
• Use voting to quickly refine
priority and approach questions
• Demos show progress, build momentum, and help newcomers
• Define features partially and take community input
52. What’s next for Community Projects
• Significant investment in Community Engineering
• Coming soon: partner internal development teams with
community contributors on core features
+
Austin, Texas
3
Kyiv, Ukraine
6
Frankfurt, Germany
1
53. Try it out!
• MSI development code
is already available
• Clone on GitHub and
follow our progress
github.com/magento-engcom/msi/wiki
54. Become a Contributor
• Want to help with MSI or another community project?
– Email Community Engineering to get started
– Not an engineer? You can also contribute to documentation!
– Other projects
• Bulk API
• Import / Export enhancements
• PHP 7.2 Compatibility
• And more!
engcom@magento.com
Hello everyone, and thank you for joining us today. My name is Mark Brinton. I’m the product manager at Magento responsible for the Multisource inventory project. I’m also joined today by Igor Miniailo, the architect for this feature. In today’s session, we will cover the story of the project and how we’ve worked with the community. Igor will also share details on some interesting technical aspects of what we have built.
But first, a little background. This project initially started because we heard many requests from merchants and partners asking for multi warehouse support in Magento. They told us we need multiple warehouses…
As a product manager, I like to summarize what we plan to build from the user’s perspective, which in this case is the merchant.
*READ CARD*
*click*
I also like to imagine what the merchant will feel like when they have this capability
Once we get any request for a feature from our users, we go through a standard validation process. We’re looking for a few key points. First, is there high interest in the feature, and is it something many merchants would benefit from? We want to make sure the request is not for an edge case scenario. To do that, we look for multiple validation signs. Does this appeal to both B2C and B2B merchants? …
With validation of the request complete, we moved to start building the feature.
And we made some progress. Here’s a screenshot of one of the original designs. We completed several important stories, including the ability to manage sources in the admin panel and assigning products and quantity to sources. We also built a priority-based selection algorithm for shipments.
However, we ran into some challenges at that point. We weren’t able to complete all the stories to release the feature.
At this point we were at a decision point. We knew the feature was important. Our market research showed merchants wanted it, but we didn’t have any internal teams available to assign to continue work. Fortunately for us however, this happened right around the time we created our Community Engineering Team…
Using community developers has several benefits for Magento
But, we still have the challenge of recruiting contributors for the project!
We started the project at a hackathon last May, just before Meet Magento Germany
But this isn’t just about how we as Magento benefit from community contributions. We worked hard to offer a compelling opportunity to participants. They had the chance to… Riccardo Tempesta at Mage Titans Italy
And of course, we kept the developers well fed in the process, which is always important. Special thanks to our sponsors on the slide here who helped out with this event.
So what happened at the hackathon? As is common for these events, we had to pitch our project against many other ideas. The good news is Multisource inventory received the most votes. I was very glad at this point, because I didn’t want to travel all the way to Leipzig and fail to recruit volunteers!
We accepted our first pull requests for the project at the event. As you can see from the slide here, we also recognized those contributors in a keynote presentation at the conference.
However, all events must come to and end. We had a great start to the project, but the real work of maintaining momentum started after we flew back to the office.
When you are in the same location, it’s easy to answer questions and discuss the best way to solve problems. But how would we do this remotely?
To manage the project remotely, we started with GitHub
Here's what it looks like. In our GitHub repository, users can always download the latest MSI code...
GitHub also has a wiki where we store project information.
We also created some channels on our Community Engineering Slack to allow for collaboration...
** preview of next slide **
Now with this basic framework in place, we needed to expand the number of contributors working on the project. Because a kickoff event had been so successful for us, we decided to double down on events and join as many as possible. Max, who runs the Community Engineering team, definitely earned a lot of frequent flyer miles to move things forward at this point
Here’s a list of events where we had MSI contributions in 2017 alone. We have continued this momentum in 2018.
The good news was we had more people contributing to the project. We were also successful in getting people who started contributing at an event to continue on the project after they returned home, with many working nights and weekends on contributions. This was good, but also introduced challenges
To solve this problem we looked for solutions that could improve what we were already doing on GitHub to manage the project. After some research we discovered ZenHub. ZenHub allows us to follow stories through from design to completion
There are also some other great benefits to ZenHub. Using ZenHub, we can
Another important thing we added was a weekly demo for the project. This allowed us to showcase the work that was being done, give individual developers the chance to be proud of their contributions, and help newcomers onboard to the project
We also added some additional meetings: a daily standup and regular grooming sessions.
Of course it's important to have high quality on any new feature we include in Magento. And MSI is the first community extension that uses the Magento Functional Testing Framework. We also develop test scenarios in a tool called Hiptest, which helps with automation. So far using this approach we also improved MFTF itself. We discovered several gaps in MFTF which are being addressed. We also....
To make the project successful, we also supported it with dedicated resources across several disciplines
I’m excited to announce the contributions for this project have exceeded our expectations. Overall, we’ve had contributions from 66 separate community members
All these contributions have created some great benefits for merchants. With MSI, merchants will be able to manage multiple location inventory from the admin panel.
And now I’ll turn it over to Igor, who will cover some of the interesting technical aspects of the MSI project
Add screenshot of cart here – make sure to use SKU-1 in screenshot
Add screenshot?
And now I’ll turn it back over to Mark, who will review some key learnings from the project
Of course the MSI project, like any project, hasn't been without challenges. Here are some of the main ones we encountered and how we tried to mitigate them. First, the challenge of attracting developers...
With the goal of continuous improvement, here are some of the lessons we have learned so far working on MSI. First, project management methods need to change as the number of contributors and the complexity of the project increases....
In the past year, we’ve made a significant investment in our Community Engineering effort. The team is now 10 people total, with 3 in our Austin office, 6 in Kiev, and one in Frankfurt Germany.
Because this is a community project, all our work is available on GitHub. Please download the code and try it out! You can follow the project progress as we go
And of course, we are always looking to welcome new contributors. If you'd like to contribute to MSI or any other community engineering project, send an email to engcom@magento.com
Q&A – for both product and architect
*We can answer both business and technical questions*