[Rakuten TechConf2014] [E-6] Rakuten Ichiba Globalization - Challenges and Solutions
•Download as PPTX, PDF•
0 likes•1,789 views
Rakuten Technology Conference 2014 "Rakuten Ichiba Globalization - Challenges and Solutions" Olivier Claquin (Rakuten)
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (19)
Similar to [Rakuten TechConf2014] [E-6] Rakuten Ichiba Globalization - Challenges and Solutions
Similar to [Rakuten TechConf2014] [E-6] Rakuten Ichiba Globalization - Challenges and Solutions (20)
More from Rakuten Group, Inc.
More from Rakuten Group, Inc. (20)
Recently uploaded
Recently uploaded (20)
[Rakuten TechConf2014] [E-6] Rakuten Ichiba Globalization - Challenges and Solutions
- 1. Rakuten Ichiba globalisation - Challenges and solutions Rakuten Technology Conference 2014 Oct. 25th 2014 Olivier Claquin Rakuten Ichiba Department http://global.rakuten.com
- 2. 2 Me Who I am French, please bear with my accent… What I was A consultant in system integration and Service Oriented Architecture What I am Since 2013, I am part of the E-Commerce Architecture group. I am currently working on integrating the Rakuten Group's e-commerce systems.
- 3. 3 The history of Rakuten Ichiba globalisation 1997: Opening of Japan Ichiba 2008: Opening of Taiwan Ichiba 2010-: Rakuten acquired several e-commerce companies around the world 2011: Japanese merchants can sell their products overseas (outside of Japan) 2012-: Rakuten started to open new E-commerce websites in new countries on a regular basis 2013-: Rakuten started to consolidate the group E-commerce systems
- 4. 4 Current Rakuten Ichiba marketplaces
- 5. 5 From a Japan-only platform to a global platform The challenges Part 1 – To a unified platform Part 2 – To a multi-language system Part 3 – To an expendable and extensible system Part 4 – To a global ‘one team’
- 6. From a Japan-only platform to a global platform Part 1 – To a unified platform Part 2 – To a multi-language system Part 3 – To an expendable and extensible system Part 4 – To a global ‘one team’
- 7. 7 The idea seems great but 1. Are the features even compatible from one system to another? 2. How well do we know our system? 3. How can we inter-connect systems?
- 8. 8 Borders between marketplace specific and global features • Each marketplace does things differently • Where does global start/end? => Our product management team manages case by case => Build a feature gaps backlog Image source: http://commons.wikimedia.org/wiki/File:Information1.svg
- 9. 9 Know yourself first • We have system metrics, but are they meaningful? • How can we compare 2 systems that use different technologies? => Go back to user stories
- 10. 10 And our user stories look like Shopper Merchant Buy a product Cancel an order Register as a member … Update product price Update order status … Mall UI Merchant UI Ichiba Backend
- 11. 11 Inter-connecting systems, but how? • In complex environments, integration often requires an additional layer (ESB, EAI, …) • It is quite nice, but we want to keep it simple and extensible => We chose to use HTTP based “APIs” in our system design • ESB: Enterprise Service Bus • EAI: Enterprise Application Integration
- 12. 12 Let’s go back in time… UI Application External System UI Application Data source Data source
- 13. 13 UI Application External System Data API Data source Data API Data source UI Application And fast forward
- 14. 14 UI Application And fast forward UI Application External System Data source Service API Data API Data API Data source Service API Caching Quality of Service …
- 15. 15 API • Did you say API? • How do we use them in our systems – They provide data to the presentation layer – Some level of abstraction is good! • How do they help integrating systems? – Simplify the technical integration
- 16. 16 We can go from System For marketplace A System For marketplace B ???
- 17. 17 To the following System A System B ? Item API Campaign API Order API Mall UI Item API Billing system ? ?
- 18. 18 Test: what, when, how? • Integration tests, but what to test and how? => 60% to 75 % of our UIs tests are automated => Elaborate end-to-end scenarios
- 19. From a Japan-only platform to a global platform Part 1 – To a unified platform Part 2 – To a multi-language system Part 3 – To an expendable and extensible system Part 4 – To a global ‘one team’
- 20. 20 Global Shopping • We want to help our merchants to sell their products in other markets => Therefore, we offer the ability to merchants to translate their items to different languages Fact: 97% of our translations are automatic!
- 21. 21 Translation IS rocket science • No translation engine is perfect so we are using a mix of several translation engines • Each of those engine has its specificities that we have to learn • We are also improving the quality of those translations (e.g.: katakana processing)
- 22. 22 Our next target: images 2個セット ５０％ＯＦＦ ポイント１０倍 Buy 1, get 1 free and x10 points
- 23. From a Japan-only platform to a global platform Part 1 – To a unified platform Part 2 – To a multi-language system Part 3 – To an expendable and extensible system Part 4 – To a global ‘one team’
- 24. 24 Expandable = we can scale • For our Global Platform, all our applications are deployed in an R-PaaS Platform • It abstracts the Operating System layer, as well as the Virtual Machine layer and the physical layer
- 25. 25 RPaaS modules rpaas instances my_application +1 Application Server O.S. (Linux) How does R-PaaS work? Load balancer Application WAR, Ruby… RPaaS modules Application Server Application WAR, Ruby… O.S. (Linux)
- 26. 26 How does R-PaaS work? RPaaS modules Application Server O.S. (Linux) Load balancer Application WAR, Ruby… RPaaS modules Application Server Application WAR, Ruby… O.S. (Linux) RPaaS modules Application Server Application WAR, Ruby… O.S. (Linux)
- 27. 27 Serving users from all over the world • We use Akamai for caching and serving customers all over the world • Because our system is based on HTTP calls, we can dispatch those components in different data centres • Our short term goal is to be able to do the same with our data storage layer => If the application supports it, scale out
- 28. 28 Extensible = we can add new features easily • As the system grows, how can we add new features? • How do I know that my new feature does not break anything? • How do I develop features that make a difference to the business? => We use event driven processing
- 29. 29 We have users and we have events Shoppers Merchants Buy a product Cancel an order Register as a member … Update product price Update order status … Mall UI Merchant UI Ichiba Backend An order has been created An order status has changed A product has been updated A new user has registered …
- 30. 30 So we have a database and we have an event bus Shoppers Merchants Buy a product Cancel an order Register as a member … Update product price Update order status … Mall UI Merchant UI Ichiba Backend An order has been created An order status has changed A product has been updated A new user has registered … Update product Update membership Update order …
- 31. 31 Events are meaningful, use them wisely • Search is expensive, generating / consuming events is not • For example, we use this pattern for: – invalidating cache – sending out notifications – rebuilding our search engine index • We can go further and bring new capabilities to our businesses
- 32. From a Japan-only platform to a global platform Part 1 – To a unified platform Part 2 – To a multi-language system Part 3 – To an expendable and extensible system Part 4 – To a global ‘one team’
- 33. 33 Language is only one problem In a global environment: • How do we assign tasks? • How do we share progress? • How do we make decisions? => Collaborative tools do help but mostly, we have to educate ourselves
- 34. 34 About the usefulness of a glossary… • Like I mentioned in part 1, each team knows their own system • The problem is, that when you have similar systems, you tend to make easy shortcuts • Those shortcuts create a lot of misunderstanding => Create a glossary
- 35. 35 About the usefulness of a glossary… Concept Meaning Glossary A table like this one Item An item is defined by a merchant […] Product A product is an item sold on a marketplace […] => Create a glossary
- 36. In a nutshell
- 37. 37 Today I suggested to Describe your system with user stories Automate your tests Offer automatic translation to merchants and shoppers Use APIs to make system integration easier Design your application for scaling out Use the event driven paradigm Learn how to work in a global environment Build a glossary
- 38. 38 olivier.claquin@mail.rakuten.com Thank you for your attention!
- 39. Bonus slides / Annexes
- 40. 40 Bonus slide – how do we track progress when working remotely • Atlassian on-demand and Atlassian on premise • Jira for issue, features tracking • Confluence for information sharing
Editor's Notes
- With all those expansions, we ended up building several E-Commerce systems. However, we are in the process of consolidating those systems. When consolidating systems that have been around from quite some time, you usually get into the following issues:
- Of course, from all marketplace perspective, the way they do business is different. However, if we want to provide a global platform, we have to make some concession. We try to make our global platform customisable by a marketplace but there are limits of what we can do. One of the big challenges of data migration is that the data topology can be different: one country could have an item with 20 images associated to it whereas another only 1. What should we chose? On this topic, we see it case by case, there is no generic way for handling this issue
- Systems tend to grow complex over time. One issue that systematically comes with all the integration works we have done was the following: "We know our system can support our needs, prove that yours can". This is a very tricky question. Of course, when migrating from one system to another, the legacy system could handle what it was handling. But a new system will have a completely different topology, architectures and concepts. The way we are trying to tackle this is to try to come to a common ground, as much as possible. If we look at a complex website, knowing only "how many users come and visit the website" is far from being enough. We need to translate Those numbers that are very contextual to a more abstract way. Some systems cache a lot, some systems do pre-rendering, some systems integrate in a different way with a search engine ...
- Legacy systems = a big box that has a lot of dependencies and that works "as it is" Our solution for that has been to promote the use of APIs between systems. We usually split our systems into the following: UI for shoppers UI for merchants UI for Business Units Open APIs that are accessed by third parties (merchants, partner companies, …) API layer, across the above layers => Our basic rule is to centralize in the API layer what can be re-used in any of the above layers. We want to code once and for all. => By standardizing the way a system communicates with the outside, we can reduce the effort for replacing a system by another. => By abstracting the data layer by an API layer, we can freely change the backend without impacting the UI components (add a caching layer, replace an RDBMS by a NoSQL solution, etc…). => In my experience, it is almost impossible to do a big-bang approach of integrating "System A" with "System B". By separating the API layer, that offers us the ability to look at one functional area after the other. We can say "First, let's integrate the order API of System A with the order system of System B".
- What we do: 60% to 75 % of UIs automated tests Then the rest of the standard package is tested manually Then, the new features that come into the platform, we test those separately In parallel, we run load and stress tests
- With all those expansions, we ended up building several E-Commerce systems. However, we are in the process of consolidating those systems. When consolidating systems that have been around from quite some time, you usually get into the following issues:
- With all those expansions, we ended up building several E-Commerce systems. However, we are in the process of consolidating those systems. When consolidating systems that have been around from quite some time, you usually get into the following issues:
- Please note that this diagram is a temporary diagram Basically, RPaaS is based on Cloud Foundry and on a top of a Virtualization layer. It provides the application with the ability of easily deploying and scaling. It also provides environment variables so we can deploy the same artifact (WAR file, ruby runtime, php modules etc…) in all the environments without having to modify them or building custom scripts. It also supports deploying a new application without any service stop. We can deploy a new application then map it to the endpoint to which external services should call. It is basically HTTP based traffic so that the UI calls API using JSON APIs over HTTP, then the APIs would connect to a database via JDBC or other protocol.
- Please note that this diagram is a temporary diagram Basically, RPaaS is based on Cloud Foundry and on a top of a Virtualization layer. It provides the application with the ability of easily deploying and scaling. It also provides environment variables so we can deploy the same artifact (WAR file, ruby runtime, php modules etc…) in all the environments without having to modify them or building custom scripts. It also supports deploying a new application without any service stop. We can deploy a new application then map it to the endpoint to which external services should call. It is basically HTTP based traffic so that the UI calls API using JSON APIs over HTTP, then the APIs would connect to a database via JDBC or other protocol.
- Is it okay to say that we have a partnership with Akamai? Regarding the data storage layer, we can of course easily split the data by marketplace. But then what happens when a merchant in Indonesia wants to sell an item in Malaysia? We could then split the data by region. But then, what happens when a merchant in Asia wants to sell an item in Europe? How do we keep the consistency of the item and inventory information? To address those challenges, we are building a data storage that is not based on relational databases because, unfortunately, those kind of features are missing from traditional relational databases
- During our design phase, we realised that our platform would be communicating with a lot of external systems (dataware house, internal search engines, …) Usually, from an E-Commerce system perspective, external systems are mostly interested in changes that occur in our platform. Those changes can be of any type: a merchant registers a new item, an item runs out of stock, an order is cancelled, and so on. However, having a lot of systems polling for changes is very expensive system wise. That is why we are doing event driven mechanisms for those cases. A system will register on a message bus saying “I am interested in being notified when an order changes”. We can then use this mechanism, without any change in the original source for: invalidating cache, sending emails, rebuilding our search engine indexes and so on and so forth
- With all those expansions, we ended up building several E-Commerce systems. However, we are in the process of consolidating those systems. When consolidating systems that have been around from quite some time, you usually get into the following issues:
- With all those expansions, we ended up building several E-Commerce systems. However, we are in the process of consolidating those systems. When consolidating systems that have been around from quite some time, you usually get into the following issues:
- With all those expansions, we ended up building several E-Commerce systems. However, we are in the process of consolidating those systems. When consolidating systems that have been around from quite some time, you usually get into the following issues: