How to migrate big, loaded, mission critical processes from monolith BPMS to Camunda, and don't screw it up! By Denis Kotov
•Download as PPTX, PDF•
0 likes•455 views
Tinkoff is the world’s largest fully online bank by customer base (Frost and Sullivan Report for 2016). We migrate a lot processes from old BPMS, include mission critical. We want share our experience with community. Presentation plan: 0. Why to migrate. 1. Split BPMS processes by product. 2. One way communications. 3. Global migration strategy. 4. Start from single activity. 5. Gateway-based feature toggling. 6. Glue activities in big process. 7. Start new processes from new system. 8. Migrate old instances (cross system migration). 9. Breathe out.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to How to migrate big, loaded, mission critical processes from monolith BPMS to Camunda, and don't screw it up! By Denis Kotov
Similar to How to migrate big, loaded, mission critical processes from monolith BPMS to Camunda, and don't screw it up! By Denis Kotov (20)
More from camunda services GmbH
More from camunda services GmbH (20)
Recently uploaded
Recently uploaded (20)
How to migrate big, loaded, mission critical processes from monolith BPMS to Camunda, and don't screw it up! By Denis Kotov
- 1. Don’t screw up migration to Camunda
- 2. We were afraid to break everything, but we couldn't live like this anymore.
- 3. About Tinkoff •One of the largest ecosystem in Russia •Bank, Travel, Mobile, Investment, Insurance, Lifestyle, etc. •Fully online •10M+ Customer base •World largest fully online bank
- 4. Old BPMS “Granny” “ex” Camunda Why to migrate
- 5. Why to migrate •Old stack •Testing •Collaborative tool •Monster app server •Reusable, but fuzzy logic
- 7. Mission-critical process •First impression from the ecosystem •~100.000 instances per month •30+ integration points •70+ activities •Fast growing backlog
- 10. Old BPMS Granny Reusable core Mortage Settlement Additional products... ... Granny Reusable core Mortage Settlement Additional products... ... Core
- 11. Boundaries and requirements •Don`t stop releases •Split monolith by product •Use modern develop stack •Zero-tolerance to lose instances
- 12. Team Refactoring
- 13. Before Web-appClient Backend Api CRM MQ Old BPMS Get business context
- 16. Process “call camunda” Old bpms
- 24. Cross-system migration Old BPMS Migration app 1. Get context and state Product camunda 2. Start 3. Terminate
- 25. Cross-system migration Top level process 2 level 3 level migration
- 26. Result layers Top level process 2 level
- 27. Result 9 people OLD BPMS = 4 people Camunda 5 months
- 28. Result
- 29. Result
- 30. Result •Incidents decreased by 14 times •Regression testing time reduced by 4 times •Update speed reduced by 8 times •Day-by-day release •Manual testing time reduced by 4 times
- 31. Result
- 32. Recommendations 1. Stop store context. 2. Stop doing the human task in the old system. 3. Add MQ. 4. Stop digging. 5. Use one-way communication. 6. Start with small activities. 7. Don’t forget about long tasks.
- 33. Дальше действовать будем мы!
Editor's Notes
- Hi, my name is Denis, I came from Moscow. I work at Tinkoff as Business Process Architect. I 'll tell the story of one of the teams at Tinkoff, who was able to get rid of outdated technology and started living. A year ago I have a stressful meeting with my colleagues and business customers. We talked about upcoming tasks, performance problems, bugs, sad developers. The customers were dissatisfied, and there were plans to make a lot of new features. But customers wasn't sure it was gonna work. The atmosphere in the team was depressing. We understood that in the existing architecture of the solution we would not be able to complete the plan. We decided to change the core of the solution to Camunda. We were afraid to break everything with major changes of solution , but we couldn't live like this anymore. The goal of my presentation is to help you stop suffering with the old technologies that make people miserable.
- Before I start you need to know what is Tinkoff. Tinkoff one of the largest ecosystem in Russia: we have a bank, Travel, Mobile, Investment, Insurance, Lifestyle services and more. Tinkoff is fully online, we haven’t branches or offices for customers. Tinkoff is largest fully online bank by customer base in the world. We provide all services remotely via mobile app, phone, social networks, and other channels. We meet our customers personally if we need to bring something. We have more than 10 million customers.
- I'm gonna say a lot to the old BPMS, sometimes I'm gonna call it grany or ex
- The team felt they were not happy with the old decision, but we decided to record it more clearly. The old technology stack does not allow the use of modern means of increasing productivity. The team would like to use Kotlin (or at least Java 8), but could not. Granny is not adapted to the possibilities of automated testing, especially in the usual manner for developers, I mean unit tests, integration tests and so on. Most of the code was tested manually, which led to bugs and long delivery time of the functionality The lack of normal branching and merging of business process schemes in EX also led to the fact that people could not develop the same business process in parallel. The old BPMS is running on a monster server that reboots for 50 minutes during the update and spits out 3 mb stack traces that can be read for 3 days in case of any errors. Due to licensing restrictions we had one big cluster where all business processes for all products work. It means that if some of our neighbors write code incorrectly, innocent other processes may break down. This is the problem of noisy neighbors. License restrictions and noisy neighbors also force us to make reusable abstractions that can be used in different processes. But they are very difficult to change if their contract is changed.
- And we can talk about the learning curve in the old BPMS. Because of the history of EX it has a lot of buttons, checkmarks, abstractions, settings, parameters. All this is described in the instructions and manuals, but they are very large. So learning curve is tough.
- Ok, I say mission critical – what that's mean? I talk about the origination process for settlement and cash service in Bank. This is the first product for every customer, that wants to join the ecosystem for enterprises. It large process, we have more thirty integration points, more than 70 activities in the process.
- In short, the process works this way - first we get information about the client from internal and external sources, then we decide whether we are ready to work with the client. Then we receive documentary confirmation of the received data and meet with the client for a dossier. After that, we activate the products in various systems
- in real life it is more complicated , it a little part of process in granyy
- As for the old solution, there were rather large pieces of general logic inside, which were reused in different processes. We decided to pull the product completely out of the old solution to achieve 100% independence. We paid for it with a bit of code duplication, but it was not so much.
- If summarize everything that I said, we got some requirements and boundaries: Don`t stop releases, because of backlog Split monolith by-product, because we don`t want noisy neighborhoods Use modern develop stack because we want to work with cool developers Zero-tolerance to lose instances, because it`s mission critical process
- We've already done complex refactoring under such conditions, and one thing we knew for sure is that we wouldn't want to be dependent on the old system after the process was transferred from the old system to the new one, because it hurts a lot to disable it. So we decided that Camunda shouldn't know anything about the old system at all.
- You need to know, that we didn’t store business objects in the context of old BPMS, we store all data in CRM. Also, you need to know that old BPMS starts with an async trigger from MQ. It was very helpful for us in refactoring. And all human task was done in CRM too.
- The first rule to refactoring – stop writing new code in the old system. All new features must be implemented in Camunda.
- When business want new feature we find activities in old bpms and and smartly rewrite it in small business process in camunda.
- We created a reusable subprocess in the old BPMS that calls Camunda. It works very simply - it takes the input variables, turns them into understandable Camunda, then calls the rest method of starting the process. And then it checks the result and takes the variables out of the context. If there is a problem, we have a treshold and a custom task to analyze errors.
- We started to do this from a small synchronous task and it works well. We moved about 15 activities to camunda, and then we face the human task. Because of nature of humans and customers some of the task can be done in weeks, we start to fall to a threshold in “Call camunda” and it was really dangerous, because we stared to see a thousands task. To fix this we start to check current activity type in camunda via rest API. If it`s async activity, we don’t trigger threshold
- Also, to have more control with migration, we stared to add gateways to old bpms for feature toggling in runtime. We control token flow via runtime variables, so we can easy turn on or off calling camunda in specific place
- And at the end of migration flow of application looks like this
- After that we have a lots small business processes in Camunda. We stared to glue it in some bigger real-world business processes, not just simple activity.
- After that we have a lots small business processes in Camunda. We stared to glue it in some bigger real-world business processes, not just simple activity.
- In the end of migration we have only one acivity on old BPMS and 1 highest level process, 5-6 second level process, and around 20 third level process, that builded from single activites. For example it is highest level process
- And finally, we switch MQ to send an event to camunda. All new application processing in a new solution. But what about old instances? Remember, we can't lose anything
- Old BPMS has REST API, when we can get the current state and small context. So we write a small application, that read the current state from old BPMS, read the context, and then creates an instance from this state in Camunda. After that terminate state in old BPMS
- Because we got around 30 process definitions in camunda in some layers we do a small migration hack – we modify some of the process to start from a specific point.
- We don’t need so many layers, so after migration we wrap it up and left 2 logic-based layer
- As a result, we boost developer efficiency for more than 2 times. Right now 4 people make work, that in the old system require 9 people
- We use every of this technology as we want
- We braked the shackles of license policy.
- We braked the shackles of license policy.
- And most importantly, people have become happier
- For all, who want to follow our path: Stop store context. Stop doing the human task in the old system. Add MQ. Stop digging. Use one-way communication. Start with small activities. Don’t forget about long tasks. It` not easy, but the cost pays off