Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Vitalii Korzh - "Exciting Migrations"
Similar to Vitalii Korzh - "Exciting Migrations" (20)
More from LogeekNightUkraine
More from LogeekNightUkraine (20)
Recently uploaded
Recently uploaded (20)
Vitalii Korzh - "Exciting Migrations"
- 2. www.luxoft.com Short introduction 2 I work on the enterprise DAM system Here are some numbers: The system uses 0.5 PB of storage It contains 40M assets It gets ~5M calls daily It handles 15 search rps It receives 500K new assets per month It is growing by 10TB It responds with 300GB of daily traffic
- 3. www.luxoft.com Where It Started We started with: ● 2 projects hosted in the same git repository ● Media that was stored in a database ● 500 tables ● 1-hour builds ● Manual deploys ● On-prem servers ● RMI-based queue mechanism ● XSLT templates ● Java 6 ● 4 clusters of Solr4 ● Data indexing that makes 1 document per second ● 1 release in 3 months 3
- 4. www.luxoft.com What Changed For last 4 years we have been undertaking many changes: ● Moved CI to gitlab ● Sped up builds slightly (takes 4 minutes) ● Built 30 services around existing application ● Dockerized everything ● Migrated to AWS ● Migrated data to S3 ● Switched from flv to mp4 ● Got rid of Oracle enterprise license ● Moved to Solr8 cloud 4
- 5. www.luxoft.com Expectation What We Know About Migrations 5 Reality
- 6. www.luxoft.com Automated Pipelines 6 Sometimes applications fail to start. We tried a lot of different methods to deliver applications bash -> expect -> ansible -> gitlab.ci -> aws. Lesson: there are a lot of methods to make things work
- 7. www.luxoft.com Put Everything into a Container 7 Not all applications can be dockerized as is. We were fighting with glassfish until it was replaced with tomcat. Lesson: if something doesn’t work, maybe you need to find a way around it.
- 8. www.luxoft.com Take it to the Cloud 8 How can hard can it be? Move all the applications and their data, update all the pipelines. Lesson: there is a plan, so just follow it. Follow evolution without revolution.
- 9. www.luxoft.com How to set your DataCenter on Fire 9 Is it hard to move files? A migration of 400TB can be done, but then people will ask what is killing the network. Sometimes servers can become overwhelmed by IOPS Lesson: remember that network capacity is limited.
- 10. www.luxoft.com That can be Handled Async 1 We can handle this without a queue Until you can’t. Some operations require too many resources Lesson: it’s easier to scale if you have some reliable buffer.
- 11. www.luxoft.com No Need to Upgrade 1 The application works perfectly We completed solr4 to solr5 migration without index optimization. We have reduced index size by 60% by moving to solr8 with optimization enabled. One stable application used solr3 until we lost all our indexes. Lesson: if you know it will collapse - be prepared.
- 12. www.luxoft.com Let’s Replace Oracle with MySql 1 The license costs so much, it’s just a db After dozens of discussions we moved to the RDS version. We adjusted some stored procedures without rewriting tons of code. Lesson: there are a lot of options how to reduce you run cost.
- 13. www.luxoft.com Don’t Waste your Money 1 We have never hit the 3rd party system limit Burn all prepaid capacity in your test run. Lesson: read documentation and get some proper metrics and be able to limit execution.
- 14. www.luxoft.com We Need Some Toggle for this Feature 1 One of our clients doesn’t like the design Sometimes, it costs more than it pays Lesson: know the real value of your choices.
- 15. www.luxoft.com Keep Track of your System 1 Nothing happens, it’s just stopped responding No one touched the system. There was just some data processing in the background. It must be something else. Lesson: have all possible metrics to be able to answer tricky questions.
- 16. www.luxoft.com Always try to push the boundaries. Just don’t forget that they exist. Evolve your Application 1