Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Dc migration


Published on

Dc migration

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Dc migration

  1. 1. Merging two live data-centers into one Anush Arvind Senior DevOps + SRE + Infrastructure Engineer, Freshworks
  2. 2. The problem Frankfurt GDPR Cost Maintenance Source: Google images
  3. 3. The idea Merge & Migrate IrelandFrankfurt
  4. 4. The idea Merge & Migrate Source: Google images
  5. 5. The challenges High volume of tickets in both DC’s Migration of all components of our infrastructure Data collision Seamless migration for customers Multiple services for similar data sources
  6. 6. The Solution Migration notice VPC Peering Storage in S3 Database ELB Redis Elasticache server Scalability of destination DC
  7. 7. Migration notice Migration notice to Ireland Customers Change DNS & IP listing to Frankfurt DC Try and ensure execution on time
  8. 8. VPC Peering Ireland FrankfurtPrivate IP
  9. 9. Storage in S3 Storage In Ireland Storage In FrankfurtMigrate Existing Storage Reroute new storage
  10. 10. Database Horizontally partitioned and no ID clash Ireland Frankfurt
  11. 11. Database When there is ID clash Source: Google images
  12. 12. ELB Ireland Frankfurt Migrate Source DC certificates
  13. 13. Redis Replication was not possible Source: Google images
  14. 14. The solution
  15. 15. The solution
  16. 16. Scale up Frankfurt Frankfurt
  17. 17. The Migration Maintenance Banner in Ireland DC Verify if incoming traffic and writes have stopped in Ireland DC Stop replication of data stores and Make Frankfurt as Master Send some traffic to Frankfurt and test Point all traffic to Frankfurt DC
  18. 18. Surprise! Some customers had still not updated their DNS Vanity URL would go to Ireland, take an extra hop and come to Frankfurt Source: Google images Source: Google images
  19. 19. Post migration Ireland Bring down source DC Document Source: Google images
  20. 20. Learnings Run multiple poller scripts to replicate live Frankfurt should accept its own and Ireland’s API keys Warm up databases before pointing it to production Source: Google images
  21. 21. Conclusion On a broader level, we had some interesting learnings for any project. On the whole, it was a tedious yet interesting project to migrate our DC and am happy to have shared the story with you all. Never be surprised by a new challenge Always have plan B Be ready to rollback Source: Google images
  22. 22. anushshiva
  23. 23. Questions?