OSDC 2016 - Hybrid Cloud - A Cloud Migration Strategy


Published on

Do you use Cloud? Why? What about the 15 year legacy of your data center? How many Enterprise vendors tried to sell you their "Hybrid Cloud" solution? What actually is a Hybrid Cloud?

Cloud computing is not just a new way of running servers or Docker containers. The interesting part of any Cloud offering are managed services that provide solutions to difficult problems. Prime examples are messaging (SNS/SQS), distributed storage (S3), managed databases (RDS) and especially turn-key solutions like managed Hadoop (EMR).

Hybrid Cloud is usually understood as a way to unify or standardize server hosting across private data centers and Public Cloud vendors. Some Hybrid Cloud solutions even go as far as providing a unified API that abstracts away all the differences between different platforms. Unfortunately that approach focuses on the lowest common denominator and effectively prevents using the advanced services that each Cloud vendor also offers. However, these services are the true value of Public Cloud vendors.

Another approach to integrating Public Cloud and private data centers is using services from both worlds depending on the problems to solve. Don't hide the cloud technologies but make it simple to use them - both from within the data center and the cloud instances. Create a bridge between the old world of the data center and the new world of the Public Cloud. A good bridge will motivate your developers to move the company to the cloud.

Based upon recent developments at ImmobilienScout24, this talk tries to suggest a sustainable Cloud migration strategy from private data centers through a Hybrid Cloud into the AWS Cloud.

Bridging the security model of the data center with the security model of AWS.
Integrating the AWS identity management (IAM) with the existing servers in the data center.
Secure communication between services running in the data center and in AWS.
Deploying data center servers and Cloud resources together.
Service discovery for services running both in the data center and AWS.
Most of the tools used are Open Source and this talk will show how they come together to support this strategy:

AWS credential provider for employees and data center servers: http://immobilienscout24.github.io/afp/
Cloud Formation automation: https://github.com/ImmobilienScout24/cfn-sphere
Compliancy with European privacy laws: https://github.com/ImmobilienScout24/aws-monocyte

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

OSDC 2016 - Hybrid Cloud - A Cloud Migration Strategy

  1. 1. www.immobilienscout24.de Berlin | 28.04.2016 | Schlomo Schapiro Systems Architect / Open Source Evangelist http://creativecommons.org/licenses/by-nd/4.0 Hybrid Cloud A Cloud Migration Strategy @schlomoschapiro go.schapiro.org/slides
  2. 2. Why should I care? The cloud is here - let's make the best out of it!
  3. 3. Our goal is to join a vibrant technical eco- system to accelerate our own innovation speed.
  4. 4. PROJECT Cloud Migration - Management View
  5. 5. Just ask about ... Timeline Budget Engineered for the CloudSecurity Resilience
  6. 6. Time & Money
  7. 7. Data Center Costs SAN Storage Server Hardware Server Hardware Core & Rack Switches SAN StorageBackup Solution Core & Rack Switchesware Backup Solution 5 years writing off BUDGET
  8. 8. Cloud Costs - Quick Migration BUDGET 1st year 2nd year 3rd year
  9. 9. Cloud Migration - Costs Journey Data Center Costs Cloud Costs Total Costs BUDGET Invest Save ROI How many years?
  10. 10. Engineering
  11. 11. Cloud = Scale Out. Automate or Die. Test Driven Development. Everything will fail.
  12. 12. Live Staging Play Search User ...
  13. 13. Internal Communication ◉ No transport encryption ◉ Trust based on IP ◉ Easy Dev/Ops access to debug and admin ports ◉ Low latency (LAN) ◉ Static service discovery works External Communication ◉ Must use HTTPS ◉ Trust based on authentication ◉ Need secure back door for debug and admin access ◉ Medium / high latency ◉ Effort for service discovery
  14. 14. Cloud Migration ≈ Microservices Migration
  15. 15. Automate Automate Automate Automate Automate Automate Automate
  16. 16. Data Center Hardware Network Storage Virtualization Operating System Application Configuration Load Balancer Automation
  17. 17. Code Cloud (AWS) Hardware Network Storage Virtualization Operating System Application Configuration Load Balancer CloudFormation EC2 VPC S3 ECS / Lambda / Bean Stalk Docker AMI ZIP / S3 ELB Route53 Cloud Front RDS / SNS / SQS / IAM / EMR Api Gateway / Dynamo DB / ...
  18. 18. Resilience
  19. 19. Cloud Formation StackRegion VPC RDS A typical web application on AWS ... Autoscaling Group EC2 EC2 EC2 ELB RDS S P O F
  20. 20. More resilience Cloud Formation Stack Region VPC RDS Autoscaling Group EC2 EC2 EC2 ELB RDS Cloud Formation Stack Region VPC RDS Autoscaling Group EC2 EC2 EC2 ELB RDS
  21. 21. Static credentials are just broken by design!
  22. 22. Static Credentials ◉ SSH keys - copy and crack at home ➨ SSH HostbasedAuthentication ➨ Consider IP trust & rsh for automation and clusters ➨ Use ssh-agent, personal keys should never leave the desktop ◉ AWS key & secret - you won't notice me using them ➨ Use temporary credentials (secret, key, token) ➨ Watch your Cloud Trail logs ◉ Username & password - thanks! ➨ Federated logins for people ➨ Certs for machines (although still static credentials) ➨ IP trust may be good enough ...
  23. 23. Private Connec- tion to DCNo Authenti- cation Perimeter Security Blind Trust Firewall = Security Federated employee login Watch logs for anomalies App is fully responsible for security Jump host for dev & admin access Local firewalls everywhere, explicit access only. AWS: Security Groups
  24. 24. Service⇔Service Communication over public Internet HTTPS only. Setup identity management for services (OAuth2)
  25. 25. Hybrid Cloud
  26. 26. Hybrid Cloud? My Virtual Machine / Docker Container can run on premise or in the cloud. 1 Use the best tool for the job: Some apps run better on premise and some apps benefit more from the cloud. Embrace Cloud services as part of our applications and integrate with them. 2
  27. 27. Hybrid Cloud Comparison Run VMs/Docker anywhere + No vendor lock in + Write once, run anywhere + Easily support multiple platforms + Unified tooling over all platforms + Unified tooling also for data center hosting + Shift workloads based on cost and demand Use best tool for the job + Benefit from external innovation + Ready-made services instead of roll-your-own + "Serverless" applications + Significantly reduce OPS + Use platform migration to refactor applications + Costs scale well with application usage + Small things are very cheap + More options to optimize costs
  28. 28. 80% 20% Benefit Work Work Benefit AWS Managed Services VM Hosting (EC2, ECS)
  29. 29. Cloud Enablement
  30. 30. A Cloud Migration Strategy 1. Establish Cloud platform besides data center 2. Integrate Cloud platform with data center 3. Build new applications into the cloud 4. Migrate existing services into the cloud 5. Repeat until done
  31. 31. 1. Establish Cloud platform besides data center 1. Solve common problems: security, compliance and cost control 2. Provide basic solution for logging, monitoring, deployment 3. Easy & secure access to Cloud platform for all employees, using temporary credentials 4. Decide upon macro architecture, e.g. many AWS accounts, communication over public Internet without VPN, OAuth2 everywhere
  32. 32. 2. Integrate Cloud platform with data center 1. Provide temporary Cloud credentials to every server 2. Provide secure communication framework between services running in the data center and in the cloud 3. Use Cloud managed services from the data center, e.g. SNS, SQS, EMR, Data Pipeline, Kinesis, SWF 4. Migrate persistent storage to Cloud where beneficial, e.g. S3, DynamoDB 5. Improve automation and gather operational experience
  33. 33. 3. Build new applications into the cloud 1. Learn working with full stack responsibility 2. Learn how to architect and develop to benefit from cloud platform 3. Learn how to optimize development and operational costs 4. Improve automation and gather operational experience
  34. 34. 4. Migrate existing services into the cloud 1. Keep total cost (data center + cloud) in check, e.g. prioritize service migrations by data center hardware replacement / investment plan 2. Prioritize cloud migration against feature development 3. Migrate application into Cloud together with new feature 4. Improve automation and gather operational experience
  35. 35. 5. Repeat until done 1. After the migration is before the next migration, e.g. to the next Cloud platform 2. "Remaining" services in data center have to pay for all the data center 3. Optimize between costs and availability requirements 4. Improve automation and gather operational experience … … … 5. Always change the running system
  36. 36. The ImmobilienScout24 Cloud Toolbox
  37. 37. The ImmobilienScout24 Cloud Toolbox ◉ Compliance: AWS resources should only run in the EU https://github.com/ImmobilienScout24/aws-monocyte ◉ Security: Provide AWS credentials to humans and machines http://immobilienscout24.github.io/afp/ ◉ Security: SSH jump host with OpenID Connect authentication https://github.com/ImmobilienScout24/c-bastion ◉ Automation: Cloud Formation cross-stack management https://github.com/ImmobilienScout24/cfn-sphere ◉ Development: Automate Python Lambda packaging https://github.com/ImmobilienScout24/pybuilder_aws_plugin go.schapiro.org/slides @schlomoschapiro www.schapiro.org/schlomo/publications