@randybias

AWS Repatriation
Bring Your Apps Back

OpenStack Summit, Hong Kong
November 5th, 2013

CCA - NoDerivs 3.0 Unpo...
@randybias
AWS Repatriation

Pub Cloud Repatriation
Bring Your Apps Back

OpenStack Summit, Hong Kong
November 5th, 2013

...
@randybias
AWS Repatriation
Public Cloud Repatriation

Apps for Hybrid Cloud
What Hybrid-ized Apps Need from Hybrid Cloud
...
Introduction
Who
OpenStack Foundation Board of Directors
Prod. OpenStack pioneer, Cloudscaling:
Part of OpenStack community since July ...
My Bias
I run an OpenStack product company
... opinions expressed here are mine
Not the foundation, not the community, jus...
FYI: State of the Stack v2
Live broadcast from OpenStack Summit
Nov 6th, 10AM Hong Kong / 6PM PST
Register here:
http://cl...
1

Why repatriate from public clouds?

2

Repatriation requirements

3

Repatriation & behavorial parity

4

Understanding...
Why Repatriate?
Public & Private Cloud

10
Public & Private Cloud Strengths

Private
Cloud

Public
Cloud
General
Purpose

Renter
Economics

Optimized
Purpose

Large ...
Public & Private Cloud Drawbacks

Private
Cloud

Public
Cloud
General
Purpose

Renter
Economics

Optimized
Purpose

Large ...
Why Repatriate from Public Clouds?
Reduce Cost, Increase Control and Ensure Compliance

13
Cost - Own the Base, Rent the Spike
Workload
Demand

Public
Cloud

Rent the
Spike

Hybrid
Cloud
Own the
Base

Private
Clou...
Control - Optimize Hardware for Workloads
CPU & IO Constrained

Memory Constrained

CPU

CPU
$$

RAM
$$$

RAM

DISK
$$$$

...
Compliance - and Security
Public clouds will close security gap
Already made exceptional progress
Many public clouds are m...
Hybrid Cloud Enables Repatriation
Manage apps on public and private cloud as a single entity

Public
Cloud
General
Purpose...
Repatriation
Requirements
Repatriation Isn’t Automatic
Requires Compatibility & Interoperability
Build private cloud to similar spec as the top publ...
Virtualization vs. Elastic Cloud

Elastic

Virtualization 2.0

20
Example: OpenStack on a Vblock
AWS doesn’t run on a Vblock and never will
Neither does GCE, RAX, Azure, or any top public ...
Example: OpenStack via OCS
50% savings over AWS over 3-5 years
Public Cloud
@ 20% Annual Growth
($17.4m over 5 Years)

Cos...
Example: OpenStack via OCS
50% savings over AWS over 3-5 years

OCS
w/ Support

AWS 1 Year Res
w/o Support

AWS On Demand
...
Top (Elastic) Clouds
Hybrid cloud only makes sense with the leaders; all elastic
Public Cloud Usage
AWS

RAX

GCE

Running...
Make Private Clouds Look Like
Major Public Elastic Clouds?
Assume application manages its own fate
Scaling, self-healing, ...
Repatriation &
Behavorial Parity
Behavior Matters
Moving apps between clouds is hard
Abstractions and APIs can only do so much

Impedance mismatches are co...
Ex 1: VM Spinup Times
Cloud #1: spins up a VM in 5 mins or less
e.g. AWS, GCE, RAX, Azure

Cloud #2: spins up a VM in 60 m...
Ex 2: Block Storage Snapshotting
Cloud #1: has incremental snapshotting
e.g. AWS, Azure

Cloud #2: has only full snapshott...
Ex 2: Missing or Different Features
Cloud #1: OpenStack w/ floating IP autoassignment
Cloud #2: OpenStack w/o auto-assignme...
Repatriation Checkpoint
Repatriation often makes sense
for Cost, Control, Security and Compliance

Repatriation isn’t auto...
Hybrid Cloud
(Explaining Systems)
Why Are We Talking About Systems?

We need a common understanding of cloud
design principles to agree on what makes a
hybr...
Definition of a System
1) A set of components
2) Integrated into a specific architecture
designed for a purpose
3) Greater t...
System Taxonomy
Input

API

How to interface
with the system

Semantics
Architecture
Output

Names & definitions
that descr...
System Example: Automobile
Input

API

Gas/brake pedals,
steering column,
climate controls, etc.

Semantics
Architecture
O...
System Example: Private Cloud
Input

API

RunInstances
AllocateAddress
CreateVolume
AttachVolume
CreateSnapshot...
Compute...
Hybrid Cloud
(systems contracts)
Compatibility vs Interoperability
Explicit vs Implicit Contracts
Input

API
Semantics

Explicit Contract
Precisely and clearly
expressed or readily
observab...
Explicit vs Implicit Contracts
API

Explicit Contract

Semantics

Input

nova boot --flavor 2 --image
e0b7734d-2331-42a3b19...
Explicit vs Implicit Contracts
Input

API

Critical for
Compatibility

When System
is a Black Box
Delivering a Service...
...
To Enable Mass Adoption...
APIs (controls & instrumentation) must be largely the same

:) :(
42
Compatibility versus Interoperability
These are NOT the same
Cars and semi-trucks (aka “lorries”) are compatible but
not i...
Compatibility versus Interoperability
All systems are like this, even clouds
AWS

GCE

AWS & GCE are highly similar
(seman...
OpenStack is
NOT a System
Definition of a System
1) A set of components
2) Integrated into a specific architecture
designed for a purpose
3) Greater t...
Definition of a System
1) A set of components
2) Integrated into a specific architecture
designed for a purpose
3) Greater t...
OpenStack Component Framework
Other stuff, you probably need/want

Topology &
Metadata

CLI tools

Dashboard

REST

Other ...
OpenStack Component Framework
Other stuff, you probably need/want

Topology &
Metadata

Which
CLI & Mgmt
CLItools
Tools?

...
OpenStack Component Framework
Very flexible like the Linux kernel
Components don’t have to be used in the
same architecture...
Integrate Components to get a System

The components
integrated into
the black box
matter (a lot)
51
Making OpenStack Compat./Interop.
with Elastic Public Clouds
Some flavors of OpenStack need to
embrace the architecture, se...
The Track Gauge Wars
OpenStack’s inherent flexibility means winning the
compatibility and interoperability race

http://sch...
The Track Gauge Wars
OpenStack’s inherent flexibility means winning the
compatibility and interoperability race
Cloud is
So...
Hybrid Cloud is
the Future
OpenStack is the Key to that Future
OpenStack’s perceived weakness is its greatest
strength
Flexibility & NOT being a pre-...
Public Cloud Compatibility and
Interoperability requires...
Same APIs (the explicit contract)
Same Behavior (the implicit ...
Hybrid Cloud Brings Both Worlds Together
Manage apps on public and private cloud as a single entity

Private
Cloud

Public...
The Hybrid-First Cloud Strategy
1) Embrace public cloud architectures
AWS, GCE, RAX, Azure, and possibly vCloud

2) Expand...
Summary
Public Cloud App Repatriation
1

Private Cloud Can Be Cost Effective
As long as it’s designed like a public cloud

2

Hybr...
Q&A
Randy Bias
CEO & Co-founder, Cloudscaling
Director, OpenStack Foundation
@randybias

engineering blog
http://simplicit...
Upcoming SlideShare
Loading in …5
×

AWS Repatriation: Bring Your Apps Back

5,440 views

Published on

Running your own infrastructure *can* be as little as half the cost of running on AWS once you are at scale. OpenStack-based cloud systems can provide the same or similar economies of scale if you leverage the lessons of AWS and GCE when building your cloud. This talk discusses the economic factors in designing a cost-efficient AWS + OpenStack hybrid cloud. We look at the issues involved in repatriating existing applications, and show a couple of real-world demonstration of tools that can assist in the repatriation process. Repatriation isn quite as simple as hitting the Easy button, but if you plan your deployment correctly, you can make it work, both technically and economically.

Published in: Technology, Business
1 Comment
3 Likes
Statistics
Notes
No Downloads
Views
Total views
5,440
On SlideShare
0
From Embeds
0
Number of Embeds
1,662
Actions
Shares
0
Downloads
72
Comments
1
Likes
3
Embeds 0
No embeds

No notes for slide

AWS Repatriation: Bring Your Apps Back

  1. 1. @randybias AWS Repatriation Bring Your Apps Back OpenStack Summit, Hong Kong November 5th, 2013 CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution* * All unlicensed or borrowed works retain their original licenses
  2. 2. @randybias AWS Repatriation Pub Cloud Repatriation Bring Your Apps Back OpenStack Summit, Hong Kong November 5th, 2013 CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution* * All unlicensed or borrowed works retain their original licenses
  3. 3. @randybias AWS Repatriation Public Cloud Repatriation Apps for Hybrid Cloud What Hybrid-ized Apps Need from Hybrid Cloud OpenStack Summit, Hong Kong November 5th, 2013 CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution* * All unlicensed or borrowed works retain their original licenses
  4. 4. Introduction
  5. 5. Who OpenStack Foundation Board of Directors Prod. OpenStack pioneer, Cloudscaling: Part of OpenStack community since July 2010 (launch) Wins: KT, Internap, LivingSocial, Seagate & others Advisor to dotCloud (docker.io) Top 10 Cloud Computing Pioneer 5
  6. 6. My Bias I run an OpenStack product company ... opinions expressed here are mine Not the foundation, not the community, just one POV I believe the pioneers to emulate are: I have run big data centers 100K+ sq ft, 1,000s of physical servers, 100s of switches 6
  7. 7. FYI: State of the Stack v2 Live broadcast from OpenStack Summit Nov 6th, 10AM Hong Kong / 6PM PST Register here: http://cloudscaling.com/stack 7
  8. 8. 1 Why repatriate from public clouds? 2 Repatriation requirements 3 Repatriation & behavorial parity 4 Understanding systems contracts 5 Compatibility vs interoperability 6 OpenStack is not a system 7 OpenStack is key to a hybrid cloud future 8 Wrapping up 8
  9. 9. Why Repatriate?
  10. 10. Public & Private Cloud 10
  11. 11. Public & Private Cloud Strengths Private Cloud Public Cloud General Purpose Renter Economics Optimized Purpose Large Resource Pool Owner Economics Direct Control 11
  12. 12. Public & Private Cloud Drawbacks Private Cloud Public Cloud General Purpose Renter Economics Optimized Purpose Large Resource Pool Owner Economics Direct Control • Expensive at scale • Limited control for optimization • 3rd party dependency & trust • Higher upfront capital outlay • IaaS architectural expertise needed • IaaS operational expertise needed 12
  13. 13. Why Repatriate from Public Clouds? Reduce Cost, Increase Control and Ensure Compliance 13
  14. 14. Cost - Own the Base, Rent the Spike Workload Demand Public Cloud Rent the Spike Hybrid Cloud Own the Base Private Cloud Utilized Capacity 1 2 3 4 5 14 6 7 Days
  15. 15. Control - Optimize Hardware for Workloads CPU & IO Constrained Memory Constrained CPU CPU $$ RAM $$$ RAM DISK $$$$ DISK $$$$ Network IO Network IO $$$ 15
  16. 16. Compliance - and Security Public clouds will close security gap Already made exceptional progress Many public clouds are more secure than the average enterprise Some security requirements may be impossible Specific hardware difficult for public clouds to implement as a solution Many won’t use public for key workloads Regional differences in regulatory compliance 16
  17. 17. Hybrid Cloud Enables Repatriation Manage apps on public and private cloud as a single entity Public Cloud General Purpose Renter Economics Large Resource Pool • Unpredictable Workloads • Scalable Data Processing • Cloudbursting Hybrid Cloud Choice Flexibility Agility + 17 Private Cloud Optimized Purpose Owner Economics Direct Control • Predictable Workloads • Sensitive Data Storage • Reverse Cloudbursting
  18. 18. Repatriation Requirements
  19. 19. Repatriation Isn’t Automatic Requires Compatibility & Interoperability Build private cloud to similar spec as the top public clouds if you want: 1 Economies of scale: Operational Costs 2 Economies of scale: HW pricing parity 3 Application portability 4 Elastic cloud architectures cattle vs. pets 19
  20. 20. Virtualization vs. Elastic Cloud Elastic Virtualization 2.0 20
  21. 21. Example: OpenStack on a Vblock AWS doesn’t run on a Vblock and never will Neither does GCE, RAX, Azure, or any top public cloud 3-5x more expensive than AWS retail price Upwards of 8x more expensive than wholesale OpenStack on a Vblock for non-cloud apps SAP, Oracle 10g, vSphere support, etc. This is the Virtualization 2.0 approach So-called “software-defined datacenter” 21
  22. 22. Example: OpenStack via OCS 50% savings over AWS over 3-5 years Public Cloud @ 20% Annual Growth ($17.4m over 5 Years) Cost / Yr 4.8 $4m $3m 4.0 3.4 2.7 2.8 $2m Private Cloud @ 20% Annual Growth ($9.6m over 5 Years) 2.2 1.8 2.3 1.6 1.4 $1m Time Yr1 Yr2 Yr3 Yr4 22 Yr5
  23. 23. Example: OpenStack via OCS 50% savings over AWS over 3-5 years OCS w/ Support AWS 1 Year Res w/o Support AWS On Demand w/o Support m1.small $13 $24 $44 m1.medium $25 $49 $88 m1.large $50 $97 $175 m1.xlarge $101 $194 $350 23
  24. 24. Top (Elastic) Clouds Hybrid cloud only makes sense with the leaders; all elastic Public Cloud Usage AWS RAX GCE Running apps Windows Azure Experimenting 0% 20% 40% 60% % of Respondents Source: RightScale State of the Cloud Report 2013 24 80% 100%
  25. 25. Make Private Clouds Look Like Major Public Elastic Clouds? Assume application manages its own fate Scaling, self-healing, data replication Commodity hardware, not gold-plated Smart apps don’t need infrastructure resiliency Operational model: cattle vs. pets In public clouds, servers die all the time & no one cares ALL of the top public clouds are elastic 25
  26. 26. Repatriation & Behavorial Parity
  27. 27. Behavior Matters Moving apps between clouds is hard Abstractions and APIs can only do so much Impedance mismatches are common Any two clouds don’t behave the same Some clouds are very very similar GCE and AWS are 95% semantically, architecturally, and behaviorally equivalent This is better than what you find with any two OpenStack clouds Yet these are proprietary software stacks Let’s look at some examples 27
  28. 28. Ex 1: VM Spinup Times Cloud #1: spins up a VM in 5 mins or less e.g. AWS, GCE, RAX, Azure Cloud #2: spins up a VM in 60 minutes e.g. SFR in France Write an app deployment framework If you build on cloud #1 first Assume 5 minutes and retry when a VM fails to show up Causes a failure on cloud #2 when I switch If you build on cloud #2 first Works on cloud #1 Now “auto-scaling” takes 1 hour response time 28
  29. 29. Ex 2: Block Storage Snapshotting Cloud #1: has incremental snapshotting e.g. AWS, Azure Cloud #2: has only full snapshotting e.g. cinder backup via OpenStack Write an app deployment framework manages persistent disks as a mechanism for advanced deployment methods (e.g. EngineYard on AWS) Build on Cloud #1 Snapshot & restore times on cloud #2 cause failures Build on Cloud #2 In testing the non-incremental snapshots mean I can’t use block storage as needed 29
  30. 30. Ex 2: Missing or Different Features Cloud #1: OpenStack w/ floating IP autoassignment Cloud #2: OpenStack w/o auto-assignment Write an app deployment framework: manages VM spinup on behalf of application Build on Cloud #1 Once I try to run on cloud #2, it all breaks because I have to add a check for whether floating IP is there and then add if not Build on Cloud #2 Probably runs on cloud #1 30
  31. 31. Repatriation Checkpoint Repatriation often makes sense for Cost, Control, Security and Compliance Repatriation isn’t automatic it requires compatibility & interoperability Cloud behavior matters How to enable compatibility and interoperability in a hybrid cloud world? 31
  32. 32. Hybrid Cloud (Explaining Systems)
  33. 33. Why Are We Talking About Systems? We need a common understanding of cloud design principles to agree on what makes a hybrid cloud possible. 33
  34. 34. Definition of a System 1) A set of components 2) Integrated into a specific architecture designed for a purpose 3) Greater than the sum of its parts 34
  35. 35. System Taxonomy Input API How to interface with the system Semantics Architecture Output Names & definitions that describe the architecture How the components fit together Behavior 35 What the system actually does
  36. 36. System Example: Automobile Input API Gas/brake pedals, steering column, climate controls, etc. Semantics Architecture Output Behavior 36 Accelerate, decelerate, turn wheels, power lights, heat/cool cabin, component monitoring, deploy air bags, etc.
  37. 37. System Example: Private Cloud Input API RunInstances AllocateAddress CreateVolume AttachVolume CreateSnapshot... Compute Networking Block Storage Object Storage Semantics Architecture Output Behavior 37 Launch a VM Get a floating IP Create a volume Attach a volume Take a snapshot...
  38. 38. Hybrid Cloud (systems contracts) Compatibility vs Interoperability
  39. 39. Explicit vs Implicit Contracts Input API Semantics Explicit Contract Precisely and clearly expressed or readily observable. Leaves nothing to inference. Architecture Output Behavior Implicit Contract Implied though not directly expressed. Inherent in the nature of something. 39
  40. 40. Explicit vs Implicit Contracts API Explicit Contract Semantics Input nova boot --flavor 2 --image e0b7734d-2331-42a3b19e-067adc0da17d --block_device_mapping vda=3195a5a7-fd0d-4ac3b919-7ba6cbe11d46:::0 myInstanceFromVolume Architecture Output Behavior Implicit Contract VM available in 4 min 95% of time 40
  41. 41. Explicit vs Implicit Contracts Input API Critical for Compatibility When System is a Black Box Delivering a Service... Output Behavior 41 Critical for Interoperability
  42. 42. To Enable Mass Adoption... APIs (controls & instrumentation) must be largely the same :) :( 42
  43. 43. Compatibility versus Interoperability These are NOT the same Cars and semi-trucks (aka “lorries”) are compatible but not interoperable Two cars of the same kind are interoperable Compatibility is a consistent interface the explicit contract Interoperability is an equivalent (and interchangeable) behavior 43
  44. 44. Compatibility versus Interoperability All systems are like this, even clouds AWS GCE AWS & GCE are highly similar (semantically, behaviorally and architecturally) This architecture is Elastic Cloud AWS & Rackspace Cloud are mostly similar RAX e.g. Rackspace control plane spans AZs, but still it’s Elastic Cloud VMW AWS & VMware clouds (like Savvis) are very dissimilar e.g. Fixed vs variable VM sizes 44
  45. 45. OpenStack is NOT a System
  46. 46. Definition of a System 1) A set of components 2) Integrated into a specific architecture designed for a purpose 3) Greater than the sum of its parts 46
  47. 47. Definition of a System 1) A set of components 2) Integrated into a specific architecture designed for a purpose 3) Greater than the sum of its parts OpenStack is a component framework w/o an integrated reference architecture 47
  48. 48. OpenStack Component Framework Other stuff, you probably need/want Topology & Metadata CLI tools Dashboard REST Other tools UI Layer REST Provisioning Orchestration Networking REST Log Aggregation Block Storage Compute Object Storage REST REST Health Monitoring DNS etc. Image Management SQL, etc. Database / KVS / Cache REST Varies Physical Network Provider Meter Data Identity Management Varies Hypervisor(s) 48 Elastic Services Layer Metering Varies External Block Provider Shared Services Layer AMQP/0MQ Queuing Data & Resource Layer
  49. 49. OpenStack Component Framework Other stuff, you probably need/want Topology & Metadata Which CLI & Mgmt CLItools Tools? Use the Dashboard Dashboard? REST Other tools UI Layer REST Provisioning Orchestration Log Aggregation Which Orchestration? Block Which Storage Networking Hypervisors Block REST Storage? or Plugin? REST Hardware? & Networking Which Compute Which REST Health Monitoring DNS etc. Where to Image Store Management Images? SQL, etc. Database / KVS / Cache REST Varies Physical Network Provider Object Storage? Meter Data Which Identity Identity Management Management? Configure the Metering Metering? Varies Hypervisor(s) 49 Object Which Storage Elastic Services Layer Varies External Block Provider Shared Services Layer AMQP/0MQ Queuing Data & Resource Layer
  50. 50. OpenStack Component Framework Very flexible like the Linux kernel Components don’t have to be used in the same architecture Private / Public Cloud Metering / Billing Platform OpenStack Components Cloud App Mgt Framework Storage Platform PaaS Platform 50
  51. 51. Integrate Components to get a System The components integrated into the black box matter (a lot) 51
  52. 52. Making OpenStack Compat./Interop. with Elastic Public Clouds Some flavors of OpenStack need to embrace the architecture, semantics and behavior of elastic clouds This will allow true Hybrid Cloud for: AWS GCE RAX Azure ... Reduced friction for Cloud Application Management tools Easy repatriation and expatriation of apps The API is the track gauge for compatibility. The rest is what you put on the tracks. 52
  53. 53. The Track Gauge Wars OpenStack’s inherent flexibility means winning the compatibility and interoperability race http://schools-wikipedia.org/wp/g/Great_Western_Railway.htm 53
  54. 54. The Track Gauge Wars OpenStack’s inherent flexibility means winning the compatibility and interoperability race Cloud is Somewhere Around Here http://schools-wikipedia.org/wp/g/Great_Western_Railway.htm 53
  55. 55. Hybrid Cloud is the Future
  56. 56. OpenStack is the Key to that Future OpenStack’s perceived weakness is its greatest strength Flexibility & NOT being a pre-integrated system means it can take many forms Some of these can be public cloud compatible (OpenStack’s future dominance depends on it) 55
  57. 57. Public Cloud Compatibility and Interoperability requires... Same APIs (the explicit contract) Same Behavior (the implicit contract) API and Behavioral Equivalency requires Same Semantics Same Architecture 56
  58. 58. Hybrid Cloud Brings Both Worlds Together Manage apps on public and private cloud as a single entity Private Cloud Public Cloud Scalable General Purpose Renter Economics Secure Optimized Purpose On Demand Hybrid Cloud Bridging the great divide! 57 Owner Economics Controlled
  59. 59. The Hybrid-First Cloud Strategy 1) Embrace public cloud architectures AWS, GCE, RAX, Azure, and possibly vCloud 2) Expand work on refstack & Tempest Public cloud interoperability & hybrid cloud 3) Embrace & extend existing AWS tests eutester, aws-compat, OpenStack tempest, NetflixOSS 58
  60. 60. Summary
  61. 61. Public Cloud App Repatriation 1 Private Cloud Can Be Cost Effective As long as it’s designed like a public cloud 2 Hybrid Cloud Provides Best of Both You probably want maximum choice 3 Hybrid Requires Behavioral Equivalency Interoperability and Compatibility are Real 4 Hybrid-first Cloud Strategy ...and OpenStack is the key to building many different hybrid cloud solutions 60
  62. 62. Q&A Randy Bias CEO & Co-founder, Cloudscaling Director, OpenStack Foundation @randybias engineering blog http://simplicityscales.com/ 61

×