Using Cloud and Continuous Delivery to
Speed Up Innovation
Adrian Cockcroft @adrianco
Cloud Expo Europe – 26th February 2014
What I learned from my time at Netflix
● Speed

wins in the marketplace

● Remove
● High

trust, low process

● Freedom
● Don’t

friction from product development

and responsibility culture

do your own undifferentiated heavy lifting

● Simple

patterns automated by tooling

2 | Battery Ventures
3 | Battery Ventures
Typical reactions to my Netflix talks…
“You guys are
crazy! Can’t
believe it”

“What Netflix is
doing won’t work”
– 2010

It only works
for ‘Unicorns’
like Netflix”

– 2009

– 2011

“We’d like to do
that but can’t”
– 2012

“We’re on our way using
Netflix OSS code”
– 2013

4 | Battery Ventures
5 | Battery Ventures
Demands on IT Increased 1000x
Compete or lose in the market!

6 | Battery Ventures
Colonel Boyd USAF, on Combat
“Get inside your
adversaries' OODA
loop to disorient them”
Observe
Orient
Decide
Act
7 | Battery Ventures
Land grab
opportunity

Engage
customers

Measure
Customers

Competitive
Move

Observe

Customer
Pain Point

Analysis
Deliver

Act

Orient
Model
Hypotheses

Implement

Decide
Commit
Resources

Plan
Response

Get Buy-in

8 | Battery Ventures
How fast can you act?

9 | Battery Ventures
Non-Cloud Product Development
Months before you find out whether the product meets the need

Business
Need

Approval
Process

• Documents
• Weeks

• Meetings
• Weeks

IaaS
Cloud

Hardware
Purchase

Software
Development

Deployment
and Testing

Customer
Feedback

• Negotiations
• Weeks

• Specifications
• Weeks

• Reports
• Weeks

• It sucks!
• Weeks

Hardware provisioning is undifferentiated heavy lifting

10 | Battery Ventures
IaaS Based Product Development
Weeks before you find out whether the product meets the need

Business Need
• Documents
• Weeks

Software
Development
• Specifications
• Weeks

Deployment and
TestingPaaS
• Reports
Cloud
• Days

Customer Feedback
• It sucks!
• Days

Software provisioning is undifferentiated heavy lifting

11 | Battery Ventures
PaaS Based Product Feature Development
Days before you find out whether the feature meets the need

Business Need
• Discussions
• Days

Software
SaaS/
Development
• CodeBPaaS
Cloud
• Days

Customer Feedback
• Fix this Bit!
• Hours

Building your own business apps is undifferentiated heavy lifting

12 | Battery Ventures
SaaS Based Business App Development
Hours before you find out whether the feature meets the need

Business Need
• GUI Builder
• Hours

Customer Feedback
• Fix this bit!
• Seconds

13 | Battery Ventures
Process Hand-Off Steps for Product Development on IaaS
Product Manager
Development Team

QA Integration Team
Operations Deploy Team
BI Analytics Team

14 | Battery Ventures
Process Hand-Off Steps for Feature Development on PaaS

Product Manager

Developer

BI Analytics Team

15 | Battery Ventures
What Happened?

Rate of
change
increased

Cost and size
and risk of
change
reduced
16 | Battery Ventures
Land grab
opportunity

Launch AB
Test

Automatic
Deploy

Incremental
Features

Measure
Customers

INNOVATION
Competitive
Move

Observe

Customer
Pain Point

Analysis

Act

Continuous
Delivery on
Cloud

Orient

BIG DATA
Model
Hypotheses

CLOUD
Decide

Plan
Response

Share Plans

CULTURE

JFDI

17 | Battery Ventures
OK, how do I get there?

18 | Battery Ventures
"This is the IT swamp draining manual for anyone who is neck deep in alligators.”
Adrian Cockcroft, Cloud Architect at Netflix

19 | Battery Ventures
Continuous Deployment for Speed
● Lean Enterprise – coming May 2014

20 | Battery Ventures
Cloud Native for High Availability
● NetflixOSS at netflix.github.com and techblog.netflix.com

● Details from Netflix team at Qcon London March 4-8 2014

21 | Battery Ventures
Cloud Native Benchmarking
Write intensive test of cross region replication capacity
16 x hi1.4xlarge SSD nodes per zone = 96 total
192 TB of SSD in six locations up and running Cassandra in 20 minutes

Test
Load

Validation
Load

1 Million reads
After 500ms
CL.ONE with no
Data loss

1 Million writes
CL.ONE (wait for one
replica to ack)

Test
Load

US-East-1 Region - Virginia

US-West-2 Region - Oregon

Zone A

Zone B

Zone C

Zone A

Zone B

Zone C

Cassandra Replicas

Cassandra Replicas

Cassandra Replicas

Cassandra Replicas

Cassandra Replicas

Cassandra Replicas

Inter-Zone Traffic

Inter-Region Traffic
Up to 9Gbits/s, 83ms

18TB
backups
from S3
Open Source Ecosystems
● Github is your company’s online resume

● Extensible platforms create ecosystems
● Give up control to get ubiquity – Apache license
● Don’t miss Simon Wardley’s Cloud Expo talk later today!

Innovate, Leverage and Commoditize

23 | Battery Ventures
Speed wins in the marketplace

24 | Battery Ventures
Any Questions? Upcoming Presentations by @adrianco
● Battery Ventures http://www.battery.com
● Adrian’s Blog http://perfcap.blogspot.com
● Netflix Tech Blog http://techblog.netflix.com
● Netflix Slideshare http://slideshare.com/netflix

● Migrating to Microservices – Qcon London - March 6th, 2014
● Monitorama Keynote Portland OR - May 7th, 2014
● GOTO Chicago Keynote May 20th, 2014
● DevOps Summit at Cloud Expo New York – June 10th, 2014
● GOTO Copenhagen/Aarhus – Denmark – Oct 25th, 2014

25 | Battery Ventures

Speeding Up Innovation

  • 1.
    Using Cloud andContinuous Delivery to Speed Up Innovation Adrian Cockcroft @adrianco Cloud Expo Europe – 26th February 2014
  • 2.
    What I learnedfrom my time at Netflix ● Speed wins in the marketplace ● Remove ● High trust, low process ● Freedom ● Don’t friction from product development and responsibility culture do your own undifferentiated heavy lifting ● Simple patterns automated by tooling 2 | Battery Ventures
  • 3.
    3 | BatteryVentures
  • 4.
    Typical reactions tomy Netflix talks… “You guys are crazy! Can’t believe it” “What Netflix is doing won’t work” – 2010 It only works for ‘Unicorns’ like Netflix” – 2009 – 2011 “We’d like to do that but can’t” – 2012 “We’re on our way using Netflix OSS code” – 2013 4 | Battery Ventures
  • 5.
    5 | BatteryVentures
  • 6.
    Demands on ITIncreased 1000x Compete or lose in the market! 6 | Battery Ventures
  • 7.
    Colonel Boyd USAF,on Combat “Get inside your adversaries' OODA loop to disorient them” Observe Orient Decide Act 7 | Battery Ventures
  • 8.
  • 9.
    How fast canyou act? 9 | Battery Ventures
  • 10.
    Non-Cloud Product Development Monthsbefore you find out whether the product meets the need Business Need Approval Process • Documents • Weeks • Meetings • Weeks IaaS Cloud Hardware Purchase Software Development Deployment and Testing Customer Feedback • Negotiations • Weeks • Specifications • Weeks • Reports • Weeks • It sucks! • Weeks Hardware provisioning is undifferentiated heavy lifting 10 | Battery Ventures
  • 11.
    IaaS Based ProductDevelopment Weeks before you find out whether the product meets the need Business Need • Documents • Weeks Software Development • Specifications • Weeks Deployment and TestingPaaS • Reports Cloud • Days Customer Feedback • It sucks! • Days Software provisioning is undifferentiated heavy lifting 11 | Battery Ventures
  • 12.
    PaaS Based ProductFeature Development Days before you find out whether the feature meets the need Business Need • Discussions • Days Software SaaS/ Development • CodeBPaaS Cloud • Days Customer Feedback • Fix this Bit! • Hours Building your own business apps is undifferentiated heavy lifting 12 | Battery Ventures
  • 13.
    SaaS Based BusinessApp Development Hours before you find out whether the feature meets the need Business Need • GUI Builder • Hours Customer Feedback • Fix this bit! • Seconds 13 | Battery Ventures
  • 14.
    Process Hand-Off Stepsfor Product Development on IaaS Product Manager Development Team QA Integration Team Operations Deploy Team BI Analytics Team 14 | Battery Ventures
  • 15.
    Process Hand-Off Stepsfor Feature Development on PaaS Product Manager Developer BI Analytics Team 15 | Battery Ventures
  • 16.
    What Happened? Rate of change increased Costand size and risk of change reduced 16 | Battery Ventures
  • 17.
    Land grab opportunity Launch AB Test Automatic Deploy Incremental Features Measure Customers INNOVATION Competitive Move Observe Customer PainPoint Analysis Act Continuous Delivery on Cloud Orient BIG DATA Model Hypotheses CLOUD Decide Plan Response Share Plans CULTURE JFDI 17 | Battery Ventures
  • 18.
    OK, how doI get there? 18 | Battery Ventures
  • 19.
    "This is theIT swamp draining manual for anyone who is neck deep in alligators.” Adrian Cockcroft, Cloud Architect at Netflix 19 | Battery Ventures
  • 20.
    Continuous Deployment forSpeed ● Lean Enterprise – coming May 2014 20 | Battery Ventures
  • 21.
    Cloud Native forHigh Availability ● NetflixOSS at netflix.github.com and techblog.netflix.com ● Details from Netflix team at Qcon London March 4-8 2014 21 | Battery Ventures
  • 22.
    Cloud Native Benchmarking Writeintensive test of cross region replication capacity 16 x hi1.4xlarge SSD nodes per zone = 96 total 192 TB of SSD in six locations up and running Cassandra in 20 minutes Test Load Validation Load 1 Million reads After 500ms CL.ONE with no Data loss 1 Million writes CL.ONE (wait for one replica to ack) Test Load US-East-1 Region - Virginia US-West-2 Region - Oregon Zone A Zone B Zone C Zone A Zone B Zone C Cassandra Replicas Cassandra Replicas Cassandra Replicas Cassandra Replicas Cassandra Replicas Cassandra Replicas Inter-Zone Traffic Inter-Region Traffic Up to 9Gbits/s, 83ms 18TB backups from S3
  • 23.
    Open Source Ecosystems ●Github is your company’s online resume ● Extensible platforms create ecosystems ● Give up control to get ubiquity – Apache license ● Don’t miss Simon Wardley’s Cloud Expo talk later today! Innovate, Leverage and Commoditize 23 | Battery Ventures
  • 24.
    Speed wins inthe marketplace 24 | Battery Ventures
  • 25.
    Any Questions? UpcomingPresentations by @adrianco ● Battery Ventures http://www.battery.com ● Adrian’s Blog http://perfcap.blogspot.com ● Netflix Tech Blog http://techblog.netflix.com ● Netflix Slideshare http://slideshare.com/netflix ● Migrating to Microservices – Qcon London - March 6th, 2014 ● Monitorama Keynote Portland OR - May 7th, 2014 ● GOTO Chicago Keynote May 20th, 2014 ● DevOps Summit at Cloud Expo New York – June 10th, 2014 ● GOTO Copenhagen/Aarhus – Denmark – Oct 25th, 2014 25 | Battery Ventures