2© 2015 Pivotal Software, Inc. All rights reserved. 2© 2015 Pivotal Software, Inc. All rights reserved.
Removing Barriers Between
Dev and Ops
It Takes a Platform
Devops @ VMworld
14 October 2015
Shahaf Azriely, Advisory Field Engineer EMEA
3© 2015 Pivotal Software, Inc. All rights reserved.
Software is Changing Industries
$6B valuation
Financial Services
$3.5B valuation
Travel & Hospitality
$19B valuation
Transportation
$3.2B Acquisition by Google
Home Automation
$20B valuation
Entertainment
$1.1B acquisition
Monsanto--Agriculture
4© 2015 Pivotal Software, Inc. All rights reserved.
Increased Expectations and Increased Efficiencies
Changing development and release disciplines
•  Agile/Developer Productivity/TDD
•  Continuous Delivery
•  DevOps
•  Cloud-native App Design
•  Culture/Talent
Time
Expectations
&
Efficiency
Innovation Gap
IaaS
Automation
DevOps, CD, Agile
Traditional IT
Startups
Enterprise
PaaS
5© 2015 Pivotal Software, Inc. All rights reserved.
“Operations is the secret sauce”
http://radar.oreilly.com/2007/10/operations-is-a-competitive-ad.html
6© 2015 Pivotal Software, Inc. All rights reserved.
the difference is platformsthe difference is third platform
7© 2015 Pivotal Software, Inc. All rights reserved.
8© 2015 Pivotal Software, Inc. All rights reserved.
Cloud Foundry is…
The world’s leading open source platform-as-a-service.
Ÿ  Supported by dozens of major organizations
Ÿ  Language and framework agnostic
Ÿ  Manages both VMs and containers
Ÿ  Orchestrates both applications and data services
Ÿ  Abstracts Infrastructure
Founded and commercialized by Pivotal, Inc.
9© 2015 Pivotal Software, Inc. All rights reserved.
An (Overly) Simple View of the World
Applications
Ÿ  Stateless
Ÿ  Run in Containers
Ÿ  Horizontally Scalable
Ÿ  Disposable
Ÿ  No permanent storage
Data Services
Ÿ  Stateful
Ÿ  Run in Virtual Machines
Ÿ  Multi-tenant
Ÿ  Diagonally scalable
Ÿ  Durable storage
10© 2015 Pivotal Software, Inc. All rights reserved.
Cloud Foundry: Applications and Services
Services (virtual machines):
managed by “BOSH”
Applications (containers):
managed by “Runtime”
11© 2015 Pivotal Software, Inc. All rights reserved.
Platform
Services (stateful):
Run in virtual machines
Applications (stateless):
Run in containers
Cloud Foundry manages transformations
between existing and desired system states,
using immutable infrastructure.
Amend
(Re)deploy
(Re)configure
(Re)connect
Monitor
Current
State Desired
State
12© 2015 Pivotal Software, Inc. All rights reserved.
The	
  Cloud	
  Foundry	
  Founda/on:	
  An	
  Open	
  Industry	
  Standard	
  
PLATINUM
GOLD
SILVER
13© 2015 Pivotal Software, Inc. All rights reserved. 13© 2015 Pivotal Software, Inc. All rights reserved.
Devops
Eliminating Barriers
14© 2015 Pivotal Software, Inc. All rights reserved.
The Cloud Platform Evolution
TRADITIONAL IAAS
Virtualization Platform
Operating System
Database
Web
Server
Messaging
Your Application
Code
Physical Servers
Database
Web
Server
Messaging
Your Application
Code
IAAS
Your Application
Code
PAAS
Virtualized
Infrastructure
PAAS
15© 2015 Pivotal Software, Inc. All rights reserved.
Application Dial Tone
Emits application “dial tone”
•  Config
•  Runtime
•  Logs
•  Metrics
•  Health Management
•  Security
•  Operations
Your Application
Code
PAAS
Virtualized
Infrastructure
16© 2015 Pivotal Software, Inc. All rights reserved.
Enabling Devops
Database
Web
Server
Messaging
Your Application
Code
PAAS
Virtualized
Infrastructure
PAAS
Platform Operations
Application Developers
Application Operations
•  Deploys platform
•  Makes standard runtimes and services
available
•  Monitors platform
•  Scales platform
(ensuring sufficient capacity)
•  Upgrades platform with zero-downtime
•  Creates deployable
artifact
•  Config Prod environment
•  Deploys application to Prod
•  Monitors application
•  Scales application (capacity)
•  Deploys new app version with
zero downtime.
17© Copyright 2013 Pivotal. All rights reserved.
18© 2015 Pivotal Software, Inc. All rights reserved.
Operations Activities and Tools
Platform Application
Deploy: Operations Manager Deploy: Application Manager, CLI,
STS Plugin, CI/CD Pipelines
Monitor: logs and metrics Monitor: Agentless APM
APM integrations
Application logs – cli, Apps Man, ELK
Scale: Operations Manager Scale: Apps Manager, cli, autoscaling
Upgrade: Canary, Rolling Upgrades
Zero Downtime
Upgrade: Blue/green, route
mappings, gradual cutover
Zero Downtime
19© 2015 Pivotal Software, Inc. All rights reserved.
Deploy
> cf push 
-p spring-music.war 
corneliastunes
20© 2015 Pivotal Software, Inc. All rights reserved.
cmt
Platform Enables Process
cmt v201cmt cmt cmt cmt v202 cmt
Prod
…
Test
Dev
…
21© 2015 Pivotal Software, Inc. All rights reserved.
The Deployment Pipeline
Commit
Tests
Integration
Tests
Deployable
Artifact
Deploy to CF
Test Env
Deploy to
Prod
•  Developer crafted (before impl.)
•  Stubs/mocks external services
•  Developer executed (before commit)
•  CI executed (after commit)
(after commit)
•  Developer or QA crafted
•  CI executed
•  Runtime context (buildpack applied)
•  Binds to test services (DB, messaging, etc.)
•  “cf push”
•  Runtime context (buildpack applied)
•  Binds to test services (DB, messaging, etc.)
•  Periodic smoke tests
•  “cf push”
•  Runtime context (buildpack applied)
•  Binds to prod services (DB,
messaging, etc.)
•  Periodic smoke tests
•  Monitoring
? ?
22© Copyright 2013 Pivotal. All rights reserved.
23© 2015 Pivotal Software, Inc. All rights reserved.
Monitoring in Cloud Foundry
• Syslog Endpoint
Configuration via
OpsMan
• Log search and
persistence such as
ELK
• Collector Component
with plugins for JMX
and DataDog
• OpsMetrics Tile and
Hyperic
• CLI: cf logs appname
• Apps Manager
• Stayup.io ELK tile
• Agentless APM
• APM integrations
Platform Applications
MetricsLogs
24© 2015 Pivotal Software, Inc. All rights reserved.
Application Logs
25© 2015 Pivotal Software, Inc. All rights reserved.
Application Metrics
26© 2015 Pivotal Software, Inc. All rights reserved.
Platform Logs
27© 2015 Pivotal Software, Inc. All rights reserved.
Platform Metrics
28© 2015 Pivotal Software, Inc. All rights reserved.
Vmware + Pivotal for Operational Excellence
vRealize
Operations
Manager Time Series DB
Pivotal
Ops Metrics
Pivotal Cloud
Foundry - ERS
29© 2015 Pivotal Software, Inc. All rights reserved.
Scale
> cf scale 
-i 5 
corneliastunes
30© 2015 Pivotal Software, Inc. All rights reserved.
Upgrade
31© 2015 Pivotal Software, Inc. All rights reserved.
Every Commit is a Candidate for Deployment
cmt v201cmt cmt cmt cmt v202 cmt
Prod
…
Test
Dev
…
32© 2015 Pivotal Software, Inc. All rights reserved.
Every Commit is a Candidate for Deployment
cmt v201cmt cmt cmt cmt v202 cmt
Prod
…
Test
…
App
Platform
CVE
33© 2015 Pivotal Software, Inc. All rights reserved.
PCF – Each Layer Upgradable with Zero Downtime
* Devs may bring a custom buildpack
runtime layer*
OS image
application layer
PCF Buildpack Apps
linux host & kernel
App container
PCF Supported
Dev Supported
Ops Supported
•  Platform Provided
•  CVEs addressed by Pivotal
•  Canary-style, zero-
downtime upgrades
•  Platform or Developer
Provided
•  CVEs addressed by Pivotal
(for Platform Provided)
•  Blue/Green deployments
•  Developer/App team
provided
•  Blue/Green deployments
34© 2015 Pivotal Software, Inc. All rights reserved.
Upgrade
35© 2015 Pivotal Software, Inc. All rights reserved.
Enabling Devops
Database
Web
Server
Messaging
Your Application
Code
PAAS
Virtualized
Infrastructure
PAAS
Platform Operations
Application Developers
Application Operations
•  Deploys platform
•  Makes standard runtimes and services
available
•  Monitors platform
•  Scales platform
(ensuring sufficient capacity)
•  Upgrades platform with zero-downtime
•  Creates deployable
artifact
•  Config Prod environment
•  Deploys application to Prod
•  Monitors application
•  Scales application (capacity)
•  Deploys new app version with
zero downtime.
36© 2015 Pivotal Software, Inc. All rights reserved. 36© 2015 Pivotal Software, Inc. All rights reserved.
And on top of all that…
… something else we’ve found in
enterprises
37Pivotal Confidential–Internal Use Only
Virtualized
Infrastructure
Infrastructure-centric Operations Model
Your Application
Code
Database
Web
Server
Messaging
IaaS+
(+automation) IT Ops Team
App Team
(Dev and Ops)
38Pivotal Confidential–Internal Use Only
Application-centric Operations
Virtualized
Infrastructure
Your Application
Code
Database
Web
Server
Messaging
IaaS+
(+automation)
IT Ops Team
App Team
Your Application
Code
PAAS
Virtualized
Infrastructure
PaaS Ops Team
App Team
(Dev and Ops)
Emits application “dial tone”
•  Configuration
•  Runtime
•  Logs
•  Metrics
•  Health Management
•  Security
39© 2015 Pivotal Software, Inc. All rights reserved. 39© Copyright 2015 Pivotal. All rights reserved.
Thank You
40© 2015 Pivotal Software, Inc. All rights reserved. 40© Copyright 2015 Pivotal. All rights reserved.
Questions?
@cdavisafc
Removing Barriers Between Dev and Ops

Removing Barriers Between Dev and Ops

  • 2.
    2© 2015 PivotalSoftware, Inc. All rights reserved. 2© 2015 Pivotal Software, Inc. All rights reserved. Removing Barriers Between Dev and Ops It Takes a Platform Devops @ VMworld 14 October 2015 Shahaf Azriely, Advisory Field Engineer EMEA
  • 3.
    3© 2015 PivotalSoftware, Inc. All rights reserved. Software is Changing Industries $6B valuation Financial Services $3.5B valuation Travel & Hospitality $19B valuation Transportation $3.2B Acquisition by Google Home Automation $20B valuation Entertainment $1.1B acquisition Monsanto--Agriculture
  • 4.
    4© 2015 PivotalSoftware, Inc. All rights reserved. Increased Expectations and Increased Efficiencies Changing development and release disciplines •  Agile/Developer Productivity/TDD •  Continuous Delivery •  DevOps •  Cloud-native App Design •  Culture/Talent Time Expectations & Efficiency Innovation Gap IaaS Automation DevOps, CD, Agile Traditional IT Startups Enterprise PaaS
  • 5.
    5© 2015 PivotalSoftware, Inc. All rights reserved. “Operations is the secret sauce” http://radar.oreilly.com/2007/10/operations-is-a-competitive-ad.html
  • 6.
    6© 2015 PivotalSoftware, Inc. All rights reserved. the difference is platformsthe difference is third platform
  • 7.
    7© 2015 PivotalSoftware, Inc. All rights reserved.
  • 8.
    8© 2015 PivotalSoftware, Inc. All rights reserved. Cloud Foundry is… The world’s leading open source platform-as-a-service. Ÿ  Supported by dozens of major organizations Ÿ  Language and framework agnostic Ÿ  Manages both VMs and containers Ÿ  Orchestrates both applications and data services Ÿ  Abstracts Infrastructure Founded and commercialized by Pivotal, Inc.
  • 9.
    9© 2015 PivotalSoftware, Inc. All rights reserved. An (Overly) Simple View of the World Applications Ÿ  Stateless Ÿ  Run in Containers Ÿ  Horizontally Scalable Ÿ  Disposable Ÿ  No permanent storage Data Services Ÿ  Stateful Ÿ  Run in Virtual Machines Ÿ  Multi-tenant Ÿ  Diagonally scalable Ÿ  Durable storage
  • 10.
    10© 2015 PivotalSoftware, Inc. All rights reserved. Cloud Foundry: Applications and Services Services (virtual machines): managed by “BOSH” Applications (containers): managed by “Runtime”
  • 11.
    11© 2015 PivotalSoftware, Inc. All rights reserved. Platform Services (stateful): Run in virtual machines Applications (stateless): Run in containers Cloud Foundry manages transformations between existing and desired system states, using immutable infrastructure. Amend (Re)deploy (Re)configure (Re)connect Monitor Current State Desired State
  • 12.
    12© 2015 PivotalSoftware, Inc. All rights reserved. The  Cloud  Foundry  Founda/on:  An  Open  Industry  Standard   PLATINUM GOLD SILVER
  • 13.
    13© 2015 PivotalSoftware, Inc. All rights reserved. 13© 2015 Pivotal Software, Inc. All rights reserved. Devops Eliminating Barriers
  • 14.
    14© 2015 PivotalSoftware, Inc. All rights reserved. The Cloud Platform Evolution TRADITIONAL IAAS Virtualization Platform Operating System Database Web Server Messaging Your Application Code Physical Servers Database Web Server Messaging Your Application Code IAAS Your Application Code PAAS Virtualized Infrastructure PAAS
  • 15.
    15© 2015 PivotalSoftware, Inc. All rights reserved. Application Dial Tone Emits application “dial tone” •  Config •  Runtime •  Logs •  Metrics •  Health Management •  Security •  Operations Your Application Code PAAS Virtualized Infrastructure
  • 16.
    16© 2015 PivotalSoftware, Inc. All rights reserved. Enabling Devops Database Web Server Messaging Your Application Code PAAS Virtualized Infrastructure PAAS Platform Operations Application Developers Application Operations •  Deploys platform •  Makes standard runtimes and services available •  Monitors platform •  Scales platform (ensuring sufficient capacity) •  Upgrades platform with zero-downtime •  Creates deployable artifact •  Config Prod environment •  Deploys application to Prod •  Monitors application •  Scales application (capacity) •  Deploys new app version with zero downtime.
  • 17.
    17© Copyright 2013Pivotal. All rights reserved.
  • 18.
    18© 2015 PivotalSoftware, Inc. All rights reserved. Operations Activities and Tools Platform Application Deploy: Operations Manager Deploy: Application Manager, CLI, STS Plugin, CI/CD Pipelines Monitor: logs and metrics Monitor: Agentless APM APM integrations Application logs – cli, Apps Man, ELK Scale: Operations Manager Scale: Apps Manager, cli, autoscaling Upgrade: Canary, Rolling Upgrades Zero Downtime Upgrade: Blue/green, route mappings, gradual cutover Zero Downtime
  • 19.
    19© 2015 PivotalSoftware, Inc. All rights reserved. Deploy > cf push -p spring-music.war corneliastunes
  • 20.
    20© 2015 PivotalSoftware, Inc. All rights reserved. cmt Platform Enables Process cmt v201cmt cmt cmt cmt v202 cmt Prod … Test Dev …
  • 21.
    21© 2015 PivotalSoftware, Inc. All rights reserved. The Deployment Pipeline Commit Tests Integration Tests Deployable Artifact Deploy to CF Test Env Deploy to Prod •  Developer crafted (before impl.) •  Stubs/mocks external services •  Developer executed (before commit) •  CI executed (after commit) (after commit) •  Developer or QA crafted •  CI executed •  Runtime context (buildpack applied) •  Binds to test services (DB, messaging, etc.) •  “cf push” •  Runtime context (buildpack applied) •  Binds to test services (DB, messaging, etc.) •  Periodic smoke tests •  “cf push” •  Runtime context (buildpack applied) •  Binds to prod services (DB, messaging, etc.) •  Periodic smoke tests •  Monitoring ? ?
  • 22.
    22© Copyright 2013Pivotal. All rights reserved.
  • 23.
    23© 2015 PivotalSoftware, Inc. All rights reserved. Monitoring in Cloud Foundry • Syslog Endpoint Configuration via OpsMan • Log search and persistence such as ELK • Collector Component with plugins for JMX and DataDog • OpsMetrics Tile and Hyperic • CLI: cf logs appname • Apps Manager • Stayup.io ELK tile • Agentless APM • APM integrations Platform Applications MetricsLogs
  • 24.
    24© 2015 PivotalSoftware, Inc. All rights reserved. Application Logs
  • 25.
    25© 2015 PivotalSoftware, Inc. All rights reserved. Application Metrics
  • 26.
    26© 2015 PivotalSoftware, Inc. All rights reserved. Platform Logs
  • 27.
    27© 2015 PivotalSoftware, Inc. All rights reserved. Platform Metrics
  • 28.
    28© 2015 PivotalSoftware, Inc. All rights reserved. Vmware + Pivotal for Operational Excellence vRealize Operations Manager Time Series DB Pivotal Ops Metrics Pivotal Cloud Foundry - ERS
  • 29.
    29© 2015 PivotalSoftware, Inc. All rights reserved. Scale > cf scale -i 5 corneliastunes
  • 30.
    30© 2015 PivotalSoftware, Inc. All rights reserved. Upgrade
  • 31.
    31© 2015 PivotalSoftware, Inc. All rights reserved. Every Commit is a Candidate for Deployment cmt v201cmt cmt cmt cmt v202 cmt Prod … Test Dev …
  • 32.
    32© 2015 PivotalSoftware, Inc. All rights reserved. Every Commit is a Candidate for Deployment cmt v201cmt cmt cmt cmt v202 cmt Prod … Test … App Platform CVE
  • 33.
    33© 2015 PivotalSoftware, Inc. All rights reserved. PCF – Each Layer Upgradable with Zero Downtime * Devs may bring a custom buildpack runtime layer* OS image application layer PCF Buildpack Apps linux host & kernel App container PCF Supported Dev Supported Ops Supported •  Platform Provided •  CVEs addressed by Pivotal •  Canary-style, zero- downtime upgrades •  Platform or Developer Provided •  CVEs addressed by Pivotal (for Platform Provided) •  Blue/Green deployments •  Developer/App team provided •  Blue/Green deployments
  • 34.
    34© 2015 PivotalSoftware, Inc. All rights reserved. Upgrade
  • 35.
    35© 2015 PivotalSoftware, Inc. All rights reserved. Enabling Devops Database Web Server Messaging Your Application Code PAAS Virtualized Infrastructure PAAS Platform Operations Application Developers Application Operations •  Deploys platform •  Makes standard runtimes and services available •  Monitors platform •  Scales platform (ensuring sufficient capacity) •  Upgrades platform with zero-downtime •  Creates deployable artifact •  Config Prod environment •  Deploys application to Prod •  Monitors application •  Scales application (capacity) •  Deploys new app version with zero downtime.
  • 36.
    36© 2015 PivotalSoftware, Inc. All rights reserved. 36© 2015 Pivotal Software, Inc. All rights reserved. And on top of all that… … something else we’ve found in enterprises
  • 37.
    37Pivotal Confidential–Internal UseOnly Virtualized Infrastructure Infrastructure-centric Operations Model Your Application Code Database Web Server Messaging IaaS+ (+automation) IT Ops Team App Team (Dev and Ops)
  • 38.
    38Pivotal Confidential–Internal UseOnly Application-centric Operations Virtualized Infrastructure Your Application Code Database Web Server Messaging IaaS+ (+automation) IT Ops Team App Team Your Application Code PAAS Virtualized Infrastructure PaaS Ops Team App Team (Dev and Ops) Emits application “dial tone” •  Configuration •  Runtime •  Logs •  Metrics •  Health Management •  Security
  • 39.
    39© 2015 PivotalSoftware, Inc. All rights reserved. 39© Copyright 2015 Pivotal. All rights reserved. Thank You
  • 40.
    40© 2015 PivotalSoftware, Inc. All rights reserved. 40© Copyright 2015 Pivotal. All rights reserved. Questions? @cdavisafc