SlideShare a Scribd company logo
1 of 46
© Copyright 2014 Pivotal. All rights reserved.
Pivotal Cloud Platform Deep Dive
Part 4: Custom Buildpacks and Data Services
1
© Copyright 2014 Pivotal. All rights reserved.
Buildpacks and Services
• Buildpacks
• Java Buildpack Deep Dive
• User-Provided Services
• Managed Services
• Demo:
• Customise Java Buildpack
2
© Copyright 2014 Pivotal. All rights reserved.
REVIEW: Deploying Applications to Cloud Foundry Runtime
3
Router
Cloud Controller
Service Broker
Node(s)
DEA
DEA
DEA
DEA
Blobstore DB
Runtime
1. Upload bits/metadata
2. Create/bind services
3. Stage app
4. Deploy app
push app
+ app MD
SC
+ =
© Copyright 2014 Pivotal. All rights reserved.
REVIEW: Stage an App
4
Router
Cloud Controller
DEA
Blobstore DB
Runtime
Detect
Compile Upload
No
System
Buildpacks
+ =
Yes
© Copyright 2014 Pivotal. All rights reserved.
Staging and Buildpacks
Buildpacks are responsible for preparing the machine
image for an application.
5
Application
Container
Runtime
Operating System
Libraries
DEA
Buildpack
}Droplet
© Copyright 2014 Pivotal. All rights reserved.
Types of Buildpacks
System
deployed with Cloud Foundry
Admin
uploaded to Cloud Foundry
BYO
specified at app push
6
© Copyright 2014 Pivotal. All rights reserved.
Managing Admin Buildpacks
7
$ cf buildpacks
$ cf create-buildpack <name> <path to bits> <position>
$ cf update-buildpack <name> [-p <path>] [-i <position>]
$ cf delete-buildpack <name>
© Copyright 2014 Pivotal. All rights reserved.
$ cf push -b
<url>
The buildpack is
referenced by a Git
URL
$ cf push -b
<name>
The admin
buildpack is
referenced by name
Buildpack selection
$ cf push
The application is
tested against
admin then system
buildpacks.
8
© Copyright 2014 Pivotal. All rights reserved.
Tested Buildpacks
https://github.com/cloudfoundry-community/cf-docs-contrib/wiki/Buildpacks
9
LanguagesContainers
Haskell
© Copyright 2014 Pivotal. All rights reserved.
Buildpack API
/bin/detect app_directory
Inspect app bits to determine buildpack applicability
/bin/compile app_directory cache_directory
Download and install runtime, container, packages, libraries; install
app bits as necessary
/bin/release app_directory
Build app start command
10
© Copyright 2014 Pivotal. All rights reserved. 11
Inspect the app bits to determine if the buildpack knows how to
handle the application
/bin/detect
Gemfile exists
package.json exists
setup.py existsOn match, return exit code 0 and
write to STDOUT a string identifying the buildpack
(often just the name of the language supported)
© Copyright 2014 Pivotal. All rights reserved.
/bin/detect
$ cf push
DEA iterates over admin and
system buildpacks calling
/bin/detect scripts
until one of them returns exit
code 0
12
$ cf push -b <url|name>
/bin/detect is
not called
© Copyright 2014 Pivotal. All rights reserved.
/bin/compile
Download and install any necessary
• runtime (Java VM, Ruby interpreter, JavaScript interpreter)
• container (web server)
• support libraries, packages, modules (Ruby gems, NPM
packages)
… and then installing the app bits into the runtime or container
13
© Copyright 2014 Pivotal. All rights reserved.
/bin/compile Caching
Runtime, container, and support packages are downloaded
from sources external to Cloud Foundry
DEA provides a location for storing downloaded artifacts to
speed subsequent staging operations
14
© Copyright 2014 Pivotal. All rights reserved.
/bin/release
Build a YAML-formatted hash with three possible keys
15
addons: []
config_vars: {}
default_process_types:
web: <start command>
On Cloud Foundry, currently only the web: value is
used to get the start command for the app
© Copyright 2014 Pivotal. All rights reserved.
Buildpacks and Services
• Buildpacks
• Java Buildpack Deep Dive
• User-Provided Services
• Managed Services
• Demo:
• Customise Java Buildpack
16
© Copyright 2014 Pivotal. All rights reserved.
Java Buildpack
Supports a variety of JVM languages, containers, and frameworks
with a modular, configurable, and extensible design
17
© Copyright 2014 Pivotal. All rights reserved.
Java Buildpack Concepts
18
Containers
How an application is run
Frameworks
Additional application
transformations
JREs
Java Runtimes
© Copyright 2014 Pivotal. All rights reserved.
Java Buildpack Concepts
19
Containers Frameworks
JREs
OpenJDK
Java main()
Tomcat
Groovy
Spring Boot CLI
Play
Spring config
Play config
Play JPA config
New Relic agent
AppDynamics agent
© Copyright 2014 Pivotal. All rights reserved.
Container Detection Criteria
20
Java main() META-INF/MANIFEST.MF exists with Main-class attribute set
Tomcat WEB-INF directory exists
Groovy
.groovy file with a main() method, or
.groovy file with no classes, or
.groovy file with a shebang (#!) declaration
Spring Boot CLI
one or more POGO .groovy files with no main() method, and
no WEB-INF directory
Spring Boot Embedded start script and lib/spring-boot-*.jar exist
Play start script and lib/play.play_*.jar exist
Ratpack start script and ratpack-core-*.jar exist
Choose zero or one
© Copyright 2014 Pivotal. All rights reserved.
Framework Detection Criteria
21
Spring spring-core*.jar exists
Play config Play application detected
Play JPA config play-java-jpa plugin exists in app
Spring Insight Insight service bound to app
New Relic agent New Relic service bound to app
AppDynamics agent AppDynamics service bound to app
Choose all that apply
© Copyright 2014 Pivotal. All rights reserved.
/bin/compile Output Example
DEA
Buildpack
DEA
-----> Downloaded app package (18M)
-----> Downloading OpenJDK 1.7.0_21 JRE (17.5s)
Expanding JRE to .java (1.4s)
-----> Downloading Auto Reconfiguration 0.7.1 (1.4s)
Modifying /WEB-INF/web.xml for Auto Reconfig
-----> Downloading Tomcat 7.0.42 (3.5s)
Expanding Tomcat to .tomcat (0.2s)
Downloading Buildpack Tomcat Support 1.1.1 (0.0s)
-----> Uploading droplet (55M)
22
© Copyright 2014 Pivotal. All rights reserved.
See What’s Going On
$ cf files <app-name> app
.buildpack-diagnostics/
.java/
.lib/
.tomcat/
META-INF/
WEB-INF/
assets/
23
Buildpack-installed runtime
Buildpack-installed support libraries
Buildpack-installed container
DEA-downloaded application files
© Copyright 2014 Pivotal. All rights reserved.
See What’s Going On
$ cf files <app-name> staging_info.yml
detected_buildpack: openjdk=1.7.0_45 tomcat=7.0.47
spring-auto-reconfiguration=0.7.2
tomcat-buildpack-support=1.1.1
start_command: JAVA_HOME=.java
JAVA_OPTS="-Dhttp.port=$PORT
-Djava.io.tmpdir=$TMPDIR -XX:MaxPermSize=52428K
-XX:OnOutOfMemoryError=./.buildpack-diagnostics/killjava
-Xmx384M -Xss1M" .tomcat/bin/catalina.sh run
24
© Copyright 2014 Pivotal. All rights reserved. 25
Two ways to customise the Java buildpack
• Configure artifacts used by standard JREs, Containers, and
Frameworks
• Extend the buildpack with your own JREs, Containers, and
Frameworks
Customisation is done by forking the buildpack
Customisation
© Copyright 2014 Pivotal. All rights reserved.
Customisation by Configuration
Configuration files in java-buildpack/config determine the behavior of a JRE,
Container, or Framework
26
# cloudfoundry/java-buildpack/config/openjdk.yml
---
version: 1.7.0_+
repository_root: "http://download.pivotal.io.s3.amazonaws.com/openjdk/{platform}/{architecture}"
memory_sizes:
memory_heuristics:
heap: 0.75
permgen: 0.1
stack: 0.05
native: 0.1
# http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/index.yml
---
1.6.0_27: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.6.0_27.tar.gz
1.7.0_21: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_21.tar.gz
1.7.0_25: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_25.tar.gz
1.8.0_M6: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.8.0_M6.tar.gz
1.8.0_M7: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.8.0_M7.tar.gz
© Copyright 2014 Pivotal. All rights reserved.
Customisation by Extension
Implement a JRE, Container, or Framework support class as one Ruby file in the
appropriate directory
(with additional support classes as necessary)
27
cloudfoundry/java-buildpack/lib/java_buildpack
jre
container
framework
© Copyright 2014 Pivotal. All rights reserved.
Customisation by Extension
Support class types have similar interfaces, following the buildpack scripts naming
conventions
28
# initialize the support class with platform information provided in context
# context includes app_dir, lib_dir, environment, java_home, java_opts,
# vcap_application, vcap_services
def initialize(context)
# return a String or an Array<String> that uniquely identifies the container/framework/jre,
# or nil
def detect
# download and unpack the container/framework/jre, and transform the application as necessary
def compile
# create and return the command to run the application with (containers) or add
# options to context[:java_opts] (frameworks)
def release
© Copyright 2014 Pivotal. All rights reserved.
Customisation by Extension
Add new support class to config/components.yml
29
---
containers:
- "JavaBuildpack::Container::Groovy"
- "JavaBuildpack::Container::JavaMain"
- "JavaBuildpack::Container::SpringBootCLI"
- "JavaBuildpack::Container::Tomcat"
- "JavaBuildpack::Container::PlayFramework"
jres:
- "JavaBuildpack::Jre::OpenJdk"
frameworks:
- "JavaBuildpack::Framework::AppDynamicsAgent"
- "JavaBuildpack::Framework::JavaOpts"
- "JavaBuildpack::Framework::NewRelicAgent"
- "JavaBuildpack::Framework::PlayAutoReconfiguration"
- "JavaBuildpack::Framework::PlayJpaPlugin"
- "JavaBuildpack::Framework::SpringAutoReconfiguration"
© Copyright 2014 Pivotal. All rights reserved.
Customisation
Much more information and documentation included in the
GitHub repository
https://github.com/cloudfoundry/java-buildpack
30
© Copyright 2014 Pivotal. All rights reserved.
Buildpacks and Services
• Buildpacks
• Java Buildpack Deep Dive
• User-Provided Services
• Managed Services
• Demo:
• Customise Java Buildpack
31
© Copyright 2014 Pivotal. All rights reserved.
Use Cases: User Provided Service Instances
Typically legacy or existing instances of a service (databases,
queues, mail, etc) where applications connect to the same instance
Credential passing when you need to inject the same credential set into
an application
32
© Copyright 2014 Pivotal. All rights reserved. 33
$ cf cups
FAILED
Incorrect Usage.
NAME:
create-user-provided-service - Make a user-provided service instance available to cf apps
ALIAS:
cups
USAGE:
cf create-user-provided-service SERVICE_INSTANCE [-p PARAMETERS] [-l SYSLOG-DRAIN-URL]
Pass comma separated parameter names to enable interactive mode:
cf create-user-provided-service SERVICE_INSTANCE -p "comma, separated, parameter, names"
Pass parameters as JSON to create a service non-interactively:
cf create-user-provided-service SERVICE_INSTANCE -p '{"name":"value","name":"value"}'
EXAMPLE:
cf create-user-provided-service oracle-db-mine -p "host, port, dbname, username, password"
cf create-user-provided-service oracle-db-mine -p '{"username":"admin","password":"pa55woRD"}'
cf create-user-provided-service my-drain-service -l syslog://example.com
OPTIONS:
-p Parameters
-l Syslog Drain Url
© Copyright 2014 Pivotal. All rights reserved.
Buildpacks and Services
• Buildpacks
• Java Buildpack Deep Dive
• User-Provided Services
• Managed Services
• Demo:
• Customise Java Buildpack
34
© Copyright 2014 Pivotal. All rights reserved.
Managed Services
Managed Services are integrated with Cloud Foundry by implementing a
documented API for which the cloud controller is the client
Service Broker is a component which implements the required API.
Service brokers advertise a catalog of service offerings and service
plans to Cloud Foundry, and receive calls from the Cloud Controller for
five functions: fetch catalog, create, bind, unbind, and delete.
35
© Copyright 2014 Pivotal. All rights reserved.
DB
Router
REVIEW: Create and Bind Services
36
Service
credentials
reserve resources
obtain connection data
CLI Cloud
Controller
Service
Broker
Data
Service
Runtime
create service (HTTP)
bind service (HTTP)
create service (HTTP)
bind service (HTTP)
© Copyright 2014 Pivotal. All rights reserved. 37
© Copyright 2014 Pivotal. All rights reserved.
Server Broker API
GET /v2/catalog – List services and plans available from this broker.
PUT /v2/service_instances/:id – Create a new service instance.
PUT /v2/service_instances/:instance_id/service_bindings/:id – Create a new
binding to a service instance.
DELETE /v2/service_instances/:instance_id/service_bindings/:id – Unbind from a
service instance.
DELETE /v2/service_instances/:id – Delete a service instance.
38
© Copyright 2014 Pivotal. All rights reserved.
Server Broker Registration
Make the service broker known to the Cloud Controller
– cf create service-broker <broker name> <username> <password> <broker base URI>
– Broker should ONLY allow access to those requestors it shared its credential with (Basic Auth)
– See: http://docs.gopivotal.com/pivotalcf/services/managing-service-brokers.html#register-broker
Make ‘plans’ accessible to users in a specific org/space
– Somewhat cumbersome: need to “hand parse” JSON to find service plan UUID as registered with
the CC
– See: http://docs.gopivotal.com/pivotalcf/services/access-control.html#make-plans-public
Need admin creds/role in order to introduce a service broker to the Cloud Controller!
39
© Copyright 2014 Pivotal. All rights reserved.
Service Broker Implementation
Service implementation is up to the service provider/developer.
Cloud Foundry only requires that the service provider implement the service
broker API.
A broker can be implemented as a separate application, or by adding the
required http endpoints to an existing service.
40
© Copyright 2014 Pivotal. All rights reserved.
Service Instance Provisioning Examples
The result of provisioning varies by service type, although there are a few common actions that
work for many services. For a MySQL service, provisioning could result in:
An empty dedicated mysqld process running on its own VM.
An empty dedicated mysqld process running in a lightweight container on a shared VM.
An empty dedicated mysqld process running on a shared VM.
An empty dedicated database, on an existing shared running mysqld.
A database with business schema already there.
A copy of a full database, for example a QA database that is a copy of the production database.
For non-data services, provisioning could just mean getting an account on an existing system.
41
© Copyright 2014 Pivotal. All rights reserved.
Service Broker Implementation
Best Practice: Each binding is represented by its own credentials =>
– T=1: create service instance
▪ Neither App-1 or App-2 has access to the service instance
– T=2: bind App-1 to service instance
▪ Only App-1 can access the service instance
– T=3: bind App-2 to service instance
▪ Both App-1 and App-2 have access to the service instance
– T=4 unbind App-1 from service instance
▪ Only App-2 can access the service instance
42
© Copyright 2014 Pivotal. All rights reserved.
Service Broker Deployment Models
Because Cloud Foundry only requires that a service implements the broker API in
order to be available to Cloud Foundry end users, many deployment models are
possible. The following are examples of valid deployment models:
Entire service (service backend + broker) packaged and deployed by BOSH
alongside Cloud Broker packaged and deployed by BOSH alongside Cloud
Foundry, rest of the service deployed and maintained by other means
Broker (and optionally service) pushed as an application to Cloud Foundry user
space Foundry
Entire service, including broker, deployed and maintained outside of Cloud Foundry
by other means
43
© Copyright 2014 Pivotal. All rights reserved.
Resources
Service Broker API: http://docs.gopivotal.com/pivotalcf/services/api.html
Managing Service Brokers:
http://docs.gopivotal.com/pivotalcf/services/managing-service-
brokers.html
Binding Credentials: http://docs.gopivotal.com/pivotalcf/services/binding-
credentials.html
Tiny sample application: https://github.com/cloudfoundry-
samples/spring-hello-env
44
© Copyright 2014 Pivotal. All rights reserved.
Buildpacks and Services
• Buildpacks
• Java Buildpack Deep Dive
• User-Provided Services
• Managed Services
• Demo:
• Customise Java Buildpack
45
© Copyright 2014 Pivotal. All rights reserved.
THANK YOU!
Get Pivotal Cloud Platform at http://network.pivotal.io!
48

More Related Content

What's hot

JavaOne 2015 CON7547 "Beyond the Coffee Cup: Leveraging Java Runtime Technolo...
JavaOne 2015 CON7547 "Beyond the Coffee Cup: Leveraging Java Runtime Technolo...JavaOne 2015 CON7547 "Beyond the Coffee Cup: Leveraging Java Runtime Technolo...
JavaOne 2015 CON7547 "Beyond the Coffee Cup: Leveraging Java Runtime Technolo...0xdaryl
 
How to implement a simple dalvik virtual machine
How to implement a simple dalvik virtual machineHow to implement a simple dalvik virtual machine
How to implement a simple dalvik virtual machineChun-Yu Wang
 
Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014biicode
 
DCEU 18: Tips and Tricks of the Docker Captains
DCEU 18: Tips and Tricks of the Docker CaptainsDCEU 18: Tips and Tricks of the Docker Captains
DCEU 18: Tips and Tricks of the Docker CaptainsDocker, Inc.
 
Docker for Developers - php[tek] 2017
Docker for Developers - php[tek] 2017Docker for Developers - php[tek] 2017
Docker for Developers - php[tek] 2017Chris Tankersley
 
Be a Happier Developer with Docker: Tricks of the Trade
Be a Happier Developer with Docker: Tricks of the TradeBe a Happier Developer with Docker: Tricks of the Trade
Be a Happier Developer with Docker: Tricks of the TradeDocker, Inc.
 
jbang: Unleash the power of Java for shell scripting
jbang: Unleash the power of Java for shell scriptingjbang: Unleash the power of Java for shell scripting
jbang: Unleash the power of Java for shell scriptingRed Hat Developers
 
Kubernetes Story - Day 1: Build and Manage Containers with Podman
Kubernetes Story - Day 1: Build and Manage Containers with PodmanKubernetes Story - Day 1: Build and Manage Containers with Podman
Kubernetes Story - Day 1: Build and Manage Containers with PodmanMihai Criveti
 
Docker for Developers - Sunshine PHP
Docker for Developers - Sunshine PHPDocker for Developers - Sunshine PHP
Docker for Developers - Sunshine PHPChris Tankersley
 
DockerCon Keynote Ben Golub
DockerCon Keynote Ben GolubDockerCon Keynote Ben Golub
DockerCon Keynote Ben GolubdotCloud
 
IBM Monitoring and Diagnostics Tools - Health Center 3.0.2
IBM Monitoring and Diagnostics Tools - Health Center 3.0.2IBM Monitoring and Diagnostics Tools - Health Center 3.0.2
IBM Monitoring and Diagnostics Tools - Health Center 3.0.2Chris Bailey
 
Docker for PHP Developers - ZendCon 2016
Docker for PHP Developers - ZendCon 2016Docker for PHP Developers - ZendCon 2016
Docker for PHP Developers - ZendCon 2016Chris Tankersley
 
WebSphere Application Server Liberty Profile and Docker
WebSphere Application Server Liberty Profile and DockerWebSphere Application Server Liberty Profile and Docker
WebSphere Application Server Liberty Profile and DockerDavid Currie
 
Building kubectl plugins with Quarkus | DevNation Tech Talk
Building kubectl plugins with Quarkus | DevNation Tech TalkBuilding kubectl plugins with Quarkus | DevNation Tech Talk
Building kubectl plugins with Quarkus | DevNation Tech TalkRed Hat Developers
 
Binary Packaging for HPC with Spack
Binary Packaging for HPC with SpackBinary Packaging for HPC with Spack
Binary Packaging for HPC with Spackinside-BigData.com
 
Paving the way to a native Sling
Paving the way to a native SlingPaving the way to a native Sling
Paving the way to a native SlingRadu Cotescu
 

What's hot (20)

JavaOne 2015 CON7547 "Beyond the Coffee Cup: Leveraging Java Runtime Technolo...
JavaOne 2015 CON7547 "Beyond the Coffee Cup: Leveraging Java Runtime Technolo...JavaOne 2015 CON7547 "Beyond the Coffee Cup: Leveraging Java Runtime Technolo...
JavaOne 2015 CON7547 "Beyond the Coffee Cup: Leveraging Java Runtime Technolo...
 
How to implement a simple dalvik virtual machine
How to implement a simple dalvik virtual machineHow to implement a simple dalvik virtual machine
How to implement a simple dalvik virtual machine
 
Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014
 
devops@cineca
devops@cinecadevops@cineca
devops@cineca
 
DCEU 18: Tips and Tricks of the Docker Captains
DCEU 18: Tips and Tricks of the Docker CaptainsDCEU 18: Tips and Tricks of the Docker Captains
DCEU 18: Tips and Tricks of the Docker Captains
 
Docker for Developers - php[tek] 2017
Docker for Developers - php[tek] 2017Docker for Developers - php[tek] 2017
Docker for Developers - php[tek] 2017
 
Be a Happier Developer with Docker: Tricks of the Trade
Be a Happier Developer with Docker: Tricks of the TradeBe a Happier Developer with Docker: Tricks of the Trade
Be a Happier Developer with Docker: Tricks of the Trade
 
jbang: Unleash the power of Java for shell scripting
jbang: Unleash the power of Java for shell scriptingjbang: Unleash the power of Java for shell scripting
jbang: Unleash the power of Java for shell scripting
 
Kubernetes Story - Day 1: Build and Manage Containers with Podman
Kubernetes Story - Day 1: Build and Manage Containers with PodmanKubernetes Story - Day 1: Build and Manage Containers with Podman
Kubernetes Story - Day 1: Build and Manage Containers with Podman
 
Docker for Developers - Sunshine PHP
Docker for Developers - Sunshine PHPDocker for Developers - Sunshine PHP
Docker for Developers - Sunshine PHP
 
DockerCon Keynote Ben Golub
DockerCon Keynote Ben GolubDockerCon Keynote Ben Golub
DockerCon Keynote Ben Golub
 
Day CRX Introduction
Day CRX IntroductionDay CRX Introduction
Day CRX Introduction
 
Apache DeltaSpike the CDI toolbox
Apache DeltaSpike the CDI toolboxApache DeltaSpike the CDI toolbox
Apache DeltaSpike the CDI toolbox
 
IBM Monitoring and Diagnostics Tools - Health Center 3.0.2
IBM Monitoring and Diagnostics Tools - Health Center 3.0.2IBM Monitoring and Diagnostics Tools - Health Center 3.0.2
IBM Monitoring and Diagnostics Tools - Health Center 3.0.2
 
Docker for PHP Developers - ZendCon 2016
Docker for PHP Developers - ZendCon 2016Docker for PHP Developers - ZendCon 2016
Docker for PHP Developers - ZendCon 2016
 
WebSphere Application Server Liberty Profile and Docker
WebSphere Application Server Liberty Profile and DockerWebSphere Application Server Liberty Profile and Docker
WebSphere Application Server Liberty Profile and Docker
 
Maven 3.0 at Øredev
Maven 3.0 at ØredevMaven 3.0 at Øredev
Maven 3.0 at Øredev
 
Building kubectl plugins with Quarkus | DevNation Tech Talk
Building kubectl plugins with Quarkus | DevNation Tech TalkBuilding kubectl plugins with Quarkus | DevNation Tech Talk
Building kubectl plugins with Quarkus | DevNation Tech Talk
 
Binary Packaging for HPC with Spack
Binary Packaging for HPC with SpackBinary Packaging for HPC with Spack
Binary Packaging for HPC with Spack
 
Paving the way to a native Sling
Paving the way to a native SlingPaving the way to a native Sling
Paving the way to a native Sling
 

Similar to Custom Buildpacks and Data Services

Part 4: Custom Buildpacks and Data Services (Pivotal Cloud Platform Roadshow)
Part 4: Custom Buildpacks and Data Services (Pivotal Cloud Platform Roadshow)Part 4: Custom Buildpacks and Data Services (Pivotal Cloud Platform Roadshow)
Part 4: Custom Buildpacks and Data Services (Pivotal Cloud Platform Roadshow)VMware Tanzu
 
Introduction to JIB and Google Cloud Run
Introduction to JIB and Google Cloud RunIntroduction to JIB and Google Cloud Run
Introduction to JIB and Google Cloud RunSaiyam Pathak
 
A Deep Dive into the Liberty Buildpack on IBM BlueMix
A Deep Dive into the Liberty Buildpack on IBM BlueMix A Deep Dive into the Liberty Buildpack on IBM BlueMix
A Deep Dive into the Liberty Buildpack on IBM BlueMix Rohit Kelapure
 
Apigee deploy grunt plugin.1.0
Apigee deploy grunt plugin.1.0Apigee deploy grunt plugin.1.0
Apigee deploy grunt plugin.1.0Diego Zuluaga
 
PHP on Heroku: Deploying and Scaling Apps in the Cloud
PHP on Heroku: Deploying and Scaling Apps in the CloudPHP on Heroku: Deploying and Scaling Apps in the Cloud
PHP on Heroku: Deploying and Scaling Apps in the CloudSalesforce Developers
 
Docker module 1
Docker module 1Docker module 1
Docker module 1Liang Bo
 
BBL Premiers pas avec Docker
BBL Premiers pas avec DockerBBL Premiers pas avec Docker
BBL Premiers pas avec Dockerkanedafromparis
 
Scaleable PHP Applications in Kubernetes
Scaleable PHP Applications in KubernetesScaleable PHP Applications in Kubernetes
Scaleable PHP Applications in KubernetesRobert Lemke
 
Kubernetes for the PHP developer
Kubernetes for the PHP developerKubernetes for the PHP developer
Kubernetes for the PHP developerPaul Czarkowski
 
No more Dockerfiles? Buildpacks to help you ship your image!
No more Dockerfiles? Buildpacks to help you ship your image!No more Dockerfiles? Buildpacks to help you ship your image!
No more Dockerfiles? Buildpacks to help you ship your image!Anthony Dahanne
 
Hints and Tips for Modularizing Existing Enterprise Applications (OSGi Commun...
Hints and Tips for Modularizing Existing Enterprise Applications (OSGi Commun...Hints and Tips for Modularizing Existing Enterprise Applications (OSGi Commun...
Hints and Tips for Modularizing Existing Enterprise Applications (OSGi Commun...Graham Charters
 
K8s, Amazon EKS - 유재석, AWS 솔루션즈 아키텍트
K8s, Amazon EKS - 유재석, AWS 솔루션즈 아키텍트K8s, Amazon EKS - 유재석, AWS 솔루션즈 아키텍트
K8s, Amazon EKS - 유재석, AWS 솔루션즈 아키텍트Amazon Web Services Korea
 
Cloud Foundry V2 | Intermediate Deep Dive
Cloud Foundry V2 | Intermediate Deep DiveCloud Foundry V2 | Intermediate Deep Dive
Cloud Foundry V2 | Intermediate Deep DiveKazuto Kusama
 
Creating Scalable JVM/Java Apps on Heroku
Creating Scalable JVM/Java Apps on HerokuCreating Scalable JVM/Java Apps on Heroku
Creating Scalable JVM/Java Apps on HerokuJoe Kutner
 
Run your Java apps on Cloud Foundry
Run your Java apps on Cloud FoundryRun your Java apps on Cloud Foundry
Run your Java apps on Cloud FoundryAndy Piper
 
Run Your Java Code on Cloud Foundry - Andy Piper (Pivotal)
Run Your Java Code on Cloud Foundry - Andy Piper (Pivotal)Run Your Java Code on Cloud Foundry - Andy Piper (Pivotal)
Run Your Java Code on Cloud Foundry - Andy Piper (Pivotal)jaxLondonConference
 

Similar to Custom Buildpacks and Data Services (20)

Part 4: Custom Buildpacks and Data Services (Pivotal Cloud Platform Roadshow)
Part 4: Custom Buildpacks and Data Services (Pivotal Cloud Platform Roadshow)Part 4: Custom Buildpacks and Data Services (Pivotal Cloud Platform Roadshow)
Part 4: Custom Buildpacks and Data Services (Pivotal Cloud Platform Roadshow)
 
Introduction to JIB and Google Cloud Run
Introduction to JIB and Google Cloud RunIntroduction to JIB and Google Cloud Run
Introduction to JIB and Google Cloud Run
 
Core Android
Core AndroidCore Android
Core Android
 
A Deep Dive into the Liberty Buildpack on IBM BlueMix
A Deep Dive into the Liberty Buildpack on IBM BlueMix A Deep Dive into the Liberty Buildpack on IBM BlueMix
A Deep Dive into the Liberty Buildpack on IBM BlueMix
 
Apigee deploy grunt plugin.1.0
Apigee deploy grunt plugin.1.0Apigee deploy grunt plugin.1.0
Apigee deploy grunt plugin.1.0
 
PHP on Heroku: Deploying and Scaling Apps in the Cloud
PHP on Heroku: Deploying and Scaling Apps in the CloudPHP on Heroku: Deploying and Scaling Apps in the Cloud
PHP on Heroku: Deploying and Scaling Apps in the Cloud
 
Container BoM Inspection with TERN
Container BoM Inspection with TERNContainer BoM Inspection with TERN
Container BoM Inspection with TERN
 
Docker module 1
Docker module 1Docker module 1
Docker module 1
 
BBL Premiers pas avec Docker
BBL Premiers pas avec DockerBBL Premiers pas avec Docker
BBL Premiers pas avec Docker
 
Scaleable PHP Applications in Kubernetes
Scaleable PHP Applications in KubernetesScaleable PHP Applications in Kubernetes
Scaleable PHP Applications in Kubernetes
 
Kubernetes for the PHP developer
Kubernetes for the PHP developerKubernetes for the PHP developer
Kubernetes for the PHP developer
 
Environment
EnvironmentEnvironment
Environment
 
No more Dockerfiles? Buildpacks to help you ship your image!
No more Dockerfiles? Buildpacks to help you ship your image!No more Dockerfiles? Buildpacks to help you ship your image!
No more Dockerfiles? Buildpacks to help you ship your image!
 
Hints and Tips for Modularizing Existing Enterprise Applications (OSGi Commun...
Hints and Tips for Modularizing Existing Enterprise Applications (OSGi Commun...Hints and Tips for Modularizing Existing Enterprise Applications (OSGi Commun...
Hints and Tips for Modularizing Existing Enterprise Applications (OSGi Commun...
 
K8s, Amazon EKS - 유재석, AWS 솔루션즈 아키텍트
K8s, Amazon EKS - 유재석, AWS 솔루션즈 아키텍트K8s, Amazon EKS - 유재석, AWS 솔루션즈 아키텍트
K8s, Amazon EKS - 유재석, AWS 솔루션즈 아키텍트
 
Cloud Foundry V2 | Intermediate Deep Dive
Cloud Foundry V2 | Intermediate Deep DiveCloud Foundry V2 | Intermediate Deep Dive
Cloud Foundry V2 | Intermediate Deep Dive
 
Creating Scalable JVM/Java Apps on Heroku
Creating Scalable JVM/Java Apps on HerokuCreating Scalable JVM/Java Apps on Heroku
Creating Scalable JVM/Java Apps on Heroku
 
2012 09-08-josug-jeff
2012 09-08-josug-jeff2012 09-08-josug-jeff
2012 09-08-josug-jeff
 
Run your Java apps on Cloud Foundry
Run your Java apps on Cloud FoundryRun your Java apps on Cloud Foundry
Run your Java apps on Cloud Foundry
 
Run Your Java Code on Cloud Foundry - Andy Piper (Pivotal)
Run Your Java Code on Cloud Foundry - Andy Piper (Pivotal)Run Your Java Code on Cloud Foundry - Andy Piper (Pivotal)
Run Your Java Code on Cloud Foundry - Andy Piper (Pivotal)
 

Recently uploaded

Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your QueriesExploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your QueriesSanjay Willie
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Strongerpanagenda
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 

Recently uploaded (20)

Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your QueriesExploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 

Custom Buildpacks and Data Services

  • 1. © Copyright 2014 Pivotal. All rights reserved. Pivotal Cloud Platform Deep Dive Part 4: Custom Buildpacks and Data Services 1
  • 2. © Copyright 2014 Pivotal. All rights reserved. Buildpacks and Services • Buildpacks • Java Buildpack Deep Dive • User-Provided Services • Managed Services • Demo: • Customise Java Buildpack 2
  • 3. © Copyright 2014 Pivotal. All rights reserved. REVIEW: Deploying Applications to Cloud Foundry Runtime 3 Router Cloud Controller Service Broker Node(s) DEA DEA DEA DEA Blobstore DB Runtime 1. Upload bits/metadata 2. Create/bind services 3. Stage app 4. Deploy app push app + app MD SC + =
  • 4. © Copyright 2014 Pivotal. All rights reserved. REVIEW: Stage an App 4 Router Cloud Controller DEA Blobstore DB Runtime Detect Compile Upload No System Buildpacks + = Yes
  • 5. © Copyright 2014 Pivotal. All rights reserved. Staging and Buildpacks Buildpacks are responsible for preparing the machine image for an application. 5 Application Container Runtime Operating System Libraries DEA Buildpack }Droplet
  • 6. © Copyright 2014 Pivotal. All rights reserved. Types of Buildpacks System deployed with Cloud Foundry Admin uploaded to Cloud Foundry BYO specified at app push 6
  • 7. © Copyright 2014 Pivotal. All rights reserved. Managing Admin Buildpacks 7 $ cf buildpacks $ cf create-buildpack <name> <path to bits> <position> $ cf update-buildpack <name> [-p <path>] [-i <position>] $ cf delete-buildpack <name>
  • 8. © Copyright 2014 Pivotal. All rights reserved. $ cf push -b <url> The buildpack is referenced by a Git URL $ cf push -b <name> The admin buildpack is referenced by name Buildpack selection $ cf push The application is tested against admin then system buildpacks. 8
  • 9. © Copyright 2014 Pivotal. All rights reserved. Tested Buildpacks https://github.com/cloudfoundry-community/cf-docs-contrib/wiki/Buildpacks 9 LanguagesContainers Haskell
  • 10. © Copyright 2014 Pivotal. All rights reserved. Buildpack API /bin/detect app_directory Inspect app bits to determine buildpack applicability /bin/compile app_directory cache_directory Download and install runtime, container, packages, libraries; install app bits as necessary /bin/release app_directory Build app start command 10
  • 11. © Copyright 2014 Pivotal. All rights reserved. 11 Inspect the app bits to determine if the buildpack knows how to handle the application /bin/detect Gemfile exists package.json exists setup.py existsOn match, return exit code 0 and write to STDOUT a string identifying the buildpack (often just the name of the language supported)
  • 12. © Copyright 2014 Pivotal. All rights reserved. /bin/detect $ cf push DEA iterates over admin and system buildpacks calling /bin/detect scripts until one of them returns exit code 0 12 $ cf push -b <url|name> /bin/detect is not called
  • 13. © Copyright 2014 Pivotal. All rights reserved. /bin/compile Download and install any necessary • runtime (Java VM, Ruby interpreter, JavaScript interpreter) • container (web server) • support libraries, packages, modules (Ruby gems, NPM packages) … and then installing the app bits into the runtime or container 13
  • 14. © Copyright 2014 Pivotal. All rights reserved. /bin/compile Caching Runtime, container, and support packages are downloaded from sources external to Cloud Foundry DEA provides a location for storing downloaded artifacts to speed subsequent staging operations 14
  • 15. © Copyright 2014 Pivotal. All rights reserved. /bin/release Build a YAML-formatted hash with three possible keys 15 addons: [] config_vars: {} default_process_types: web: <start command> On Cloud Foundry, currently only the web: value is used to get the start command for the app
  • 16. © Copyright 2014 Pivotal. All rights reserved. Buildpacks and Services • Buildpacks • Java Buildpack Deep Dive • User-Provided Services • Managed Services • Demo: • Customise Java Buildpack 16
  • 17. © Copyright 2014 Pivotal. All rights reserved. Java Buildpack Supports a variety of JVM languages, containers, and frameworks with a modular, configurable, and extensible design 17
  • 18. © Copyright 2014 Pivotal. All rights reserved. Java Buildpack Concepts 18 Containers How an application is run Frameworks Additional application transformations JREs Java Runtimes
  • 19. © Copyright 2014 Pivotal. All rights reserved. Java Buildpack Concepts 19 Containers Frameworks JREs OpenJDK Java main() Tomcat Groovy Spring Boot CLI Play Spring config Play config Play JPA config New Relic agent AppDynamics agent
  • 20. © Copyright 2014 Pivotal. All rights reserved. Container Detection Criteria 20 Java main() META-INF/MANIFEST.MF exists with Main-class attribute set Tomcat WEB-INF directory exists Groovy .groovy file with a main() method, or .groovy file with no classes, or .groovy file with a shebang (#!) declaration Spring Boot CLI one or more POGO .groovy files with no main() method, and no WEB-INF directory Spring Boot Embedded start script and lib/spring-boot-*.jar exist Play start script and lib/play.play_*.jar exist Ratpack start script and ratpack-core-*.jar exist Choose zero or one
  • 21. © Copyright 2014 Pivotal. All rights reserved. Framework Detection Criteria 21 Spring spring-core*.jar exists Play config Play application detected Play JPA config play-java-jpa plugin exists in app Spring Insight Insight service bound to app New Relic agent New Relic service bound to app AppDynamics agent AppDynamics service bound to app Choose all that apply
  • 22. © Copyright 2014 Pivotal. All rights reserved. /bin/compile Output Example DEA Buildpack DEA -----> Downloaded app package (18M) -----> Downloading OpenJDK 1.7.0_21 JRE (17.5s) Expanding JRE to .java (1.4s) -----> Downloading Auto Reconfiguration 0.7.1 (1.4s) Modifying /WEB-INF/web.xml for Auto Reconfig -----> Downloading Tomcat 7.0.42 (3.5s) Expanding Tomcat to .tomcat (0.2s) Downloading Buildpack Tomcat Support 1.1.1 (0.0s) -----> Uploading droplet (55M) 22
  • 23. © Copyright 2014 Pivotal. All rights reserved. See What’s Going On $ cf files <app-name> app .buildpack-diagnostics/ .java/ .lib/ .tomcat/ META-INF/ WEB-INF/ assets/ 23 Buildpack-installed runtime Buildpack-installed support libraries Buildpack-installed container DEA-downloaded application files
  • 24. © Copyright 2014 Pivotal. All rights reserved. See What’s Going On $ cf files <app-name> staging_info.yml detected_buildpack: openjdk=1.7.0_45 tomcat=7.0.47 spring-auto-reconfiguration=0.7.2 tomcat-buildpack-support=1.1.1 start_command: JAVA_HOME=.java JAVA_OPTS="-Dhttp.port=$PORT -Djava.io.tmpdir=$TMPDIR -XX:MaxPermSize=52428K -XX:OnOutOfMemoryError=./.buildpack-diagnostics/killjava -Xmx384M -Xss1M" .tomcat/bin/catalina.sh run 24
  • 25. © Copyright 2014 Pivotal. All rights reserved. 25 Two ways to customise the Java buildpack • Configure artifacts used by standard JREs, Containers, and Frameworks • Extend the buildpack with your own JREs, Containers, and Frameworks Customisation is done by forking the buildpack Customisation
  • 26. © Copyright 2014 Pivotal. All rights reserved. Customisation by Configuration Configuration files in java-buildpack/config determine the behavior of a JRE, Container, or Framework 26 # cloudfoundry/java-buildpack/config/openjdk.yml --- version: 1.7.0_+ repository_root: "http://download.pivotal.io.s3.amazonaws.com/openjdk/{platform}/{architecture}" memory_sizes: memory_heuristics: heap: 0.75 permgen: 0.1 stack: 0.05 native: 0.1 # http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/index.yml --- 1.6.0_27: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.6.0_27.tar.gz 1.7.0_21: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_21.tar.gz 1.7.0_25: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_25.tar.gz 1.8.0_M6: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.8.0_M6.tar.gz 1.8.0_M7: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.8.0_M7.tar.gz
  • 27. © Copyright 2014 Pivotal. All rights reserved. Customisation by Extension Implement a JRE, Container, or Framework support class as one Ruby file in the appropriate directory (with additional support classes as necessary) 27 cloudfoundry/java-buildpack/lib/java_buildpack jre container framework
  • 28. © Copyright 2014 Pivotal. All rights reserved. Customisation by Extension Support class types have similar interfaces, following the buildpack scripts naming conventions 28 # initialize the support class with platform information provided in context # context includes app_dir, lib_dir, environment, java_home, java_opts, # vcap_application, vcap_services def initialize(context) # return a String or an Array<String> that uniquely identifies the container/framework/jre, # or nil def detect # download and unpack the container/framework/jre, and transform the application as necessary def compile # create and return the command to run the application with (containers) or add # options to context[:java_opts] (frameworks) def release
  • 29. © Copyright 2014 Pivotal. All rights reserved. Customisation by Extension Add new support class to config/components.yml 29 --- containers: - "JavaBuildpack::Container::Groovy" - "JavaBuildpack::Container::JavaMain" - "JavaBuildpack::Container::SpringBootCLI" - "JavaBuildpack::Container::Tomcat" - "JavaBuildpack::Container::PlayFramework" jres: - "JavaBuildpack::Jre::OpenJdk" frameworks: - "JavaBuildpack::Framework::AppDynamicsAgent" - "JavaBuildpack::Framework::JavaOpts" - "JavaBuildpack::Framework::NewRelicAgent" - "JavaBuildpack::Framework::PlayAutoReconfiguration" - "JavaBuildpack::Framework::PlayJpaPlugin" - "JavaBuildpack::Framework::SpringAutoReconfiguration"
  • 30. © Copyright 2014 Pivotal. All rights reserved. Customisation Much more information and documentation included in the GitHub repository https://github.com/cloudfoundry/java-buildpack 30
  • 31. © Copyright 2014 Pivotal. All rights reserved. Buildpacks and Services • Buildpacks • Java Buildpack Deep Dive • User-Provided Services • Managed Services • Demo: • Customise Java Buildpack 31
  • 32. © Copyright 2014 Pivotal. All rights reserved. Use Cases: User Provided Service Instances Typically legacy or existing instances of a service (databases, queues, mail, etc) where applications connect to the same instance Credential passing when you need to inject the same credential set into an application 32
  • 33. © Copyright 2014 Pivotal. All rights reserved. 33 $ cf cups FAILED Incorrect Usage. NAME: create-user-provided-service - Make a user-provided service instance available to cf apps ALIAS: cups USAGE: cf create-user-provided-service SERVICE_INSTANCE [-p PARAMETERS] [-l SYSLOG-DRAIN-URL] Pass comma separated parameter names to enable interactive mode: cf create-user-provided-service SERVICE_INSTANCE -p "comma, separated, parameter, names" Pass parameters as JSON to create a service non-interactively: cf create-user-provided-service SERVICE_INSTANCE -p '{"name":"value","name":"value"}' EXAMPLE: cf create-user-provided-service oracle-db-mine -p "host, port, dbname, username, password" cf create-user-provided-service oracle-db-mine -p '{"username":"admin","password":"pa55woRD"}' cf create-user-provided-service my-drain-service -l syslog://example.com OPTIONS: -p Parameters -l Syslog Drain Url
  • 34. © Copyright 2014 Pivotal. All rights reserved. Buildpacks and Services • Buildpacks • Java Buildpack Deep Dive • User-Provided Services • Managed Services • Demo: • Customise Java Buildpack 34
  • 35. © Copyright 2014 Pivotal. All rights reserved. Managed Services Managed Services are integrated with Cloud Foundry by implementing a documented API for which the cloud controller is the client Service Broker is a component which implements the required API. Service brokers advertise a catalog of service offerings and service plans to Cloud Foundry, and receive calls from the Cloud Controller for five functions: fetch catalog, create, bind, unbind, and delete. 35
  • 36. © Copyright 2014 Pivotal. All rights reserved. DB Router REVIEW: Create and Bind Services 36 Service credentials reserve resources obtain connection data CLI Cloud Controller Service Broker Data Service Runtime create service (HTTP) bind service (HTTP) create service (HTTP) bind service (HTTP)
  • 37. © Copyright 2014 Pivotal. All rights reserved. 37
  • 38. © Copyright 2014 Pivotal. All rights reserved. Server Broker API GET /v2/catalog – List services and plans available from this broker. PUT /v2/service_instances/:id – Create a new service instance. PUT /v2/service_instances/:instance_id/service_bindings/:id – Create a new binding to a service instance. DELETE /v2/service_instances/:instance_id/service_bindings/:id – Unbind from a service instance. DELETE /v2/service_instances/:id – Delete a service instance. 38
  • 39. © Copyright 2014 Pivotal. All rights reserved. Server Broker Registration Make the service broker known to the Cloud Controller – cf create service-broker <broker name> <username> <password> <broker base URI> – Broker should ONLY allow access to those requestors it shared its credential with (Basic Auth) – See: http://docs.gopivotal.com/pivotalcf/services/managing-service-brokers.html#register-broker Make ‘plans’ accessible to users in a specific org/space – Somewhat cumbersome: need to “hand parse” JSON to find service plan UUID as registered with the CC – See: http://docs.gopivotal.com/pivotalcf/services/access-control.html#make-plans-public Need admin creds/role in order to introduce a service broker to the Cloud Controller! 39
  • 40. © Copyright 2014 Pivotal. All rights reserved. Service Broker Implementation Service implementation is up to the service provider/developer. Cloud Foundry only requires that the service provider implement the service broker API. A broker can be implemented as a separate application, or by adding the required http endpoints to an existing service. 40
  • 41. © Copyright 2014 Pivotal. All rights reserved. Service Instance Provisioning Examples The result of provisioning varies by service type, although there are a few common actions that work for many services. For a MySQL service, provisioning could result in: An empty dedicated mysqld process running on its own VM. An empty dedicated mysqld process running in a lightweight container on a shared VM. An empty dedicated mysqld process running on a shared VM. An empty dedicated database, on an existing shared running mysqld. A database with business schema already there. A copy of a full database, for example a QA database that is a copy of the production database. For non-data services, provisioning could just mean getting an account on an existing system. 41
  • 42. © Copyright 2014 Pivotal. All rights reserved. Service Broker Implementation Best Practice: Each binding is represented by its own credentials => – T=1: create service instance ▪ Neither App-1 or App-2 has access to the service instance – T=2: bind App-1 to service instance ▪ Only App-1 can access the service instance – T=3: bind App-2 to service instance ▪ Both App-1 and App-2 have access to the service instance – T=4 unbind App-1 from service instance ▪ Only App-2 can access the service instance 42
  • 43. © Copyright 2014 Pivotal. All rights reserved. Service Broker Deployment Models Because Cloud Foundry only requires that a service implements the broker API in order to be available to Cloud Foundry end users, many deployment models are possible. The following are examples of valid deployment models: Entire service (service backend + broker) packaged and deployed by BOSH alongside Cloud Broker packaged and deployed by BOSH alongside Cloud Foundry, rest of the service deployed and maintained by other means Broker (and optionally service) pushed as an application to Cloud Foundry user space Foundry Entire service, including broker, deployed and maintained outside of Cloud Foundry by other means 43
  • 44. © Copyright 2014 Pivotal. All rights reserved. Resources Service Broker API: http://docs.gopivotal.com/pivotalcf/services/api.html Managing Service Brokers: http://docs.gopivotal.com/pivotalcf/services/managing-service- brokers.html Binding Credentials: http://docs.gopivotal.com/pivotalcf/services/binding- credentials.html Tiny sample application: https://github.com/cloudfoundry- samples/spring-hello-env 44
  • 45. © Copyright 2014 Pivotal. All rights reserved. Buildpacks and Services • Buildpacks • Java Buildpack Deep Dive • User-Provided Services • Managed Services • Demo: • Customise Java Buildpack 45
  • 46. © Copyright 2014 Pivotal. All rights reserved. THANK YOU! Get Pivotal Cloud Platform at http://network.pivotal.io! 48

Editor's Notes

  1. Quick overview of what we’re going to cover End up with a demo showing modifying a java build pack
  2. Just to recap on how apps get deployed to CF Stage app done by the DEA, pulling uploaded app bits plus build packs from the blobstore, compiles into droplets
  3. This shows the process the DEA uses as it goes through the available buildpacls to work out what is needed to support execution of the uploaded code app + metadata + buildpack compiled into droplet and uploaded to containers in DEAs
  4. This slide goes into more detail about what is container within a droplet. Note that we rely on the DEA to provide the base OS, but the droplet contains everything we need to execute the uploaded app Highlight link between buildpack, DEA, and application container here refers to something like Tomcat as a Java application container - OS container provided by DEA to isolate all of this
  5. Three types of build pack - admin and BYO allow you to override or curate your own Maximum flexibility while still maintaining control over platform
  6. Simple commands via the cf CLI to create, update and delete build packs Uploaded build packs have precedence over admin buildpacks, which have precedence over system ones
  7. Git != GitHub - use any public Git provider or local Git repositories, enables local curation and control to comply with security/audit/compliance issues if you have them Git repository must be visible on the network that CF is running on - public CF requires public Git repo, private CF can use private or public Git repo
  8. All community-contributed and maintained - there’s a wide range of support for lots of different languages, containers, etc. Existing buildpacks are a great place to start when creating a new buildpack from scratch.
  9. Simple buildpack API uses three main scripts Written in bash shell scripts or Ruby
  10. What do we need to find to show that we are using a particular language? Java intentionally left off this list because the CF Java buildpack has more complex detection criteria
  11. If app is deployed just with CF push, iterate over all admin and system build packs called detect scripts until one of them strikes gold If you specify a build pack as part of your cf push command, no need to call the detect script - you’ve told PCF what build pack to use
  12. This is where most of the buildpack work is done - the compile script pulls all the needed component together - give more package manager examples - Python PIP, Perl CPAN Installing the app may mean copying or moving the app files from the location DEA put them in, or symlinking them to a different location
  13. External source can include custom packages in S3 buckets or public mirrors for public CF, or internal download locations for private CF. Again, lots of flexibility while maintaining control of the platform
  14. The release script provides feedback metadata back to Cloud Foundry indicating how the application should be executed.
  15. Java buildpack is a bit more complex because it provides a lot more ‘stuff’
  16. Java buildpack gives us three main things - containers, frameworks, and the JREs Frameworks - loose definition, basically anything that is orthogonal to the container
  17. And examples of each of those three components
  18. Detection is typically based on file existence and content of files, also makes it easy to modify and extend This list is likely to grow to support other containers like Spring XD.
  19. As with containers, detection is typically based on file existence and content of files. Can also be done based on services bound to an app, using app bindings in yaml metadata file uploaded alongside code.
  20. Middle part is of interest - where the build pack gets assembled Get the runtime Then the framework (auto reconfig part of the Spring and Play Framework auto reconfig) Then the container
  21. Use the cf files command to query what all the parts are that have been assembled to support your running app The buildpack-installed locations are decisions made by the Java buildpack - other buildpacks will put things in other places.
  22. Can also use the cf files command to query the created yaml file with all the discovered info used to stage the app Useful for tracking down specific component versions used in an app
  23. Awesome documentation is in the github repo, well worth a read Can configure/customise the already included components in the default java build pack Or you can fork it and extend it by adding in your own components or customisations
  24. version and repository_root are common to all components that download artifacts other options are specific to the component 1. consult <component-name>.yml (openjdk.yml, first line of lower window) 2. read repository_root/index.yml (first line of upper window) 3. look up the specified version in the index (version in red in lower window, first arrow) 4. download the matching artifact (highlighted in red in upper window, first arrow points to this)
  25. One class is sufficient for most extensions - eg. a .rb in the framework directory to add a staging timestamp - example we’ll use in the demo Existing support classes serve as good examples and easy to examine the files to see how things have been implemented
  26. if detect returns nil, compile and release will not be called only one container support class can return non-nil from detect
  27. After you’ve added in your support classes, add them to the main components yaml config file The buildpack will iterate over all components, applying them as necessary
  28. Java buildpack is probably the best example to use when trying to understand how to customise or extend a build pack - most complete documentation and examples
  29. Even legacy and existing services can also be exposed via a Managed Service Example: multiple applications connect to the same legacy oracle database and therefore just need connection info and credentials
  30. cups - create-user-provided-service options pretty self-explanatory
  31. Managed services are implemented via service brokers, which allow greater control over remote services
  32. Shows the process of creating a service and then binding to it
  33. An example using the MySQL Service Broker - different plans with different levels of service Note how create-service maps to provisioning an instance on the relevant plan, delete-service removes instance
  34. In both ‘service instance creation’ and ‘service binding creation’, org/space/plan identifiers are provided to the ‘service broker’ This allows the broker flexibility what to do - for example, different options available to users/developers depending on which org/space or plans have been selected
  35. Need to register the service broker with the cloud controller - eg. via Ops Manager Then make specific plans available to spaces - this can be as coarse or fine-grained as needed Gives a high degree of control over access to and consumption of external services, gives existing teams the ‘warm fuzzies’ when integrating PCF into existing infrastructure
  36. CF just needs the service broker API to be in place - what happens behind the scenes, how you interface with a service or present it to PCF, is up to you
  37. Examples of how a ‘service instance’ could be provisioned behind the Service Broker. These could be managed - for example - on a space or plan basis
  38. app-1 and app-2 have different credentials
  39. All of this means you have a great degree of flexibility and control over how you deploy a service to PCF