More Related Content Similar to Cloud Foundry Open Tour - London (20) Cloud Foundry Open Tour - London2. agenda
• cloud foundry anniversary recap
• linux of the cloud
© 2012 VMware, Inc. All rights reserved 2
4. the new cloudfoundry.org
cloudfoundry.com
production updated ~2x/week
github
github.com/cloudfoundry
on +2 and ✓ change pushed to github
reviewers +1/-1 Gerrit Code Review +2/-2 committers
reviews.cloudfoundry.org
test verification score ✓ ✗ git push triggers test execution
Jenkins CI
ci.cloudfoundry.org
© 2012 VMware, Inc. All rights reserved 4
5. 80% of work is below the water line
• Developer Visible Features
• services
• frameworks, runtimes
• tools and ide‟s
• micro cloud foundry
• Contributor Visible Features
• re-visiting architectural issues
• extensive hardening
• algorithm tuning
• scalability enhancements
• operational enhancements
© 2012 VMware, Inc. All rights reserved 5
6. app lifecycle management: at launch
• awesome conceptual architecture,
logical flows start/stop instances
• simplistic design provided a solid
conceptual foundation cloud
• scalability, robustness, operational controller health
efficiency, and extensibility was manager
inadequate
cc - db actual
get expected
• deconstruction/refactoring state
state
• enable multiple independently
scalable components NFS
• eliminate “boot storms” at scale dea
resources, droplets,
• tune the hot data paths
packages
• apply learning from the beta
• across the board and to all major fetch
system components droplets
© 2012 VMware, Inc. All rights reserved 6
7. app lifecycle management: +1yr
v2
cloud
stager health
v2
controller New!
manager
cc - db staging
jobs New!
package
uaa - db cacheNew! v2*
New!
dea
redis
New!
uaa - AuthN blobstore
staging logs New!
New!
more new code than old code
© 2012 VMware, Inc. All rights reserved 7
8. app execution (DEA): launch and +1yr
• Major internal rewrite
• Validating isolation model in Jenkins
• Live in CF.com ~30d dea.rb
container api
nat and firewall
dea.rb
private network
apps
app
private fs
warden container*
* also running in our Jenkins workers
© 2012 VMware, Inc. All rights reserved 8
9. router: at launch and +1yr
http request http request
nginx nginx
(+lua extension)
proxied lookup
request node:port
upstream
router.rb
locator svc
proxied proxied
nats request nats request
<app>.cf.com <app>.cf.com
node:port node:port
dea dea
© 2012 VMware, Inc. All rights reserved 9
10. production cloud foundry clusters
cloudfoundry.com
• 500 – 5,000 VMs
• 40+ unique node types
• 75+ unique software packages
• 75+ unique environments
• 2x/week cf.com updates
• 24x7x365 non-stop operation
• No-downtime deployments
• Reliable, robust, repeatable
deployments, updates,
capacity adjustments
• Small teams manage many
instances production, staging, stress, qa, dev
Google style problem Google style solution
© 2012 VMware, Inc. All rights reserved 10
11. cloud foundry BOSH
• cloud foundry BOSH is an open source tool-chain for
release engineering, deployment, and lifecycle management
of large scale distributed services
• prescriptive way of creating releases, managing systems and services
• it is not a collection of shell scripts, not a pile of Perl
• built to deploy/manage production-class, large scale clusters
• production-class cloud foundry clusters: 500+ VMs, 40+ jobs, 75+
packages
• multi-node, multi-tier, complex clusters: e.g., our Gerrit/Jenkins Cluster
• built for devops usage and scale by a crack team of veterans
• a project, not a product: command line interface, YAML, etc.
• continuous improvement, iterative development, rough edges
github.com/cloudfoundry/bosh
© 2012 VMware, Inc. All rights reserved 11
12. BOSH: under the hood
BOSH User
“BOSH is deployed by BOSH”
bosh cli director healthmon
cloudfoundry.com
db redis nats
workers blobs
active jobs
agent
IaaS CPI stemcell disk
© 2012 VMware, Inc. All rights reserved 12
13. IaaS neutral by design
vSphere: battle tested
implementation, thousands of deployments CPI: code complete
vCloud Director: “work in progress”, 2H 2012 functional status: “work in progress”
Cloud Foundry BOSH
Cloud Provider Interface (CPI)
contribute: github.com/cloudfoundry/bosh
© 2012 VMware, Inc. All rights reserved 13
14. linux of the cloud
© 2012 VMware, Inc. All rights reserved 14
15. linux of the cloud
most widely used OS generation in computing
the primary os of an era
© 2012
© 2012 VMware, Inc. All rights reserved VMware, Inc. All rights reserved 15
16. value proposition of a “primary os”
• developers can target the broadest possible market
with highly leveraged portable code
• applications have a place to reliably run
• customers can use the os knowing they have the
broadest choice of applications
• there is enough of a market to support a healthy eco-
system where money is changing hands
• OSS, portable OS‟s magnify all of the positive effects
listed above, commercial friendly licensing is a further
magnifier (Apache2/MIT)
© 2012 VMware, Inc. All rights reserved 16
17. linux of the cloud
© 2012 VMware, Inc. All rights reserved 17
18. recent generations in computing
CP/M MS/DOS WINDOWS NT
micro‟s 8080, z-80 pc‟s x86 family
workgroup, servers
OS/400, Novell, NT
Version 6,7 SunOS, 4.x BSD Linux
mini‟s pdp-11 workstations sun datacenter server x86 vm
super-mini‟s vax, sun, convex, sequent
VMS, Solaris, 4.x BSD
© 2012 VMware, Inc. All rights reserved 18
19. linux du jour summary
• CP/M == linux of the micro-computer era
• MS/DOS == linux of the pc era
• Windows NT == linux of the 32/64bit pc era
• Unix v6/v7, 4.2BSD == linux of the mini-computer era
• SunOS == linux of the workstation era
• Linux == linux of the datacenter server era
??? ?? == linux of the cloud
© 2012 VMware, Inc. All rights reserved 19
20. linux of the cloud
© 2012 VMware, Inc. All rights reserved 20
21. what the os provides
• ability to create and launch applications
• with an appropriate amount of isolation
• with an appropriate level of resource management
• with a high degree of application portability
• app will run on any instance of the os
• apps are largely isolated from hardware
• allow application to interact with environment
• devices
• other computers and services
• modern and relevant to current problems and scale
• foundation capable of supporting an ecosystem
© 2012 VMware, Inc. All rights reserved 21
22. traditional os layering
shell libraries & APIs user apps
system call interface
core operating system
hardware abstraction layer
physical hardware
© 2012 VMware, Inc. All rights reserved 22
23. OS largely defined by its syscall layer
System Info, Misc Mgmt:
- swapon(2), gettimeofday (2)
- mov ah,01Bh; int 21h; drvinfo
- NtEnumerateBootEntries
Process & Memory Mgmt: File, Device, I/O System:
- fork(2), exec(2), sbrk(2), - open(2), readv(2), etc.
- getpagesize(2), sigpause(2) - mov ah,021h; int 21h; write
- NtOpenFile
OS
- mov ah,04Bh; int 21h; exec
- mov ah,031h; int 21h; tsr syscall
Networking:
- socket(2), select(2), bind(2)
- NtCreateThread, NtTerminate - mov ax,5E00h; int 21h; hostname
- NtCreateSection, NtLockVM - NtDeviceIoControlFile
Hardware Abstraction Layer
© 2012 VMware, Inc. All rights reserved 23
24. CP/M
console app to launch apps and built in
pcp
commands (pip, etc.)
bdos function0 - 112
file, file system, device access
bdos
ability to launch applications
raw device access apis
bios
disk/block access
8080, z-80
- shell
- core
os - hal
© 2012 VMware, Inc. All rights reserved - hardware
24
25. CP/M
• operating system was portable across a range of micro-
computers
• bios abstracted away uniqueness for the OS layer
• large market of 8080/z-80 machines had an OS
• apps like Wordstar, Multiplan, dBase II, made real $$
• bdos/bios architecture allowed decent hardware
innovation
• bank-switched multi-user systems
• better serial ports, devices, etc.
© 2012 VMware, Inc. All rights reserved 25
26. MS-DOS
console app to launch apps and built in
command.com
commands (dir, type, etc.)
int 21h
file, file system, device access (prn, aux, com)
pcdos
ability to launch applications, mild memory mgmt
raw device access
ibm pc-bios
pc io, disk, graphics
apps coded to hardware as needed
8086+
- shell
- core
os - hal
© 2012 VMware, Inc. All rights reserved - hardware
26
27. MS-DOS – PC-DOS
• started diverse and portable (MS-DOS), morphed to
IBM compatible and PC-DOS
• pc rom-bios was extremely complete and accessible,
grew well with each advancement in hw
• virtually all x86 (including 8088) ran PC-DOS with a
cloned rom-bios
• massive market for ISV software, PC-DOS was the DOS
powering Windows 95…
• IBM PC compatibility was all that mattered
• huge market of machines to run PC-DOS/Windows16
• in exchange for highly constrained pc innovation
© 2012 VMware, Inc. All rights reserved 27
28. WINDOWS NT
32-bit shell, based on win32 base api
command.exe
ntdll/zwapi,ntapi
file, file system, device access, networking
NT Executive/Kernel
smp, process and threading, rich memory mgmt
raw device access like busses, interrupt
hal
controllers, timers, etc.
hardware access by kernel and hal only
i386+
- shell
- core
os - hal
© 2012 VMware, Inc. All rights reserved - hardware
28
29. WINDOWS NT
• 1988 – smp, pre-emptive kernel, highly portable
• high degree of portability across all modern laptops,
desktops, x86 based systems. portable across other
cpu architectures (but all of those proved niche…)
• NT HAL allowed a wide degree of freedom for hardware
guys (at high and low end)
• huge market for apps from Office to games and
everything in between
“Windows runs best in a VM. One that you can
throwaway when windows gets sick and infected
with registry cancer”
© 2012 VMware, Inc. All rights reserved 29
30. Unix, Linux
advanced shell with great set of commands
sh
section(2)
file, file system, device access, networking
kernel
smp, process and threading, rich memory mgmt
raw device access like busses, interrupt
arch modules
controllers, timers, etc. (asm/*, arch/*)
hardware access by kernel and arch only
68000 – x86++
- shell
- core
os - hal
© 2012 VMware, Inc. All rights reserved - hardware
30
31. Unix/Linux
• Version 6 – 1975, Version 7 – 1979: pdp-11 focused,
variations for lsi-11.
• V7 – first highly portable unix: (68010, Z8000, 8086,
Vax, Eclipse, Sun, etc.) Networking, demand paged mm,
non-pre-emptive kernel.
• 4.2BSD – 1983 networking, demand paging, perf
• hal layer implemented by convention, as needed, in
custom arch/* and asm/* source code. solid portable
core so design is extremely well exercised.
• Linux – 1991 == the endgame for Unix. open source,
unix-like core, portable, licensing compatible with large
webfarms
© 2012 VMware, Inc. All rights reserved 31
33. os layering review
shell libraries & APIs user apps
system call interface
core operating system
hardware abstraction layer
physical hardware
© 2012 VMware, Inc. All rights reserved 33
34. the role of IaaS
• programmable, on-demand, custom configured hw
• old-school BIOS API: create vga device
• mov ah,0; mov al,12h; int 10h
• cloud-era BIOS API: create quad-core, 16GB server
• Bosh::Cloud.create_stemcell(path, props)
IaaS == the BIOS of the cloud
IaaS du jour aws OpenStack
HAL vSphere google Azure
cloudstack eucalyptus Hyper-V
physical multi-blade, multi-core, large memory x86,
hardware unlimited net, storage, dc
© 2012 VMware, Inc. All rights reserved 34
35. aws == AMI of the cloud?
© 2012 VMware, Inc. All rights reserved 35
36. cloud os layering
shell:
shell - modern, scriptable, cli, thin layer above REST API
- webui, control panel and IDE integrations
system call
core os:
interface
- system call layer implemented as REST endpoint
- ability to create and launch apps
core os - portable across all IaaS hardware abstractions
- OSS with liberal licensing (MIT/Apache2)
IaaS Equalizer Interface - CPI
HAL IaaS: aws, vSphere, OpenStack, eucalyptus, etc.
physical multi-blade, multi-core, large memory x86,
hardware unlimited net, storage, dc
© 2012 VMware, Inc. All rights reserved 36
37. cloud foundry os layering
vmc shell:
- scriptable CLI
- thin UI layer above REST API
- STS plugin for integration into Eclipse, Cloud9 IDE, etc.
vcap core:
- REST control api, uniform syscall API across all cloud foundry instances
- highly portable application model, URL addressable apps
- service layer: uniform access to files, databases, kv stores, etc. uniform
- IaaS/HAL abstraction orchestrated by BOSH
- OSS with liberal licensing Apache2
aws OpenStack vSphere
Azure
google eucalyptus cloudstack
Hyper-V
multi-blade, multi-core, large memory x86, unlimited net, storage, dc
© 2012 VMware, Inc. All rights reserved 37
38. cloud foundry syscall
REST control API:
app management
service management
user/system management
app management: service management:
- http url addressable apps - next-gen “I/O” system
- simple horizontal scale out - unified discovery/binding
- REST api OS - REST api
- vmc push; vmc update syscall - vmc services
- vmc stop; vmc start - vmc create-service
- vmc mem; vmc map - vmc delete-service
- vmc instances - vmc bind-service
- vmc {files, logs, crashlogs} - vmc unbind-service
Hardware Abstraction Layer
© 2012 VMware, Inc. All rights reserved 38
39. cloud foundry is a modern cloud os
• cloud-centric application launch API
• horizontally scalable instances, full memory control
• http addressable and background apps
• highly portable, end-to-end OS
• BOSH – tool-chain for release engineering, deployment,
and lifecycle management of large scale distributed
systems. CPI based infrastructure abstraction
• VCAP – Cloud Foundry core (the green box). OSS,
infrastructure neutral
• next generation service-based “I/O” system API
• service-independent: discovery, binding, lifecycle API
• connects apps to services: ~open(„/dev/mongoDB‟)
• OSS with liberal Apache2/MIT licensing
© 2012 VMware, Inc. All rights reserved 39
40. generations in computing
Cloud Foundry ???
cloud vSphere, aws, OpenStack, eucalyptus, Azure, Hyper-V, cloudstack, google
CP/M MS/DOS WINDOWS NT
micro‟s 8080, z-80 pc‟s x86 family
workgroup, servers
OS/400, Novell, NT
Version 6,7 SunOS, 4.x BSD Linux
mini‟s pdp-11 workstations sun datacenter server x86 vm
super-mini‟s vax, sun, convex, sequent
VMS, Solaris, 4.x BSD
© 2012 VMware, Inc. All rights reserved 40