• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Cloud Foundry Open Tour - London
 

Cloud Foundry Open Tour - London

on

  • 2,606 views

This talk includes a brief review of the Cloud Foundry Anniversary event on 4/11/2012, and then goes deeply into the idea of Cloud Foundry as the "linux of the cloud"

This talk includes a brief review of the Cloud Foundry Anniversary event on 4/11/2012, and then goes deeply into the idea of Cloud Foundry as the "linux of the cloud"

Statistics

Views

Total Views
2,606
Views on SlideShare
2,582
Embed Views
24

Actions

Likes
5
Downloads
74
Comments
0

6 Embeds 24

http://lanyrd.com 15
https://twimg0-a.akamaihd.net 5
http://us-w1.rockmelt.com 1
https://si0.twimg.com 1
http://a0.twimg.com 1
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Cloud Foundry Open Tour - London Cloud Foundry Open Tour - London Presentation Transcript

    • cloud foundry update mark lucovsky vp of engineering, cloud foundry
    • agenda • cloud foundry anniversary recap • linux of the cloud© 2012 VMware, Inc. All rights reserved 2
    • cloudfoundry.org© 2012 VMware, Inc. All rights reserved 3
    • the new cloudfoundry.org cloudfoundry.com production updated ~2x/week github github.com/cloudfoundry on +2 and ✓ change pushed to github reviewers +1/-1 Gerrit Code Review +2/-2 committers reviews.cloudfoundry.org test verification score ✓ ✗ git push triggers test execution Jenkins CI ci.cloudfoundry.org© 2012 VMware, Inc. All rights reserved 4
    • 80% of work is below the water line • Developer Visible Features • services • frameworks, runtimes • tools and ide‟s • micro cloud foundry • Contributor Visible Features • re-visiting architectural issues • extensive hardening • algorithm tuning • scalability enhancements • operational enhancements© 2012 VMware, Inc. All rights reserved 5
    • app lifecycle management: at launch • awesome conceptual architecture, logical flows start/stop instances • simplistic design provided a solid conceptual foundation cloud • scalability, robustness, operational controller health efficiency, and extensibility was manager inadequate cc - db actual get expected • deconstruction/refactoring state state • enable multiple independently scalable components NFS • eliminate “boot storms” at scale dea resources, droplets, • tune the hot data paths packages • apply learning from the beta • across the board and to all major fetch system components droplets© 2012 VMware, Inc. All rights reserved 6
    • app lifecycle management: +1yr v2 cloud stager health v2 controller New! manager cc - db staging jobs New! package uaa - db cacheNew! v2* New! dea redis New! uaa - AuthN blobstore staging logs New! New! more new code than old code© 2012 VMware, Inc. All rights reserved 7
    • app execution (DEA): launch and +1yr • Major internal rewrite • Validating isolation model in Jenkins • Live in CF.com ~30d dea.rb container api nat and firewall dea.rb private network apps app private fs warden container* * also running in our Jenkins workers© 2012 VMware, Inc. All rights reserved 8
    • router: at launch and +1yr http request http request nginx nginx (+lua extension) proxied lookup request node:port upstream router.rb locator svc proxied proxied nats request nats request <app>.cf.com <app>.cf.com   node:port node:port dea dea© 2012 VMware, Inc. All rights reserved 9
    • production cloud foundry clusters cloudfoundry.com • 500 – 5,000 VMs • 40+ unique node types • 75+ unique software packages • 75+ unique environments • 2x/week cf.com updates • 24x7x365 non-stop operation • No-downtime deployments • Reliable, robust, repeatable deployments, updates, capacity adjustments • Small teams manage many instances production, staging, stress, qa, dev Google style problem  Google style solution© 2012 VMware, Inc. All rights reserved 10
    • cloud foundry BOSH • cloud foundry BOSH is an open source tool-chain for release engineering, deployment, and lifecycle management of large scale distributed services • prescriptive way of creating releases, managing systems and services • it is not a collection of shell scripts, not a pile of Perl • built to deploy/manage production-class, large scale clusters • production-class cloud foundry clusters: 500+ VMs, 40+ jobs, 75+ packages • multi-node, multi-tier, complex clusters: e.g., our Gerrit/Jenkins Cluster • built for devops usage and scale by a crack team of veterans • a project, not a product: command line interface, YAML, etc. • continuous improvement, iterative development, rough edges github.com/cloudfoundry/bosh© 2012 VMware, Inc. All rights reserved 11
    • BOSH: under the hood BOSH User “BOSH is deployed by BOSH” bosh cli director healthmon cloudfoundry.com db redis nats workers blobs active jobs agent IaaS CPI stemcell disk© 2012 VMware, Inc. All rights reserved 12
    • IaaS neutral by design vSphere: battle tested implementation, thousands of deployments CPI: code complete vCloud Director: “work in progress”, 2H 2012 functional status: “work in progress” Cloud Foundry BOSH Cloud Provider Interface (CPI) contribute: github.com/cloudfoundry/bosh© 2012 VMware, Inc. All rights reserved 13
    • linux of the cloud© 2012 VMware, Inc. All rights reserved 14
    • linux of the cloud most widely used OS generation in computing the primary os of an era © 2012© 2012 VMware, Inc. All rights reserved VMware, Inc. All rights reserved 15
    • value proposition of a “primary os” • developers can target the broadest possible market with highly leveraged portable code • applications have a place to reliably run • customers can use the os knowing they have the broadest choice of applications • there is enough of a market to support a healthy eco- system where money is changing hands • OSS, portable OS‟s magnify all of the positive effects listed above, commercial friendly licensing is a further magnifier (Apache2/MIT)© 2012 VMware, Inc. All rights reserved 16
    • linux of the cloud© 2012 VMware, Inc. All rights reserved 17
    • recent generations in computing CP/M MS/DOS WINDOWS NT micro‟s 8080, z-80 pc‟s x86 family workgroup, servers OS/400, Novell, NT Version 6,7 SunOS, 4.x BSD Linux mini‟s pdp-11 workstations sun datacenter server x86 vm super-mini‟s vax, sun, convex, sequent VMS, Solaris, 4.x BSD© 2012 VMware, Inc. All rights reserved 18
    • linux du jour summary • CP/M == linux of the micro-computer era • MS/DOS == linux of the pc era • Windows NT == linux of the 32/64bit pc era • Unix v6/v7, 4.2BSD == linux of the mini-computer era • SunOS == linux of the workstation era • Linux == linux of the datacenter server era ??? ?? == linux of the cloud© 2012 VMware, Inc. All rights reserved 19
    • linux of the cloud© 2012 VMware, Inc. All rights reserved 20
    • what the os provides • ability to create and launch applications • with an appropriate amount of isolation • with an appropriate level of resource management • with a high degree of application portability • app will run on any instance of the os • apps are largely isolated from hardware • allow application to interact with environment • devices • other computers and services • modern and relevant to current problems and scale • foundation capable of supporting an ecosystem© 2012 VMware, Inc. All rights reserved 21
    • traditional os layering shell libraries & APIs user apps system call interface core operating system hardware abstraction layer physical hardware© 2012 VMware, Inc. All rights reserved 22
    • OS largely defined by its syscall layer System Info, Misc Mgmt: - swapon(2), gettimeofday (2) - mov ah,01Bh; int 21h; drvinfo - NtEnumerateBootEntries Process & Memory Mgmt: File, Device, I/O System: - fork(2), exec(2), sbrk(2), - open(2), readv(2), etc. - getpagesize(2), sigpause(2) - mov ah,021h; int 21h; write - NtOpenFile OS - mov ah,04Bh; int 21h; exec - mov ah,031h; int 21h; tsr syscall Networking: - socket(2), select(2), bind(2) - NtCreateThread, NtTerminate - mov ax,5E00h; int 21h; hostname - NtCreateSection, NtLockVM - NtDeviceIoControlFile Hardware Abstraction Layer© 2012 VMware, Inc. All rights reserved 23
    • CP/M console app to launch apps and built in pcp commands (pip, etc.) bdos function0 - 112 file, file system, device access bdos ability to launch applications raw device access apis bios disk/block access 8080, z-80  - shell  - core os - hal © 2012 VMware, Inc. All rights reserved  - hardware 24
    • CP/M • operating system was portable across a range of micro- computers • bios abstracted away uniqueness for the OS layer • large market of 8080/z-80 machines had an OS • apps like Wordstar, Multiplan, dBase II, made real $$ • bdos/bios architecture allowed decent hardware innovation • bank-switched multi-user systems • better serial ports, devices, etc.© 2012 VMware, Inc. All rights reserved 25
    • MS-DOS console app to launch apps and built in command.com commands (dir, type, etc.) int 21h file, file system, device access (prn, aux, com) pcdos ability to launch applications, mild memory mgmt raw device access ibm pc-bios pc io, disk, graphics apps coded to hardware as needed 8086+  - shell  - core os - hal © 2012 VMware, Inc. All rights reserved  - hardware 26
    • MS-DOS – PC-DOS • started diverse and portable (MS-DOS), morphed to IBM compatible and PC-DOS • pc rom-bios was extremely complete and accessible, grew well with each advancement in hw • virtually all x86 (including 8088) ran PC-DOS with a cloned rom-bios • massive market for ISV software, PC-DOS was the DOS powering Windows 95… • IBM PC compatibility was all that mattered • huge market of machines to run PC-DOS/Windows16 • in exchange for highly constrained pc innovation© 2012 VMware, Inc. All rights reserved 27
    • WINDOWS NT 32-bit shell, based on win32 base api command.exe ntdll/zwapi,ntapi file, file system, device access, networking NT Executive/Kernel smp, process and threading, rich memory mgmt raw device access like busses, interrupt hal controllers, timers, etc. hardware access by kernel and hal only i386+  - shell  - core os - hal © 2012 VMware, Inc. All rights reserved  - hardware 28
    • WINDOWS NT • 1988 – smp, pre-emptive kernel, highly portable • high degree of portability across all modern laptops, desktops, x86 based systems. portable across other cpu architectures (but all of those proved niche…) • NT HAL allowed a wide degree of freedom for hardware guys (at high and low end) • huge market for apps from Office to games and everything in between “Windows runs best in a VM. One that you can throwaway when windows gets sick and infected with registry cancer”© 2012 VMware, Inc. All rights reserved 29
    • Unix, Linux advanced shell with great set of commands sh section(2) file, file system, device access, networking kernel smp, process and threading, rich memory mgmt raw device access like busses, interrupt arch modules controllers, timers, etc. (asm/*, arch/*) hardware access by kernel and arch only 68000 – x86++  - shell  - core os - hal © 2012 VMware, Inc. All rights reserved  - hardware 30
    • Unix/Linux • Version 6 – 1975, Version 7 – 1979: pdp-11 focused, variations for lsi-11. • V7 – first highly portable unix: (68010, Z8000, 8086, Vax, Eclipse, Sun, etc.) Networking, demand paged mm, non-pre-emptive kernel. • 4.2BSD – 1983 networking, demand paging, perf • hal layer implemented by convention, as needed, in custom arch/* and asm/* source code. solid portable core so design is extremely well exercised. • Linux – 1991 == the endgame for Unix. open source, unix-like core, portable, licensing compatible with large webfarms© 2012 VMware, Inc. All rights reserved 31
    • cloud era© 2012 VMware, Inc. All rights reserved 32
    • os layering review shell libraries & APIs user apps system call interface core operating system hardware abstraction layer physical hardware© 2012 VMware, Inc. All rights reserved 33
    • the role of IaaS • programmable, on-demand, custom configured hw • old-school BIOS API: create vga device • mov ah,0; mov al,12h; int 10h • cloud-era BIOS API: create quad-core, 16GB server • Bosh::Cloud.create_stemcell(path, props) IaaS == the BIOS of the cloud IaaS du jour aws OpenStack HAL vSphere google Azure cloudstack eucalyptus Hyper-V physical multi-blade, multi-core, large memory x86, hardware unlimited net, storage, dc© 2012 VMware, Inc. All rights reserved 34
    • aws == AMI of the cloud?© 2012 VMware, Inc. All rights reserved 35
    • cloud os layering shell: shell - modern, scriptable, cli, thin layer above REST API - webui, control panel and IDE integrations system call core os: interface - system call layer implemented as REST endpoint - ability to create and launch apps core os - portable across all IaaS hardware abstractions - OSS with liberal licensing (MIT/Apache2) IaaS Equalizer Interface - CPI HAL IaaS: aws, vSphere, OpenStack, eucalyptus, etc. physical multi-blade, multi-core, large memory x86, hardware unlimited net, storage, dc© 2012 VMware, Inc. All rights reserved 36
    • cloud foundry os layering vmc shell: - scriptable CLI - thin UI layer above REST API - STS plugin for integration into Eclipse, Cloud9 IDE, etc. vcap core: - REST control api, uniform syscall API across all cloud foundry instances - highly portable application model, URL addressable apps - service layer: uniform access to files, databases, kv stores, etc. uniform - IaaS/HAL abstraction orchestrated by BOSH - OSS with liberal licensing Apache2 aws OpenStack vSphere Azure google eucalyptus cloudstack Hyper-V multi-blade, multi-core, large memory x86, unlimited net, storage, dc© 2012 VMware, Inc. All rights reserved 37
    • cloud foundry syscall REST control API: app management service management user/system management app management: service management: - http url addressable apps - next-gen “I/O” system - simple horizontal scale out - unified discovery/binding - REST api OS - REST api - vmc push; vmc update syscall - vmc services - vmc stop; vmc start - vmc create-service - vmc mem; vmc map - vmc delete-service - vmc instances - vmc bind-service - vmc {files, logs, crashlogs} - vmc unbind-service Hardware Abstraction Layer© 2012 VMware, Inc. All rights reserved 38
    • cloud foundry is a modern cloud os • cloud-centric application launch API • horizontally scalable instances, full memory control • http addressable and background apps • highly portable, end-to-end OS • BOSH – tool-chain for release engineering, deployment, and lifecycle management of large scale distributed systems. CPI based infrastructure abstraction • VCAP – Cloud Foundry core (the green box). OSS, infrastructure neutral • next generation service-based “I/O” system API • service-independent: discovery, binding, lifecycle API • connects apps to services: ~open(„/dev/mongoDB‟) • OSS with liberal Apache2/MIT licensing© 2012 VMware, Inc. All rights reserved 39
    • generations in computing Cloud Foundry ??? cloud vSphere, aws, OpenStack, eucalyptus, Azure, Hyper-V, cloudstack, google CP/M MS/DOS WINDOWS NT micro‟s 8080, z-80 pc‟s x86 family workgroup, servers OS/400, Novell, NT Version 6,7 SunOS, 4.x BSD Linux mini‟s pdp-11 workstations sun datacenter server x86 vm super-mini‟s vax, sun, convex, sequent VMS, Solaris, 4.x BSD© 2012 VMware, Inc. All rights reserved 40
    • www.cloudfoundry.com/jobs© 2012 VMware, Inc. All rights reserved 41