Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
Ubiquitous Computing and AmI Smart Environments
Next
Download to read offline and view in fullscreen.

Share

The Architecture of Continuous Innovation - OSCON 2015

Download to read offline

For many years, the gold standard of business strategy has been the mantra “Sustainable competitive advantage.” But the world has changed. Moving forward, the mantra for survival must be “Continuous innovation.”

In this talk, I will take the audience inside the architectural foundation of a modern cloud native platform. I’ll walk through the tools they’ll use to deliver on the promise of continuous innovation — tools such as Docker, Lattice, Puppet, and Cloud Foundry. And I’ll show examples of how to use those tools to deliver the speed and portability businesses need to thrive in a cloud native world.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

The Architecture of Continuous Innovation - OSCON 2015

  1. 1. Chip Childers | @chipchilders VP Technology | Cloud Foundry Foundation The Architecture of Continuous Innovation
  2. 2. Continuous Innovation
  3. 3. Application Patterns Are Changing
  4. 4. Microservices are great. Per Martin Fowler they lead to specific requirements: rapid provisioning basic monitoring rapid application deployment devops culture
  5. 5. • Use declarative formats for setup automation, to minimize time and cost for new developers joining the project; • Have a clean contract with the underlying OS, offering maximum portability between execution environments; • Are suitable for deployment on modern cloud platforms, obviating the need for servers and systems administration; • Minimize divergence between development and production, enabling continuous deployment for maximum agility; • And can scale up without significant changes to tooling, architecture, or development practices.
  6. 6. But even that’s not enough…
  7. 7. • Role based access to resources: the right people should be able to do things and the wrong people shouldn’t • Run specified bits on demand: take code, put it together with all the rest of the things it needs and and get it running • Coordinate cross service configurations: in a service oriented world, services need to be configured to connect with each other • Route public requests to running bits: the next big thing needs access to the internet • Read and write persistent data: data has to live somewhere • Add and remove resources: scaling is a great problem to have, but still • Isolate resources and failures without isolation and decoupling, that is one big distributed single point of failure • Measure performance/health: can’t manage what you don’t measure • Detect and determine failure: sometimes, things get real… but how do you know • Recover failures: someone is going to have to clean this mess • Work tomorrow: when everything you’ve thought to be true has been shown not to
  8. 8. Containers Are Awesome, but Not Enough
  9. 9. Cloud Native Application Platform
  10. 10. Platforms on Platforms • Better SLAs • Flexibility • Speed • Availability • Faster Time To Market • Mobile + Data Services • Agile and Iterative • Leverage OSS • Continuous Delivery • No Downtime • Instant scaling • Consistency & Automation App Dev App OpsIaaS
  11. 11. Understanding Cloud Native Application Platforms .war .jar dependencies libraries service manifest App App App LB DB Multi-server run time environment(s) .tar.gz Turning this: Into this:
  12. 12. Unit of Value IaaS == Virtual Machine • Opaque to the system • Orchestration is post-hoc • System changes are imperative (“launch” stuff) App Platform == Application • Containers are transparent • Lifecycle is fully managed • System changes are declarative (manifest.yml)
  13. 13. Removing Developer and Operational Constraints BUILD APPLICATION PUSH FIRST RELEASE MAINTAIN APPLICATION UPDATE APPLICATIONS RETIRE APPLICATIONS • Auto-detect frameworks • Link to App Platform • Self-service deploy • Dynamic routing • A/B versioning • Live upgrades • Self-service removal • Elastic scale • Integrated HA • Log aggregation • Policy and Auth
  14. 14. Prescriptive Assembly CHRONOS runC scheduler.next container.next
  15. 15. Prescriptive Assembly CHRONOS runC scheduler.next gorouter CloudController Auth Loggregator Staging Buildpacks BOSH Service Broker Diego Linux Windows Docker etcd Core Services container.next
  16. 16. gorouter CloudController Auth Loggregator Staging Buildpacks BOSH Service Broker Diego Linux Windows Docker etcd Core Services
  17. 17. Let’s talk about Diego ? ? ?
  18. 18. ?
  19. 19. ? DIEGO is a distributed system that orchestrates containerized workloads
  20. 20. ? DIEGO a distributed system that orchestrates containerized workloads Cells Brain BBS (currently etcd)
  21. 21. ? Cells Brain BBS (currently etcd) scheduler DIEGO a distributed system that orchestrates containerized workloads
  22. 22. ? Cells Brain BBS (currently etcd) scheduler DIEGO a distributed system that orchestrates containerized workloads
  23. 23. ? Cells Brain BBS (currently etcd) scheduler DIEGO a distributed system that orchestrates containerized workloads
  24. 24. ? Cells Brain BBS (currently etcd) scheduler DIEGO a distributed system that orchestrates containerized workloads
  25. 25. ? Cells Brain BBS (currently etcd) health-monitor DIEGO a distributed system that orchestrates containerized workloads
  26. 26. ? Cells Brain BBS (currently etcd) health-monitor DIEGO a distributed system that orchestrates containerized workloads
  27. 27. ? Cells Brain BBS (currently etcd) health-monitor DIEGO a distributed system that orchestrates containerized workloads
  28. 28. ? Cells Brain BBS (currently etcd) health-monitor DIEGO a distributed system that orchestrates containerized workloads
  29. 29. ? Cells Brain BBS (currently etcd) health-monitor DIEGO a distributed system that orchestrates containerized workloads
  30. 30. ? Cells Brain BBS (currently etcd) health-monitor DIEGO a distributed system that orchestrates containerized workloads
  31. 31. ? Cells Brain BBS (currently etcd) health-monitor DIEGO a distributed system that orchestrates containerized workloads
  32. 32. ? DIEGO runs one-off tasks long running processes a distributed system that orchestrates containerized workloads
  33. 33. ? Task a unit of work runs at most once DIEGO runs a distributed system that orchestrates containerized workloads long running processes
  34. 34. ? Task LRP a unit of work runs at most once N long-running instances distributed across cells for HA monitored & restarted DIEGO runs a distributed system that orchestrates containerized workloads
  35. 35. ? generic, platform independent, abstraction DIEGO runs a distributed system that orchestrates containerized workloads Task LRP
  36. 36. ? generic, platform independent, abstraction DIEGO runs a distributed system that orchestrates containerized workloads Task LRP
  37. 37. ? working today DIEGO runs a distributed system that orchestrates containerized workloads generic, platform independent, abstraction Task LRP
  38. 38. ? DIEGO runs a distributed system that orchestrates containerized workloads successful abstraction Task LRP working today
  39. 39. …confusion
  40. 40. …confusion = ?
  41. 41. …confusion = ?
  42. 42. …confusion ? ?
  43. 43. isolation ? ? ? ?
  44. 44. isolation shared resources kernel resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 ? ?
  45. 45. isolation CPU kernel resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 ? ?
  46. 46. isolation resource isolation namespace isolation CPU processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 ? ?
  47. 47. isolation resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 CPU ? ?
  48. 48. isolation resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 CPU ? ?
  49. 49. isolation resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 cgroups CPU ? ?
  50. 50. isolation resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 cgroups processD processE processF CPU ? ?
  51. 51. isolation shared resources kernel resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 ? ?
  52. 52. isolation kernel resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 ProcessID ? ?
  53. 53. isolation resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 PID 2 3 4 5 6 7 ? ?
  54. 54. isolation resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 PID 2 3 4 5 6 7 ? ?
  55. 55. isolation resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 PID 2 3 4 5 6 7 ? ?
  56. 56. isolation resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 PID 2 3 4 5 6 7 PID namespace ? ?
  57. 57. isolation resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 PID 2 3 4 5 6 7 PID namespace ? ?
  58. 58. isolation resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 PID 2 3 4 2 2 3 PID namespace ? ?
  59. 59. isolation resource isolation namespace isolation processA processB processC processD processE processFtenant 1 tenant 2 tenant 3 PID shared resources kernel Network Mount User namespaces ? ?
  60. 60. = isolation User Network cgroups PID ? ? ? ?
  61. 61. = isolation PID User Network cgroups ? ? ? ?
  62. 62. = isolation PID User Network cgroups + contents ? ? ? ?
  63. 63. = isolation PID User Network cgroups + contents + processes ? ? ? ?
  64. 64. =? ? ? ?
  65. 65. Tasks LRPs in ? ?
  66. 66. Tasks LRPs in Garden ? ?
  67. 67. Garden allows Diego to programmatically say “make me a container” “put this in it” “then run this” via a platform-agnostic API ? ?
  68. 68. Garden allows Diego’s abstractions to be flexible ? ?
  69. 69. cf push ? ?
  70. 70. app source code Task staging cf push? ?
  71. 71. cf push compiled asset app + app-specific dependencies assumes a particular execution context cflinuxfs2 ? ?
  72. 72. cf push ? ? ?
  73. 73. cf push LRP ? ?
  74. 74. cf push cflinuxfs2 preloaded rootfs ? ?
  75. 75. cf push cflinuxfs2 preloaded rootfs download droplet ? ?
  76. 76. cf push cflinuxfs2 preloaded rootfs download droplet start command ? ?
  77. 77. cf push Droplet LRP { memory: 128mb, rootfs: “preloaded:cflinuxfs2”, setup: <download-droplet>, run: {metadata}.start-command } ? ?
  78. 78. cf push Droplet LRP { memory: 128mb, rootfs: “preloaded:cflinuxfs2”, setup: <download-droplet>, run: {metadata}.start-command } ? ?
  79. 79. cf push { memory: 128mb, rootfs: “preloaded:cflinuxfs2”, setup: <download-droplet>, run: {metadata}.start-command } Droplet LRP ? ?
  80. 80. cf push { memory: 128mb, rootfs: “preloaded:cflinuxfs2”, setup: <download-droplet>, run: {metadata}.start-command } Droplet LRP ? ?
  81. 81. cf push ? ?
  82. 82. cf push-docker ? ?
  83. 83. cf push-docker? ?
  84. 84. cf push-docker docker image ? ?
  85. 85. cf push-docker docker image docker metadata ? ?
  86. 86. cf push-docker docker image docker metadata docker registry } ? ?
  87. 87. Docker LRP { memory:128mb, rootfs: “docker://docker-image”, run: {docker metadata}.start-command } cf push-docker? ?
  88. 88. cf push-docker docker image docker metadata docker registry } ? ?
  89. 89. Docker LRP { memory:128mb, rootfs: “docker://docker-image”, run: {docker metadata}.start-command } cf push-docker? ?
  90. 90. Docker LRP { memory:128mb, rootfs: “docker://docker-image”, run: {docker metadata}.start-command } cf push-docker? ?
  91. 91. Docker LRP { memory:128mb, rootfs: “docker://docker-image”, run: {docker metadata}.start-command } cf push-docker? ?
  92. 92. ? ? ?
  93. 93. ? (anything) (anything) ? ?
  94. 94. ? ? ?
  95. 95. cf push-docker ? ?
  96. 96. cf push -stack windows ? ?
  97. 97. Garden-Windows resource isolation kernel job object disk quotas namespace isolation user profiles Host Web Core (an isolated IIS instance) Garden-Linux resource isolation cgroups namespace isolation PID Network User Mount ? ?
  98. 98. collaborating with Microsoft Garden-Windows ? ?
  99. 99. Garden-Windows provides a container experience for Windows 2012 that will only get better with Windows 2016 allows us to build a cf push experience ? ?
  100. 100. Garden-Linux Garden-Windows ? ? ?
  101. 101. Garden API ? ?
  102. 102. .net LRP { memory: 128mb, rootfs: “preloaded:windows2012R2”, setup: <download-application> run: {metadata}.start-command } ? ?
  103. 103. .net LRP { memory: 128mb, rootfs: “preloaded:windows2012R2”, setup: <download-application> run: {metadata}.start-command } ? ?
  104. 104. .net LRP { memory: 128mb, rootfs: “preloaded:windows2012R2”, setup: <download-application> run: {metadata}.start-command } ? ?
  105. 105. .net LRP { memory: 128mb, rootfs: “preloaded:windows2012R2”, setup: <download-application> run: {metadata}.start-command } ? ?
  106. 106. .net LRP { memory: 128mb, rootfs: “preloaded:windows2012R2”, setup: <download-application> run: {metadata}.start-command } ? ?
  107. 107. 3 different contexts ? ?
  108. 108. 1 cluster ? ?
  109. 109. Cloud Controller DEA cf push stage DEA DEA DEA run ? Current CF Architecture (Simplified)
  110. 110. Cloud Controllercf push stage run app-specific generic ?
  111. 111. Cloud Controllercf push stage run CC Bridge Cells BrainBBS ReceptorAPI ?
  112. 112. Cloud Controller CC Bridge generic consumer ? Cells BrainBBS ReceptorAPI
  113. 113. Cloud Controller CC Bridge BrainBBS generic consumer other consumers? ? Cells BrainBBS ReceptorAPI
  114. 114. Cells BrainBBS Task or LRP ReceptorAPI
  115. 115. Cells BrainBBS Task or LRP meh ReceptorAPI
  116. 116. Cells BrainBBS Task or LRP gorouter http traffic ReceptorAPI
  117. 117. Cells BrainBBS Task or LRP gorouter http traffic loggregator logs ReceptorAPI
  118. 118. vagrant up
  119. 119. vagrant up terraform apply
  120. 120. vagrant up terraform apply ltc create <app>
  121. 121. lattice.cf
  122. 122. lattice.cf Local VM
  123. 123. lattice.cf Local VM AWS Digital Ocean Google Cloud Platform OpenStack
  124. 124. ?Why ?
  125. 125. ? CC UAA Diego Loggregator Gorouter Buildpacks Services BOSH
  126. 126. ? CC UAA Diego Loggregator Gorouter Buildpacks Services BOSH
  127. 127. ? CC UAA Diego Loggregator Gorouter Buildpacks Services BOSH
  128. 128. ? CC UAA Diego Loggregator Gorouter Buildpacks Services BOSH single-tenant
  129. 129. ? CC UAA Diego Loggregator Gorouter Buildpacks* Services BOSH docker single-tenant
  130. 130. ? CC UAA Diego Loggregator Gorouter Buildpacks* Services BOSH BYOS docker single-tenant
  131. 131. ? CC UAA Diego Loggregator Gorouter Buildpacks* Services BOSH no rolling upgrades BYOS docker single-tenant
  132. 132. ? ?Why
  133. 133. ? …is a useful low-barrier solution to real-world problems …makes exploring Diego easy …is a softer onramp to the CF tech stack …allows us to efficiently prototype new ideas for Diego’s future Lattice…
  134. 134. WHEN?
  135. 135. “rewrite the DEA” Diego’s scope is much more than
  136. 136. Diego is running in production on PWS Managing ~5% of the load
  137. 137. Diego is in beta while we validate performance at O(~100s) of cells secure Diego’s internal components
  138. 138. Start using it alongside the DEAs now and give us feedback
  139. 139. Diego should be out of beta within Q3 (probably)
  140. 140. Then what?
  141. 141. Placement Constraints top of backlog post-beta
  142. 142. cf ssh <app/index> working now, CLI support on the way shell access, port forwarding, scp
  143. 143. TCP Routing
  144. 144. Private Docker Registry
  145. 145. Support for persistence (a long term goal)
  146. 146. Container-Container networking (a long term goal)
  147. 147. Condenser lightweight buildpacks for Lattice
  148. 148. ? ? ?
  149. 149. A Cloud Foundry is a place of practice for continuous innovation. noun pragmatic cathedral Chip Childers | @chipchilders VP Technology | Cloud Foundry Foundation
  • devrimdevrim71

    Feb. 2, 2016
  • IanMorrison4

    Sep. 2, 2015
  • csterwa

    Jul. 24, 2015

For many years, the gold standard of business strategy has been the mantra “Sustainable competitive advantage.” But the world has changed. Moving forward, the mantra for survival must be “Continuous innovation.” In this talk, I will take the audience inside the architectural foundation of a modern cloud native platform. I’ll walk through the tools they’ll use to deliver on the promise of continuous innovation — tools such as Docker, Lattice, Puppet, and Cloud Foundry. And I’ll show examples of how to use those tools to deliver the speed and portability businesses need to thrive in a cloud native world.

Views

Total views

1,415

On Slideshare

0

From embeds

0

Number of embeds

53

Actions

Downloads

34

Shares

0

Comments

0

Likes

3

×