Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Couchbase Containers with Bare Metal Performance – Couchbase Live New York 2015


Published on

Containerization, spurred on by Docker’s rise, is swiftly establishing itself as the future of computing. Containers promise a shorter, faster path from development to deploy, and may also improve performance and cost efficiencies. Yet, achieving the promised efficiencies and convenience, especially with applications that require sophisticated networking, remains challenging. Worse, many operators compromise container performance by running them with an unnecessary hardware hypervisor layer that slows performance and increases costs. In this session, Joyent CTO Bryan Cantrill will demonstrate how we can enjoy the simplicity and performance or running Couchbase on bare metal while also achieving the benefits promised by containerization. Joyent’s Triton container-native platform gives applications like Couchbase full access to the network stack and fast I/O that you’d expect of bare metal, combined with the convenience of a Docker-native API to launch those containers across the entire data center. See Bryan deploy, rebalance, and benchmark a high performance Couchbase cluster spread across the data center with just a few Docker commands.

Published in: Software
  • Be the first to comment

Couchbase Containers with Bare Metal Performance – Couchbase Live New York 2015

  1. 1. Couchbase Containers with Bare Metal Performance CTO Bryan Cantrill @bcantrill
  2. 2. Elastic infrastructure, circa Dot Com Boom • In the late 1990s, the only way to meaningfully scale a database was up — and the physical infrastructure had to scale with it • This was excruciatingly expensive — and became a non-starter in the post-apocalyptic nuclear winter of the early 2000s...
  3. 3. Elastic infrastructure, circa Dot Com Bust • The rise of rack-and-stack commodity servers brought with it new distributed software architectures like memcached that were designed to scale across many machines • The rise of these architectures afforded new operational possibilities: if the computer itself is a commodity, why buy it at all? Why not rent from someone who runs it cheaper and better? • From the perspective of compute providers, economies of scale could only be realized if hardware is shared across tenants... • Multi-tenancy demands virtualization, but where in the stack to virtualize?
  4. 4. Hardware-level virtualization? • The historical answer to virtualization — since the 1960s — has been to virtualize the hardware: • A virtual machine is presented upon which each tenant runs an operating system that they choose (and must manage) • There are as many operating systems on a machine as tenants! • Can run entire legacy stacks unmodified... • ...but operating systems are heavy and don’t play well with others with respect to resources like DRAM, CPU, I/O devices, etc. • Hardware-level virtualization limits tenancy and performance!
  5. 5. Platform-level virtualization? • Virtualizing at the application platform layer addresses the tenancy challenges of hardware virtualization, and presents a much more nimble (& developer friendly!) abstraction... • ...but at the cost of dictating abstraction to the developer • This is the “Google App Engine” problem: developers are in a straightjacket where toy programs are easy — but sophisticated applications are impossible • Virtualizing at the application platform layer poses many other challenges with respect to security, containment, etc.
  6. 6. OS-level virtualization? • Virtualizing at the operating system level began with the crude filesystem virtualization of chroot in Seventh Edition Unix • chroot originated with Bill Joy, but specifics are blurry; according to Kirk McKusick, via Poul-Henning Kamp and Robert Watson:
  7. 7. OS-level virtualization • Seeking to provide a security mechanism, FreeBSD extended chroot into jails:
  8. 8. • To provide workload consolidation, Sun introduced complete operating system virtualization with zones (née Project Kevlar) OS-level virtualization
  9. 9. OS-level virtualization! • A single operating system (i.e. a single kernel) allows for efficient use of hardware resources, maximizing tenancy and performance • Disjoint instances, known as containers, are securely isolated and virtualized by the operating system • Gives tenants what appears to be a virtual machine (albeit a very fast one) on which to run higher-level software • Containers combine the developer ease of platform-level virtualization with the generality of hardware-level virtualization!
  10. 10. OS-level virtualization at Joyent • Joyent runs OS containers in the cloud via SmartOS — and we have run containers in multi-tenant production since ~2006 • Adding support for hardware-based virtualization circa 2011 strengthened our resolve with respect to OS-based virtualization • We emphasized their operational characteristics — performance, elasticity, tenancy — and for many years, we were a lone voice...
  11. 11. Containers as PaaS foundation? • Some saw the power of OS containers to facilitate up-stack platform-as-a-service abstractions • For example, dotCloud — a platform-as-a-service provider — built their PaaS on OS containers • Struggling as a PaaS, dotCloud pivoted — and open sourced their container-based orchestration layer...
  12. 12. ...and Docker was born
  13. 13. Docker revolution • Docker has used the rapid provisioning + shared underlying filesystem of containers to allow developers to think operationally • Developers can encode deployment procedures via an image • Images can be reliably and reproducibly deployed as a container • Images can be quickly deployed — and re-deployed • Docker will do to apt what apt did to tar
  14. 14. Broader container revolution • The Docker model has pointed to the future of containers • Docker’s challenges today are largely operational: network virtualization, persistence, security, etc. • Security concerns are not due to Docker per se, but rather to the architectural limitations of the Linux “container” substrate • For multi-tenancy, state-of-the-art for Docker containers is to run in hardware virtual machines (!!) • Deploying OS containers in hardware virtual machines negates their economic advantage!
  15. 15. Container-native infrastructure? • SmartOS has been container-native since its inception — and running in multi-tenant, internet-facing production for many years • Can we achieve an ideal world that combines the development model of Docker with the container-native model of SmartOS? • This would be the best of all worlds: agility of Docker coupled with production-proven security and on-the-metal performance of SmartOS containers • But there are some obvious obstacles...
  16. 16. Docker + SmartOS: Linux binaries? • First (obvious) problem: while it has been designed to be cross- platform, Docker is Linux-centric — and the encyclopedia of Docker images will likely forever remain Linux binaries • SmartOS is Unix — but it isn’t Linux… • Fortunately, Linux itself is really “just” the kernel — which only has one interface: the system call table • We resurrected (and finished) a Sun technology for Linux system call emulation, LX-branded zones, the technical details of which are beyond the scope of this presentation...
  17. 17. LX-branded zones: tl;dr
  18. 18. Docker + SmartOS: Provisioning? • With the binary problem being tackled, focus turned to the mechanics of integrating Docker with SmartOS provisioning • Provisioning a SmartOS zone operates via the global zone that represents the control plane of the machine • docker is a single binary that functions as both client and server — and with too much surface area to run in the global zone, especially for a public cloud • docker has also embedded Go- and Linux-isms that we did not want in the global zone; we needed to find a different approach...
  19. 19. Aside: The power of an interface
  20. 20. Aside: The power of an interface
  21. 21. Aside: The power of an interface
  22. 22. Aside: The power of an interface
  23. 23. Docker Remote API • While docker is a single binary that can run on the client or the server, it does not run in both at once… • docker (the client) communicates with docker (the server) via the Docker Remote API • The Docker Remote API is expressive, modern and robust (i.e. versioned), allowing for docker to communicate with Docker backends that aren’t docker • The clear approach was therefore to implement a Docker Remote API endpoint for SmartDataCenter, our (open source!) orchestration software for SmartOS
  24. 24. Triton: Docker + SmartOS/SmartDataCenter • In March, we launched Triton, which combines SmartOS and SmartDataCenter with our Docker Remote API endpoint • With Triton, the notion of a Docker host is virtualized: to the Docker client, the datacenter is a large Docker host • One never allocates VMs with Triton; all Triton containers are run directly on-the-metal • All of the components to Triton are open source: you can download and install SmartDataCenter and run it yourself • Triton is currently general available on the Joyent Public Cloud!
  25. 25. Container landscape • It is becoming broadly clear that containers are the future of application development and (especially) deployment • But the upstack ramifications are entirely unclear — there are many rival frameworks for service discovery, deployment, etc. • The rival frameworks are all open source: • Unlikely to be winner-take-all • Productive mutation is not just possible but highly likely • Triton takes a deliberately modular approach: the container as general-purpose foundation, not prescriptive framework
  26. 26. Containers and Couchbase • Couchbase is particularly appropriate for containers: its scale-out architecture demands elastic infrastructure — and its use cases demand on-the-metal performance • But hardware-virtualized Docker hosts undermine the efficacy of containers — and force an allocation-oriented disposition instead of allowing a consumption-oriented one • The Triton model represents Couchbase containers without compromise: like Couchbase itself, the infrastructure can grow as needed — while still delivering bare-metal performance!
  27. 27. Future of containers • For nearly a decade, we have believed that OS-virtualized containers represent the future of computing — and with the rise of microservices + Docker, this is no longer controversial • But to achieve the full promise of containers, they must run directly on-the-metal — multi-tenant security is a constraint! • The virtual machine is a vestigial abstraction; we must reject container-based infrastructure that implicitly assumes it • Triton represents our belief that containers needn’t compromise: multi-tenant security, operational elasticity and on-the-metal performance!