Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Openshift/Kubernetes integration with Apache YARN


Published on

Openshift/Kubernetes integration with Apache YARN
Shows how YARN allows Big Data/Hadoop and other workloads to run side by side on same cluster

Published in: Technology
  • Be the first to comment

Openshift/Kubernetes integration with Apache YARN

  1. 1. Page1 © Hortonworks Inc. 2014 OpenShift scheduling docker containers in YARN via Kubernetes
  2. 2. Page2 © Hortonworks Inc. 2014 Static website Web frontendUser DB Queue Analytics DB Development VM QA server Public Cloud Contributor’ s laptop Docker: a shipping container system for codeMultiplicityofStacks Multiplicityof hardware environments Production Cluster Customer Data Center Doservicesandapps interact appropriately? CanImigrate smoothlyandquickly …that can be manipulated using standard operations and run consistently on virtually any hardware platform An engine that enables any payload to be encapsulated as a lightweight, portable, self-sufficient container…
  3. 3. Page3 © Hortonworks Inc. 2014 Why are Docker containers lightweight? HDP 2.1 Hortonworks Data Platform I/O performance comparable to Bare Metal
  4. 4. Page4 © Hortonworks Inc. 2014 Kubernetes – container orchestrator • Service for container cluster management • Allows deploying and managing applications running on multiple hosts using docker • Open sourced by Google • Supports GCE, CoreOS, Azure, vSphere • Used to manage Docker containers as a default implementation • Master – maintain state of Kubernetes Server runtime • Scheduler, API server, registries, storage • Minions – represent the host were containers created • Kubelet – manage pod and container lifecycle
  5. 5. Page5 © Hortonworks Inc. 2014 OpenShift • Red Hat’s platform as a service for applications in the cloud that supports both public and private cloud • Provides high level abstraction for applications on top of containers allowing easy scaling, service discovery, and deployment • Enable Docker image authors to easily deliver reusable application components, including highly available databases, monitoring and log aggregation tools, service discovery platforms, and prepackaged web applications • Allow developers to deeply customize their runtime environments while preserving operational support at scale for those applications
  6. 6. Page6 © Hortonworks Inc. 2014 6 Kubernetes/YARN Integration YARN Node Manager YARN Node Manager Kubernetes AppMaster
  7. 7. Page7 © Hortonworks Inc. 2014 Demo: OpenShift scheduling docker container in YARN Running in docker
  8. 8. Page8 © Hortonworks Inc. 2014 Demo: 1. Launch Docker container on YARN via OpenShift #openshift kube create pods –c demo.json #cat demo.json
  9. 9. Page9 © Hortonworks Inc. 2014 Demo: 2. Enter into container and confirm web server started #openshift kube create pods –c demo.json
  10. 10. Page10 © Hortonworks Inc. 2014 Demo: 3. Webapp on Openshift interacting with Hadoop
  11. 11. Page11 © Hortonworks Inc. 2014 Demo: 4. Launch other workloads on YARN YARN allows Big Data and other workloads to run side by side on same cluster
  12. 12. Page12 © Hortonworks Inc. 2014 Questions?