Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Docker & aPaaS: Enterprise Innovation and Trends for 2015

5,409 views

Published on

WaveMaker Webinar: Cloud-based App Development and Docker: Trends to watch out for in 2015 - http://www.wavemaker.com/news/webinar-cloud-app-development-and-docker-trends/

CIOs, IT planners and developers at a growing number of organizations are taking advantage of the simplicity and productivity benefits of cloud application development. With Docker technology, cloud-based app development or aPaaS (Application Platform as a Service) is only becoming more disruptive − forcing organizations to rethink how they handle innovation, time-to-market pressures, and IT workloads.

Published in: Technology
  • Be the first to comment

Docker & aPaaS: Enterprise Innovation and Trends for 2015

  1. 1. Docker & aPaaS: Enterprise Innovation and Trends for 2015 Samir Adams Ghosh, CEO Anurag Parashar, Technical Architect WaveMaker, Inc.
  2. 2. Presenters 2 Technical Architect Samir Ghosh CEO Anurag Parashar WaveMaker, Inc. • Headquartered in Mountain View, CA • Enterprise Software Platform for Custom App Delivery
  3. 3. Enterprises Facing Increasing Challenges • Global Competition • Scarce, Expensive Resources • Complex Supply Chains • Social-media Empowered Customers • Demanding Consumerized Employees • Disruptive Business Models • Governance and Compliance
  4. 4. Successful Enterprises Enable Agility Democratize Decision Making Foster Innovation IBM CIO Study – Successful Companies… • Build a workplace strategy and roadmap that drives innovation and productivity • Enable secure anytime, anywhere access instead of restricting usage
  5. 5. Custom Applications are Key Enablers Making information more broadly available Capturing and refining competitive and innovative processes Gather input and interaction from broader constituents Improved accessibility = faster decision making Automation for consistency and speed Improved customer service
  6. 6. Competitive advantage: Get the top-tier right 6 High Volume of Change. Experimentation. Systems of Innovation Incremental Change Systems of Differentiation Data. Low change apps. Systems of Record ”Pace Layering” - Gartner
  7. 7. Why new Applications are important? 7 Software Is Eating the World Mark Andreesen, August 2011* All New Business Strategy is powered by New Software *Read more at http://online.wsj.com/news/articles/SB10001424053111903480904576512250915629460
  8. 8. But there is a problem… 8
  9. 9. Enterprise App Gaps Build Run Business Pro Developer IT Ops 9 Long-tail app / Shadow IT problem Pro dev doesn’t want Self-service vs. Control/Legacy lock-in to RAD tools
  10. 10. Trends in Enterprise App Delivery Development Integration Deployment 10 Rapid API Application Development (RAAD) App Optimized Cloud (Docker containers) Microservices Architecture
  11. 11. Trends in Enterprise App Delivery Development Integration Deployment 11 Rapid API Application Development (RAAD) App Optimized Cloud (Docker containers) Microservices Architecture
  12. 12. Rapid Developers Business Expert Pro Developer • Non-programmer • Needs app quickly • Cannot wait for IT • Time constrained • Limited front end experience • Needs to extend
  13. 13. Modern RAAD Visual Development •WYSIWYG, Drag & Drop, Live Data Standards Based Code • AngularJS, Java/JEE, Spring, Hibernate Auto Device Responsiveness Easy Services Consumption Re-usable Components Avoid Vendor Lock-in •Open Source, Support Extendable and Maintainable • Eclipse, Maven Data Visualization SaaS & Desktop • Evaluate using data behind firewall Social Team Development •Multi-dev, feeds, follow Version Control • Git, BitBucket, Apache SVN 1-Click Deployment
  14. 14. Poll How should business users create custom apps for their needs? • They shouldn’t • Let them choose the tools of their choice • Provide company approved tool(s) 14
  15. 15. Trends in Enterprise App Delivery Development Integration Deployment 15 Rapid API Application Development (RAAD) App Optimized Cloud (Docker containers) Microservices Architecture
  16. 16. APIs are becoming Ubiquitous 16 11,927 APIs Sept 2014
  17. 17. API Success 17
  18. 18. Poll How easy is it to leverage APIs within your organization (or, if you’re a vendor, in the average enterprises you work with)? • Impossible • Difficult even for the Pros • Fine if you’re a Professional Developer • Possible for even non-Professional Developers • Super easy 18
  19. 19. Trends in Enterprise App Delivery Development Integration Deployment 19 Rapid API Application Development (RAAD) App Optimized Cloud (Docker containers) Microservices Architecture
  20. 20. Challenges Implementing Docker for enterprise application delivery
  21. 21. Problem – A lot of scripts http://blog.docker.com/2013/08/paas-present-and-future/
  22. 22. Problem – A lot of scripts • Combination of technologies (Java, Node, Ruby …), – Layers (DB, Web app, LB …), – Targets (dev machine, staging, production …) – Platforms (linux, windows, cloud …) • What?! Microsoft will also support Docker? • Consistent delivery is challenging, – Even with best devops tools: Puppet, Chef, Ansible ...
  23. 23. Ideal solution = run exactly the same image everywhere
  24. 24. VM vs Container. • Containers are “orders of magnitude better” than virtual machines. • Shared OS • “Multi-tenancy at OS level” • Building VMs take mins to hrs • Launching VMs takes minutes vs sub-seconds for container. 24 Phil Wainewright, Diginomica 2-Jul-2014
  25. 25. Virtualization Approaches Compared. Hypervisor Vs. Container 25 More Juice Same Glass Hypervisor-based VM Docker on bare metal Multiple-OS-layers
  26. 26. Virtualization Approaches Compared. Application density 26 Reserve Capacity App#1 App Stack Reserve space could hold other apps App#2 App could grow using available capacity App#1 Shared App Stack Hypervisor-based VM Docker on bare metal
  27. 27. What is Container? • Not a new concept – Solaris Zones – BSD Jails – Warden containers for CloudFoundry – LXC • Isolated OS process with its own process space, network interface, user space • Share kernel with host
  28. 28. What is Container? • Isolation with namespaces for – pid, mnt, net, uts , ipc, user • Control with cgroups for – memory, cpu, blkio, devices
  29. 29. Why Docker? • Commoditize containers – Usable tools and APIs • Copy-on-write and layers for quick provisioning • Create and share the images • Standard format for container • Reproducible images with DockerFile
  30. 30. Use Case Dockerfile Source Code Repository Docker Engine Docker Registry Build Docker Host 2 OS (Linux) Container A Container B Container C Container A Push Search Pull Run Host 1 OS (Linux)
  31. 31. What does Docker provide? • Compose application stack with Docker file • Ability to run more than one Docker container on VM • Lightweight and provision new containers in seconds • Portable and can be run on any hardware • Complete isolation for app and its stack • Keep the application footprint on the disk minimal – By separating out read-only O/S & software stack directories from writeable app-specific directories • Docker Registry public / private (portability) 31
  32. 32. Is Docker a cloud platform? 32 No, Docker only provides necessary tooling to build your own private cloud It is up-to the Enterprise or the Platform Vendor to get the solution right for the IT and Developer needs, bridging that gap.
  33. 33. What does it take to use Docker to build your cloud platform?
  34. 34. Orchestration – Why? • Micro Services and integration – Easy upgrades and fine grained control • Optimized Utilization • Fault Tolerance • Scalability
  35. 35. Optimized Utilization 35 Hibernate container, freeing the resources utilized App #3 App #3 App #2 App #1 Stack v1.0 App #4 App #1 Stack v1.0 Activate new app container from saved app data 1 Passivate app 2 contents to disk for later activation 3
  36. 36. Fault Tolerance 36 App #3 App #2 App #1 Stack v1.0 App #3 App #4 App #1 Stack v1.0 App #4 App #2 App #1 Stack v1.0 Should be resilient to app & stack failures, as well as VM or hardware problems
  37. 37. Horizontal Scaling & Replication 37 App #2 App #1 Stack v1.0 App #2 App #1 Stack v1.0 Load balance app requests across containers
  38. 38. Orchestration tools • CoreOS/Fleet • Kubernetes from Google • Apache Mesos / Marathon • Consul from Hashicorp • Geard from Redhat • Helios • Centurian • Fig • Shipper • … Docker libswarm - Standardized APIs for Docker Orchestration (and more) - Adaptors for actual implementations - Far from finalized
  39. 39. Orchestration - Bootstrapping Docker • Setting up and managing Docker – Boot2docker for developer machine – Evolving lightweight OS like CoreOS provides Docker by default. Can be initialized with PXE boot. – On other infrastructure like public clouds, OpenStack, VMWare or a single VM setup using Vagrant/Virtual Box • Separate set of instructions. – “docker hosts” command proposal • https://github.com/docker/docker/issues/8681 • Plugins/adaptors for different providers
  40. 40. Orchestration - Communication • Create and manage a multi container service – Docker acquired Fig, plans to include it in Docker product. But multi-host support still evolving. • Inter container communication – Docker provides port mapping for configuring container port to the external port on the machine/VM (-P or –p options) – Docker also has container linking for local containers (link based on container name and environment properties). Static links as of now. – Ambassador pattern for solving static nature of links and providing fault tolerance. – Missing multi host inter container communication with docker container linking. – Docker Networking Proposals • https://github.com/docker/docker/issues/8951 - Multi host networking • https://github.com/docker/docker/issues/8952 - Network Drivers • https://github.com/docker/docker/issues/7468 - Dynamic links • https://github.com/docker/docker/issues/7467 - Upgrade network model • Enable communication among containers running on different machines. – Link containers using Open vSwitch.
  41. 41. Orchestration - Cluster Management • Scheduled containers on a bunch of distributed resources – Constraints (MySQL master and slave not running on the same node) – Scaling resources • Distributed configuration management • Service Discovery • Distributed init system • Logging / Monitoring / Alerting
  42. 42. Orchestration -Schedule containers • Optimized container placement Enterprise have different needs – Heterogeneous workloads – Limit h/w resources • Constraints (e.g. MySQL master / slave containers should not run on same host) • Docker Clustering Proposal https://github.com/aluzzardi/docker/blob/clustering-proposal/ docs/sources/userguide/cluster.md • Relevant products – CoreOS Fleet – Kubernetes – Mesos (can write custom scheduler)
  43. 43. Orchestration - Distributed configuration management Consistent configuration across cluster, distributed locking and consensus. • Master Election – RAFT / Paxos • Relevant products – etcd – Consul – Zookeeper
  44. 44. Orchestration - Service Discovery • Env variables – static • Mounted configuration – still need to reload files • DNS – how will restart with new IP work? • Ambassador Pattern – Still using env variables, but more dynamic and a proxy container can be restarted. • Other products – etcd (CoreOS and Kubernetes use this) – Consul – HAProxy(Marathon) – confd (watcher for etcd, env variable, consul) • Service Events
  45. 45. Orchestration - Cluster Service Manager • For fault tolerance and scalability • Like systemd for cluster • Relevant products – Fleet for CoreOS – Marathon for Mesos – Kubernetes
  46. 46. 46 Managing and migrating data volumes Container Platform Stack Host VM App Data resides in an attached device/volume Volume Layer Container Platform Stack Host VM Move & swap containers 1. Portable containers enable seamless Application Stack upgrades separating Application data from Stack & libraries 2. Containers restored quickly in the event of Host VM failures 3. Allows replication of Containers for scalability Failed VM & Container
  47. 47. Recovery and Backup • File systems which makes snapshotting easier – AUFS file based snapshots – BTRFS other block snapshotting file systems – OverlayFS • Still need to create integration for backup infrastructure and recovery semantics
  48. 48. 48 Recovery and Backup App #3 App #3 App #2 App #1 Stack v1.0 App #2 App #1 Stack v1.0 Keep data transfers for snapshots minimal and hence a faster recovery time Incremental snapshots at time intervals 1 2 OverlayFS/AUFS/BTRFS provides support for incremental snapshots
  49. 49. Upgrades • Support for rolling upgrades for 0-downtime upgrades in redundant distributed deployments
  50. 50. Application Stack Upgrades 50 Create a container with new version of application stack App #2 App #1 Stack v1.0 App #2 App #1 Stack v2.0
  51. 51. Application versions & migration 51 App #1 version2 App #1 Stack v1.0 Swap out older version after successful migration Use same stack for guaranteed stability
  52. 52. Logging / Monitoring / Alerting • cAdvisor for Kubernetes • libswarm (API for aggregation)
  53. 53. Application Stack creation and management • Address complexity of creation of application stack – Developers need not be aware of docker image creation and management – Multi-image orchestrated application stacks need to be made reusable. – Panamax enabling application templates sharing through a central repository. • Better abstraction is needed for fine tuning existing stacks.
  54. 54. Security • Implementation needs to ensure end to end security. • Not giving host root access to containers. • Docker support – SELinux – AppArmor – Seccomp • Application Isolation, disable inter container communication. • Multi-tenancy
  55. 55. Docker Related Infrastructure Technologies
  56. 56. Mesos • Popular open source cluster manager from apache • Enables building resource-efficient distributed systems • Efficient dynamic resource isolation and sharing across multiple distributed applications e.g. Hadoop, Spark, Memcache • Production Tested Application Scaling to 10,000+ nodes
  57. 57. Mesos http://mesos.apache.org/documentation/latest/mesos-architecture/
  58. 58. Mesos Data Center or cloud With Mesos and related technologies • Apache Mesos • Apache ZooKeeper • Mesosphere Marathon • Chronos • HAProxy
  59. 59. Kubernetes Builds upon Google’s more than decade experience with GAE • Manages containers across multiple hosts • Provides mechanisms for deployment, maintenance and scaling application • Multi-cloud support • Pluggable architecture • Self-healing – auto restart, auto-placement, auto-replication
  60. 60. Kubernetes https://github.com/GoogleCloudPlatform/kubernetes/blob/master/DESIGN.md
  61. 61. CoreOS • CoreOS Linux – Minimum Linux OS – Automatic clustering – Automatic updates for security 61 • fleet – cluster-level init system – etcd + systemd https://coreos.com/using-coreos/clustering/
  62. 62. Service Discovery with etcd • etcd provides a service registry for containers • Service metadata can be looked up from etcd • etcd cluster allows for service lookup across a cluster of machines • Services can listen on dependent service lifecycle events, e.g.: Start, stop, update etc. 62 https://coreos.com/using-coreos/
  63. 63. Flocker • Flocker is a data volume manager • Flocker can run multiple containers on multiple nodes. • Connect to any node on a Flocker cluster and traffic is routed to the node hosting the appropriate container (based on port). • Flocker - Application State – Flocker manages ZFS filesystem as Docker volumes. It attaches them to your containers. – Flocker provides tools for copying those volumes between nodes. – If an application container is moved from one node to another, Flocker automatically moves the volume with it.
  64. 64. Flocker https://docs.clusterhq.com/en/latest/introduction.html
  65. 65. Technology Landscape - Summary • Address evolving problems • Fast moving technology space • No standardization PaaS can provide abstraction to handle this change
  66. 66. Docker is not a ready-made solution For Enterprise IT or DevOps to manage their application infrastructure using Docker, they need to understand a complex set of things: – Linux images – Read-Write & Layered file systems – Disk snapshots and other recovery mechanisms – Managing firewall and port bridges – Network configuration for containers – Setting up monitoring – Managing software stacks & their configurations – Automating deployment processes 66
  67. 67. A good Docker PaaS should simplify above complexities, e.g. Simplified Docker Container Management • Console on top of Docker • Provision and Manage Containers and App Stacks within Containers Effective Scheduling and Resource Utilization • Automatic Container Passivation, Hibernation, and Activation Comprehensive Monitoring • Container, Application, Deployment, Instance, Usage Role-based Access Control • For container management operations Data and Incremental Snapshots • Efficient backup support & faster recovery … 67
  68. 68. Poll How likely are you to explore Docker and/or Docker-architected PaaS solutions? • Neither • Docker yes, but probably don’t need Docker PaaS • Docker PaaS, yes. 68
  69. 69. Our Findings • WaveMaker RAAD Studio: millions of downloads • In Beta, we hosted thousands of custom Java apps across 90+ countries via our Docker PaaS on AWS. • We found: – 30x greater VM density – 50-100x performance (e.g., 2 sec vs. 5 min app start in VM) • E.g., Enables hibernation – Literally, 80% cost savings! 69
  70. 70. Summary • Custom apps are key to enabling agility and innovation in the enterprise. • Platform should leverage RAD + Microservices + Docker. • Docker is ideal for optimizing custom app workloads. • A well-architected Docker management platform is needed to manage complexity and extract value out of Docker. 70
  71. 71. Q&A Docker aPaaS trial: WaveMakerOnline.com Thank You Contact us: info@wavemaker.com WaveMaker.com

×