Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

BigTop vm and docker provisioner

2,851 views

Published on

A quick introduction to BigTop's VM and Docker provisioner

Published in: Software
  • Be the first to comment

BigTop vm and docker provisioner

  1. 1. BigTop VM and Docker Provisioner Evans Ye 2015.2.17
  2. 2. Apache Bigtop is a project for… • Packaging • Packages Hadoop ecosystem • Testing • integration/interoperability testing framework • Virtualization • vagrant/docker based hadoop provisioning • Deployment • puppet recipes for included components
  3. 3. What does vagrant-puppet do?
  4. 4. 1) Create a set of VMs by vagrant box 2/17/2015 Vagrant Box (Centos, Ubuntu,…) Virtualbox VMs
  5. 5. 2) Install puppet agents 2/17/2015
  6. 6. 3) Deploy Hadoop by bigtop puppet (masterless) 2/17/2015 puppet apply -d --modulepath="bigtop-deploy/puppet/modules:/etc/puppet/modules" bigtop-deploy/puppet/manifests/site.pp puppet apply… puppet apply… puppet apply…
  7. 7. One click auto deployment 2/17/2015 $ cd bigtop-deploy/vm/vagrant-puppet-vm $ vagrant up
  8. 8. And the docker-puppet?
  9. 9. Similar to vagrant-puppet but spin up a cluster on top of Linux containers 2/17/2015 $ cd bigtop-deploy/vm/vagrant-puppet-docker $ ./docker-hadoop.sh --build-image --create 3
  10. 10. Some prerequisites (see each README.md) vagrant-puppet docker-puppet 2/17/2015 Confidential | Copyright 2013 TrendMicro Inc. 2
  11. 11. Use cases • for BigTop • to do bigtop integration tests • to test bigtop puppet recipes • integrates packaging, deployment, and testing all into one package (demonstrate the BigTop life cycle) • for Contributors & Users • run round trip test locally (from source code to a testing cluster) • easy-to-get Hadoop cluster environment 2/17/2015 Confidential | Copyright 2013 TrendMicro Inc. 2
  12. 12. Execute BigTop smoke-tests • vagrant-puppet • README.md • vagrantconfig.yaml • set run_smoke_tests: true • Vagrantfile • docker-puppet • ./docker-hadoop.sh --smoke-tests 2/17/2015 Confidential | Copyright 2013 TrendMicro Inc. 2

×