SlideShare a Scribd company logo
By Nicolas Poggi @ni_po
provider
July, 2014
Sources: https://github.com/npoggi/vagrant-docker
Docker Barcelona:
+
[+ ]
About Nicolas Poggi @ni_po
Previous:
Current: Organizing s:
Motivation
X
About Vagrant [+ puppet]
 It’s all about AUTOMATION
 Designed to build and setup replicable, isolated, [DEV] envs
 Isolation (similar to docker)
 Package code + environment
 Multiplatform
 Multiple Dev Envs
 Distributable
 Avoids having a central DEV server/s
 Keep different versions
 Simplifies Configuration Management
 (Also Docker’s)
 About puppet
 World’s favorite Configuration Management tool
About Docker
 Light-weight replacement for VMs in several scenarios
 Also for Vagrant
 Based on Linux containers
 Adds abstraction layer
 GIT like for OS images
 Every step creates a different image
 Docker hub integrates with github [and bitbucket]
 Similar to Vagrant
 Vagrantfile ~= Dockerfile
 CONS
 Complex commands
Docker basics
Command Description
 docker
 docker pull
 docker images
 docker ps
 docker run
 docker rm/rmi
 Lists commands
 Downloads container images
 Lists available images
 Lists running containers
 Executes a command in a container
 Deletes images
Docker tutorial
 https://github.com/harbur/docker-workshop
 By Dimitris Kapanidis @spiddy
 Initial Steps:
1. docker version
2. docker pull ubuntu
3. docker pull nginx
4. git clone https://github.com/harbur/docker-workshop
Docker advantages
 Speed
 lightweight
 Isolation
 Automation
 Portability
 Migration
 … more to come
Why Vagrant + Docker provider?
 Same reason on “why Docker”
 Get rid of the VM
 Free system resources
 No reserved RAM memory or CPU/s
 Get back control of your own machine!
 Same reasons on “why Vagrant”
 Best way to learn puppet, so a good way to learn docker
 Vagrant takes care of many manual chores
 …
 Makes sense!
 You are dealing with DEV environments
 Fewer security concerns
 Limited resources on laptops
 Keep active tens of different DEV environments
 …and vagrant up anything!
 Including docker
vagrant up vs. sudo docker run -d -name image_name -p 8080:80 -t user/repo
Requirements
 1.) Get a real open-source GNU OS
 …With the latest kernel, i.e., Archlinux
 Sorry Mac and Windows…
 Don’t cry, boot2dockerto the rescue
 Vagrant fires up the VM on non supported platforms
 Latest Vagrant version
 v1.6 >
 Latest Docker version
 v1.0 >
 Root access (for Docker)
Vagrant providers
 Official providers
 Virtualbox (DEFAULT)
 VMWare
 Amazon
 …
Docker
 YES!
 Other providers
Basic Vagrant commands
Vagrant Docker provider
 vagrant
 up
 provision
 ssh
 halt/suspend
 vagrant
 docker-logs
 docker-run
NOTE: You can still use all of docker commands!
Docker provider for Vagrant
1.) Docker images 2.) Dockerfile
 Use a pre-built docker
image
 Pros
 Easy
 Faster to deploy
 No need to deal with the
Docker file
 Cons
 Less control
 Public images
 Build your own (w/ Vagrant)
 Cons
 More complex
 Slow to start
 (then is cached)
 Pros
 Custom made
 Privately managed
 For free
Two alternative routes
Building a docker image in Vagrant
The hard way
Dockerfile, the requisites for Vagrant
1. Import image FROM repo:tag
2. Create vagrant user
1. create password
2. sudo permissions and password-less
3. Configure SSH
1. Setup keys
4. Install base packages
5. Install Configuration Management system
1. Puppet, etc…
6. Expose SSH port
7. Run SSH as a daemon
Sample Docker file for vagrant
 https://github.com/npoggi/vagrant-docker/blob/master/Dockerfile
The Vagrantfile
 Optional: Setup docker a default provider
 Select
 If to build from a Dockerfile
 Or use a prebuilt docker image
 Set has.ssh to true
 Rest as usual in a Vagrantfile
 Ports, mounts
 Config management i.e., puppet
 Example
 https://github.com/npoggi/vagrant-docker/blob/master/Vagrantfile
Building, running, setting up, deploying, etc.
just
vagrant up
Use prebuilt docker images for Vagrant
The easy way
Examples at:
https://hub.docker.com/u/npoggi/
https://registry.hub.docker.com/search?q=vagrant
Other ways to create images
1. Use docker
2. Docker Hub
1. Link your github project (public or private (at a cost))
2. Select Automate build
3. Point to your Docker file
4. And … Wait for the build to complete!
Saving state [and time] - Docker Hub
 Get container CONTAINER_ID
 docker ps
 Commit image
 docker commit [–m “my comment”] CONTAINER_ID user/repo[:tag]
 (had permissions problems to the main repo)
 Push image
 docker push user/repo[:tag]
Try it yourself
 Install requirements (latest vagrant and docker)
 Basic usage:
1. git clone https://github.com/npoggi/vagrant-docker.git
2. cd vagrant-docker
3. sudo vagrant up
 Test: http://localhost:8080/
 Advanced usage:
 For a vanilla image built using the same docker file:
 DOCKER_IMAGE='npoggi/vagrant-docker‘ sudo vagrant up --provision
 For a pre-provisioned image with the puppet config (saves time)
 DOCKER_IMAGE='npoggi/vagrant-docker-provisioned' sudo vagrant up --provision
Time improvements
Vagrant virtualbox (default) Vagrant docker
 vagrant up: 30.6s
 vagrant halt: 7.1s
 vagrant up: 2.6s
 Vagrant halt: 2.3
Gotchas! …and annoyances
 Docker by default requires root permissions!
 Be ready for BUGS!!!
 Vagrant Docker provider less stable
 Doesn’t save to run docker commands occasionally
 Docker too…
 Renames in machines are a problem (manual work)
 Delete orphaned images with complex commands
 …
 Vagrant package command only for virtualbox
 Docker image downloads quite slow…
 …there is still Mac and Windows
Other considerations and Alternatives
 Can Docker replace vagrant use?
 Yes, but…
 Docker abstraction alternative to Vagrant:
 FIG http://orchardup.github.io/fig/
 Alternatives to Docker in Vagrant
 vagrant-lxc plugin (not official)
vs.
Future steps for the source code
 Multiple Vagrant-Docker machines
 Light-weight OS
 Boot2Docker, CoreOS, Project Atomic
 Vagrant + Docker provider+ Docker
 Inception kick!
 Docker patterns
 Terraform (for PROD)
Tips and tricks
 Update to the latest versions (i.e., github)
 Docker by default requires root permissions!
 Add your user to the docker group
 sudo usermod -a -G docker your_user; newgrp docker; #or relogin
 Change the default image dir (to control repo size)
 Edit the service description add
 -g /home/user/docker_images
 Delete “untagged/orphaned images”
 docker rmi $(docker images -a | grep "^<none>" | awk '{print $3}')
 Enable IPv4 forwarding to allow internet connections
inside the containers
Final notes
 Get back you DEV ENV!
 Speedup development
 R&D
+
[+ ]
 Code
 https://github.com/npoggi/vagrant-docker
 Docker images
 https://registry.hub.docker.com/u/npoggi/
Credits and References
 Vagrant specific
 Main site: http://www.vagrantup.com/
 Alternative to Docker: https://github.com/fgrehm/vagrant-lxc
 PROD Vagrant-like: http://www.terraform.io/
 Docker specific
 Docker: https://www.docker.com/
 Docker hub: https://registry.hub.docker.com
 Light-weight VM:
https://github.com/boot2docker/boot2docker
 Fig (Vagrant alternative): http://orchardup.github.io/fig/
 Puppet labs: http://puppetlabs.com/

More Related Content

What's hot

Vagrant vs Docker
Vagrant vs DockerVagrant vs Docker
Vagrant vs Docker
jchase50
 
Installaling Puppet Master and Agent
Installaling Puppet Master and AgentInstallaling Puppet Master and Agent
Installaling Puppet Master and Agent
Ranjit Avasarala
 
Locally it worked! virtualizing docker
Locally it worked! virtualizing dockerLocally it worked! virtualizing docker
Locally it worked! virtualizing docker
Sascha Brinkmann
 
Learn docker in 90 minutes
Learn docker in 90 minutesLearn docker in 90 minutes
Learn docker in 90 minutes
Larry Cai
 
Vagrant-Overview
Vagrant-OverviewVagrant-Overview
Vagrant-Overview
Crifkin
 
An Introduction to Vagrant and Docker
An Introduction to Vagrant and DockerAn Introduction to Vagrant and Docker
An Introduction to Vagrant and Docker
Scott Lowe
 
Using Docker with Puppet - PuppetConf 2014
Using Docker with Puppet - PuppetConf 2014Using Docker with Puppet - PuppetConf 2014
Using Docker with Puppet - PuppetConf 2014
Puppet
 
It Works On My Machine: Vagrant for Software Development
It Works On My Machine: Vagrant for Software DevelopmentIt Works On My Machine: Vagrant for Software Development
It Works On My Machine: Vagrant for Software Development
Carlos Perez
 
Docker - From Walking To Running
Docker - From Walking To RunningDocker - From Walking To Running
Docker - From Walking To Running
Giacomo Vacca
 
Continuous delivery with Jenkins, Docker and Mesos/Marathon - jbcnconf
Continuous delivery with Jenkins, Docker and Mesos/Marathon - jbcnconfContinuous delivery with Jenkins, Docker and Mesos/Marathon - jbcnconf
Continuous delivery with Jenkins, Docker and Mesos/Marathon - jbcnconf
Julia Mateo
 
Continuous delivery with jenkins, docker and exoscale
Continuous delivery with jenkins, docker and exoscaleContinuous delivery with jenkins, docker and exoscale
Continuous delivery with jenkins, docker and exoscale
Julia Mateo
 
Docker by Example - Basics
Docker by Example - Basics Docker by Example - Basics
Docker by Example - Basics
Ganesh Samarthyam
 
A Hands-on Introduction to Docker
A Hands-on Introduction to DockerA Hands-on Introduction to Docker
A Hands-on Introduction to Docker
CodeOps Technologies LLP
 
Dockerfile Basics | Docker workshop #2 at twitter, 2013-11-05
Dockerfile Basics | Docker workshop #2 at twitter, 2013-11-05Dockerfile Basics | Docker workshop #2 at twitter, 2013-11-05
Dockerfile Basics | Docker workshop #2 at twitter, 2013-11-05
dotCloud
 
Deploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGH
Deploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGHDeploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGH
Deploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGH
Erica Windisch
 
Vagrant For DevOps
Vagrant For DevOpsVagrant For DevOps
Vagrant For DevOps
Lalatendu Mohanty
 
From development environments to production deployments with Docker, Compose,...
From development environments to production deployments with Docker, Compose,...From development environments to production deployments with Docker, Compose,...
From development environments to production deployments with Docker, Compose,...
Jérôme Petazzoni
 
Docker
DockerDocker
Docker
Chen Chun
 
Docker by Example - Basics
Docker by Example - Basics Docker by Example - Basics
Docker by Example - Basics
CodeOps Technologies LLP
 
Docker orchestration
Docker orchestrationDocker orchestration
Docker orchestration
Open Source Consulting
 

What's hot (20)

Vagrant vs Docker
Vagrant vs DockerVagrant vs Docker
Vagrant vs Docker
 
Installaling Puppet Master and Agent
Installaling Puppet Master and AgentInstallaling Puppet Master and Agent
Installaling Puppet Master and Agent
 
Locally it worked! virtualizing docker
Locally it worked! virtualizing dockerLocally it worked! virtualizing docker
Locally it worked! virtualizing docker
 
Learn docker in 90 minutes
Learn docker in 90 minutesLearn docker in 90 minutes
Learn docker in 90 minutes
 
Vagrant-Overview
Vagrant-OverviewVagrant-Overview
Vagrant-Overview
 
An Introduction to Vagrant and Docker
An Introduction to Vagrant and DockerAn Introduction to Vagrant and Docker
An Introduction to Vagrant and Docker
 
Using Docker with Puppet - PuppetConf 2014
Using Docker with Puppet - PuppetConf 2014Using Docker with Puppet - PuppetConf 2014
Using Docker with Puppet - PuppetConf 2014
 
It Works On My Machine: Vagrant for Software Development
It Works On My Machine: Vagrant for Software DevelopmentIt Works On My Machine: Vagrant for Software Development
It Works On My Machine: Vagrant for Software Development
 
Docker - From Walking To Running
Docker - From Walking To RunningDocker - From Walking To Running
Docker - From Walking To Running
 
Continuous delivery with Jenkins, Docker and Mesos/Marathon - jbcnconf
Continuous delivery with Jenkins, Docker and Mesos/Marathon - jbcnconfContinuous delivery with Jenkins, Docker and Mesos/Marathon - jbcnconf
Continuous delivery with Jenkins, Docker and Mesos/Marathon - jbcnconf
 
Continuous delivery with jenkins, docker and exoscale
Continuous delivery with jenkins, docker and exoscaleContinuous delivery with jenkins, docker and exoscale
Continuous delivery with jenkins, docker and exoscale
 
Docker by Example - Basics
Docker by Example - Basics Docker by Example - Basics
Docker by Example - Basics
 
A Hands-on Introduction to Docker
A Hands-on Introduction to DockerA Hands-on Introduction to Docker
A Hands-on Introduction to Docker
 
Dockerfile Basics | Docker workshop #2 at twitter, 2013-11-05
Dockerfile Basics | Docker workshop #2 at twitter, 2013-11-05Dockerfile Basics | Docker workshop #2 at twitter, 2013-11-05
Dockerfile Basics | Docker workshop #2 at twitter, 2013-11-05
 
Deploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGH
Deploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGHDeploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGH
Deploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGH
 
Vagrant For DevOps
Vagrant For DevOpsVagrant For DevOps
Vagrant For DevOps
 
From development environments to production deployments with Docker, Compose,...
From development environments to production deployments with Docker, Compose,...From development environments to production deployments with Docker, Compose,...
From development environments to production deployments with Docker, Compose,...
 
Docker
DockerDocker
Docker
 
Docker by Example - Basics
Docker by Example - Basics Docker by Example - Basics
Docker by Example - Basics
 
Docker orchestration
Docker orchestrationDocker orchestration
Docker orchestration
 

Similar to Vagrant + Docker provider [+Puppet]

Docker for Deep Learning (Andrea Panizza)
Docker for Deep Learning (Andrea Panizza)Docker for Deep Learning (Andrea Panizza)
Docker for Deep Learning (Andrea Panizza)
MeetupDataScienceRoma
 
Docker Ecosystem on Azure
Docker Ecosystem on AzureDocker Ecosystem on Azure
Docker Ecosystem on Azure
Patrick Chanezon
 
Dockerizing a Symfony2 application
Dockerizing a Symfony2 applicationDockerizing a Symfony2 application
Dockerizing a Symfony2 application
Roman Rodomansky
 
Using Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and JenkinsUsing Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and Jenkins
Micael Gallego
 
Running the Oracle SOA Suite Environment in a Docker Container
Running the Oracle SOA Suite Environment in a Docker ContainerRunning the Oracle SOA Suite Environment in a Docker Container
Running the Oracle SOA Suite Environment in a Docker Container
Guido Schmutz
 
codemotion-docker-2014
codemotion-docker-2014codemotion-docker-2014
codemotion-docker-2014
Carlo Bonamico
 
Develop with docker 2014 aug
Develop with docker 2014 augDevelop with docker 2014 aug
Develop with docker 2014 aug
Vincent De Smet
 
Docker and containers - Presentation Slides by Priyadarshini Anand
Docker and containers - Presentation Slides by Priyadarshini AnandDocker and containers - Presentation Slides by Priyadarshini Anand
Docker and containers - Presentation Slides by Priyadarshini Anand
PRIYADARSHINI ANAND
 
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
Codemotion
 
Docker
DockerDocker
Docker for .NET Developers
Docker for .NET DevelopersDocker for .NET Developers
Docker for .NET Developers
Taswar Bhatti
 
Dockers & kubernetes detailed - Beginners to Geek
Dockers & kubernetes detailed - Beginners to GeekDockers & kubernetes detailed - Beginners to Geek
Dockers & kubernetes detailed - Beginners to Geek
wiTTyMinds1
 
Docker From Scratch
Docker From ScratchDocker From Scratch
Docker From Scratch
Giacomo Vacca
 
Primi passi con Docker - ItalianCoders - 12-01-2021
Primi passi con Docker - ItalianCoders - 12-01-2021Primi passi con Docker - ItalianCoders - 12-01-2021
Primi passi con Docker - ItalianCoders - 12-01-2021
Alessandro Mignogna
 
Docker in Action
Docker in ActionDocker in Action
Docker in Action
Alper Kanat
 
Docker navjot kaur
Docker navjot kaurDocker navjot kaur
Docker navjot kaur
Navjot Kaur
 
Docker introduction - Part 1
Docker introduction - Part 1Docker introduction - Part 1
Docker introduction - Part 1
Alessandro Mignogna
 
Docker and the Container Ecosystem
Docker and the Container EcosystemDocker and the Container Ecosystem
Docker and the Container Ecosystem
psconnolly
 
Docker for mere mortals
Docker for mere mortalsDocker for mere mortals
Docker for mere mortals
Henryk Konsek
 
Academy PRO: Docker. Part 2
Academy PRO: Docker. Part 2Academy PRO: Docker. Part 2
Academy PRO: Docker. Part 2
Binary Studio
 

Similar to Vagrant + Docker provider [+Puppet] (20)

Docker for Deep Learning (Andrea Panizza)
Docker for Deep Learning (Andrea Panizza)Docker for Deep Learning (Andrea Panizza)
Docker for Deep Learning (Andrea Panizza)
 
Docker Ecosystem on Azure
Docker Ecosystem on AzureDocker Ecosystem on Azure
Docker Ecosystem on Azure
 
Dockerizing a Symfony2 application
Dockerizing a Symfony2 applicationDockerizing a Symfony2 application
Dockerizing a Symfony2 application
 
Using Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and JenkinsUsing Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and Jenkins
 
Running the Oracle SOA Suite Environment in a Docker Container
Running the Oracle SOA Suite Environment in a Docker ContainerRunning the Oracle SOA Suite Environment in a Docker Container
Running the Oracle SOA Suite Environment in a Docker Container
 
codemotion-docker-2014
codemotion-docker-2014codemotion-docker-2014
codemotion-docker-2014
 
Develop with docker 2014 aug
Develop with docker 2014 augDevelop with docker 2014 aug
Develop with docker 2014 aug
 
Docker and containers - Presentation Slides by Priyadarshini Anand
Docker and containers - Presentation Slides by Priyadarshini AnandDocker and containers - Presentation Slides by Priyadarshini Anand
Docker and containers - Presentation Slides by Priyadarshini Anand
 
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
 
Docker
DockerDocker
Docker
 
Docker for .NET Developers
Docker for .NET DevelopersDocker for .NET Developers
Docker for .NET Developers
 
Dockers & kubernetes detailed - Beginners to Geek
Dockers & kubernetes detailed - Beginners to GeekDockers & kubernetes detailed - Beginners to Geek
Dockers & kubernetes detailed - Beginners to Geek
 
Docker From Scratch
Docker From ScratchDocker From Scratch
Docker From Scratch
 
Primi passi con Docker - ItalianCoders - 12-01-2021
Primi passi con Docker - ItalianCoders - 12-01-2021Primi passi con Docker - ItalianCoders - 12-01-2021
Primi passi con Docker - ItalianCoders - 12-01-2021
 
Docker in Action
Docker in ActionDocker in Action
Docker in Action
 
Docker navjot kaur
Docker navjot kaurDocker navjot kaur
Docker navjot kaur
 
Docker introduction - Part 1
Docker introduction - Part 1Docker introduction - Part 1
Docker introduction - Part 1
 
Docker and the Container Ecosystem
Docker and the Container EcosystemDocker and the Container Ecosystem
Docker and the Container Ecosystem
 
Docker for mere mortals
Docker for mere mortalsDocker for mere mortals
Docker for mere mortals
 
Academy PRO: Docker. Part 2
Academy PRO: Docker. Part 2Academy PRO: Docker. Part 2
Academy PRO: Docker. Part 2
 

More from Nicolas Poggi

Benchmarking Elastic Cloud Big Data Services under SLA Constraints
Benchmarking Elastic Cloud Big Data Services under SLA ConstraintsBenchmarking Elastic Cloud Big Data Services under SLA Constraints
Benchmarking Elastic Cloud Big Data Services under SLA Constraints
Nicolas Poggi
 
Correctness and Performance of Apache Spark SQL
Correctness and Performance of Apache Spark SQLCorrectness and Performance of Apache Spark SQL
Correctness and Performance of Apache Spark SQL
Nicolas Poggi
 
State of Spark in the cloud (Spark Summit EU 2017)
State of Spark in the cloud (Spark Summit EU 2017)State of Spark in the cloud (Spark Summit EU 2017)
State of Spark in the cloud (Spark Summit EU 2017)
Nicolas Poggi
 
The state of Hive and Spark in the Cloud (July 2017)
The state of Hive and Spark in the Cloud (July 2017)The state of Hive and Spark in the Cloud (July 2017)
The state of Hive and Spark in the Cloud (July 2017)
Nicolas Poggi
 
The state of Spark in the cloud
The state of Spark in the cloudThe state of Spark in the cloud
The state of Spark in the cloud
Nicolas Poggi
 
Using BigBench to compare Hive and Spark (Long version)
Using BigBench to compare Hive and Spark (Long version)Using BigBench to compare Hive and Spark (Long version)
Using BigBench to compare Hive and Spark (Long version)
Nicolas Poggi
 
Using BigBench to compare Hive and Spark (short version)
Using BigBench to compare Hive and Spark (short version)Using BigBench to compare Hive and Spark (short version)
Using BigBench to compare Hive and Spark (short version)
Nicolas Poggi
 
Accelerating HBase with NVMe and Bucket Cache
Accelerating HBase with NVMe and Bucket CacheAccelerating HBase with NVMe and Bucket Cache
Accelerating HBase with NVMe and Bucket Cache
Nicolas Poggi
 
The state of SQL-on-Hadoop in the Cloud
The state of SQL-on-Hadoop in the CloudThe state of SQL-on-Hadoop in the Cloud
The state of SQL-on-Hadoop in the Cloud
Nicolas Poggi
 
sudoers: Benchmarking Hadoop with ALOJA
sudoers: Benchmarking Hadoop with ALOJAsudoers: Benchmarking Hadoop with ALOJA
sudoers: Benchmarking Hadoop with ALOJA
Nicolas Poggi
 
Benchmarking Hadoop and Big Data
Benchmarking Hadoop and Big DataBenchmarking Hadoop and Big Data
Benchmarking Hadoop and Big Data
Nicolas Poggi
 
The case for Hadoop performance
The case for Hadoop performanceThe case for Hadoop performance
The case for Hadoop performance
Nicolas Poggi
 

More from Nicolas Poggi (12)

Benchmarking Elastic Cloud Big Data Services under SLA Constraints
Benchmarking Elastic Cloud Big Data Services under SLA ConstraintsBenchmarking Elastic Cloud Big Data Services under SLA Constraints
Benchmarking Elastic Cloud Big Data Services under SLA Constraints
 
Correctness and Performance of Apache Spark SQL
Correctness and Performance of Apache Spark SQLCorrectness and Performance of Apache Spark SQL
Correctness and Performance of Apache Spark SQL
 
State of Spark in the cloud (Spark Summit EU 2017)
State of Spark in the cloud (Spark Summit EU 2017)State of Spark in the cloud (Spark Summit EU 2017)
State of Spark in the cloud (Spark Summit EU 2017)
 
The state of Hive and Spark in the Cloud (July 2017)
The state of Hive and Spark in the Cloud (July 2017)The state of Hive and Spark in the Cloud (July 2017)
The state of Hive and Spark in the Cloud (July 2017)
 
The state of Spark in the cloud
The state of Spark in the cloudThe state of Spark in the cloud
The state of Spark in the cloud
 
Using BigBench to compare Hive and Spark (Long version)
Using BigBench to compare Hive and Spark (Long version)Using BigBench to compare Hive and Spark (Long version)
Using BigBench to compare Hive and Spark (Long version)
 
Using BigBench to compare Hive and Spark (short version)
Using BigBench to compare Hive and Spark (short version)Using BigBench to compare Hive and Spark (short version)
Using BigBench to compare Hive and Spark (short version)
 
Accelerating HBase with NVMe and Bucket Cache
Accelerating HBase with NVMe and Bucket CacheAccelerating HBase with NVMe and Bucket Cache
Accelerating HBase with NVMe and Bucket Cache
 
The state of SQL-on-Hadoop in the Cloud
The state of SQL-on-Hadoop in the CloudThe state of SQL-on-Hadoop in the Cloud
The state of SQL-on-Hadoop in the Cloud
 
sudoers: Benchmarking Hadoop with ALOJA
sudoers: Benchmarking Hadoop with ALOJAsudoers: Benchmarking Hadoop with ALOJA
sudoers: Benchmarking Hadoop with ALOJA
 
Benchmarking Hadoop and Big Data
Benchmarking Hadoop and Big DataBenchmarking Hadoop and Big Data
Benchmarking Hadoop and Big Data
 
The case for Hadoop performance
The case for Hadoop performanceThe case for Hadoop performance
The case for Hadoop performance
 

Recently uploaded

快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
3a0sd7z3
 
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
rtunex8r
 
Discover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to IndiaDiscover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to India
davidjhones387
 
Securing BGP: Operational Strategies and Best Practices for Network Defenders...
Securing BGP: Operational Strategies and Best Practices for Network Defenders...Securing BGP: Operational Strategies and Best Practices for Network Defenders...
Securing BGP: Operational Strategies and Best Practices for Network Defenders...
APNIC
 
Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?
Paul Walk
 
HijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process HollowingHijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process Hollowing
Donato Onofri
 
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
3a0sd7z3
 
Bengaluru Dreamin' 24 - Personal Branding
Bengaluru Dreamin' 24 - Personal BrandingBengaluru Dreamin' 24 - Personal Branding
Bengaluru Dreamin' 24 - Personal Branding
Tarandeep Singh
 
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
APNIC
 
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
k4ncd0z
 
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
thezot
 
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
xjq03c34
 

Recently uploaded (12)

快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
 
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
 
Discover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to IndiaDiscover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to India
 
Securing BGP: Operational Strategies and Best Practices for Network Defenders...
Securing BGP: Operational Strategies and Best Practices for Network Defenders...Securing BGP: Operational Strategies and Best Practices for Network Defenders...
Securing BGP: Operational Strategies and Best Practices for Network Defenders...
 
Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?
 
HijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process HollowingHijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process Hollowing
 
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
 
Bengaluru Dreamin' 24 - Personal Branding
Bengaluru Dreamin' 24 - Personal BrandingBengaluru Dreamin' 24 - Personal Branding
Bengaluru Dreamin' 24 - Personal Branding
 
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
 
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
 
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
 
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
 

Vagrant + Docker provider [+Puppet]

  • 1. By Nicolas Poggi @ni_po provider July, 2014 Sources: https://github.com/npoggi/vagrant-docker Docker Barcelona: + [+ ]
  • 2. About Nicolas Poggi @ni_po Previous: Current: Organizing s:
  • 4. X About Vagrant [+ puppet]  It’s all about AUTOMATION  Designed to build and setup replicable, isolated, [DEV] envs  Isolation (similar to docker)  Package code + environment  Multiplatform  Multiple Dev Envs  Distributable  Avoids having a central DEV server/s  Keep different versions  Simplifies Configuration Management  (Also Docker’s)  About puppet  World’s favorite Configuration Management tool
  • 5. About Docker  Light-weight replacement for VMs in several scenarios  Also for Vagrant  Based on Linux containers  Adds abstraction layer  GIT like for OS images  Every step creates a different image  Docker hub integrates with github [and bitbucket]  Similar to Vagrant  Vagrantfile ~= Dockerfile  CONS  Complex commands
  • 6. Docker basics Command Description  docker  docker pull  docker images  docker ps  docker run  docker rm/rmi  Lists commands  Downloads container images  Lists available images  Lists running containers  Executes a command in a container  Deletes images
  • 7. Docker tutorial  https://github.com/harbur/docker-workshop  By Dimitris Kapanidis @spiddy  Initial Steps: 1. docker version 2. docker pull ubuntu 3. docker pull nginx 4. git clone https://github.com/harbur/docker-workshop
  • 8. Docker advantages  Speed  lightweight  Isolation  Automation  Portability  Migration  … more to come
  • 9. Why Vagrant + Docker provider?  Same reason on “why Docker”  Get rid of the VM  Free system resources  No reserved RAM memory or CPU/s  Get back control of your own machine!  Same reasons on “why Vagrant”  Best way to learn puppet, so a good way to learn docker  Vagrant takes care of many manual chores  …  Makes sense!  You are dealing with DEV environments  Fewer security concerns  Limited resources on laptops  Keep active tens of different DEV environments  …and vagrant up anything!  Including docker vagrant up vs. sudo docker run -d -name image_name -p 8080:80 -t user/repo
  • 10. Requirements  1.) Get a real open-source GNU OS  …With the latest kernel, i.e., Archlinux  Sorry Mac and Windows…  Don’t cry, boot2dockerto the rescue  Vagrant fires up the VM on non supported platforms  Latest Vagrant version  v1.6 >  Latest Docker version  v1.0 >  Root access (for Docker)
  • 11. Vagrant providers  Official providers  Virtualbox (DEFAULT)  VMWare  Amazon  … Docker  YES!  Other providers
  • 12. Basic Vagrant commands Vagrant Docker provider  vagrant  up  provision  ssh  halt/suspend  vagrant  docker-logs  docker-run NOTE: You can still use all of docker commands!
  • 13. Docker provider for Vagrant 1.) Docker images 2.) Dockerfile  Use a pre-built docker image  Pros  Easy  Faster to deploy  No need to deal with the Docker file  Cons  Less control  Public images  Build your own (w/ Vagrant)  Cons  More complex  Slow to start  (then is cached)  Pros  Custom made  Privately managed  For free Two alternative routes
  • 14. Building a docker image in Vagrant The hard way
  • 15. Dockerfile, the requisites for Vagrant 1. Import image FROM repo:tag 2. Create vagrant user 1. create password 2. sudo permissions and password-less 3. Configure SSH 1. Setup keys 4. Install base packages 5. Install Configuration Management system 1. Puppet, etc… 6. Expose SSH port 7. Run SSH as a daemon
  • 16. Sample Docker file for vagrant  https://github.com/npoggi/vagrant-docker/blob/master/Dockerfile
  • 17. The Vagrantfile  Optional: Setup docker a default provider  Select  If to build from a Dockerfile  Or use a prebuilt docker image  Set has.ssh to true  Rest as usual in a Vagrantfile  Ports, mounts  Config management i.e., puppet  Example  https://github.com/npoggi/vagrant-docker/blob/master/Vagrantfile
  • 18. Building, running, setting up, deploying, etc. just vagrant up
  • 19. Use prebuilt docker images for Vagrant The easy way Examples at: https://hub.docker.com/u/npoggi/ https://registry.hub.docker.com/search?q=vagrant
  • 20. Other ways to create images 1. Use docker 2. Docker Hub 1. Link your github project (public or private (at a cost)) 2. Select Automate build 3. Point to your Docker file 4. And … Wait for the build to complete!
  • 21. Saving state [and time] - Docker Hub  Get container CONTAINER_ID  docker ps  Commit image  docker commit [–m “my comment”] CONTAINER_ID user/repo[:tag]  (had permissions problems to the main repo)  Push image  docker push user/repo[:tag]
  • 22. Try it yourself  Install requirements (latest vagrant and docker)  Basic usage: 1. git clone https://github.com/npoggi/vagrant-docker.git 2. cd vagrant-docker 3. sudo vagrant up  Test: http://localhost:8080/  Advanced usage:  For a vanilla image built using the same docker file:  DOCKER_IMAGE='npoggi/vagrant-docker‘ sudo vagrant up --provision  For a pre-provisioned image with the puppet config (saves time)  DOCKER_IMAGE='npoggi/vagrant-docker-provisioned' sudo vagrant up --provision
  • 23. Time improvements Vagrant virtualbox (default) Vagrant docker  vagrant up: 30.6s  vagrant halt: 7.1s  vagrant up: 2.6s  Vagrant halt: 2.3
  • 24. Gotchas! …and annoyances  Docker by default requires root permissions!  Be ready for BUGS!!!  Vagrant Docker provider less stable  Doesn’t save to run docker commands occasionally  Docker too…  Renames in machines are a problem (manual work)  Delete orphaned images with complex commands  …  Vagrant package command only for virtualbox  Docker image downloads quite slow…  …there is still Mac and Windows
  • 25. Other considerations and Alternatives  Can Docker replace vagrant use?  Yes, but…  Docker abstraction alternative to Vagrant:  FIG http://orchardup.github.io/fig/  Alternatives to Docker in Vagrant  vagrant-lxc plugin (not official) vs.
  • 26. Future steps for the source code  Multiple Vagrant-Docker machines  Light-weight OS  Boot2Docker, CoreOS, Project Atomic  Vagrant + Docker provider+ Docker  Inception kick!  Docker patterns  Terraform (for PROD)
  • 27. Tips and tricks  Update to the latest versions (i.e., github)  Docker by default requires root permissions!  Add your user to the docker group  sudo usermod -a -G docker your_user; newgrp docker; #or relogin  Change the default image dir (to control repo size)  Edit the service description add  -g /home/user/docker_images  Delete “untagged/orphaned images”  docker rmi $(docker images -a | grep "^<none>" | awk '{print $3}')  Enable IPv4 forwarding to allow internet connections inside the containers
  • 28. Final notes  Get back you DEV ENV!  Speedup development  R&D + [+ ]  Code  https://github.com/npoggi/vagrant-docker  Docker images  https://registry.hub.docker.com/u/npoggi/
  • 29. Credits and References  Vagrant specific  Main site: http://www.vagrantup.com/  Alternative to Docker: https://github.com/fgrehm/vagrant-lxc  PROD Vagrant-like: http://www.terraform.io/  Docker specific  Docker: https://www.docker.com/  Docker hub: https://registry.hub.docker.com  Light-weight VM: https://github.com/boot2docker/boot2docker  Fig (Vagrant alternative): http://orchardup.github.io/fig/  Puppet labs: http://puppetlabs.com/