FullStack Developers Israel
Docking micro services
with

Haggai	
  Philip	
  Zagury	
  
28.1.2014
Google Campus T.A

Hoste...
WHO AM I ?
Haggai Philip Zagury,
DevOps Engineer

	
   	
   }
{
● 
● 
● 
● 

Continuous integration
Continuous delivery
It Operations...
“ 	
  I am a member of Tikal's
DevOps/ALM group. With over
15 members, we meet, share,
contribute and code together
on a m...
WHO WE ARE?
We help companies build,
deliver, deploy, manage
and optimize their products.
OUR EXPERTISE
Where are we
going today
	
  	
  

text
How CM & Deployment changed
Between
Monolithic & SOA / MSA
Stack => Stacks
More Technologies
More Technologies

More Services (API’s)
More Technologies

More Services (API’s)

More Teams
ch team with
Ea

it’s own
“madness”
MONOLITHIC
app deployment

FABRIC	
  

}

1	
  …	
  n	
  

*	
  n	
  
1	
  …	
  n	
  
MONOLITHIC style for SOA/MSA

FABRIC	
  

}

service	
  A	
  
*	
  n	
  
service	
  B	
  

service	
  C	
  
MONOLITHIC style for SOA/MSA
Team	
  /	
  Service	
  A	
  
MONOLITHIC style for SOA/MSA
Team	
  /	
  Service	
  B	
  
MONOLITHIC style for SOA/MSA
Team	
  /	
  Service	
  C	
  
MONOLITHIC style for SOA/MSA

FABRIC	
  

}

*	
  n	
  
text

MONOLITHIC “style”

System provisioning (& OS provisioning)
•  Much more “base images” in order to save time
	
   De...
text

MONOLITHIC “style”

I	
  need	
  xyz	
  installed	
  
• Kernel	
  version	
  not	
  supported	
  
• Other	
  compone...
text

More images == GB/$$/PERF
Between 100MB & nGB

Cost in storage … [ e.g. S3 ]
Cost in performance [ VMware …]
From library dependency)
From library dependency
runtime directory (encapsulation)
From library dependency
runtime directory (encapsulation)
“.service” (hybrid)
Choose 1 tool for the job ?!

FABRIC	
  

}

service	
  A	
  
*	
  n	
  
service	
  B	
  

service	
  C	
  
Containers	
  
•  OSLV	
  -­‐	
  OperaJng	
  System	
  Level	
  VirtualizaJon	
  (link)	
  
•  API	
  &	
  tooling,	
  whi...
What are containers anyway ?
text

Linux Containers (LXC) - Why ?

Isn't there enough container tech ?
•  Solaris Zones (containers - link)
	
   Vserve...
text

Linux Containers (LXC) - Why ?

Features
•  Kernel namespaces [ isolated processes, network etc ]
	
  
•  Chroot & S...
text

Revolution – Hard/Software

From	
  Rack	
  servers	
  =>	
  Blade	
  
So what’s this DOCKER and
why do I need it ?
Why DOCKER ?
Why?
•  A wrapper for LXC
•  An abstraction layer for LXC + features
So Why not “plain old” LXC ?
•  Portable...
text

Docker ( & LXC ) Solve !

ISOLATION
•  Daemon per container
	
  

Any	
  version	
  is	
  supported	
  
text

Docker ( & LXC ) Solve !

ISOLATION
•  Daemon per container
	
  
SECURITY	
  
•  Container == Independent ( user/gro...
text

Docker ( & LXC ) Solve !

ISOLATION
•  Daemon per container
	
  
SECURITY	
  
•  Container == Independent ( user/gro...
VM	
  vs	
  Container	
  

• 
• 
• 
• 

No hypervisor layer
No lib duplication
Shared kernel
VMS are “heavy”

• 
• 
• 
• 
...
Docker - lightweight

• 
• 

Reuse kernel
Add functionality to a
container, version it, share it
Micro service example

Docker	
  

•  ROR	
  front	
  end	
  
•  Key-­‐value	
  store	
  
Host	
  /	
  VM	
  	
  
Workflow(s)
The developer workflow
●  How do we test locally ? { if running on
windows / OSX }
●  Define an interface with operations ...
Vagrant & Docker
Vagrant.configure("2") do |config|
config.vm.box = "dummy" config.vm.provider :docker do |docker|
docker....
Fast,	
  isolated	
  development	
  
environments	
  using	
  Docker.	
  
•  Define your application’s environment
•  OS
•...
Search & Get an image
docker	
  search	
  <keyword>	
  
	
  

root@docker-­‐poc:/tmp#	
  docker	
  search	
  centos*6	
  
...
Define your own
Dockerfile	
  -­‐>	
  Redis	
  server	
  running	
  in	
  a	
  container	
  	
  
#	
  Docker	
  Image/tag	
...
Docker - Choose base
docker	
  pull	
  user/container-­‐name	
  
	
  

root@docker-­‐poc:/tmp#	
  docker	
  pull	
  saltst...
Docker build
build	
  from	
  Dockerfile	
  	
  
	
  
docker	
  build	
  .	
  
	
  
Step	
  1	
  :	
  FROM	
  ubuntu:12.10	...
Docker build
…	
  
Processing	
  triggers	
  for	
  ureadahead	
  ...	
  
	
  -­‐-­‐-­‐>	
  ba4030995701	
  
Step	
  4	
  ...
Docker tag & push
docker	
  tag	
  a6dd4adbb425	
  localhost:5000/redis_hagzag	
  
	
  
	
  
docker	
  push	
  localhost:5...
The Deployment workflow
●  Provide docker-registry service / interface
●  Monitoring & Logging facilities
●  Data binding ...
Our service

Docker	
  

•  Using	
  –name	
  &	
  -­‐link	
  
•  Linking	
  containers	
  by	
  
reference	
  (not	
  ip)...
Docker run & ps
docker	
  run	
  -­‐name	
  redis	
  -­‐d	
  a6dd4adbb425	
  
	
  
	
  
	
  
docker	
  ps	
  

CONTAINER	
...
Docker inspect
docker	
  inspect	
  redis	
  
	
  
	
  
	
  
What we achieved ?
docker	
  push	
  <reg-­‐name>/app-­‐1	
  

docker	
  pull	
  <reg-­‐name>/app-­‐1	
  

take	
  tag	
  ...
Evolving with Docker
DevEnv	
  
• 
• 
• 
• 

FIG	
  
Vagrant	
  –	
  buggy	
  
Chef-­‐docker	
  (hgps://github.com/bflad/ch...
Search for “Dockerfile”
A nodejs container …
Heroku like with Docker = Dokku

hgps://github.com/progrium/dokku	
  
Heroku like
with LXC + Chef = Diez

hgp://deis.io/	
  

hgps://github.com/opdemand/deis	
  
To Summarize
•  Very promising & almost J production ready
•  A great complementary to existing CM tooling
•  Simplifies ...
Thank You
Haggai	
  Philip	
  Zagury	
  
Email:	
  hagzag@Jkalk.com	
  
Docking your services_with_docker
Docking your services_with_docker
Docking your services_with_docker
Upcoming SlideShare
Loading in...5
×

Docking your services_with_docker

3,385

Published on

Published in: Technology
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,385
On Slideshare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
31
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide

Docking your services_with_docker

  1. 1. FullStack Developers Israel Docking micro services with Haggai  Philip  Zagury   28.1.2014 Google Campus T.A Hosted by:
  2. 2. WHO AM I ?
  3. 3. Haggai Philip Zagury, DevOps Engineer     } { ●  ●  ●  ●  Continuous integration Continuous delivery It Operations Configuration management over 10 years of DevOps expertise
  4. 4. “  I am a member of Tikal's DevOps/ALM group. With over 15 members, we meet, share, contribute and code together on a monthly basis “
  5. 5. WHO WE ARE?
  6. 6. We help companies build, deliver, deploy, manage and optimize their products.
  7. 7. OUR EXPERTISE
  8. 8. Where are we going today     text
  9. 9. How CM & Deployment changed Between Monolithic & SOA / MSA
  10. 10. Stack => Stacks
  11. 11. More Technologies
  12. 12. More Technologies More Services (API’s)
  13. 13. More Technologies More Services (API’s) More Teams
  14. 14. ch team with Ea it’s own “madness”
  15. 15. MONOLITHIC app deployment FABRIC   } 1  …  n   *  n   1  …  n  
  16. 16. MONOLITHIC style for SOA/MSA FABRIC   } service  A   *  n   service  B   service  C  
  17. 17. MONOLITHIC style for SOA/MSA Team  /  Service  A  
  18. 18. MONOLITHIC style for SOA/MSA Team  /  Service  B  
  19. 19. MONOLITHIC style for SOA/MSA Team  /  Service  C  
  20. 20. MONOLITHIC style for SOA/MSA FABRIC   } *  n  
  21. 21. text MONOLITHIC “style” System provisioning (& OS provisioning) •  Much more “base images” in order to save time   Deployment takes much longer (ad hoc configuration) •  •  Consolidate in order to save time •  Backup & Restore ? doesn’t save time :( •  Security ?
  22. 22. text MONOLITHIC “style” I  need  xyz  installed   • Kernel  version  not  supported   • Other  component's  depend  on  that   • Wait  for  next  release  /  OS  upgrade  
  23. 23. text More images == GB/$$/PERF Between 100MB & nGB Cost in storage … [ e.g. S3 ] Cost in performance [ VMware …]
  24. 24. From library dependency)
  25. 25. From library dependency runtime directory (encapsulation)
  26. 26. From library dependency runtime directory (encapsulation) “.service” (hybrid)
  27. 27. Choose 1 tool for the job ?! FABRIC   } service  A   *  n   service  B   service  C  
  28. 28. Containers   •  OSLV  -­‐  OperaJng  System  Level  VirtualizaJon  (link)   •  API  &  tooling,  which  enable  *nix  users  to  easily   create  and  manage  system  or  applicaJon   containers.    
  29. 29. What are containers anyway ?
  30. 30. text Linux Containers (LXC) - Why ? Isn't there enough container tech ? •  Solaris Zones (containers - link)   Vserver •  •  Openvz •  Chroot Why now ? •  Solaris not widely used as linux/freebsd …   Linux kernel support ( >= 2.6.27 ) •  •  Application segmentation •  We really need it !!! => “.service” era
  31. 31. text Linux Containers (LXC) - Why ? Features •  Kernel namespaces [ isolated processes, network etc ]   •  Chroot & Seccomp (isolation) •  Control groups (a.k.a cgroups)   Limitation   • Only Linux !
  32. 32. text Revolution – Hard/Software From  Rack  servers  =>  Blade  
  33. 33. So what’s this DOCKER and why do I need it ?
  34. 34. Why DOCKER ? Why? •  A wrapper for LXC •  An abstraction layer for LXC + features So Why not “plain old” LXC ? •  Portable deployments across machines •  •  •  LXC alone doesn't guarantee that ! Docker build - a “build tool” designed for portability Application centric / OS centric [ Docker’s API ] •  SHA-1 (git like) based versioning •  •  DRY / Reuse - 1 base image for many applications Sharing - index (global) or registry (private / on prem)
  35. 35. text Docker ( & LXC ) Solve ! ISOLATION •  Daemon per container   Any  version  is  supported  
  36. 36. text Docker ( & LXC ) Solve ! ISOLATION •  Daemon per container   SECURITY   •  Container == Independent ( user/group/service etc) •  New version == new container ( not toe trading …)
  37. 37. text Docker ( & LXC ) Solve ! ISOLATION •  Daemon per container   SECURITY   •  Container == Independent ( user/group/service etc) •  New version == new container ( not toe trading …) PORTABILITY   •  Container on DEV machine => to production •  Deploy from private registry •  Rollback == latest -1
  38. 38. VM  vs  Container   •  •  •  •  No hypervisor layer No lib duplication Shared kernel VMS are “heavy” •  •  •  •  5-10 x Faster Startup time VMS are “heavy” Better utilize HW (cloud)
  39. 39. Docker - lightweight •  •  Reuse kernel Add functionality to a container, version it, share it
  40. 40. Micro service example Docker   •  ROR  front  end   •  Key-­‐value  store   Host  /  VM    
  41. 41. Workflow(s)
  42. 42. The developer workflow ●  How do we test locally ? { if running on windows / OSX } ●  Define an interface with operations ?
  43. 43. Vagrant & Docker Vagrant.configure("2") do |config| config.vm.box = "dummy" config.vm.provider :docker do |docker| docker.image = "your/image:tag" docker.cmd = ["/path/to/your", "command"] end end vagrant  plugin  install  docker-­‐provider     -­‐  docker  friendly  vagrant  image  
  44. 44. Fast,  isolated  development   environments  using  Docker.   •  Define your application’s environment •  OS •  Packages •  Configuration ! etc •  Number of machines ? •  Define a container via Dockerfile •  Use that Dockerfile to define your environment (via yaml file) web:   workfl o w      build:  .      links:        -­‐  db      ports:        -­‐  8000:8000   db:      image:  hagzag/pgsql  
  45. 45. Search & Get an image docker  search  <keyword>     root@docker-­‐poc:/tmp#  docker  search  centos*6   NAME                                                              DESCRIPTION                                                                                STARS          OFFICIAL      TRUSTED   saltstack/centos-­‐6                                                                                                                                        0                                                  [OK]   salgest/centos-­‐6                                                                                                                                          0                                                  [OK]   saltstack/centos-­‐6-­‐minimal                                                                                                                      1                                                  [OK]   leifw/tokumx-­‐buildslave-­‐centos-­‐6                                                                                                          0                                                  [OK]   tenforward/centos-­‐i386                            CentOS  6  32bit  image                                                        0   hansode/rpmbuilder-­‐rhel6                        CentOS-­‐6  with  rpmdevtools                                  0   ...   hgp://index.Docker.io  
  46. 46. Define your own Dockerfile  -­‐>  Redis  server  running  in  a  container     #  Docker  Image/tag   FROM                  ubuntu:12.10   #  command(s)  to  execute  on  container     RUN                        apt-­‐get  update   RUN                        apt-­‐get  -­‐y  install  redis-­‐server   #  what  port  to  listen  on   EXPOSE            6379   #  once  container  is  acJve  what  binary  to  run   ENTRYPOINT      ["/usr/bin/redis-­‐server"]  
  47. 47. Docker - Choose base docker  pull  user/container-­‐name     root@docker-­‐poc:/tmp#  docker  pull  saltstack/centos-­‐6-­‐minimal   Pulling  repository  saltstack/centos-­‐6-­‐minimal   aca320b373f2:  Download  complete   f2f28f99c5fd:  Download  complete   bf9724189396:  Download  complete   e7adb01c55f6:  Download  complete   a3f13a39bbbe:  Download  complete       Git  style  “tags”   Salt  –  inside  …  
  48. 48. Docker build build  from  Dockerfile       docker  build  .     Step  1  :  FROM  ubuntu:12.10    -­‐-­‐-­‐>  b750fe79269d   Step  2  :  RUN  apt-­‐get  update    -­‐-­‐-­‐>  Running  in  0d768rc284d   Fetched  9813  kB  in  20s  (481  kB/s)    -­‐-­‐-­‐>  46a6f0556e96   Step  3  :  RUN  apt-­‐get  -­‐y  install  redis-­‐server    -­‐-­‐-­‐>  Running  in  5ea88c37d21f   The  following  extra  packages  will  be  installed:      libjemalloc1   The  following  NEW  packages  will  be  installed:      libjemalloc1  redis-­‐server   0  upgraded,  2  newly  installed,  0  to  remove  and  0  not  upgraded.   Need  to  get  319  kB  of  archives.    
  49. 49. Docker build …   Processing  triggers  for  ureadahead  ...    -­‐-­‐-­‐>  ba4030995701   Step  4  :  EXPOSE  6379    -­‐-­‐-­‐>  Running  in  24720beda74b    -­‐-­‐-­‐>  6fdf06372117   Step  5  :  ENTRYPOINT  ["/usr/bin/redis-­‐server"]    -­‐-­‐-­‐>  Running  in  c9b9480840ad    -­‐-­‐-­‐>  a6dd4adbb425   Successfully  built  a6dd4adbb425     docker  images   REPOSITORY                                      TAG                                  IMAGE  ID                        CREATED                          VIRTUAL  SIZE   <none>                                              <none>                            a6dd4adbb425                8  minutes  ago              297.2  MB  
  50. 50. Docker tag & push docker  tag  a6dd4adbb425  localhost:5000/redis_hagzag       docker  push  localhost:5000/redis_hagzag   The  push  refers  to  a  repository  [localhost:5000/redis_hagzag]  (len:  1)   Sending  image  list   Pushing  repository  localhost:5000/redis_hagzag  (1  tags)   27cf78414709:  Image  successfully  pushed   b750fe79269d:  Image  successfully  pushed   46a6f0556e96:  Image  successfully  pushed   ba4030995701:  Image  successfully  pushed   6fdf06372117:  Image  successfully  pushed   a6dd4adbb425:  Image  successfully  pushed   Pushing  tags  for  rev  [a6dd4adbb425]  on  {hgp://localhost:5000/v1/repositories/ redis_hagzag/tags/latest}  
  51. 51. The Deployment workflow ●  Provide docker-registry service / interface ●  Monitoring & Logging facilities ●  Data binding / persistent configuration
  52. 52. Our service Docker   •  Using  –name  &  -­‐link   •  Linking  containers  by   reference  (not  ip)   Host  /  VM     build  run  +  -­‐name,  build  run  +  -­‐link  tag  =  complete  “.service”  on  a  single  node  
  53. 53. Docker run & ps docker  run  -­‐name  redis  -­‐d  a6dd4adbb425         docker  ps   CONTAINER  ID                IMAGE                              COMMAND                                CREATED                          STATUS                            PORTS                                        NAMES   9026507ef675                a6dd4adbb425    /usr/bin/redis-­‐serve      12  minutes  ago            Up  12  minutes              6379/tcp                                   redis   7e88dcb96856                registry:0.6.1            /bin/sh  -­‐c  cd  /docke      9  days  ago                    Up  40  minutes              0.0.0.0:5000-­‐>5000/ tcp      condescending_thompson    
  54. 54. Docker inspect docker  inspect  redis        
  55. 55. What we achieved ? docker  push  <reg-­‐name>/app-­‐1   docker  pull  <reg-­‐name>/app-­‐1   take  tag  “latest”  of  app  A   In container responsibility ●  Latest code ●  Dependencies Out container responsibility ●  Security & Remote access ●  Logging ●  Monitoring ●  Networking Immutability ? - not just yet … but we are getting close
  56. 56. Evolving with Docker DevEnv   •  •  •  •  FIG   Vagrant  –  buggy   Chef-­‐docker  (hgps://github.com/bflad/chef-­‐docker)     Chef  Docker  registry  ( hgp://community.opscode.com/cookbooks/docker-­‐registry)     OpsEnv   •  Chef-­‐docker  (hgps://github.com/bflad/chef-­‐docker)     •  Chef  Docker  registry  ( hgp://community.opscode.com/cookbooks/docker-­‐registry)     •  Puppet  docker  (hgp://forge.puppetlabs.com/garethr/docker)     •  DOTCLOUDS  (focke  authors)  –  About  to  base  PASS  based  on  Docker  
  57. 57. Search for “Dockerfile”
  58. 58. A nodejs container …
  59. 59. Heroku like with Docker = Dokku hgps://github.com/progrium/dokku  
  60. 60. Heroku like with LXC + Chef = Diez hgp://deis.io/   hgps://github.com/opdemand/deis  
  61. 61. To Summarize •  Very promising & almost J production ready •  A great complementary to existing CM tooling •  Simplifies deployment (I know it doesn’t seem so)   text
  62. 62. Thank You Haggai  Philip  Zagury   Email:  hagzag@Jkalk.com  
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×