Cooking Perl with Chef

4,595 views
4,473 views

Published on

Reliable and scalable applications need repeatable, automated application deployment. Configuration management tools like Chef, Puppet and others make it easy to deploy an entire application stack, but support for Perl applications has lagged behind other popular, dynamic languages.

The Perl community has responded to these challenges with tools like perlbrew, local::lib, carton and others to make it easier to manage an application and its dependencies in isolation. This presentation will show you how to make those tools work with Chef for complete automation of Perl application deployment.

Published in: Technology
0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,595
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
42
Comments
0
Likes
8
Embeds 0
No embeds

No notes for slide

Cooking Perl with Chef

  1. 1. Cooking Perl with Chef David Golden @xdg http://perlchef.com/ July 2012
  2. 2. Configuration Management(e.g. chef, puppet, cfengine, ...)
  3. 3. Unknown state ↓ Target state
  4. 4. New machine ↓Deployed app
  5. 5. Infrastructure as code automated! repeatable! testable! (no manual steps, checklist, etc.)
  6. 6. One tool to deploy the whole stack(DB, caching, messaging, ...)
  7. 7. But wait!Isnt that hard to do for Perl apps?
  8. 8. Perl applications are complex
  9. 9. Dependency hell
  10. 10. App = perl + CPAN + your code
  11. 11. CHI DateTime DBI JSONApp = perl + CPAN + your code Moose Plack POE Try::Tiny ...
  12. 12. CHI DateTime DBI JSONApp = perl + CPAN + your code Moose Plack POE Try::Tiny ... your application is the versioned set of all its compontents
  13. 13. CHI DateTime DBI JSONAppv1.0.0 = perl + CPAN + your code Moose Plack POE Try::Tiny ... your application is the versioned set of all its compontents
  14. 14. CHI DateTime DBI JSONAppv1.0.0 = Perl + CPAN + your code v5.14.2 Moose Plack POE Try::Tiny ... your application is the versioned set of all its compontents
  15. 15. 0.55 0.76 1.622 2.53Appv1.0.0 = Perl + CPAN + your code v5.14.2 2.0603 0.9989 1.354 0.11 ... your application is the versioned set of all its compontents
  16. 16. 0.55 0.76 1.622 2.53Appv1.0.0 = Perl + CPAN + your code v5.14.2 v1.0 2.0603 0.9989 1.354 0.11 ... your application is the versioned set of all its compontents
  17. 17. 0.55 0.76 1.622 2.53Appv1.0.0 = Perl + CPAN + your code v5.14.2 v1.0 2.0603 0.9989 1.354 0.11 ... change one piece...
  18. 18. 0.55 0.76 1.622 2.53Appv1.0.0 = Perl + CPAN + your code v5.16.0 v1.0 2.0603 0.9989 1.354 0.11 ... change one piece...
  19. 19. 0.55 0.76 1.622 2.53Appv1.0.1 = Perl + CPAN + your code v5.16.0 v1.02.0603 0.9989 1.354 0.11 ... … and you have a new version of your application
  20. 20. Repeatable deployment means...
  21. 21. Repeatable deployment means... ... the same Perl
  22. 22. Repeatable deployment means... ... the same Perl ... the same modules
  23. 23. Repeatable deployment means... ... the same Perl ... the same modules ... the same code
  24. 24. Repeatable deployment means... ... the same Perl ... the same modules ... the same code ... on demand
  25. 25. Easy...
  26. 26. If we have the right tools
  27. 27. Use one-size fits all distribution packagers like apt, yum, etc...? (How much do you like your system perl?!)
  28. 28. This problem is not unique to Perl
  29. 29. Lets be inspired by Larry
  30. 30. [larry hat pic]
  31. 31. Or better yet, Paul
  32. 32. YARRR!
  33. 33. Great hackers steal!
  34. 34. Great hackers steal ideas
  35. 35. Consider Chef...
  36. 36. Chef ❤ Ruby(written in ruby; various ruby app stacks)
  37. 37. Chef ❤ Python(virtualenv; pip; django apps)
  38. 38. Common patterns emerge
  39. 39. python → virtualenv + pip ruby → rvm + Bundler
  40. 40. Weve built tools like these, too
  41. 41. Kang-min Liu (gugod) Matt S Trout (mst)Tatsuhiko Miyagawa(and a big community helping them)
  42. 42. perlbrew – multiple perl manager Matt S Trout (mst) Tatsuhiko Miyagawa (and a big community helping them)
  43. 43. perlbrew – multiple perl managerlocal::lib – custom @INC manager Tatsuhiko Miyagawa (and a big community helping them)
  44. 44. perlbrew – multiple perl manager local::lib – custom @INC managercarton – versioned dependency installer (and a big community helping them)
  45. 45. So we have the pieces
  46. 46. Repeatable deployment in five parts
  47. 47. Repeatable deployment in five parts application-specific Perl application-specific @INC path versioned application code versioned module dependencies automate the previous four
  48. 48. Repeatable deployment in five parts perlbrew application-specific @INC path versioned application code versioned module dependencies automate the previous four
  49. 49. Repeatable deployment in five parts perlbrew local::lib versioned application code versioned module dependencies automate the previous four
  50. 50. Repeatable deployment in five parts perlbrew local::lib git versioned module dependencies automate the previous four
  51. 51. Repeatable deployment in five parts perlbrew local::lib git carton automate the previous four
  52. 52. Repeatable deployment in five parts perlbrew local::lib git carton @&$%!
  53. 53. [swedish chef FAIL pic]
  54. 54. No support in Chef...
  55. 55. So I implemented it
  56. 56. (In Ruby)
  57. 57. (After I learned some Ruby)
  58. 58. Now... Chef ❤ Perl (perlbrew; local::lib; carton)
  59. 59. Time for a quick Chef glossary... (see http://wiki.opscode.com/)
  60. 60. “Cookbook”A collection of components to configurea particular applicationTypically includes recipes, providers,templates, etc.(CPAN analogy → “distribution”)
  61. 61. “Recipe”Component applied that deploys anapplication or serviceTypically declarative, specifying desiredresources and associated configuration
  62. 62. “Resource”An abstraction of something to be deployed
  63. 63. “Provider”Platform-specific implementation to deliver aresource
  64. 64. “Node”A host computer managed with ChefOften means the configuration file thatdefines recipes, attributes and roles thatdefine the target state of a host
  65. 65. “Attribute”A variable used in a recipe and/or providerthat customizes the configuration of aresourceAttributes have defaults, but can becustomized for nodes or roles
  66. 66. “Role”A collection of recipes and attributes used toapply common configuration across multiplenodes
  67. 67. Summary... cookbooks include recipes and providersroles, recipes and attributes get applied to nodesrecipes specify desired resources and customize them with attributes providers do the work of deploying resources
  68. 68. I wrote two Perl Chef cookbooksfor the Chef community repository (which is like CPAN circa 1996 or so) http://community.opscode.com/
  69. 69. 1. perlbrew – for managing perls2. carton – for deploying appsAlso available here: https://github.com/dagolden/perl-chef
  70. 70. perlbrew cookbook resources: perlbrew_perl – install a perl perlbrew_lib – create a local::lib perlbrew_cpanm – install modules to perl or lib perlbrew_run – run shell commands under a particular perlbrew and/or lib
  71. 71. carton cookbook resource: carton_app – deploy an app with carton – start in directory with the app source – configure for a specific perlbrew perl – install versioned dependencies with carton – create a runit service for the app – start the app
  72. 72. Time for an example:Deploying a “Hello World” Plack app https://github.com/dagolden/zzz-hello-world
  73. 73. Steps for creating Hello World 1. Write the application 2. Use carton to create a carton.lock file with versioned dependency info 3. Write a simple cookbook for the application 4. Check it all into git 5. Deploy the application with Chef
  74. 74. $ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm
  75. 75. $ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm
  76. 76. use strict;use warnings;use ZZZ::Hello::World;my $app = sub { ZZZ::Hello::World->run_psgi(@_) }; (this Plack app just invokes a simple module)
  77. 77. $ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm
  78. 78. use 5.008001;use strict;use warnings;package ZZZ::Hello::World;our $VERSION = "1.0";use Plack::Request;sub run_psgi { my $self = shift; my $req = Plack::Request->new(shift); my $res = $req->new_response(200); $res->content_type(text/html); $res->body(<<"HERE");<html><head><title>Hello World</title></head><body><p>Hello World. It is @{[scalar localtime]}</p>...</body></html>HERE return $res->finalize;}1; (the module just returns some dynamic HTML)
  79. 79. $ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm
  80. 80. use inc::Module::Install;name ZZZ-Hello-World;version 1.0;requires Plack;requires Starman;WriteAll; (the Makefile.PL also includes deployment dependencies like Starman)
  81. 81. During development, carton installsdependencies locally and creates a versioneddependency file called carton.lock$ carton install# installs dependencies into a local directory# creates carton.lock if it doesnt exist# carton.lock is a JSON file of dependency info
  82. 82. During deployment, carton installs dependenciesfrom carton.lock and runs the app with them$ carton install# installs dependencies into a local directory$ carton exec ­Ilib ­­ starman ­p 8080 app.psgi# runs the app using carton installed deps
  83. 83. $ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm
  84. 84. # carton.lock JSON{ "modules" : { "Class::Inspector" : { "dist" : "Class-Inspector-1.27", "module" : "Class::Inspector", "pathname" : "A/AD/ADAMK/Class-Inspector-1.27.tar.gz", ... }. "Data::Dump" : { ... }, "Devel::StackTrace" : { ... }, "Encode::Locale" : { ... }, ...} (carton.lock associates module names to specific versions of those module)
  85. 85. $ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm
  86. 86. # perlbrew to execute with default[hello-world][perl_version] = perl-5.16.0 # Install directory, repo and tag default[hello-world][deploy_dir] = /opt/hello-world default[hello-world][deploy_repo] = https://github.com/dagolden/zzz-hello-world.git default[hello-world][deploy_tag] = master # Service user/group/port default[hello-world][user] = "nobody" default[hello-world][group] = "nogroup" default[hello-world][port] = 8080(attributes are variables used in the recipe; can be customized per-node during deployment)
  87. 87. $ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm
  88. 88. include_recipe cartonpackage git-coregit node[hello-world][deploy_dir] do repository node[hello-world][deploy_repo] reference node[hello-world][deploy_tag] notifies :restart, "carton_app[hello-world]"endcarton_app "hello-world" do perlbrew node[hello-world][perl_version] command "starman -p #{node[hello-world][port]} app.psgi" cwd node[hello-world][deploy_dir] user node[hello-world][user] group node[hello-world][group]endcarton_app "hello-world" do action :startend (recipe ensures carton and git are available...)
  89. 89. include_recipe cartonpackage git-coregit node[hello-world][deploy_dir] do repository node[hello-world][deploy_repo] reference node[hello-world][deploy_tag] notifies :restart, "carton_app[hello-world]"endcarton_app "hello-world" do perlbrew node[hello-world][perl_version] command "starman -p #{node[hello-world][port]} app.psgi" cwd node[hello-world][deploy_dir] user node[hello-world][user] group node[hello-world][group]endcarton_app "hello-world" do action :startend (git resource specifies where application code goes...)
  90. 90. include_recipe cartonpackage git-coregit node[hello-world][deploy_dir] do repository node[hello-world][deploy_repo] reference node[hello-world][deploy_tag] notifies :restart, "carton_app[hello-world]"endcarton_app "hello-world" do perlbrew node[hello-world][perl_version] command "starman -p #{node[hello-world][port]} app.psgi" cwd node[hello-world][deploy_dir] user node[hello-world][user] group node[hello-world][group]endcarton_app "hello-world" do action :startend (attributes parameterize the resource statement...)
  91. 91. include_recipe cartonpackage git-coregit node[hello-world][deploy_dir] do repository node[hello-world][deploy_repo] reference node[hello-world][deploy_tag] notifies :restart, "carton_app[hello-world]"endcarton_app "hello-world" do perlbrew node[hello-world][perl_version] command "starman -p #{node[hello-world][port]} app.psgi" cwd node[hello-world][deploy_dir] user node[hello-world][user] group node[hello-world][group]endcarton_app "hello-world" do action :startend (carton_app resources installs deps and sets up runit service...)
  92. 92. include_recipe cartonpackage git-coregit node[hello-world][deploy_dir] do repository node[hello-world][deploy_repo] reference node[hello-world][deploy_tag] notifies :restart, "carton_app[hello-world]"endcarton_app "hello-world" do perlbrew node[hello-world][perl_version] command "starman -p #{node[hello-world][port]} app.psgi" cwd node[hello-world][deploy_dir] user node[hello-world][user] group node[hello-world][group]endcarton_app "hello-world" do action :startend (again, attributes parameterize the resource...)
  93. 93. include_recipe cartonpackage git-coregit node[hello-world][deploy_dir] do repository node[hello-world][deploy_repo] reference node[hello-world][deploy_tag] notifies :restart, "carton_app[hello-world]"endcarton_app "hello-world" do perlbrew node[hello-world][perl_version] command "starman -p #{node[hello-world][port]} app.psgi" cwd node[hello-world][deploy_dir] user node[hello-world][user] group node[hello-world][group]endcarton_app "hello-world" do action :startend (finally, the resource is idempotently started...)
  94. 94. These files – and the Perl Chef cookbooks – are all you need
  95. 95. Enough code... lets see how to deploy it
  96. 96. Steps for deployment of Hello World 1. Set up a Vagrant virtual machine 2. Prepare Pantry to manage Chef Solo 3. Get Hello World cookbook and dependencies 4. Configure virtual machine for Hello World 5. Deploy
  97. 97. Steps for deployment of Hello World 1. Set up a Vagrant virtual machine 2. Prepare Pantry to manage Chef Solo 3. Get Hello World cookbook and dependencies 4. Configure virtual machine for Hello World 5. Deploy
  98. 98. Vagrant is a tool for managing virtual machines “Can I have a VirtualBox now, please?”
  99. 99. Vagrant is a tool for managing virtual machines$ vagrant box add base http://files.vagrantup.com/lucid32.box$ vagrant init$ vagrant up
  100. 100. Vagrant is great for testing Chef deployment (and other things, besides)
  101. 101. Steps for deployment of Hello World 1. Set up a Vagrant virtual machine 2. Prepare Pantry to manage Chef Solo 3. Get Hello World cookbook and dependencies 4. Configure virtual machine for Hello World 5. Deploy
  102. 102. Chef Solo is Chef without a central configuration server (good for demos and smaller deployments)
  103. 103. Chef – you push config data to Chef Server – nodes run Chef Client to pull config from Chef Server and execute itChef Solo – you push config data to nodes – you run Chef Solo remotely
  104. 104. One advantage of Chef Solo... Your config repo is canonical(i.e. you dont have to track what youve pushed to the central server)
  105. 105. One dis-advantage of Chef Solo...Manual rsync/ssh required (yuck!)
  106. 106. Steps for deployment of Hello World 1. Set up a Vagrant virtual machine 2. Prepare Pantry to manage Chef Solo 3. Get Hello World cookbook and dependencies 4. Configure virtual machine for Hello World 5. Deploy
  107. 107. Pantry is a tool for automating Chef Solo
  108. 108. Pantry is a tool for automating Chef Solo$ pantry create node server.example.com$ pantry apply node server.example.com --role web --recipe myapp$ pantry sync node server.example.com
  109. 109. Pantry is written in Perl and available on CPAN (Similar to pocketknife [Ruby] and littlechef [Python])
  110. 110. Finally, a demonstration... Screencast available athttp://youtu.be/H93rt-KtwBE
  111. 111. Steps for deployment of Hello World 1. Set up a Vagrant virtual machine 2. Prepare Pantry to manage Chef Solo 3. Get Hello World cookbook and dependencies 4. Configure virtual machine for Hello World 5. Deploy
  112. 112. $ vagrant init# create config file$ vim Vagrantfile# edit to forward local port 8080 to# virtual machine port 8080$ vagrant up# launch it$ vagrant ssh# check that its up, then exit
  113. 113. Steps for deployment of Hello World 1. Set up a Vagrant virtual machine 2. Prepare Pantry to manage Chef Solo 3. Get Hello World cookbook and dependencies 4. Configure virtual machine for Hello World 5. Deploy
  114. 114. $ pantry init# create directories to hold Chef Solo config$ pantry create node vagrant     ­­host localhost     ­­port 8080          ­­user vagrant# create a node config file$ ssh­add ~/.vagrant.d/insecure_private_key# allow pantry to ssh to vagrant machine  (Important tip: remove the insecure_private_key after you no longer need it because Github chokes on it.)
  115. 115. Steps for deployment of Hello World 1. Set up a Vagrant virtual machine 2. Prepare Pantry to manage Chef Solo 3. Get Hello World cookbook and dependencies 4. Configure virtual machine for Hello World 5. Deploy
  116. 116. Four cookbooks must be downloadedand copied to the cookbooks directory – hello-world – carton – perlbrew – runit
  117. 117. Steps for deployment of Hello World 1. Set up a Vagrant virtual machine 2. Prepare Pantry to manage Chef Solo 3. Get Hello World cookbook and dependencies 4. Configure virtual machine for Hello World 5. Deploy
  118. 118. $ pantry apply node vagrant ­­recipe hello­world# apply recipe to node configuration$ pantry apply node vagrant ­­default     hello­world.perl_version=perl­5.14.2# override a default attribute$ pantry show node vagrant# see the resulting JSON config file{   "hello­world" : {      "perl_version" : "perl­5.14.2"   },   "run_list" : [      "recipe[hello­world]"   ],   "name" : "vagrant",   "pantry_user" : "vagrant",   "pantry_port" : "2222",   "pantry_host" : "localhost"}
  119. 119. Steps for deployment of Hello World 1. Set up a Vagrant virtual machine 2. Prepare Pantry to manage Chef Solo 3. Get Hello World cookbook and dependencies 4. Configure virtual machine for Hello World 5. Deploy
  120. 120. $ pantry sync node vagrant# ... wait for everything to deploy ...# then load browser and test it!
  121. 121. It works
  122. 122. You can do this, too
  123. 123. Dont be afraid. Try it out. Get involved. tutorial and screencast → http://perlchef.com mailing list → perl-devops-subscribe@perl.org

×