Cooking Perl with Chef

Cooking Perl with Chef David Golden @xdg http://perlchef.com/ July 2012

Configuration Management(e.g. chef, puppet, cfengine, ...)

Unknown state ↓ Target state

New machine ↓Deployed app

Infrastructure as code automated! repeatable! testable! (no manual steps, checklist, etc.)

One tool to deploy the whole stack(DB, caching, messaging, ...)

But wait!Isnt that hard to do for Perl apps?

Perl applications are complex

Dependency hell

App = perl + CPAN + your code

CHI DateTime DBI JSONApp = perl + CPAN + your code Moose Plack POE Try::Tiny ...

CHI DateTime DBI JSONApp = perl + CPAN + your code Moose Plack POE Try::Tiny ... your application is the versioned set of all its compontents

CHI DateTime DBI JSONAppv1.0.0 = perl + CPAN + your code Moose Plack POE Try::Tiny ... your application is the versioned set of all its compontents

CHI DateTime DBI JSONAppv1.0.0 = Perl + CPAN + your code v5.14.2 Moose Plack POE Try::Tiny ... your application is the versioned set of all its compontents

0.55 0.76 1.622 2.53Appv1.0.0 = Perl + CPAN + your code v5.14.2 2.0603 0.9989 1.354 0.11 ... your application is the versioned set of all its compontents

0.55 0.76 1.622 2.53Appv1.0.0 = Perl + CPAN + your code v5.14.2 v1.0 2.0603 0.9989 1.354 0.11 ... your application is the versioned set of all its compontents

0.55 0.76 1.622 2.53Appv1.0.0 = Perl + CPAN + your code v5.14.2 v1.0 2.0603 0.9989 1.354 0.11 ... change one piece...

0.55 0.76 1.622 2.53Appv1.0.0 = Perl + CPAN + your code v5.16.0 v1.0 2.0603 0.9989 1.354 0.11 ... change one piece...

0.55 0.76 1.622 2.53Appv1.0.1 = Perl + CPAN + your code v5.16.0 v1.02.0603 0.9989 1.354 0.11 ... … and you have a new version of your application

Repeatable deployment means...

Repeatable deployment means... ... the same Perl

Repeatable deployment means... ... the same Perl ... the same modules

Repeatable deployment means... ... the same Perl ... the same modules ... the same code

Repeatable deployment means... ... the same Perl ... the same modules ... the same code ... on demand

Easy...

If we have the right tools

Use one-size fits all distribution packagers like apt, yum, etc...? (How much do you like your system perl?!)

This problem is not unique to Perl

Lets be inspired by Larry

[larry hat pic]

Or better yet, Paul

YARRR!

Great hackers steal!

Great hackers steal ideas

Consider Chef...

Chef ❤ Ruby(written in ruby; various ruby app stacks)

Chef ❤ Python(virtualenv; pip; django apps)

Common patterns emerge

python → virtualenv + pip ruby → rvm + Bundler

Weve built tools like these, too

Kang-min Liu (gugod) Matt S Trout (mst)Tatsuhiko Miyagawa(and a big community helping them)

perlbrew – multiple perl manager Matt S Trout (mst) Tatsuhiko Miyagawa (and a big community helping them)

perlbrew – multiple perl managerlocal::lib – custom @INC manager Tatsuhiko Miyagawa (and a big community helping them)

perlbrew – multiple perl manager local::lib – custom @INC managercarton – versioned dependency installer (and a big community helping them)

So we have the pieces

Repeatable deployment in five parts

Repeatable deployment in five parts application-specific Perl application-specific @INC path versioned application code versioned module dependencies automate the previous four

Repeatable deployment in five parts perlbrew application-specific @INC path versioned application code versioned module dependencies automate the previous four

Repeatable deployment in five parts perlbrew local::lib versioned application code versioned module dependencies automate the previous four

Repeatable deployment in five parts perlbrew local::lib git versioned module dependencies automate the previous four

Repeatable deployment in five parts perlbrew local::lib git carton automate the previous four

Repeatable deployment in five parts perlbrew local::lib git carton @&$%!

[swedish chef FAIL pic]

No support in Chef...

So I implemented it

(In Ruby)

(After I learned some Ruby)

Now... Chef ❤ Perl (perlbrew; local::lib; carton)

Time for a quick Chef glossary... (see http://wiki.opscode.com/)

“Cookbook”A collection of components to configurea particular applicationTypically includes recipes, providers,templates, etc.(CPAN analogy → “distribution”)

“Recipe”Component applied that deploys anapplication or serviceTypically declarative, specifying desiredresources and associated configuration

“Resource”An abstraction of something to be deployed

“Provider”Platform-specific implementation to deliver aresource

“Node”A host computer managed with ChefOften means the configuration file thatdefines recipes, attributes and roles thatdefine the target state of a host

“Attribute”A variable used in a recipe and/or providerthat customizes the configuration of aresourceAttributes have defaults, but can becustomized for nodes or roles

“Role”A collection of recipes and attributes used toapply common configuration across multiplenodes

Summary... cookbooks include recipes and providersroles, recipes and attributes get applied to nodesrecipes specify desired resources and customize them with attributes providers do the work of deploying resources

I wrote two Perl Chef cookbooksfor the Chef community repository (which is like CPAN circa 1996 or so) http://community.opscode.com/

1. perlbrew – for managing perls2. carton – for deploying appsAlso available here: https://github.com/dagolden/perl-chef

perlbrew cookbook resources: perlbrew_perl – install a perl perlbrew_lib – create a local::lib perlbrew_cpanm – install modules to perl or lib perlbrew_run – run shell commands under a particular perlbrew and/or lib

carton cookbook resource: carton_app – deploy an app with carton – start in directory with the app source – configure for a specific perlbrew perl – install versioned dependencies with carton – create a runit service for the app – start the app

Time for an example:Deploying a “Hello World” Plack app https://github.com/dagolden/zzz-hello-world

Steps for creating Hello World 1. Write the application 2. Use carton to create a carton.lock file with versioned dependency info 3. Write a simple cookbook for the application 4. Check it all into git 5. Deploy the application with Chef

$ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm

use strict;use warnings;use ZZZ::Hello::World;my $app = sub { ZZZ::Hello::World->run_psgi(@_) }; (this Plack app just invokes a simple module)

use 5.008001;use strict;use warnings;package ZZZ::Hello::World;our $VERSION = "1.0";use Plack::Request;sub run_psgi { my $self = shift; my $req = Plack::Request->new(shift); my $res = $req->new_response(200); $res->content_type(text/html); $res->body(<<"HERE");<html><head><title>Hello World</title></head><body><p>Hello World. It is @{[scalar localtime]}</p>...</body></html>HERE return $res->finalize;}1; (the module just returns some dynamic HTML)

use inc::Module::Install;name ZZZ-Hello-World;version 1.0;requires Plack;requires Starman;WriteAll; (the Makefile.PL also includes deployment dependencies like Starman)

During development, carton installsdependencies locally and creates a versioneddependency file called carton.lock$ carton install# installs dependencies into a local directory# creates carton.lock if it doesnt exist# carton.lock is a JSON file of dependency info

During deployment, carton installs dependenciesfrom carton.lock and runs the app with them$ carton install# installs dependencies into a local directory$ carton exec Ilib starman p 8080 app.psgi# runs the app using carton installed deps

# carton.lock JSON{ "modules" : { "Class::Inspector" : { "dist" : "Class-Inspector-1.27", "module" : "Class::Inspector", "pathname" : "A/AD/ADAMK/Class-Inspector-1.27.tar.gz", ... }. "Data::Dump" : { ... }, "Devel::StackTrace" : { ... }, "Encode::Locale" : { ... }, ...} (carton.lock associates module names to specific versions of those module)

# perlbrew to execute with default[hello-world][perl_version] = perl-5.16.0 # Install directory, repo and tag default[hello-world][deploy_dir] = /opt/hello-world default[hello-world][deploy_repo] = https://github.com/dagolden/zzz-hello-world.git default[hello-world][deploy_tag] = master # Service user/group/port default[hello-world][user] = "nobody" default[hello-world][group] = "nogroup" default[hello-world][port] = 8080(attributes are variables used in the recipe; can be customized per-node during deployment)

include_recipe cartonpackage git-coregit node[hello-world][deploy_dir] do repository node[hello-world][deploy_repo] reference node[hello-world][deploy_tag] notifies :restart, "carton_app[hello-world]"endcarton_app "hello-world" do perlbrew node[hello-world][perl_version] command "starman -p #{node[hello-world][port]} app.psgi" cwd node[hello-world][deploy_dir] user node[hello-world][user] group node[hello-world][group]endcarton_app "hello-world" do action :startend (recipe ensures carton and git are available...)

include_recipe cartonpackage git-coregit node[hello-world][deploy_dir] do repository node[hello-world][deploy_repo] reference node[hello-world][deploy_tag] notifies :restart, "carton_app[hello-world]"endcarton_app "hello-world" do perlbrew node[hello-world][perl_version] command "starman -p #{node[hello-world][port]} app.psgi" cwd node[hello-world][deploy_dir] user node[hello-world][user] group node[hello-world][group]endcarton_app "hello-world" do action :startend (git resource specifies where application code goes...)

include_recipe cartonpackage git-coregit node[hello-world][deploy_dir] do repository node[hello-world][deploy_repo] reference node[hello-world][deploy_tag] notifies :restart, "carton_app[hello-world]"endcarton_app "hello-world" do perlbrew node[hello-world][perl_version] command "starman -p #{node[hello-world][port]} app.psgi" cwd node[hello-world][deploy_dir] user node[hello-world][user] group node[hello-world][group]endcarton_app "hello-world" do action :startend (attributes parameterize the resource statement...)

include_recipe cartonpackage git-coregit node[hello-world][deploy_dir] do repository node[hello-world][deploy_repo] reference node[hello-world][deploy_tag] notifies :restart, "carton_app[hello-world]"endcarton_app "hello-world" do perlbrew node[hello-world][perl_version] command "starman -p #{node[hello-world][port]} app.psgi" cwd node[hello-world][deploy_dir] user node[hello-world][user] group node[hello-world][group]endcarton_app "hello-world" do action :startend (carton_app resources installs deps and sets up runit service...)

include_recipe cartonpackage git-coregit node[hello-world][deploy_dir] do repository node[hello-world][deploy_repo] reference node[hello-world][deploy_tag] notifies :restart, "carton_app[hello-world]"endcarton_app "hello-world" do perlbrew node[hello-world][perl_version] command "starman -p #{node[hello-world][port]} app.psgi" cwd node[hello-world][deploy_dir] user node[hello-world][user] group node[hello-world][group]endcarton_app "hello-world" do action :startend (again, attributes parameterize the resource...)

include_recipe cartonpackage git-coregit node[hello-world][deploy_dir] do repository node[hello-world][deploy_repo] reference node[hello-world][deploy_tag] notifies :restart, "carton_app[hello-world]"endcarton_app "hello-world" do perlbrew node[hello-world][perl_version] command "starman -p #{node[hello-world][port]} app.psgi" cwd node[hello-world][deploy_dir] user node[hello-world][user] group node[hello-world][group]endcarton_app "hello-world" do action :startend (finally, the resource is idempotently started...)

These files – and the Perl Chef cookbooks – are all you need

Enough code... lets see how to deploy it

Steps for deployment of Hello World 1. Set up a Vagrant virtual machine 2. Prepare Pantry to manage Chef Solo 3. Get Hello World cookbook and dependencies 4. Configure virtual machine for Hello World 5. Deploy

Vagrant is a tool for managing virtual machines “Can I have a VirtualBox now, please?”

Vagrant is a tool for managing virtual machines$ vagrant box add base http://files.vagrantup.com/lucid32.box$ vagrant init$ vagrant up

Vagrant is great for testing Chef deployment (and other things, besides)

Chef Solo is Chef without a central configuration server (good for demos and smaller deployments)

Chef – you push config data to Chef Server – nodes run Chef Client to pull config from Chef Server and execute itChef Solo – you push config data to nodes – you run Chef Solo remotely

One advantage of Chef Solo... Your config repo is canonical(i.e. you dont have to track what youve pushed to the central server)

One dis-advantage of Chef Solo...Manual rsync/ssh required (yuck!)

Pantry is a tool for automating Chef Solo

Pantry is a tool for automating Chef Solo$ pantry create node server.example.com$ pantry apply node server.example.com --role web --recipe myapp$ pantry sync node server.example.com

Pantry is written in Perl and available on CPAN (Similar to pocketknife [Ruby] and littlechef [Python])

Finally, a demonstration... Screencast available athttp://youtu.be/H93rt-KtwBE

$ vagrant init# create config file$ vim Vagrantfile# edit to forward local port 8080 to# virtual machine port 8080$ vagrant up# launch it$ vagrant ssh# check that its up, then exit

$ pantry init# create directories to hold Chef Solo config$ pantry create node vagrant host localhost port 8080 user vagrant# create a node config file$ sshadd ~/.vagrant.d/insecure_private_key# allow pantry to ssh to vagrant machine (Important tip: remove the insecure_private_key after you no longer need it because Github chokes on it.)

Four cookbooks must be downloadedand copied to the cookbooks directory – hello-world – carton – perlbrew – runit

$ pantry apply node vagrant recipe helloworld# apply recipe to node configuration$ pantry apply node vagrant default helloworld.perl_version=perl5.14.2# override a default attribute$ pantry show node vagrant# see the resulting JSON config file{ "helloworld" : { "perl_version" : "perl5.14.2" }, "run_list" : [ "recipe[helloworld]" ], "name" : "vagrant", "pantry_user" : "vagrant", "pantry_port" : "2222", "pantry_host" : "localhost"}

$ pantry sync node vagrant# ... wait for everything to deploy ...# then load browser and test it!

It works

You can do this, too

Dont be afraid. Try it out. Get involved. tutorial and screencast → http://perlchef.com mailing list → perl-devops-subscribe@perl.org

Cooking Perl with Chef

by David Golden on Jul 29, 2012

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 1

Statistics

Cooking Perl with Chef Presentation Transcript