Talk by Patrick McDonnell (@mcdonnps) at #ChefConf 2012
Chef makes it so easy to change configuration en masse that it can be dangerous if not used with certain precautions and in accordance with a well thought out testing workflow. In our use of Chef at Etsy, we have devised many in-house best practices in response to failures which have helped greatly in avoiding catastrophic outages. This talk will focus on mistakes we've made and how we've avoided repeating them by enforcing standards in cookbooks, testing changes before rollout through the use of environments and in conjunction with the Spork plugin for Knife, and linting cookbooks with Foodcritic. I'll also talk about using handlers intelligently to monitor Chef runs and how to generate reports from the myriad data available in CouchDB.
Community Cookbooks & further resources - Fundamentals Webinar Series Part 6Chef
Part 6 of a 6 part series introducing you to the fundamentals of Chef.
This session includes an introducing Community Cookbooks and some additional resources.
After viewing this webinar you will be able to:
- Find, preview, and download cookbooks from the Chef Community site
- Use knife to work with the Community Site API
- Download, extract, examine and implement cookbooks from the Community site
Video of this webinar can be found at the following URL
https://www.youtube.com/watch?v=ovTIeS3kx4g&list=PL11cZfNdwNyPnZA9D1MbVqldGuOWqbumZ
Community Cookbooks & further resources - Fundamentals Webinar Series Part 6Chef
Part 6 of a 6 part series introducing you to the fundamentals of Chef.
This session includes an introducing Community Cookbooks and some additional resources.
After viewing this webinar you will be able to:
- Find, preview, and download cookbooks from the Chef Community site
- Use knife to work with the Community Site API
- Download, extract, examine and implement cookbooks from the Community site
Video of this webinar can be found at the following URL
https://www.youtube.com/watch?v=ovTIeS3kx4g&list=PL11cZfNdwNyPnZA9D1MbVqldGuOWqbumZ
Common configuration with Data Bags - Fundamentals Webinar Series Part 4Chef
Part 4 of a 6 part series introducing you to the fundamentals of Chef.
This session includes an introducing Data Bags & Data Bag Items
After viewing this webinar you will be able to:
- Use Data Bags for data-driven recipes
- Use multiple recipes for a node's run list
Video of this webinar can be found at the following URL
https://www.youtube.com/watch?v=fS_yrFNSL9w&list=PL11cZfNdwNyPnZA9D1MbVqldGuOWqbumZ
Node setup, resource, and recipes - Fundamentals Webinar Series Part 2Chef
Part 2 of a 6 part series introducing you to the fundamentals of Chef.
This session includes:
* Node Setup
* Chef Resources and Recipes
After viewing this webinar you will be able to:
- Login to the node in your Chef Training Lab
- Install Chef nodes using "knife bootstrap"
- Explain how knife bootstrap configures a node to use the - Organization created in the previous section
- Explain the basic configuration needed to run chef-client
- Describe in detail what a cookbook is
- Create a new cookbook
- Explain what a recipe is
- Describe how to use the package, service, and template - resources
- Upload a cookbook to the Chef Server
- Explain what a run list is, and how to set it for a node - via knife
- Explain the output of a chef-client run
Video of this webinar can be found at the following URL
https://www.youtube.com/watch?v=S5lHUpzoCYo&list=PL11cZfNdwNyPnZA9D1MbVqldGuOWqbumZ
How to do Test Driven Development to write Chef cookbooks?
This slide-deck explains how to use TDD and tools of the trade, to develop cookbooks, Unit Tests and Integration Tests.
Source code is available on GitHub: https://github.com/siso/cheftdd-cookbook
When most people talk about automating infrastructure, they focus on things like consistency, scalability, and flexibility. While fine goals, we recently converted several projects to Chef for both systems AND application deployment, and found that, with a little work, these tools could also help you enable better software quality assurance, load modeling, and even improve resource allocation.
By sharing cookbooks across projects, we were able to standardize practices and eliminate arbitrary differences, while using parameterization to perfectly isolate the special needs of each project. This allowed us to transfer knowledge among staff much more quickly. Pulling in and parameterizing application state – database contents, website assets, uploaded content – allowed us to spin up new environments with as much or as little state as needed. Integrating with Vagrant and Jenkins, we were then able to use chef to treat the entire image – system and application – as a test fixture. As each engineer (ops or dev) has visibility into the whole stack, we can more easily move people between dev and ops, or between projects.
Node object and roles - Fundamentals Webinar Series Part 3Chef
Part 3 of a 6 part series introducing you to the fundamentals of Chef.
This session includes:
* Node object
* Chef roles
After viewing this webinar you will be able to:
- Explain what the node object represents in Chef
- Show details about a node
- Describe what node attributes are
- Retrieve a node attribute
- Describe where and how attributes are set
- Explain the attribute merge order and precedence rules
- Declare an attribute with a recipe and set its value
- Explain what Roles are, and how they are used to provide -larity
- Discuss the Role JSON DSL
- Explain how merge order affects the precedence hierarchy
Video of this webinar can be found at the following URL
https://www.youtube.com/watch?v=nQogf89hgnM&list=PL11cZfNdwNyPnZA9D1MbVqldGuOWqbumZ
This presentation was held at the DevOps Meetup Frankfurt on 2014/12/08. It describes some tools and practices for testing your infrastructure provisioned with Chef.
Infrastructure Automation with Chef & Ansiblewajrcs
What is Infrastructure and why you should automate it?
Typical Infrastructure
Benefits
CMS/ Automation
Chef / Terminologies / Disadvantages
Ansible / Disadvantages
Demo
Summary
Author: Waqar Alamgir; Twitter @wajrcs
Common configuration with Data Bags - Fundamentals Webinar Series Part 4Chef
Part 4 of a 6 part series introducing you to the fundamentals of Chef.
This session includes an introducing Data Bags & Data Bag Items
After viewing this webinar you will be able to:
- Use Data Bags for data-driven recipes
- Use multiple recipes for a node's run list
Video of this webinar can be found at the following URL
https://www.youtube.com/watch?v=fS_yrFNSL9w&list=PL11cZfNdwNyPnZA9D1MbVqldGuOWqbumZ
Node setup, resource, and recipes - Fundamentals Webinar Series Part 2Chef
Part 2 of a 6 part series introducing you to the fundamentals of Chef.
This session includes:
* Node Setup
* Chef Resources and Recipes
After viewing this webinar you will be able to:
- Login to the node in your Chef Training Lab
- Install Chef nodes using "knife bootstrap"
- Explain how knife bootstrap configures a node to use the - Organization created in the previous section
- Explain the basic configuration needed to run chef-client
- Describe in detail what a cookbook is
- Create a new cookbook
- Explain what a recipe is
- Describe how to use the package, service, and template - resources
- Upload a cookbook to the Chef Server
- Explain what a run list is, and how to set it for a node - via knife
- Explain the output of a chef-client run
Video of this webinar can be found at the following URL
https://www.youtube.com/watch?v=S5lHUpzoCYo&list=PL11cZfNdwNyPnZA9D1MbVqldGuOWqbumZ
How to do Test Driven Development to write Chef cookbooks?
This slide-deck explains how to use TDD and tools of the trade, to develop cookbooks, Unit Tests and Integration Tests.
Source code is available on GitHub: https://github.com/siso/cheftdd-cookbook
When most people talk about automating infrastructure, they focus on things like consistency, scalability, and flexibility. While fine goals, we recently converted several projects to Chef for both systems AND application deployment, and found that, with a little work, these tools could also help you enable better software quality assurance, load modeling, and even improve resource allocation.
By sharing cookbooks across projects, we were able to standardize practices and eliminate arbitrary differences, while using parameterization to perfectly isolate the special needs of each project. This allowed us to transfer knowledge among staff much more quickly. Pulling in and parameterizing application state – database contents, website assets, uploaded content – allowed us to spin up new environments with as much or as little state as needed. Integrating with Vagrant and Jenkins, we were then able to use chef to treat the entire image – system and application – as a test fixture. As each engineer (ops or dev) has visibility into the whole stack, we can more easily move people between dev and ops, or between projects.
Node object and roles - Fundamentals Webinar Series Part 3Chef
Part 3 of a 6 part series introducing you to the fundamentals of Chef.
This session includes:
* Node object
* Chef roles
After viewing this webinar you will be able to:
- Explain what the node object represents in Chef
- Show details about a node
- Describe what node attributes are
- Retrieve a node attribute
- Describe where and how attributes are set
- Explain the attribute merge order and precedence rules
- Declare an attribute with a recipe and set its value
- Explain what Roles are, and how they are used to provide -larity
- Discuss the Role JSON DSL
- Explain how merge order affects the precedence hierarchy
Video of this webinar can be found at the following URL
https://www.youtube.com/watch?v=nQogf89hgnM&list=PL11cZfNdwNyPnZA9D1MbVqldGuOWqbumZ
This presentation was held at the DevOps Meetup Frankfurt on 2014/12/08. It describes some tools and practices for testing your infrastructure provisioned with Chef.
Infrastructure Automation with Chef & Ansiblewajrcs
What is Infrastructure and why you should automate it?
Typical Infrastructure
Benefits
CMS/ Automation
Chef / Terminologies / Disadvantages
Ansible / Disadvantages
Demo
Summary
Author: Waqar Alamgir; Twitter @wajrcs
Cooking Perl with Chef: Real World Tutorial with JitterbugDavid Golden
This tutorial provides a command-by-command walk-through for deploying the Jitterbug continuous integration application using the Chef configuration management tool
Vous avez besoin d'une stratégie pour déployer votre infrastructure ? Je vais vous donner une méthode qui vient du monde du développement. Dans le but d'avoir une plate-forme stable.
Décrivez ce dont vous avez besoin dans votre future recette "Je veux un serveur HTTP" est testez avec un outil du type BDD (Développement Piloté par les Fonctionnalités).
Maintenant, vous allez devoir de décrire (avec un langage de plus bas niveau) les prérequis dont vous avez besoin pour cette fonctionnalité (vérifier si NGINX est installé).
Vous basculez donc dans le mode TDD (Développement Piloté par les Tests). Quand votre recette est prête, vous allez l'ajouter à votre gestionnaire de sources. Et votre système d'Intégration Continue va tester votre recette à chaque mise à jour.
Comme pour un système de développement. Je vous l'ai dit “Infrastructure as code”
Atmosphere 2014: Really large scale systems configuration - Phil DibowitzPROIDEA
For many years, Facebook managed its systems with cfengine2. With many individual clusters over 10k nodes in size, a slew of different constantly-changing system configurations, and small teams, this system was showing its age and the complexity was steadily increasing, limiting its effectiveness and usability. It was difficult to integrate with internal systems, testing was often impractical, and it provided no isolation of configurations, among many other problems. After an extensive evaluation of the tools and paradigms in modern systems configuration management – open source, proprietary, and a potential home-grown solution – we built a system based on the open-source project Chef. The evaluation process involved understanding the direction we wanted to take in managing the next many iterations of systems, clusters, and teams. More importantly, we evaluated the various paradigms behind effective configuration management and the different kinds of scale they provide. What we ended up with is an extremely flexible system that allows a tiny team to manage an incredibly large number of systems with a variety of unique configuration needs. In this talk we will look at the paradigms behind the system we built, the software we chose and why, and the system we built using that software. Further, we will look at how the philosophies we followed can apply to anyone wanting to scale their systems infrastructure.
Phil Dibowitz - Phil Dibowitz has been working in systems engineering for 12 years and is currently a production engineer at Facebook. Initially, he worked on the traffic infrastructure team, automating load balancer configuration management, as well as designing and building the production IPv6 infrastructure. He now leads the team responsible for rebuilding the configuration management system from the ground up. Prior to Facebook, he worked at Google, where he managed the large Gmail environment, and at Ticketmaster, where he co-authored and open sourced a configuration management tool called Spine. He also contributes to, and maintains, various open source projects and has spoken at conferences and LUG’s on a variety of topics from Path MTU Discovery to X509.
Introduction to Infrastructure as Code & Automation / Introduction to ChefAll Things Open
All Things Open 2014 - Day 2
Thursday, October 23rd, 2014
Nathen Harvey
Co-Host of Food Fight Show & Director of Community and #LEARNCHEF / Chef
DevOps
Introduction to Infrastructure as Code & Automation / Introduction to Chef
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
6. How Do You Enforce This?
• Documented standards and communicated best practices
• Robust testing workflow
• Environments
• Knife Plugins
• Linting with rules derived from standards
• Foodcritic
8. How We Use Environments
• Three environments: production, development, testing
• Testing is unconstrained
• Test nodes are depooled and “flipped” to the testing environment,
then repooled and analyzed
• Test nodes are then flipped back to production
9. Working with Environments
• knife-flip by Etsy engineer Jon Cowie
(https://github.com/jonlives/knife-flip)
% knife node flip somenode.etsy.com testing
% knife role flip SomeRole testing
• knife-bulkchangeenvironment (https://github.com/jonlives/knife-
bulkchangeenvironment)
% knife node bulk_change_environment testing production
10. Keeping Environments in Sync
• knife-env-diff by Etsy engineer John Goulah
• Get it at https://github.com/jgoulah/knife-env-diff
% knife environment diff development production
diffing environment development against production
cookbook: hadoop
development version: = 0.1.0
production version: = 0.1.8
cookbook: mysql
development version: = 0.2.4
production version: = 0.2.5
11. Introducing Knife Spork
• Knife plugin providing a testing/versioning workflow
• Authored by Jon Cowie
• Get it at https://github.com/jonlives/knife-spork
12. Spork Features
• Four stage process
• Check: Look at versioning info for a cookbook
• Bump: Automatically increment the cookbook’s version number
• Upload: Knife upload and freeze
• Promote: Set environment constraints equal to specified version
14. % knife spork check foodcritic
Checking versions for cookbook foodcritic...
Current local version: 0.0.4
Remote versions (Max. 5 most recent only):
*0.0.4, frozen
0.0.3, frozen
0.0.2, unfrozen
0.0.1, frozen
DANGER: Your local cookbook has same version number as the
starred version above!
Please bump your local version or you won't be able to
upload.
15. % knife spork bump foodcritic
Loaded config file /home/pmcdonnell/git/chef-repo/config/
spork-config.yml...
Loaded config file /etc/spork-config.yml...
Pulling latest changes from git
Pulling latest changes from git submodules (if any)
Bumping patch level of the foodcritic cookbook from 0.0.4 to
0.0.5
Git add'ing /home/pmcdonnell/git/chef-repo/cookbooks/
foodcritic/metadata.rb
17. % knife spork promote foodcritic --remote
Pulling latest changes from git
Checking that foodcritic version 0.0.5 exists on the server
before promoting (any error means it hasn't been uploaded
yet)...
foodcritic version 0.0.5 found on server!
Environment: production
Adding version constraint foodcritic = 0.0.5
Saving changes into production.json
Git add'ing /home/pmcdonnell/git/chef-repo/environments/
production.json
Uploading production to server
18. WARNING: You're about to promote changes to several
cookbooks:
logrotate: = 0.1.24 changed to = 0.1.23
foodcritic: = 0.0.4 changed to = 0.0.5
Are you sure you want to continue? (Y/N) n
You said no, so I'm done here.
Would you like to reset your local production.json to match
the server?? (Y/N) y
Git add'ing /home/pmcdonnell/git/chef-repo/environments/
production.json
production.json reset.
19. Spork’s Logging Mechanisms
• Irccat: Logs to IRC channel (https://github.com/RJ/irccat)
[11:35:33] <irccat> CHEF: pmcdonnell uploaded and froze cookbook ldap version 0.1.27
[11:35:43] <irccat> CHEF: pmcdonnell uploaded environment production
https://github.etsycorp.com/gist/376967
[11:35:43] <irccat> CHEF: pmcdonnell uploaded environment development
https://github.etsycorp.com/gist/376968
• Graphite: promote --remote sends to deploys.chef metric
• Gist: Added to irccat notifications on promote --remote
Environment production uploaded at 2012-05-15 18:35:42 UTC by pmcdonnell
Constraints updated on server in this version:
ldap: = 0.1.26 changed to = 0.1.27
21. Foodcritic
• A lint tool for Chef cookbooks written by Andrew Crump
(http://acrmp.github.com/foodcritic/)
• Comes with a good set of default rules and is very easily extensible
• To enable in spork config:
foodcritic:
enabled: true
fail_tags: [any]
tags: [foo]
include_rules: [/home/me/myrules]
22. Etsy’s Rules
• A work in progress, but newly open-sourced at
https://github.com/etsy/foodcritic-rules
• Our rules are “style”-tagged rules that serve to enforce what we
consider to be best practices in our environment
• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
• ETSY003 - Execute resource used to run curl or wget commands
• ETSY004 - Execute resource defined without conditional or action :nothing
• ETSY005 - Action :restart sent to a core service
23. Rule Resulting from Image Outage
• ETSY005 - Action :restart sent to a core service
• Trippable services include httpd, mysql, memcached, postgresql-server
% foodcritic -t etsy -I ~/git/chef-repo/config/rules.rb ~/
git/chef-repo/cookbooks/apache
ETSY005: Action :restart sent to a core service:
/home/pmcdonnell/git/chef-repo/cookbooks/apache/recipes/
default.rb:39
28. Resulting Foodcritic Rule
• ETSY001 - Package or yum_package resource used with :upgrade action
• Enforces always using :install
% foodcritic -t etsy -I ~/git/chef-repo/config/rules.rb ~/
git/chef-repo/cookbooks/memcache
ETSY001: Package or yum_package resource used with :upgrade
action: /home/pmcdonnell/git/chef-repo/cookbooks/memcache/
recipes/default.rb:20
29. Resulting Foodcritic Rule
20 package "memcached" do
21 action :upgrade
22 end
Changed to:
20 package "memcached" do
21 version "1.4.2-1.fc10"
22 action :install
23 end
31. Using Handlers
• Etsy’s handlers (https://github.com/etsy/chef-handlers)
• Log failures to IRC
[10:52:03] <irccat> Chef run failed on dev-dbtasks01.ny4dev.etsy.com
[10:52:03] <irccat> https://github.etsycorp.com/gist/371229
• Graph aggregated metrics with Graphite
• Graph chef “deploys”
32. Graph with Graphite
• Metrics reporting made possible by knife-lastrun, authored by
John Goulah (https://github.com/jgoulah/knife-lastrun)
• Provides a handler and knife plugin for reporting on the most recent
chef run, storing data as node attributes
• Elapsed, starting, and ending time
• Exit code status
• Backtrace/exception information
33.
34.
35. % dsh -g all -c -M 'grep "Chef Run complete in" /var/log/
chef/client.log | head -n 3' 2>&1 | tee /tmp/tee && grep
'Chef Run complete' /tmp/tee | sort -n -k +13 | tail -5
dn0035.doop: [Mon, 14 May 2012 03:21:07 +0000] INFO: Chef
Run complete in 512.936813012 seconds
dn0004.doop: [Mon, 14 May 2012 04:28:03 +0000] INFO: Chef
Run complete in 677.423964906 seconds
dn0006.doop: [Mon, 14 May 2012 04:29:51 +0000] INFO: Chef
Run complete in 770.231469266 seconds
dn0025.doop: [Mon, 14 May 2012 04:26:13 +0000] INFO: Chef
Run complete in 787.183615612 seconds
dn0030.doop: [Mon, 14 May 2012 04:30:42 +0000] INFO: Chef
Run complete in 848.586507872 seconds
36.
37.
38.
39.
40. Finding Run Time Outliers
• Knife doesn’t currently support Lucene’s NumericRangeQuery
• Elapsed time is a floating point number, but we can only match it as a
string due to query limitations in knife
• Work around it with knife search -a
41. % knife search node 'elapsed:[200 TO 225]' -a
lastrun.runtimes.elapsed
4 items found
id: cent6-vmtemplate.ny4dev.etsy.com
lastrun.runtimes.elapsed: 21.642378406
id: sandboxmisc01.ny4.etsy.com
lastrun.runtimes.elapsed: 211.749555
id: smardenfeld.vm.ny4dev.etsy.com
lastrun.runtimes.elapsed: 22.184596
id: bob0120.vm.ny4dev.etsy.com
lastrun.runtimes.elapsed: 21.348335354
42. % knife node lastrun sandboxmisc01.ny4.etsy.com
Status failed
Elapsed Time 211.78604
Start Time 2012-05-15 07:43:18 +0000
End Time 2012-05-15 07:46:50 +0000
Backtrace
Omitted for brevity
Exception
Chef::Exceptions::Package: package[diffutils]
(installerz::diffutils line 1) had an error: Yum failed -
#<Process::Status: pid 21293 exit 1> - returns: ["yum-dump
Repository Error: Cannot retrieve repository metadata
(repomd.xml) for repository: PostgreSQL-8.3-x86_64. Please
verify its path and try againn"]
43. What Did Chef Just Do?
• chefrecentupdates by Etsy engineer Laurie Denness
(https://github.com/lozzd/ChefScripts)
% chefrecentupdates
...
1 resources updated in /var/log/chef/client.log-20120505.gz:
[Fri, 04 May 2012 17:49:42 +0000]
INFO: cookbook_file[/usr/bin/gist]
...
45. Knife Preflight
• By Jon Cowie (https://github.com/jonlives/knife-preflight)
% knife preflight memcache::datacache
Searching for nodes containing memcache::datacache in their expanded
run_list...
4 Nodes found
datacache03.ny4.etsy.com
datacache04.ny4.etsy.com
datacache01.ny4.etsy.com
datacache02.ny4.etsy.com
Searching for roles containing memcache::datacache in their run_list...
1 Roles found
Datacache
Found 4 nodes and 1 roles using the specified search criteria
47. “Out-of-Band” Management
• dsh (distributed shell) works even if Chef server is down
• Etsy’s dsh groups are managed by Chef and generated from the list of
nodes corresponding to each role
48. Configs Bundled with Packages
• Be careful with configs distributed with packages overwriting Chef
configs
• They must be replaced by Chef before restarting services, so watch
out for resource order
49. Jon will be at Velocity!
• Workshop: Michelin Starred Cooking with Chef
• 11:00am Monday, 06/25/2012
• Topics
• Team-wide familiarity and understanding
• Critical approach and experimentation with workflows
• Plugin writing 101
50. We’re Hiring!
http://www.etsy.com/careers
• TONS of engineering positions open!
• Especially looking for a talented network engineer; referrals welcome!