The document discusses configuration management strategies like manual configuration, scripting, file distribution, and declarative syntax using tools like Chef. It explains key Chef concepts like recipes, resources, and how Chef clients converge systems to a desired state defined by run lists and roles. Chef manages infrastructure as code by treating configurations like any other code and reconstructing operations from code repositories.
The document provides a quick introduction to Chef, a tool for configuration management. It discusses key Chef concepts like resources, recipes, cookbooks, roles, and attributes. Resources are the basic building blocks in Chef and represent things like packages, files, and services. Recipes contain collections of resources to configure systems. Cookbooks contain recipes, files, and other code to manage related components. Roles define the recipes and cookbooks that should be applied to nodes. Attributes are used to parameterize resources and provide node details.
Serverspec and Sensu - Testing and Monitoring collidem_richardson
This document discusses using RSpec tests to check server configurations and integrating those tests with the monitoring tool Sensu. It recommends running RSpec tests frequently via Sensu to simplify monitoring and overlap with traditional checks. It outlines an iteration where Sensu runs RSpec tests and reports individual test results and summaries to provide monitoring of server configurations.
Steamlining your puppet development workflowTomas Doran
The document discusses ways to streamline a Puppet development workflow including using revision control, running Puppet in noop or automatic mode, moving changes slowly through testing and using branches, reporting on changes, and implementing testing strategies like unit testing with rspec-puppet and integration testing with serverspec. It also recommends tools like Foreman, Norman, Puppetfile, and Jenkins to improve testing and deployment.
Dennis Matotek, Technical Lead Platforms at Experian Hitwise Australia, gave an excellent presentation on setting up puppet using vagrant, puppet and testing, including a full demo of rspec-puppet and Jenkins.
Chasing AMI - Building Amazon machine images with Puppet, Packer and JenkinsTomas Doran
Using puppet when configuring EC2 machines seems a natural fit. However bringing up new machines from a community image with puppet is not trivial and can be slow, and so not useful for auto-scaling.
The cloud also offers a solution to ongoing server maintenance, allowing you to launch fresh instances whenever you upgrade your applications (Immutable or Phoenix servers). However to predictably succeed, you need to freeze the puppet code alongside the application version for deployment.
The solution to these issues is generating custom machine images (AMIs) with your software inlined. This talk will cover Yelp's use of a Packer, Jenkins and Puppet for generating AMIs. This will include how we deal with issues like bootstrapping, getting canonical information about a machine's environment and cluster state at launch time, as well as supporting immutable/phoenix servers in combination with more traditional long lived servers inside our hybrid cloud infrastructure.
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!Puppet
The document describes how to build a modern infrastructure using Puppet modules. It discusses setting up MCollective for orchestration, Sensu for monitoring, Logstash for logging, and Jenkins for continuous integration. A Puppet module called moderninfra is demonstrated that defines the architecture and installs/configures all of the required components including RabbitMQ, Elasticsearch, and Kibana. The full infrastructure can then be built out across multiple nodes by writing Hiera data and node definitions.
Chef for OpenStack - OpenStack Fall 2012 SummitMatt Ray
Chef for OpenStack is a collaborative project for the deployment and management of OpenStack clouds. This is an overview of the status of the project at the OpenStack Fall 2012 Summit
The document provides a quick introduction to Chef, a tool for configuration management. It discusses key Chef concepts like resources, recipes, cookbooks, roles, and attributes. Resources are the basic building blocks in Chef and represent things like packages, files, and services. Recipes contain collections of resources to configure systems. Cookbooks contain recipes, files, and other code to manage related components. Roles define the recipes and cookbooks that should be applied to nodes. Attributes are used to parameterize resources and provide node details.
Serverspec and Sensu - Testing and Monitoring collidem_richardson
This document discusses using RSpec tests to check server configurations and integrating those tests with the monitoring tool Sensu. It recommends running RSpec tests frequently via Sensu to simplify monitoring and overlap with traditional checks. It outlines an iteration where Sensu runs RSpec tests and reports individual test results and summaries to provide monitoring of server configurations.
Steamlining your puppet development workflowTomas Doran
The document discusses ways to streamline a Puppet development workflow including using revision control, running Puppet in noop or automatic mode, moving changes slowly through testing and using branches, reporting on changes, and implementing testing strategies like unit testing with rspec-puppet and integration testing with serverspec. It also recommends tools like Foreman, Norman, Puppetfile, and Jenkins to improve testing and deployment.
Dennis Matotek, Technical Lead Platforms at Experian Hitwise Australia, gave an excellent presentation on setting up puppet using vagrant, puppet and testing, including a full demo of rspec-puppet and Jenkins.
Chasing AMI - Building Amazon machine images with Puppet, Packer and JenkinsTomas Doran
Using puppet when configuring EC2 machines seems a natural fit. However bringing up new machines from a community image with puppet is not trivial and can be slow, and so not useful for auto-scaling.
The cloud also offers a solution to ongoing server maintenance, allowing you to launch fresh instances whenever you upgrade your applications (Immutable or Phoenix servers). However to predictably succeed, you need to freeze the puppet code alongside the application version for deployment.
The solution to these issues is generating custom machine images (AMIs) with your software inlined. This talk will cover Yelp's use of a Packer, Jenkins and Puppet for generating AMIs. This will include how we deal with issues like bootstrapping, getting canonical information about a machine's environment and cluster state at launch time, as well as supporting immutable/phoenix servers in combination with more traditional long lived servers inside our hybrid cloud infrastructure.
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!Puppet
The document describes how to build a modern infrastructure using Puppet modules. It discusses setting up MCollective for orchestration, Sensu for monitoring, Logstash for logging, and Jenkins for continuous integration. A Puppet module called moderninfra is demonstrated that defines the architecture and installs/configures all of the required components including RabbitMQ, Elasticsearch, and Kibana. The full infrastructure can then be built out across multiple nodes by writing Hiera data and node definitions.
Chef for OpenStack - OpenStack Fall 2012 SummitMatt Ray
Chef for OpenStack is a collaborative project for the deployment and management of OpenStack clouds. This is an overview of the status of the project at the OpenStack Fall 2012 Summit
This document discusses how to deploy OpenStack using Opscode Chef. It provides an overview of Chef and how it can be used to configure OpenStack infrastructure as code. Key topics covered include the Chef framework, tools, API, community, OpenStack cookbooks on GitHub, common resources like recipes, roles and data bags, and a roadmap for improving support of OpenStack components like Nova and Swift.
Sensu and Sensibility - Puppetconf 2014Tomas Doran
As the Yelp infrastructure and engineering team grew, so did the pain of managing Nagios. Problems like splitting alerting across multiple teams, providing high availability and managing nagios systems in multiple environments had become pressing. As we grew towards a service oriented architecture and pushed some services out into the cloud, we rapidly needed more automated monitoring configuration.
An evolutionary solution wasn’t going to solve all of our problems, we needed to revolutionize our monitoring. Sensu is built from the ground up to solve many of our issues and be easy to extend.
This talk covers our puppet ‘monitoring_check’ API (that sets up monitoring for our services within puppet), how and why we deploy Sensu and our custom handlers and escalations, along with how we provide automatic ‘self service’ monitoring for dynamic services and how we deal with the challenges posed by the more ephemeral nature of cloud architectures.
The document discusses setting up high availability for PostgreSQL 9.0 including streaming replication between a master and slave, configuration of the master and slave, using Corosync for clustering, backups, monitoring, and automation with Puppet. It provides an overview of the environment and goals, as well as details on specific components like the Corosync configuration, custom OCF resource, and monitoring tools.
The document discusses using Chef to deploy Django applications to real-world infrastructures. It begins by introducing Chef and its main components, including resources, recipes, roles, and treating infrastructure as code. It then covers using Python-specific tools with Chef and provides a case study on Packaginator, a Python packaging tool.
TXLF: Chef- Software Defined Infrastructure Today & TomorrowMatt Ray
The open source configuration management and automation framework Chef is used to configure, deploy and manage infrastructure of every sort. In addition to managing Linux, Windows and many other operating systems; Chef may be used to manage network hardware and storage systems. This session will provide an overview of the concepts and capabilities of Chef and discuss upcoming projects and how they fit into the Chef ecosystem.
This document discusses deploying fully automated, highly available Django application stacks using Chef. Chef is an infrastructure automation tool that allows users to define infrastructure as code. It enables deploying and managing multi-tier Django application stacks across servers through recipes, which are collections of resources that define what software and configuration should exist on nodes. Chef recipes can be extended with Ruby code and configured dynamically based on node attributes and data from searches. Cookbooks package recipes for reuse across environments. Roles are used to describe nodes and assign recipes and attributes. Data bags store arbitrary configuration data.
Manage and Deploy your sites with DrushAmazee Labs
Drush is not only awesome for managing your local Drupal site, with site aliases you can manage Drupal sites on remote servers without logging in via SSH!
With Drush Deploy you even can deploy sites on multiple servers fully automated with one tool we all aready know: Drush, no other additional library like Capistrano needed!
In this Session we will present how to setup Drush to work with remote sites, what is possible and what we use in our daily business.
Then we will dig deeper into Drush Deploy: how it works, what it does and show how to set it up to deploy a drupal site on multiple servers with a single command, with automated backups, database updates, cache clears and even rollback functionalities.
A talk about Open Source logging and monitoring tools, using the ELK stack (ElasticSearch, Logstash, Kibana) to aggregate logs, how to track metrics from systems and logs, and how Drupal.org uses the ELK stack to aggregate and process billions of logs a month.
Verifying your Ansible Roles using Docker, Test Kitchen and ServerspecEdmund Dipple
In this presentation I demonstrated how simple it is to develop Ansible roles in a test driven way, using the faster feedback that docker enables and the easily readable Serverspec test framework, with Test Kitchen helping to bring all the tools together.
Puppet getting started will show the different components used in puppet environments, starting with facter and puppet to different webinterfaces like puppet enterprise console and foreman. It will also cover an exemplary design for scaling the puppet master and for development livecycle of modules. Furthermore an example for design of modules will be given.
This document provides an overview of automated server deployment and configuration using Ansible. It discusses traditional server provisioning processes versus modern approaches using infrastructure as code and configuration management software. It introduces key concepts in Ansible like idempotence and provides examples of installing Apache web server using Ansible playbooks and modules. The document recommends Ansible as an easy to learn configuration management tool and outlines steps to get started, including installing Ansible, configuring inventory files, using modules and writing playbooks. It also discusses using Ansible to manage Docker images and containers.
Puppet is an IT automation tool that allows system administrators to define a desired configuration for servers and automatically enforce that configuration, ensuring consistency across infrastructure; it works by analyzing configuration scripts, comparing the actual server state to the desired state, and executing scripts to make any necessary changes; key components of Puppet include modules, node configuration files, templates, and resource types for common system configuration tasks.
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013Puppet
"Testing for Ops: Going Beyond the Manifest" by Christopher Webber, Infrastructure Engineer, Demand Media.
Presentation Overview: This talk aims to show the value of rspec-puppet for those who come from a more Ops-centric background. The focus will be on using tests to go beyond just rewriting manifests in rspec. Instead the focus will be on scenarios like: - Are the baseline security measures in place? - Do the differences between dev and prod get reflected? - Are the config elements that are core to the application present? In addition, tests will help to be a place to help document the oddities of our configurations and ensuring that minor changes don't result in catastrophe.
Speaker Bio: After beginning his career at UC Riverside supporting enterprise operations and bioinformatics research, Chris is now rocking being an infrastructure engineer at Demand Media in Santa Monica. He currently supports large high-traffic sites like eHow.com, LiveSTRONG.com, and Cracked.com. Chris enjoys attending local meetups, writing new Puppet modules, and creating small tools to make his team's lives a little easier. Find him on Twitter as @cwebber.
This is a presentation demonstrating how Sensu is used at Yelp to support dynamic infrastructure, and promote self-service monitoring among teams.
Video Part 1: https://vimeo.com/92770954
Video Part 2: https://vimeo.com/92838680
Debugging Hive with Hadoop-in-the-Cloud by David Chaiken of AltiscaleData Con LA
Anyone who has used Hadoop knows that jobs sometimes get stuck. Hadoop is powerful, and it’s experiencing a tremendous rate of innovation, but it also has many rough edges. As Hadoop practitioners we all spend a lot of effort dealing with these rough edges in order to keep Hadoop and Hadoop jobs running well for our customers and/or organizations. For this session, we will look at a typical problem encountered by a Hadoop user, and discuss its implications for the future of Hadoop development. We will also go through the solution to this kind of problem using step-by-step instructions and the specific code we used to identify the issue. As a community, we need to work together to improve this kind of experience for our industry. Now that Hadoop 2 has been shipped, we believe the Hadoop community will be able to focus its energies on rounding off rough edges like these, and this session should provide advanced users with some tools and strategies to identify issues with jobs and how to keep these running smoothly.
Bootstrapping Puppet and Application Deployment - PuppetConf 2013Puppet
"Bootstrapping Puppet and Application Deployment" by Robert de Macedo Soares, Application Security Engineer, BusinessWire.
Presentation Overview: A dive into the problems faced when first launching Puppet across existing, heterogeneous servers, outlining possible solutions using our experience as an example. In addition, this session will touch on application management and deployment using subversion and rake tasks, what works and what is a little rough around the edges.
Speaker Bio: Robert is an engineer who has spent the past several years attempting to automate away the need for the work that he does. Focusing on server automation and security work for BusinessWire, Robert also develops web services such as tee.ms, a chat service, and designs and develops games. Trism, which he co-designed, was nominated for Cellular Game of the Year by the Academy of Interactive Arts & Sciences in the 2009 Interactive Achievement Awards.
This document summarizes a workshop on network automation tools including Chef and Zero Touch Provisioning.
The agenda includes demonstrating ZTP to boot three bare metal switches, using Chef to orchestrate the baseline configuration of two switches and enforce configuration statements, creating a VXLAN tunnel between two leaf switches using Cisco's CVX, and starting an Opendaylight controller to configure Openflow on switches.
The workshop will require some Virtualbox experience and a notebook with at least 4GB RAM and 10GB storage. Software needed includes Virtualbox, Hypervisor, and virtualization solutions. Attendees should be DevOps engineers interested in the network side of DevOps.
The workshop will prepare VMs, demonstrate
Chef for OpenStack: OpenStack Spring Summit 2013Matt Ray
This document provides an overview of using Chef to deploy and manage OpenStack. It discusses why Chef is useful for infrastructure as code and its declarative interface. The document outlines the current state of the Chef for OpenStack project, including contributors, available cookbooks, and roadmap. It promotes the project as a way to collaboratively deploy OpenStack in a standardized, automated way and reduce fragmentation.
Tomas Doran presented on their implementation of Logstash at TIM Group to process over 55 million messages per day. Their applications are all Java/Scala/Clojure and they developed their own library to send structured log events as JSON to Logstash using ZeroMQ for reliability. They index data in Elasticsearch and use it for metrics, alerts and dashboards but face challenges with data growth.
Vagrant allows users to define and manage virtual machine environments in code. It provides tools to easily create consistent development environments that match production. With Vagrant, users can define virtual machines and provision them automatically using tools like Chef, Puppet, or shell scripts. This allows development environments to match production and avoids issues caused by differences between environments. Vagrant saves time by automating environment setup and ensuring environments are consistent.
Australian OpenStack User Group August 2012: Chef for OpenStackMatt Ray
This document discusses how Chef can be used to deploy and manage OpenStack infrastructure. It provides an overview of Chef's capabilities for infrastructure automation including defining servers, applications, and databases through code and templates. The document also describes the Chef for OpenStack project which provides cookbooks for deploying OpenStack components like Keystone, Glance, Nova, etc. It promotes the community around automating OpenStack deployment and management with Chef.
This document discusses Chef for OpenStack, which uses Chef to automate the deployment and management of OpenStack. It provides concise summaries of key points:
- Chef for OpenStack includes cookbooks for common OpenStack components like Keystone, Glance, Nova, and Swift that can be used to programmatically deploy and manage OpenStack infrastructure.
- The Chef community contributes to OpenStack-related cookbooks that are open source and available on GitHub, helping reduce fragmentation and encourage collaboration around automating OpenStack.
- Chef allows OpenStack infrastructures to be treated like code, with configurations version controlled and deployments that can be automatically reconstructed from backups, improving supportability and portability across environments.
This document discusses how to deploy OpenStack using Opscode Chef. It provides an overview of Chef and how it can be used to configure OpenStack infrastructure as code. Key topics covered include the Chef framework, tools, API, community, OpenStack cookbooks on GitHub, common resources like recipes, roles and data bags, and a roadmap for improving support of OpenStack components like Nova and Swift.
Sensu and Sensibility - Puppetconf 2014Tomas Doran
As the Yelp infrastructure and engineering team grew, so did the pain of managing Nagios. Problems like splitting alerting across multiple teams, providing high availability and managing nagios systems in multiple environments had become pressing. As we grew towards a service oriented architecture and pushed some services out into the cloud, we rapidly needed more automated monitoring configuration.
An evolutionary solution wasn’t going to solve all of our problems, we needed to revolutionize our monitoring. Sensu is built from the ground up to solve many of our issues and be easy to extend.
This talk covers our puppet ‘monitoring_check’ API (that sets up monitoring for our services within puppet), how and why we deploy Sensu and our custom handlers and escalations, along with how we provide automatic ‘self service’ monitoring for dynamic services and how we deal with the challenges posed by the more ephemeral nature of cloud architectures.
The document discusses setting up high availability for PostgreSQL 9.0 including streaming replication between a master and slave, configuration of the master and slave, using Corosync for clustering, backups, monitoring, and automation with Puppet. It provides an overview of the environment and goals, as well as details on specific components like the Corosync configuration, custom OCF resource, and monitoring tools.
The document discusses using Chef to deploy Django applications to real-world infrastructures. It begins by introducing Chef and its main components, including resources, recipes, roles, and treating infrastructure as code. It then covers using Python-specific tools with Chef and provides a case study on Packaginator, a Python packaging tool.
TXLF: Chef- Software Defined Infrastructure Today & TomorrowMatt Ray
The open source configuration management and automation framework Chef is used to configure, deploy and manage infrastructure of every sort. In addition to managing Linux, Windows and many other operating systems; Chef may be used to manage network hardware and storage systems. This session will provide an overview of the concepts and capabilities of Chef and discuss upcoming projects and how they fit into the Chef ecosystem.
This document discusses deploying fully automated, highly available Django application stacks using Chef. Chef is an infrastructure automation tool that allows users to define infrastructure as code. It enables deploying and managing multi-tier Django application stacks across servers through recipes, which are collections of resources that define what software and configuration should exist on nodes. Chef recipes can be extended with Ruby code and configured dynamically based on node attributes and data from searches. Cookbooks package recipes for reuse across environments. Roles are used to describe nodes and assign recipes and attributes. Data bags store arbitrary configuration data.
Manage and Deploy your sites with DrushAmazee Labs
Drush is not only awesome for managing your local Drupal site, with site aliases you can manage Drupal sites on remote servers without logging in via SSH!
With Drush Deploy you even can deploy sites on multiple servers fully automated with one tool we all aready know: Drush, no other additional library like Capistrano needed!
In this Session we will present how to setup Drush to work with remote sites, what is possible and what we use in our daily business.
Then we will dig deeper into Drush Deploy: how it works, what it does and show how to set it up to deploy a drupal site on multiple servers with a single command, with automated backups, database updates, cache clears and even rollback functionalities.
A talk about Open Source logging and monitoring tools, using the ELK stack (ElasticSearch, Logstash, Kibana) to aggregate logs, how to track metrics from systems and logs, and how Drupal.org uses the ELK stack to aggregate and process billions of logs a month.
Verifying your Ansible Roles using Docker, Test Kitchen and ServerspecEdmund Dipple
In this presentation I demonstrated how simple it is to develop Ansible roles in a test driven way, using the faster feedback that docker enables and the easily readable Serverspec test framework, with Test Kitchen helping to bring all the tools together.
Puppet getting started will show the different components used in puppet environments, starting with facter and puppet to different webinterfaces like puppet enterprise console and foreman. It will also cover an exemplary design for scaling the puppet master and for development livecycle of modules. Furthermore an example for design of modules will be given.
This document provides an overview of automated server deployment and configuration using Ansible. It discusses traditional server provisioning processes versus modern approaches using infrastructure as code and configuration management software. It introduces key concepts in Ansible like idempotence and provides examples of installing Apache web server using Ansible playbooks and modules. The document recommends Ansible as an easy to learn configuration management tool and outlines steps to get started, including installing Ansible, configuring inventory files, using modules and writing playbooks. It also discusses using Ansible to manage Docker images and containers.
Puppet is an IT automation tool that allows system administrators to define a desired configuration for servers and automatically enforce that configuration, ensuring consistency across infrastructure; it works by analyzing configuration scripts, comparing the actual server state to the desired state, and executing scripts to make any necessary changes; key components of Puppet include modules, node configuration files, templates, and resource types for common system configuration tasks.
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013Puppet
"Testing for Ops: Going Beyond the Manifest" by Christopher Webber, Infrastructure Engineer, Demand Media.
Presentation Overview: This talk aims to show the value of rspec-puppet for those who come from a more Ops-centric background. The focus will be on using tests to go beyond just rewriting manifests in rspec. Instead the focus will be on scenarios like: - Are the baseline security measures in place? - Do the differences between dev and prod get reflected? - Are the config elements that are core to the application present? In addition, tests will help to be a place to help document the oddities of our configurations and ensuring that minor changes don't result in catastrophe.
Speaker Bio: After beginning his career at UC Riverside supporting enterprise operations and bioinformatics research, Chris is now rocking being an infrastructure engineer at Demand Media in Santa Monica. He currently supports large high-traffic sites like eHow.com, LiveSTRONG.com, and Cracked.com. Chris enjoys attending local meetups, writing new Puppet modules, and creating small tools to make his team's lives a little easier. Find him on Twitter as @cwebber.
This is a presentation demonstrating how Sensu is used at Yelp to support dynamic infrastructure, and promote self-service monitoring among teams.
Video Part 1: https://vimeo.com/92770954
Video Part 2: https://vimeo.com/92838680
Debugging Hive with Hadoop-in-the-Cloud by David Chaiken of AltiscaleData Con LA
Anyone who has used Hadoop knows that jobs sometimes get stuck. Hadoop is powerful, and it’s experiencing a tremendous rate of innovation, but it also has many rough edges. As Hadoop practitioners we all spend a lot of effort dealing with these rough edges in order to keep Hadoop and Hadoop jobs running well for our customers and/or organizations. For this session, we will look at a typical problem encountered by a Hadoop user, and discuss its implications for the future of Hadoop development. We will also go through the solution to this kind of problem using step-by-step instructions and the specific code we used to identify the issue. As a community, we need to work together to improve this kind of experience for our industry. Now that Hadoop 2 has been shipped, we believe the Hadoop community will be able to focus its energies on rounding off rough edges like these, and this session should provide advanced users with some tools and strategies to identify issues with jobs and how to keep these running smoothly.
Bootstrapping Puppet and Application Deployment - PuppetConf 2013Puppet
"Bootstrapping Puppet and Application Deployment" by Robert de Macedo Soares, Application Security Engineer, BusinessWire.
Presentation Overview: A dive into the problems faced when first launching Puppet across existing, heterogeneous servers, outlining possible solutions using our experience as an example. In addition, this session will touch on application management and deployment using subversion and rake tasks, what works and what is a little rough around the edges.
Speaker Bio: Robert is an engineer who has spent the past several years attempting to automate away the need for the work that he does. Focusing on server automation and security work for BusinessWire, Robert also develops web services such as tee.ms, a chat service, and designs and develops games. Trism, which he co-designed, was nominated for Cellular Game of the Year by the Academy of Interactive Arts & Sciences in the 2009 Interactive Achievement Awards.
This document summarizes a workshop on network automation tools including Chef and Zero Touch Provisioning.
The agenda includes demonstrating ZTP to boot three bare metal switches, using Chef to orchestrate the baseline configuration of two switches and enforce configuration statements, creating a VXLAN tunnel between two leaf switches using Cisco's CVX, and starting an Opendaylight controller to configure Openflow on switches.
The workshop will require some Virtualbox experience and a notebook with at least 4GB RAM and 10GB storage. Software needed includes Virtualbox, Hypervisor, and virtualization solutions. Attendees should be DevOps engineers interested in the network side of DevOps.
The workshop will prepare VMs, demonstrate
Chef for OpenStack: OpenStack Spring Summit 2013Matt Ray
This document provides an overview of using Chef to deploy and manage OpenStack. It discusses why Chef is useful for infrastructure as code and its declarative interface. The document outlines the current state of the Chef for OpenStack project, including contributors, available cookbooks, and roadmap. It promotes the project as a way to collaboratively deploy OpenStack in a standardized, automated way and reduce fragmentation.
Tomas Doran presented on their implementation of Logstash at TIM Group to process over 55 million messages per day. Their applications are all Java/Scala/Clojure and they developed their own library to send structured log events as JSON to Logstash using ZeroMQ for reliability. They index data in Elasticsearch and use it for metrics, alerts and dashboards but face challenges with data growth.
Vagrant allows users to define and manage virtual machine environments in code. It provides tools to easily create consistent development environments that match production. With Vagrant, users can define virtual machines and provision them automatically using tools like Chef, Puppet, or shell scripts. This allows development environments to match production and avoids issues caused by differences between environments. Vagrant saves time by automating environment setup and ensuring environments are consistent.
Australian OpenStack User Group August 2012: Chef for OpenStackMatt Ray
This document discusses how Chef can be used to deploy and manage OpenStack infrastructure. It provides an overview of Chef's capabilities for infrastructure automation including defining servers, applications, and databases through code and templates. The document also describes the Chef for OpenStack project which provides cookbooks for deploying OpenStack components like Keystone, Glance, Nova, etc. It promotes the community around automating OpenStack deployment and management with Chef.
This document discusses Chef for OpenStack, which uses Chef to automate the deployment and management of OpenStack. It provides concise summaries of key points:
- Chef for OpenStack includes cookbooks for common OpenStack components like Keystone, Glance, Nova, and Swift that can be used to programmatically deploy and manage OpenStack infrastructure.
- The Chef community contributes to OpenStack-related cookbooks that are open source and available on GitHub, helping reduce fragmentation and encourage collaboration around automating OpenStack.
- Chef allows OpenStack infrastructures to be treated like code, with configurations version controlled and deployments that can be automatically reconstructed from backups, improving supportability and portability across environments.
Achieving Infrastructure Portability with ChefMatt Ray
Deploying to the cloud has made it easy to run large numbers of servers, but users may become dissatisfied with their particular cloud platform for reasons such as price, support and performance. There are a number of vendor lock-ins to avoid, this talk discusses how to do so with the open source configuration management and infrastructure automation platform Chef. Chef makes it easy to deploy to nearly every public and private cloud platform as well as virtualized and physical servers. Chef may also be used to deploy cloud infrastructures such as OpenStack, Eucalyptus or CloudStack. By abstracting away the platform, infrastructure becomes portable and you are free to deploy wherever necessary.
Chef is an open source configuration management and service integration automation tool that has been integral to a number of large successful OpenStack deployments. This talk will provide a brief introduction to Chef and why it frequently the configuration tool of choice for large deployments and discuss the use of Chef within the OpenStack ecosystem (development, testing, deploying and managing the installation). Chef also provides the ability to manage the instances running on top of Nova through the knife-openstack plugin.
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014Miguel Zuniga
This document discusses configuration management tools Puppet and Chef in the cloud. It provides an overview of what configuration management is and why it is more painful in cloud environments where resources are dynamic. It then covers using infrastructure as code and discusses Puppet and Chef architectures, code examples, and how to use them in a masterless configuration in the cloud. Key aspects covered include using repositories to manage code, rebuilding on failure, and dynamically updating config files using knife to search inventory in Chef.
This is story of our journey from SaltStack to Puppet and beyond. This talk will answer following questions:
- why we moved from SaltStack
- why Puppet was chosen
- how to use Puppet OpenSource in painless way
- which orchestration tool to use with Puppet
- what is next
Kevin Smith is the Director of Server Engineering at Opscode and has been developing software for 17 years including 7 years with Erlang. He discusses infrastructure as code, configuration management with Chef, and how Chef can be used in large environments. Specifically, he covers how Chef uses recipes, roles, attributes and resources to declaratively configure nodes. He also discusses how the Chef server and clients interact and how search is used. Finally, he notes how Chef is open source and has a large community contributing cookbooks and tools to support deployments of all sizes.
under the covers -- chef in 20 minutes or lesssarahnovotny
Learn how to automate your infrastructure to make more time for fun things. In this rapid fire intro to Chef, an open source provisioning and automation platform, we'll touch on the strengths of it's flexible architecture as well as showing some concrete and simple starting points on your path to become an executive chef.
This document provides a top ten list of tips for improving PHP and web application performance. They include tweaking realpath cache settings, using offline processing whenever possible, writing efficient SQL queries, not executing queries in loops, caching data, using a content delivery network, and using APC caching with apc.stat set to 0. The tips cover optimizing PHP, database, and infrastructure performance.
Chef for OpenStack provides a framework for automatically deploying and managing OpenStack using Chef. It includes cookbooks for common OpenStack components like Keystone, Glance, Nova, etc. that allow OpenStack to be deployed in an automated, repeatable way. The project has many corporate contributors and aims to reduce fragmentation and encourage collaboration in deploying OpenStack. It provides a Chef repository, documentation, and community support through various channels. The goal is to make deploying and managing OpenStack infrastructure as code a reality.
What's new in Docker - InfraKit - Docker Meetup Berlin 2016Patrick Chanezon
This document provides an overview of Docker and its products and initiatives:
1. Docker provides tools for container isolation using Linux kernel features like namespaces and cgroups. It also utilizes image layers for packaging applications.
2. Docker's products focus on the developer experience through tools like Docker for Mac/Windows, as well as orchestration with Swarm mode and services in Docker 1.12.
3. For operations, Docker provides tools to integrate with load balancers, templates, and other infrastructure through products like Docker Universal Control Plane and Docker Cloud. Docker is building tools to program infrastructure as code.
Rekcurd is a tool for deploying and managing machine learning models on Kubernetes. It provides a dashboard for uploading models, switching between models without stopping services, and versioning models. Rekcurd integrates with Kubernetes for high availability, auto-scaling, load balancing and auto-healing of machine learning services. The presentation demonstrates how Rekcurd addresses common tasks in serving machine learning models on Kubernetes.
Practical introduction to dev ops with chefLeanDog
The document provides an introduction to DevOps using Chef. It discusses configuration management and deployment automation. It introduces key Chef concepts like nodes, resources, recipes and cookbooks. It demonstrates using Chef recipes to configure a sample Ubuntu application server with Apache, Python, Django and PostgreSQL. The recipes install packages, create a virtualenv, install dependencies and configure the application using Chef resources and Ruby code.
Apache Spark the Hard Way: Challenges with Building an On-Prem Spark Analytic...Spark Summit
This document discusses building an on-premise analytics pipeline using Spark. It summarizes challenges faced including sharing a cluster for different environments, orchestrating multi-step jobs, performance concerns due to resource contention, debugging issues, and lack of development tooling. Solutions proposed include using Docker to isolate environments, Luigi for job orchestration, optimizing resource allocation, logging to Graylog, and developing custom tools. The next steps suggested are moving to the cloud to simplify development and enable broader insights.
Austin OpenStack Meetup December 2012 presentation. The first part of the session was Chef for OpenStack, the second was Q&A about AT&T's OpenStack private cloud deployments to multiple data centers.
Large-scaled Deploy Over 100 Servers in 3 MinutesHiroshi SHIBATA
Large-scaled Deploy Over 100 Servers in 3 Minutes
Deployment strategy for next generation involves creating OS images using Packer and cloud-init that allow deploying over 100 servers within 3 minutes through automation. The strategy uses Puppet for configuration management and builds minimal and application-specific images to reduce bootstrap time. All deployment operations are implemented through a CLI tool for rapid and automated scaling.
Taking Spinnaker for a spin @ London DevOps Meetup 36aleonhardt
This document summarizes an engineer's experience setting up and experimenting with Netflix's Spinnaker continuous delivery tool on a local Kubernetes cluster. The engineer and their team set up Kubernetes using Docker for Mac, installed Halyard to manage Spinnaker, and deployed Spinnaker services. They took a look around the Spinnaker and Kubernetes dashboards. An attempt was made to deploy sample applications but it initially failed due to incorrect assumptions about how Spinnaker works. After reviewing documentation, they were able to deploy applications a little better but more time is needed to fully understand Spinnaker. Key learnings included the importance of documentation and that Spinnaker works in a push rather than pull model.
This document discusses open-source logging and monitoring tools. It provides an overview of log formats from common daemons like Apache, Nginx, and Varnish. It then discusses shipping logs with Rsyslog and aggregating logs using the ELK stack, including Logstash, Elasticsearch, and Kibana. The document includes configuration examples and demonstrates parsing log data with Grok.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Driving Business Innovation: Latest Generative AI Advancements & Success StorySafe Software
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Infrastructure Challenges in Scaling RAG with Custom AI modelsZilliz
Building Retrieval-Augmented Generation (RAG) systems with open-source and custom AI models is a complex task. This talk explores the challenges in productionizing RAG systems, including retrieval performance, response synthesis, and evaluation. We’ll discuss how to leverage open-source models like text embeddings, language models, and custom fine-tuned models to enhance RAG performance. Additionally, we’ll cover how BentoML can help orchestrate and scale these AI components efficiently, ensuring seamless deployment and management of RAG systems in the cloud.
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Speck&Tech
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceIndexBug
Imagine a world where machines not only perform tasks but also learn, adapt, and make decisions. This is the promise of Artificial Intelligence (AI), a technology that's not just enhancing our lives but revolutionizing entire industries.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Building Production Ready Search Pipelines with Spark and MilvusZilliz
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
5. Infrastructure
"It is common to think in terms of individual machines
rather than view an entire infrastructure as a
combined whole"
“A good infrastructure, whether departmental,
divisional, or enterprise-wide, is a single loosely-
coupled virtual machine, with hundreds or
thousands of hard drives and CPU's.”
-- Bootstrapping an Infrastructure USENIX LISA ’98
http://www.infrastructures.org/papers/bootstrap/bootstrap.html
6. .... as code!
• Programmatically provision and
configure
• Treat like any other code base
• Reconstruct operations from code
repository, data backup, and bare
metal resources.
http://www.flickr.com/photos/louisb/4555295187/
7. Considerations
• Infrastructure changes over time
• Entropy
• Changing business requirements
http://www.flickr.com/photos/seatbelt67/502255276/
10. Manual
Configuration
• Labor intensive
• Error prone
• Hard to reproduce
• Unsustainable
http://www.flickr.com/photos/pureimaginations/4805330106/
11. Scripting
• Typically very brittle
• Throw away, one off scripts
• grep sed awk perl
• curl | bash
http://www.flickr.com/photos/40389360@N00/2428706650/
12. File
Distribution
• NFS mounts
• rdist
• scp-on-a-for-loop
• rsync on cron
http://www.flickr.com/photos/walkadog/4317655660
13. This used to be
awesome
for i in `cat servers.txt` ; do scp ntp.conf root@$i:/etc/
ntpd.conf ; done
for i in `cat servers.txt` ; do ssh root@$i /etc/init.d/ntpd
restart ; done
for i in `cat servers.txt` ; do ssh root@$i chkconfig ntpd
on ; done
• ^ does not scale
http://www.flickr.com/photos/alexerde/3479006495
14. Declarative
Syntax
• Define policy
• Say what, not how
• Abstract interface to
resources
• Enables some interesting
behavior
http://www.flickr.com/photos/bixentro/2591838509/
16. Declarative Syntax
package "ntp" do
action :install
Idempotence
•
end
cookbook_file "/etc/ntp.conf" do
You’ll hear this a lot
source "ntp.conf"
owner "root" • Property of declarative
group "root" interface
mode 0644
action :create
notifies :restart, “service[ntpd]”
• Eliminates brittleness of
end
scripting
service "ntpd" do
action [:enable,:start]
• Identity function: f(x)=x
end
17. Declarative Syntax
while true do
package "ntp" do
action :install
Idempotence
•
end
cookbook_file "/etc/ntp.conf" do
You’ll hear this a lot
source "ntp.conf"
owner "root" • Property of declarative
group "root" interface
mode 0644
action :create
notifies :restart, “service[ntpd]”
• Eliminates brittleness of
end
scripting
service "ntpd" do
action [:enable,:start]
• Identity function: f(x)=x
end • Safe to repeat
end
18. Declarative Syntax
Convergence
• Agents “converge” a system to
desired state
• Repetition inches closer to
desired state
• It eventually gets there
• SCIENCE!
http://www.flickr.com/photos/tolomea/4852616645/
19. Declarative Syntax
Convergence
service "ntpd" do
action [:enable,:start]
ignore_failure true
end
cookbook_file "/etc/ntp.conf" do • Agents “converge” a system to
source "ntp.conf" desired state
owner "root"
group "root"
mode 0644 • Repetition inches closer to
action :create desired state
•
notifies :restart, “service[ntpd]”
ignore_failure true It eventually gets there
•
end
SCIENCE!
package "ntp" do
action :install
ignore_failure true
end
20. Declarative Syntax
# echo “boom” > /etc/ntp.conf ;
chef-client
Convergence
$ grep server /etc/ntp.conf | head -n 1
us.pool.ntp.org
$ ps -e | grep ntp
• Fights entropy, unauthorized
1799 ? 00:00:00 ntpd
changes, and gingivitis
# /etc/init.d/ntpd stop ; chef-client • Update function inputs to deal
with changing requirements
ps -e | grep ntp
1822 ? 00:00:00 ntpd
21. Config Generation
• Often made by hand (still!?)
• Stop that.
• Generate them based on
database content
• Infrastructures evolve
http://www.flickr.com/photos/jabella/4753170413/
33. Generate configs
• Centralized generation
• Version control!
• Distribute with packages, Chef,
git, whatever.
http://www.flickr.com/photos/ssoosay/5126146763/
34. Generate configs
• Local generation directly on nodes
• Reduces management complexity
• No need to distribute
• Version control the programs instead
http://www.flickr.com/photos/ssoosay/5126146763/
36. All That Stuff
• Declarative interface to resources
• Database of nodes and their roles
• Grab remote configs
• Generate configs locally
37. and more!
• Data Driven Infrastructure
• Use APIs to obtain data
• chef-server, SQL, anything.
• Feed resources parameters
• IPs, FQDNs, memory sizes,
• Templates, package, firewall
rules
38. Architecture
• Code Repository
• Chef Server
• Chef Clients
• Data Bags
• Recipes and Cookbooks
• Roles and Run Lists
http://www.flickr.com/photos/boedker/3871267007
39. Code Repository
• Version control
• Development
workflows
• Sharing is Caring
40. Chef Server
Server
Server
chef-server
Server
Server
• Upload from laptop
with knife
RESTful API
Cookbook
Cookbook
Cookbook
Data Bag
Knife Knife
Role
Knife
41. Chef Clients
Server
Server
chef-server
Server
Server
Knife
• Clients are API users
• Read RESTful API Knife
• Write
• Search
chef-client
chef-client chef-client chef-client chef-client
42. Chef Clients
Server
Server
someara.pub chef-server
Server
Server
jtimberman.pub
node5.fqdn.pub
Knife someara.pem
• Clients are API users
• Public keys on server RESTful API Knife jtimberman.pem
• Private keys local to
machines
chef-client
chef-client chef-client chef-client chef-client
node5.fqdn.pem
43. Run Lists
Server
Server
chef-server
Server
Server
Ohai!
API chef-client
Give me
recipe[ntp::client]
ntp
node
client.rb
44. Run Lists
Server
Server
chef-server
Server
Server
Ohai!
chef-client
API Give me
“ntp::client”,
ntp “openssh::server”
openssh
node
client.rb
server.rb
45. Run Lists
Server
Server
chef-server
Server
Server
Ohai!
chef-client Give me
API
“recipe[ntp::client]”,
ntp “recipe[openssh::server]”,
“recipe[apache]”,
openssh
node “recipe[php]”
client.rb apache
server.rb php
default.rb
default.rb
46. Roles
Server
Server
chef-server
Server
Server
Role Recipe
API
Role
Role Recipe
Role Recipe
Recipe
Knife
Recipe
Recipe
Recipe
47. Roles
Server
Server
chef-server
Server
Server
chef-client Ohai!
API
Give me
ntp “role[base]”,
“role[webserver]”
openssh
node
client.rb apache
server.rb php
default.rb
default.rb
48. Roles
Server
Server
chef-server
Server
Server
ntp
openssh
chef-client
API client.rb apache
php
server.rb “role[webserver]”
default.rb
ntp default.rb node
openssh
chef-client
client.rb mysql
server.rb
server.rb
“role[database]”
node
49. Bootstrapping
nodes
• Get chef-client installed
• Write run list to a file
• “Press go”
http://www.flickr.com/photos/liftarn/1447521121/
51. Bootstrapping nodes
{
"kernel": {
"machine": "x86_64",
"name": "Darwin",
"os": "Darwin",
"version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010;
•
root:xnu-1504.7.4~1/RELEASE_I386",
Ohai generates a JSON },
"release": "10.4.0"
attributes list "platform_version": "10.6.4",
•
"platform": "mac_os_x",
Run list and attributes are "platform_build": "10F569",
"domain": "local",
combined into a Node object "os": "darwin",
"current_user": "mray",
•
"ohai_time": 1278602661.60043,
Can be viewed and "os_version": "10.4.0",
"uptime": "18 days 17 hours 49 minutes 18 seconds",
searched through API "ipaddress": "10.13.37.116",
"hostname": "morbo",
"fqdn": "morbomorbo.local",
"uptime_seconds": 1619358
}
52. Bootstrapping nodes
• Run list is requested
• Cookbooks downloaded
• Recipes executed
• Node saved to chef-server
http://www.flickr.com/photos/architopher/457885721
53. Cookbooks
and Recipes
• Cookbooks contain recipes
• And everything they need to
work
• Templates, files, custom
resources, etc
http://www.flickr.com/photos/shutterhacks/4474421855/
54. Cookbooks
$ tree -a cookbooks/haproxy/
README.md
attributes
default.rb
• Cookbooks contain recipes metadata.rb
• And everything they need to recipes
work app_lb.rb
default.rb
• Templates, files, custom templates
resources, etc default
haproxy-app_lb.cfg.erb
haproxy-default.erb
haproxy.cfg.erb
55. Recipes
package "haproxy" do
action :install
end
template "/etc/default/haproxy" do
source "haproxy-default.erb"
• Recipes contain lists of owner "root"
group "root"
resources mode 0644
notifies :restart, "service[haproxy]"
end
service "haproxy" do
action [:enable, :start]
end
57. Resources
package "apache2" do
version "2.2.11-2ubuntu2.6"
action :install
end
template "/etc/apache2/apache2.conf" do
source "apache2.conf.erb"
owner "root"
group "root"
mode 0644
action :create
end
58. Resources
package "apache2" do
•
version "2.2.11-2ubuntu2.6"
Have a type action :install
end
template "/etc/apache2/apache2.conf" do
source "apache2.conf.erb"
owner "root"
group "root"
mode 0644
action :create
end
59. Resources
package "apache2" do
•
version "2.2.11-2ubuntu2.6"
Have a type action :install
•
end
Have a name
template "/etc/apache2/apache2.conf" do
source "apache2.conf.erb"
owner "root"
group "root"
mode 0644
action :create
end
60. Resources
package "apache2" do
•
version "2.2.11-2ubuntu2.6"
Have a type action :install
•
end
Have a name
•
template "/etc/apache2/apache2.conf" do
Have parameters source "apache2.conf.erb"
owner "root"
group "root"
mode 0644
action :create
end
61. Resources
package "apache2" do
•
version "2.2.11-2ubuntu2.6"
Have a type action :install
•
end
Have a name
•
template "/etc/apache2/apache2.conf" do
Have parameters source "apache2.conf.erb"
owner "root"
• Take action to put the resource group "root"
mode 0644
in the declared state action :create
end
63. Searching
• All object in Chef server are
indexed by Solr
http://www.flickr.com/photos/fotos_medem/3399096196/
64. Searching
• All object in Chef server are
indexed by Solr
• Can search through the API
http://www.flickr.com/photos/fotos_medem/3399096196/
65. Searching
• All object in Chef server are
indexed by Solr
• Can search through the API
• From knife and in recipes
http://www.flickr.com/photos/fotos_medem/3399096196/
66. Searching
• All object in Chef server are
indexed by Solr
• Can search through the API
• From knife and in recipes
• Returns an array of JSON Node
objects
http://www.flickr.com/photos/fotos_medem/3399096196/