Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013Puppet
"Running at Scale: Practical Performance Tuning with Puppet" by Sam Kottler Engineer, Red Hat.
Presentation Overview: This session will talk about some production issues I've seen running Puppet in large environments. From how to manage a single master with hundreds of hosts to real-life patterns for building high availability clusters that scale to 10's of thousands of agents. Another important topic that will be covered is how to deploy networked filesystems that perform well under high load and streaming files to many hosts simultaneously.
Speaker Bio: Sam Kottler is a software engineer in the Virtualization R&D group at Red Hat. He's helped build infrastructure for leading startups, including Digg.com, Acquia, and Venmo and is a contributor to Puppet, the Fedora Project, Drupal, and the Rubygems.org. Sam speaks around the world on the topics of internet security, systems automation, and software architecture.
Configuration Management - Finding the tool to fit your needsSaltStack
This presentation was originally given by Joseph Hall, SaltStack senior engineer, at the combined Montreal Python and DevOps Montreal meet up on April 14, 2014. Here is the talk abstract: In ye olde days of web, a company might manage a handful of servers, each manually and frequently tuned and re-tuned to the company's needs. Those days are gone. Server farms now dominate, and it is no longer reasonable to manage individual servers by hand. Various configuration management tools have stepped in to help the modern engineer, but which to choose? It is not an easy question, and canned pitches from sales people are unlikely to take into account all of your variables. This talk will attempt to discuss The Big Four objectively, and from what angles they approach the task at hand.
Continuously-Integrated Puppet in a Dynamic EnvironmentPuppet
This talk will show how we deploy Puppet without a Puppetmaster on an autoscaling Amazon Web Services infrastructure. Key points of interest: - Masterless Puppet - Use of Jenkins for Puppet manifest testing and environment promotion (test->staging->production) - Puppet integration with Amazon CloudFormation
Sam Bashton
Director, Bashton Ltd
After working for a number of Internet Service Providers, Sam founded Bashton Ltd in 2004. Focussing exclusively on Linux and Open Source software, Sam and his team provide consultancy, support and 24/7 infrastructure management for a number of high-traffic websites. A serial early adopter, Sam has travelled the world providing training and consultancy and generally spreading the Open Source message. Sam lives in Manchester, UK.
SaltConf14 - Brendan Burns, Google - Management at Google ScaleSaltStack
As a leading developer of highly scalable, large-scale Web services, Google was forced early on to develop systems to support the deployment and management of diverse workloads at an immense scale. As the broader developer community embraces cloud technologies we see significant parallels between the internal management infrastructure which Google has built over the last decade, and open source management technologies of today. This talk will describe Google's experience in managing large-scale compute services, draw parallels to open source efforts underway today, and sketch out how our past experience shapes our future development of the Google Cloud Platform.
This session will be an overview of highly available components that can be deployed with Puppet Enterprise. It will focus on some of the current Beta support in PuppetDB as well as tips and tricks from the professional services department. The session will cover field solutions ( both supported and unsupported ) that allow architectures to be designed that align with different levels of high availability across the services that support running puppet on agent nodes during an outage of your primary puppet infrastructure.
Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013Puppet
"Running at Scale: Practical Performance Tuning with Puppet" by Sam Kottler Engineer, Red Hat.
Presentation Overview: This session will talk about some production issues I've seen running Puppet in large environments. From how to manage a single master with hundreds of hosts to real-life patterns for building high availability clusters that scale to 10's of thousands of agents. Another important topic that will be covered is how to deploy networked filesystems that perform well under high load and streaming files to many hosts simultaneously.
Speaker Bio: Sam Kottler is a software engineer in the Virtualization R&D group at Red Hat. He's helped build infrastructure for leading startups, including Digg.com, Acquia, and Venmo and is a contributor to Puppet, the Fedora Project, Drupal, and the Rubygems.org. Sam speaks around the world on the topics of internet security, systems automation, and software architecture.
Configuration Management - Finding the tool to fit your needsSaltStack
This presentation was originally given by Joseph Hall, SaltStack senior engineer, at the combined Montreal Python and DevOps Montreal meet up on April 14, 2014. Here is the talk abstract: In ye olde days of web, a company might manage a handful of servers, each manually and frequently tuned and re-tuned to the company's needs. Those days are gone. Server farms now dominate, and it is no longer reasonable to manage individual servers by hand. Various configuration management tools have stepped in to help the modern engineer, but which to choose? It is not an easy question, and canned pitches from sales people are unlikely to take into account all of your variables. This talk will attempt to discuss The Big Four objectively, and from what angles they approach the task at hand.
Continuously-Integrated Puppet in a Dynamic EnvironmentPuppet
This talk will show how we deploy Puppet without a Puppetmaster on an autoscaling Amazon Web Services infrastructure. Key points of interest: - Masterless Puppet - Use of Jenkins for Puppet manifest testing and environment promotion (test->staging->production) - Puppet integration with Amazon CloudFormation
Sam Bashton
Director, Bashton Ltd
After working for a number of Internet Service Providers, Sam founded Bashton Ltd in 2004. Focussing exclusively on Linux and Open Source software, Sam and his team provide consultancy, support and 24/7 infrastructure management for a number of high-traffic websites. A serial early adopter, Sam has travelled the world providing training and consultancy and generally spreading the Open Source message. Sam lives in Manchester, UK.
SaltConf14 - Brendan Burns, Google - Management at Google ScaleSaltStack
As a leading developer of highly scalable, large-scale Web services, Google was forced early on to develop systems to support the deployment and management of diverse workloads at an immense scale. As the broader developer community embraces cloud technologies we see significant parallels between the internal management infrastructure which Google has built over the last decade, and open source management technologies of today. This talk will describe Google's experience in managing large-scale compute services, draw parallels to open source efforts underway today, and sketch out how our past experience shapes our future development of the Google Cloud Platform.
This session will be an overview of highly available components that can be deployed with Puppet Enterprise. It will focus on some of the current Beta support in PuppetDB as well as tips and tricks from the professional services department. The session will cover field solutions ( both supported and unsupported ) that allow architectures to be designed that align with different levels of high availability across the services that support running puppet on agent nodes during an outage of your primary puppet infrastructure.
Creating SaltStack State data with PyobjectsEvan Borgstrom
Pyobjects is an alternative renderer that allows you to author SaltStack state data in pure Python using a Pythonic API.
This presentation takes an in-depth look at the motivation behind creating the SaltStack Pyobjects renderer and cover how to use it, and best practices.
OpenNebula Conf 2014: CentOS, QA an OpenNebula - Christoph GaluschkaNETWAYS
CentOS, the Community Enterprise OS, uses Opennebula as virtualization plattform for its automated QA-process. The opennebula setup consists of 3 nodes, all running CentOS-6, who handle the following tasks:
– sunstone as cloud controller
– local mirror/DNS-Server/http-Server for the VMs to pull in packages
– one VM to run a jenkins instance to launch the various tests (ci.de.centos.org)
– nginx on the cloud controller to forward http traffic to the jenkins VM
A public git repository (http://www.gitorious.org/testautomation) is used to allow whoever wants to contribute to pull the current test suite – t_functional, a series of bash scripts used to do funtional tests of various applications, binaries, configuration files and Trademark issues. As new tests are added to the repo via personal clones and merge requests, those tests first need to complete a test run via jenkins. Each test run currently consists of 4 VMs (one for each arch for C5 and C6 – C7 to come), which run the complete test suite. All VMs used for theses tests are instantiated and torn down on demand, whenever the call to testrun a personal clone is issued (via IRC).
Once completed successfully, the request is merged into the main repo. The jenkins node monitors this repository and which automatically triggers another complete test run.
Besides these triggered test runs, the test suite is automatically triggered daily to run. This is used to verify functionality of published updates – a handfull of failty updates have allready been discovered this way.
Besides t_functional, the Linux Test Project Suite of tests is also run on a daily basis, also to verify functionality of the OS and all updates.
The third setup is used to test the available and functional integrity of published docker images for CentOS.
All these tests are later – during the QA-phase of a point release – used to verify functionality of new packages inside the CentOS QA-Setup.
KubeCon EU 2016: Full Automatic Database: PostgreSQL HA with KubernetesKubeAcademy
Why pay for always-on relational database service when you can deploy it yourself so easily? This demo-heavy talk will show off a deceptively simple high availability stack for PostgreSQL, using Docker, Etcd, Kubernetes, Patroni and Atomic. Not only is this open source solution ready to go to give you HA Postgres right now, it represents an approach which can be adapted to other relational databases with replication.
Sched Link: http://sched.co/6BV4
Puppet Camp Berlin 2015: Andrea Giardini | Configuration Management @ CERN: G...NETWAYS
In 2011, CERN decided to start using Puppet as main tool for development, machines configuration and provisioning as replacement of Quattor.
Since then the infrastructure has changed a lot, the "Agile infrastructure" project evolved is a series of tools and softwares that currently allow more than 10.000 nodes to be configured and provisioned following custom definitions.
Foreman, Git, Openstack and our homemade librarian Jens are only a few of the tools that will be described during the talk, that aims to give an overview about the current workflow for machines lifecycle at CERN.
This talk will cover how Puppet allows us to deal with several hundred of installations a day and, at the same time, provide highly customizable machine configurations for service owners.
SaltConf14 - Saurabh Surana, HP Cloud - Automating operations and support wit...SaltStack
Using SaltStack to automate enterprise IT operations and support capabilities is not as well documented as the more traditional SaltStack use cases. This session will show how the HP Cloud team runs a secure and reliable SaltStack automation environment by writing Salt states and modules to simplify day-to-day operations and support while extending SaltStack capabilities through dynamic states and modules. The talk will also show how to protect sensitive information and safe guard against user errors.
SaltConf14 - Craig Sebenik, LinkedIn - SaltStack at Web ScaleSaltStack
This talk will focus on the unique challenges of managing Web scale and an application stack that lives on tens of thousands of servers spread across multiple data centers. Learn more about LinkedIn's unique topology, about the development of an efficient build environment, and hear more about LinkedIn plans for a deployment system based on Salt. Also, all of the software that runs LinkedIn sends a LOT of data. In order to stay ahead of this tidal wave of data, the team must address scale challenges seen in very few environments through efficient use of monitoring and metrics systems. This talk will highlight best practices and user training necessary for the use of SaltStack in large environments.
SaltConf14 - Matthew Williams, Flowroute - Salt Virt for Linux contatiners an...SaltStack
This SaltConf14 talk by Matthew Williams of Flowroute shows the power of Salt Virt and Runner for creating and managing VMs and Linux containers. A demonstration of the Salt lxc module shows the simplicity with which containers and VMs can be created and configured.
SaltConf 2014 keynote - Thomas Jackson, LinkedIn
Safety with Power tools
As infrastructure scales, simple tasks become increasingly difficult. For large infrastructures to be manageable, we use automation. But automation, like any power tool, comes with its own set of risks and challenges. Automation should be handled like production code, and great care should be exercised with power tools. This talk will cover how SaltStack is used at LinkedIn and offer tips and tricks for automating management with SaltStack at massive scale including a look at LinkedIn-inspired Salt features such as blacklist and pre-req states. It will also cover Salt master and minion instrumentation and a compilation of how not to use Salt.
How Can OpenNebula Fit Your Needs: A European Project FeedbackNETWAYS
BonFIRE is an european project which aims at providing a ”multi-site cloud facility for applications, services and systems research and experimentation”. Grouping different research cloud providers behind a common set of tools, APIs and services, it enables users to run their experiment against a heterogeneous set of infrastructure, hypervisors, networks, etc …
BonFIRE, and thus the (OpenNebula) testbeds, provide a relatively small set of images used to boot VMs. However, the experimental nature of BonFIRE projects results in a big ”turnover” of running VMs. Lot of VMs are used for a time period between a few hours and a few days, and an experiment startup can trigger deployment of many VMs at same time on a small set of OpenNebula workers, which does not correspond to usual Cloud workflow.
Default OpenNebula is not optimized for such usecase (small amount of worker nodes, high VMs turnover). However, thanks to its ability to be easily modified at each level of a Cloud deployment workflow, OpenNebula has been tuned to make it fit better with BonFIRE deployment process. This presentation will explain how to change OpenNebula TM and VMM to improve the parrallel deployment of many VMs in a short amount of time, reducing time needed to deploy an experiment to its lowest without lot of expensive hardware.
The complexity of a typical OpenNebula installation brings a special set of challenges on the monitoring side. In this talk, I will show monitoring of a full stack of from the physical servers to storage layer and ONE daemon. Providing an aggregated view of this information allows you see the real impact of a certain failure. I would like to also present a use case for a “closed-loop” setup where new VMs are automatically added to the monitoring without human intervention, allowing for an efficient approach to monitoring the services a OpenNebula setup provides.
OpenNebula Conf 2014 | Puppet and OpenNebula - David LutterkortNETWAYS
Many facets of using an IaaS cloud like OpenNebula can be greatly
simplified by using a configuration management tool such as Puppet. This
includes the management of hosts as well as the management of cloud
resources such as virtual machines and networks. Of course, Puppet can also
play an important role in the management of the actual workload of virtual
machine instances. Besides using it in the traditional, purely agent-based
way, it is also possible to use Puppet during the building of machine
images. This serves two purposes: firstly, it speeds up the initial Puppet
run when an instance is launched off that image, sometimes quite
dramatically. Secondly, it supports operating immutable infrastructure
without losing Puppet’s benefits to organize and simplify the description
of the entire infrastructure.
This talk will show how Puppet can be used by adminsitrators to manage
OpenNebula hosts, and by users to manage their infrastructure as well as
how to use Puppet during image builds.
Sydney based cloud consultancy Cloudten's Richard Tomkinson shows how masterless Puppet can be used in concert with AWS's services including Lambda to automate server builds and manage code deployments
This talk is a followup to Deploying systemd at scale that was presented at systemd.conf 2016, and covers the aftermath of the migration of our fleet to CentOS 7. Now that systemd is available everywhere, we found more and more services that started adopting it for their deployment, leveraging its features and occasionally exposing interesting behaviors. At the same time, we've been able to hone our process for integrating and rolling out new versions of systemd on the fleet, and started building tooling to manage and monitor it at scale.
De-centralise and Conquer: Masterless Puppet in a Dynamic EnvironmentPuppet
"De-centralise and Conquer: Masterless Puppet in a dynamic environment" by Sam Bashton of Bashton Ltd., at Puppet Camp London 2013. Learn about upcoming Puppet Camps at http://puppetlabs.com/community/puppet-camp/
Linux Container Brief for IEEE WG P2302Boden Russell
A brief into to Linux Containers presented to IEEE working group P2302 (InterCloud standards and portability). This deck covers:
- Definitions and motivations for containers
- Container technology stack
- Containers vs Hypervisor VMs
- Cgroups
- Namespaces
- Pivot root vs chroot
- Linux Container image basics
- Linux Container security topics
- Overview of Linux Container tooling functionality
- Thoughts on container portability and runtime configuration
- Container tooling in the industry
- Container gaps
- Sample use cases for traditional VMs
Overall, a bulk of this deck is covered in other material I have posted here. However there are a few new slides in this deck, most notability some thoughts on container portability and runtime config.
Creating SaltStack State data with PyobjectsEvan Borgstrom
Pyobjects is an alternative renderer that allows you to author SaltStack state data in pure Python using a Pythonic API.
This presentation takes an in-depth look at the motivation behind creating the SaltStack Pyobjects renderer and cover how to use it, and best practices.
OpenNebula Conf 2014: CentOS, QA an OpenNebula - Christoph GaluschkaNETWAYS
CentOS, the Community Enterprise OS, uses Opennebula as virtualization plattform for its automated QA-process. The opennebula setup consists of 3 nodes, all running CentOS-6, who handle the following tasks:
– sunstone as cloud controller
– local mirror/DNS-Server/http-Server for the VMs to pull in packages
– one VM to run a jenkins instance to launch the various tests (ci.de.centos.org)
– nginx on the cloud controller to forward http traffic to the jenkins VM
A public git repository (http://www.gitorious.org/testautomation) is used to allow whoever wants to contribute to pull the current test suite – t_functional, a series of bash scripts used to do funtional tests of various applications, binaries, configuration files and Trademark issues. As new tests are added to the repo via personal clones and merge requests, those tests first need to complete a test run via jenkins. Each test run currently consists of 4 VMs (one for each arch for C5 and C6 – C7 to come), which run the complete test suite. All VMs used for theses tests are instantiated and torn down on demand, whenever the call to testrun a personal clone is issued (via IRC).
Once completed successfully, the request is merged into the main repo. The jenkins node monitors this repository and which automatically triggers another complete test run.
Besides these triggered test runs, the test suite is automatically triggered daily to run. This is used to verify functionality of published updates – a handfull of failty updates have allready been discovered this way.
Besides t_functional, the Linux Test Project Suite of tests is also run on a daily basis, also to verify functionality of the OS and all updates.
The third setup is used to test the available and functional integrity of published docker images for CentOS.
All these tests are later – during the QA-phase of a point release – used to verify functionality of new packages inside the CentOS QA-Setup.
KubeCon EU 2016: Full Automatic Database: PostgreSQL HA with KubernetesKubeAcademy
Why pay for always-on relational database service when you can deploy it yourself so easily? This demo-heavy talk will show off a deceptively simple high availability stack for PostgreSQL, using Docker, Etcd, Kubernetes, Patroni and Atomic. Not only is this open source solution ready to go to give you HA Postgres right now, it represents an approach which can be adapted to other relational databases with replication.
Sched Link: http://sched.co/6BV4
Puppet Camp Berlin 2015: Andrea Giardini | Configuration Management @ CERN: G...NETWAYS
In 2011, CERN decided to start using Puppet as main tool for development, machines configuration and provisioning as replacement of Quattor.
Since then the infrastructure has changed a lot, the "Agile infrastructure" project evolved is a series of tools and softwares that currently allow more than 10.000 nodes to be configured and provisioned following custom definitions.
Foreman, Git, Openstack and our homemade librarian Jens are only a few of the tools that will be described during the talk, that aims to give an overview about the current workflow for machines lifecycle at CERN.
This talk will cover how Puppet allows us to deal with several hundred of installations a day and, at the same time, provide highly customizable machine configurations for service owners.
SaltConf14 - Saurabh Surana, HP Cloud - Automating operations and support wit...SaltStack
Using SaltStack to automate enterprise IT operations and support capabilities is not as well documented as the more traditional SaltStack use cases. This session will show how the HP Cloud team runs a secure and reliable SaltStack automation environment by writing Salt states and modules to simplify day-to-day operations and support while extending SaltStack capabilities through dynamic states and modules. The talk will also show how to protect sensitive information and safe guard against user errors.
SaltConf14 - Craig Sebenik, LinkedIn - SaltStack at Web ScaleSaltStack
This talk will focus on the unique challenges of managing Web scale and an application stack that lives on tens of thousands of servers spread across multiple data centers. Learn more about LinkedIn's unique topology, about the development of an efficient build environment, and hear more about LinkedIn plans for a deployment system based on Salt. Also, all of the software that runs LinkedIn sends a LOT of data. In order to stay ahead of this tidal wave of data, the team must address scale challenges seen in very few environments through efficient use of monitoring and metrics systems. This talk will highlight best practices and user training necessary for the use of SaltStack in large environments.
SaltConf14 - Matthew Williams, Flowroute - Salt Virt for Linux contatiners an...SaltStack
This SaltConf14 talk by Matthew Williams of Flowroute shows the power of Salt Virt and Runner for creating and managing VMs and Linux containers. A demonstration of the Salt lxc module shows the simplicity with which containers and VMs can be created and configured.
SaltConf 2014 keynote - Thomas Jackson, LinkedIn
Safety with Power tools
As infrastructure scales, simple tasks become increasingly difficult. For large infrastructures to be manageable, we use automation. But automation, like any power tool, comes with its own set of risks and challenges. Automation should be handled like production code, and great care should be exercised with power tools. This talk will cover how SaltStack is used at LinkedIn and offer tips and tricks for automating management with SaltStack at massive scale including a look at LinkedIn-inspired Salt features such as blacklist and pre-req states. It will also cover Salt master and minion instrumentation and a compilation of how not to use Salt.
How Can OpenNebula Fit Your Needs: A European Project FeedbackNETWAYS
BonFIRE is an european project which aims at providing a ”multi-site cloud facility for applications, services and systems research and experimentation”. Grouping different research cloud providers behind a common set of tools, APIs and services, it enables users to run their experiment against a heterogeneous set of infrastructure, hypervisors, networks, etc …
BonFIRE, and thus the (OpenNebula) testbeds, provide a relatively small set of images used to boot VMs. However, the experimental nature of BonFIRE projects results in a big ”turnover” of running VMs. Lot of VMs are used for a time period between a few hours and a few days, and an experiment startup can trigger deployment of many VMs at same time on a small set of OpenNebula workers, which does not correspond to usual Cloud workflow.
Default OpenNebula is not optimized for such usecase (small amount of worker nodes, high VMs turnover). However, thanks to its ability to be easily modified at each level of a Cloud deployment workflow, OpenNebula has been tuned to make it fit better with BonFIRE deployment process. This presentation will explain how to change OpenNebula TM and VMM to improve the parrallel deployment of many VMs in a short amount of time, reducing time needed to deploy an experiment to its lowest without lot of expensive hardware.
The complexity of a typical OpenNebula installation brings a special set of challenges on the monitoring side. In this talk, I will show monitoring of a full stack of from the physical servers to storage layer and ONE daemon. Providing an aggregated view of this information allows you see the real impact of a certain failure. I would like to also present a use case for a “closed-loop” setup where new VMs are automatically added to the monitoring without human intervention, allowing for an efficient approach to monitoring the services a OpenNebula setup provides.
OpenNebula Conf 2014 | Puppet and OpenNebula - David LutterkortNETWAYS
Many facets of using an IaaS cloud like OpenNebula can be greatly
simplified by using a configuration management tool such as Puppet. This
includes the management of hosts as well as the management of cloud
resources such as virtual machines and networks. Of course, Puppet can also
play an important role in the management of the actual workload of virtual
machine instances. Besides using it in the traditional, purely agent-based
way, it is also possible to use Puppet during the building of machine
images. This serves two purposes: firstly, it speeds up the initial Puppet
run when an instance is launched off that image, sometimes quite
dramatically. Secondly, it supports operating immutable infrastructure
without losing Puppet’s benefits to organize and simplify the description
of the entire infrastructure.
This talk will show how Puppet can be used by adminsitrators to manage
OpenNebula hosts, and by users to manage their infrastructure as well as
how to use Puppet during image builds.
Sydney based cloud consultancy Cloudten's Richard Tomkinson shows how masterless Puppet can be used in concert with AWS's services including Lambda to automate server builds and manage code deployments
This talk is a followup to Deploying systemd at scale that was presented at systemd.conf 2016, and covers the aftermath of the migration of our fleet to CentOS 7. Now that systemd is available everywhere, we found more and more services that started adopting it for their deployment, leveraging its features and occasionally exposing interesting behaviors. At the same time, we've been able to hone our process for integrating and rolling out new versions of systemd on the fleet, and started building tooling to manage and monitor it at scale.
De-centralise and Conquer: Masterless Puppet in a Dynamic EnvironmentPuppet
"De-centralise and Conquer: Masterless Puppet in a dynamic environment" by Sam Bashton of Bashton Ltd., at Puppet Camp London 2013. Learn about upcoming Puppet Camps at http://puppetlabs.com/community/puppet-camp/
Linux Container Brief for IEEE WG P2302Boden Russell
A brief into to Linux Containers presented to IEEE working group P2302 (InterCloud standards and portability). This deck covers:
- Definitions and motivations for containers
- Container technology stack
- Containers vs Hypervisor VMs
- Cgroups
- Namespaces
- Pivot root vs chroot
- Linux Container image basics
- Linux Container security topics
- Overview of Linux Container tooling functionality
- Thoughts on container portability and runtime configuration
- Container tooling in the industry
- Container gaps
- Sample use cases for traditional VMs
Overall, a bulk of this deck is covered in other material I have posted here. However there are a few new slides in this deck, most notability some thoughts on container portability and runtime config.
OSDC 2015: Bernd Mathiske | Why the Datacenter Needs an Operating SystemNETWAYS
Developers are moving away from their host-based patterns and adopting a new mindset around the idea that the datacenter is the computer. It?s quickly becoming a mainstream model that you can view a warehouse full of servers as a single computer (with terabytes of memory and tens of thousands of cores). There is a key missing piece, which is an operating system for the datacenter (DCOS), which would provide the same OS functionality and core OS abstractions across thousands of machines that an OS provides on a single machine today. In this session, we will discuss:
How the abstraction of an OS has evolved over time and can cleanly scale to spand thousands of machines in a datacenter.
How key open source technologies like the Apache Mesos distributed systems kernel provide the key underpinnings for a DCOS.
How developers can layer core system services on top of a distributed systems kernel, including an init system (Marathon), cron (Chronos), service discovery (DNS), and storage (HDFS)
What would the interface to the DCOS look like? How would you use it?
How you would install and operate datacenter services, including Apache Spark, Apache Cassandra, Apache Kafka, Apache Hadoop, Apache YARN, Apache HDFS, and Google's Kubernetes.
How will developers build datacenter-scale apps, programmed against the datacenter OS like it?s a single machine?
Planning For High Performance Web ApplicationYue Tian
This slide is prepared for Beijing Open Party (a monthly unconference in Beijing China). And it's covered some important points when you are building a scalable web sites. And few page of this slide is in Chinese.
ContainerDays Boston 2015: "CoreOS: Building the Layers of the Scalable Clust...DynamicInfraDays
Slides from Barak Michener's talk "CoreOS: Building the Layers of the Scalable Cluster for Containers" at ContainerDays Boston 2015: http://dynamicinfradays.org/events/2015-boston/programme.html#layers
A GitOps model for High Availability and Disaster Recovery on EKSWeaveworks
Enterprises today require high availability and disaster recovery for critical business systems. One of the advantages Kubernetes can bring to the table is greater reliability and stability. When disaster strikes, cluster or application recovery should be quick and dependable.
Paul Curtis, Principal Solutions Architect at Weaveworks will demonstrate how to leverage Weave Kubernetes Platform and GitOps to create disaster recovery plans and highly available clusters with minimal effort on EKS.
In this webinar you will learn:
The 4 principles of GitOps (operations by pull request)
How to build for reproducibility, security and scale with EKS from the start
GitOps driven cluster and cluster lifecycle management with WKP
Containerization is more than the new Virtualization: enabling separation of ...Jérôme Petazzoni
Docker offers a new, lightweight approach to application
portability. Applications are shipped using a common container format,
and managed with a high-level API. Their processes run within isolated
namespaces which abstract the operating environment, independently of
the distribution, versions, network setup, and other details of this
environment.
This "containerization" has often been nicknamed "the new
virtualization". But containers are more than lightweight virtual
machines. Beyond their smaller footprint, shorter boot times, and
higher consolidation factors, they also bring a lot of new features
and use cases which were not possible with classical virtual machines.
We will focus on one of those features: separation of operational
concerns. Specifically, we will demonstrate how some fundamental tasks
like logging, remote access, backups, and troubleshooting can be
entirely decoupled from the deployment of applications and
services. This decoupling results in independent, smaller, simpler
moving parts; just like microservice architectures break down large
monolithic apps in more manageable components.
The Why and How of HPC-Cloud Hybrids with OpenStack - Lev Lafayette, Universi...OpenStack
Audience Level
Intermediate
Synopsis
High performance computing and cloud computing have traditionally been seen as separate solutions to separate problems, dealing with issues of performance and flexibility respectively. In a diverse research environment however, both sets of compute requirements can occur. In addition to the administrative benefits in combining both requirements into a single unified system, opportunities are provided for incremental expansion.
The deployment of the Spartan cloud-HPC hybrid system at the University of Melbourne last year is an example of such a design. Despite its small size, it has attracted international attention due to its design features. This presentation, in addition to providing a grounding on why one would wish to build an HPC-cloud hybrid system and the results of the deployment, provides a complete technical overview of the design from the ground up, as well as problems encountered and planned future developments.
Speaker Bio
Lev Lafayette is the HPC and Training Officer at the University of Melbourne. Prior to that he worked at the Victorian Partnership for Advanced Computing for several years in a similar role.
OpenNebulaconf2017US: Paying down technical debt with "one" dollar bills by ...OpenNebula Project
In addition to providing bare-metal access to large amounts of compute FAS Research Computing (FASRC) at Harvard also builds and fully maintains custom virtual machines tailored to faculty and researchers needs including lab websites, portals, databases, project development environments, and more both locally and on public clouds. Recently FASRC converted its internal VM infrastructure from a completely home-made KVM cluster to a more robust and reliable system powered by OpenNebula and Ceph configured with public cloud integration. Over the years as the number of VMs grew our home-made solution started to show signs of wear and tear with respect to scheduling, provisioning, management, inventory, and performance. Our new deployment improves on all of these areas and provides APIs and features that both help us serve clients more efficiently and improve our internal processes for testing new system configurations and dynamically spinning up resources for continous integration and deployment. Our new VM infrastructure deployment is fully automated via puppet and has been used to provision a multi-datacenter, fault-tolerant, VM infrastructure with a multi-tiered back-up system and robust VM and virtual disk monitoring. We will describe our internal system architecture and deployment, challenges we faced, and innovations we made along the way while deploying OpenNebula and Ceph. We will also discuss a new client-facing OpenNebula cloud deployment we’re currently beta testing with select users where users have full control over the creation and configuration of their VMs on FASRC compute resources via the OpenNebula dashboard and APIs.
Sanger, upcoming Openstack for Bio-informaticiansPeter Clapham
Delivery of a new Bio-informatics infrastructure at the Wellcome Trust Sanger Center. We include how to programatically create, manage and provide providence for images used both at Sanger and elsewhere using open source tools and continuous integration.
Similar to Puppet Camp Chicago 2014: Running Multiple Puppet Masters (Beginner) (20)
Automating it management with Puppet + ServiceNowPuppet
As the leading IT Service Management and IT Operations Management platform in the marketplace, ServiceNow is used by many organizations to address everything from self service IT requests to Change, Incident and Problem Management. The strength of the platform is in the workflows and processes that are built around the shared data model, represented in the CMDB. This provides the ‘single source of truth’ for the organization.
Puppet Enterprise is a leading automation platform focused on the IT Configuration Management and Compliance space. Puppet Enterprise has a unique perspective on the state of systems being managed, constantly being updated and kept accurate as part of the regular Puppet operation. Puppet Enterprise is the automation engine ensuring that the environment stays consistent and in compliance.
In this webinar, we will explore how to maximize the value of both solutions, with Puppet Enterprise automating the actions required to drive a change, and ServiceNow governing the process around that change, from definition to approval. We will introduce and demonstrate several published integration points between the two solutions, in the areas of Self-Service Infrastructure, Enriched Change Management and Automated Incident Registration.
Simplified Patch Management with Puppet - Oct. 2020Puppet
Does your company struggle with patching systems? If so, you’re not alone — most organizations have attempted to solve this issue by cobbling together multiple tools, processes, and different teams, which can make an already complicated issue worse.
Puppet helps keep hosts healthy, secure and compliant by replacing time-consuming and error prone patching processes with Puppet’s automated patching solution.
Join this webinar to learn how to do the following with Puppet:
Eliminate manual patching processes with pre-built patching automation for Windows and Linux systems.
Gain visibility into patching status across your estate regardless of OS with new patching solution from the PE console.
Ensure your systems are compliant and patched in a healthy state
How Puppet Enterprise makes patch management easy across your Windows and Linux operating systems.
Presented by: Margaret Lee, Product Manager, Puppet, and Ajay Sridhar, Sr. Sales Engineer, Puppet.
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
Check out the webinar slides to learn more about how XfilesPro transforms Salesforce document management by leveraging its world-class applications. For more details, please connect with sales@xfilespro.com
If you want to watch the on-demand webinar, please click here: https://www.xfilespro.com/webinars/salesforce-document-management-2-0-smarter-faster-better/
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Anthony Dahanne
Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ?
Venez le découvrir lors de cette session ignite
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Globus
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics.
To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/
Experience our free, in-depth three-part Tendenci Platform Corporate Membership Management workshop series! In Session 1 on May 14th, 2024, we began with an Introduction and Setup, mastering the configuration of your Corporate Membership Module settings to establish membership types, applications, and more. Then, on May 16th, 2024, in Session 2, we focused on binding individual members to a Corporate Membership and Corporate Reps, teaching you how to add individual members and assign Corporate Representatives to manage dues, renewals, and associated members. Finally, on May 28th, 2024, in Session 3, we covered questions and concerns, addressing any queries or issues you may have.
For more Tendenci AMS events, check out www.tendenci.com/events
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteGoogle
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-pilot-review/
AI Pilot Review: Key Features
✅Deploy AI expert bots in Any Niche With Just A Click
✅With one keyword, generate complete funnels, websites, landing pages, and more.
✅More than 85 AI features are included in the AI pilot.
✅No setup or configuration; use your voice (like Siri) to do whatever you want.
✅You Can Use AI Pilot To Create your version of AI Pilot And Charge People For It…
✅ZERO Manual Work With AI Pilot. Never write, Design, Or Code Again.
✅ZERO Limits On Features Or Usages
✅Use Our AI-powered Traffic To Get Hundreds Of Customers
✅No Complicated Setup: Get Up And Running In 2 Minutes
✅99.99% Up-Time Guaranteed
✅30 Days Money-Back Guarantee
✅ZERO Upfront Cost
See My Other Reviews Article:
(1) TubeTrivia AI Review: https://sumonreview.com/tubetrivia-ai-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamtakuyayamamoto1800
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Globus
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
Accelerate Enterprise Software Engineering with PlatformlessWSO2
Key takeaways:
Challenges of building platforms and the benefits of platformless.
Key principles of platformless, including API-first, cloud-native middleware, platform engineering, and developer experience.
How Choreo enables the platformless experience.
How key concepts like application architecture, domain-driven design, zero trust, and cell-based architecture are inherently a part of Choreo.
Demo of an end-to-end app built and deployed on Choreo.
Accelerate Enterprise Software Engineering with Platformless
Puppet Camp Chicago 2014: Running Multiple Puppet Masters (Beginner)
1. Jeffrey Miller
Sr. Systems Administrator
The University of Iowa: ITS – EI – SST – IT
Email: Jeff-L-Miller@uiowa.edu
Freenode: millerjl1701
2. StartedVAX mainframes, Sun SPARC systems in
college
Chemistry/Physics teacher…WGS Pro Linux
In 2001, sys admin for the UI Department of
Chemistry doing various things
Now, a infrastructure admin for the UI campus IT
group doing cross-platform stuff
Puppet infrastructure
Red Hat Satellite / Spacewalk
SC: OM,VMM
22+Years in the MN ARNG going various places
4. No infrastructure required other than content
distribution… which you probably already
have..
puppet apply….
All nodes have access to the modules and
manifests for your environment
Multiple zones? Multiple data centers?
Configuration churn?
Test Driven Development administration…
5. All machines have access to the entire
environment… perhaps your security office
has some concerns?
Reporting is limited
No exported resources
Exported resources allow nodes to share
information with each other
Infrastructure as code
6. Can’t I just get by with a single puppet
master?
Yes, without a doubt absolutely… maybe…
9. This is the holy grail of running puppet open
source….
maybe…
10. Should the servers be physical or virtual?
If you have an virtualization infrastructure
already, use it…
If you don’t have one, build it and then use
it… perhaps puppet can help you with that…
11. Leverage redundancy and resilience of the
VMware infrastructure
Ability to scale as needed quickly as more
systems are deployed with puppet
Flexibility in providing puppet infrastructure
to non-central units
Flexibility to test and roll through upgrades as
they come out from PuppetLabs
12. Java application (ok… it’s clojure inside a JVM…
just ask Deepak) that data generated by Puppet
with a postgresql database backend
Stores the most recent set of facts, most recent
catalog and by default 14 days of reports
Provides a very robust API for access to
information
Great for exported resources! (don’t even think
about using storeconfigs)
14. There isn’t a golden “though shall must
always have more than one at this level”
Number of resources declared in manifests
and infrastructure?
Exported resources?
Number of nodes?
Time to compile catalog?
Time between runs?
How are other administrative groups using
puppet?
17. Methods for getting the manifests modules out
to the puppet masters
Software distribution (RPM, Jenkins testing, etc.)
Puppetfile management with R10K, puppet- librarian,
puppet-librarian-simple
Puppet librarian
Run your own forge! (Pulp)
git pull via cron jobs…
NFS share…
Puppet 3.6 with the new path is a potential issue
for migration in this module…
22. theForeman and Dashboard use the same
postgresql DB backend (2 cores, 8 GB Ram)
theForeman web frontend: 2 cores, 4 GB Ram
Dashboard frontend: 2 cores, 8 GB Ram (have
puppet master and puppetdb installed on
same server for reporting access)
YMMV on any of these specs
23. LB (mod_proxy) splits
traffic to appropriate PM
Puppet Master (PM) can
serve out a single or
multiple environments via
apache/mod_passenger
The CA CRL is distributed
to the LB and PMs
All reports are forwarded
to foreman and dashboard
Shared file bucket across
all puppet masters via NFS
24. Leverage redundancy
and resilience of the
VMware infrastructure
Can quickly scale as
needed for systems or
environments
Security and partitioning
of environments
Safely test and roll
through server upgrades
Fast resource additions
to reduce chokepoints
25. A single load balancer
in front (hardware or
software)
A single server running
PuppetDB
A single server running
postgresql
Reporting?
27. Running puppet as a service for multiple
groups with different administrative
boundaries
Everything is just peachy for getting
manifests out and reports separated with
RBAC! except…
PuppetDB: All for one and one for all… full
access to any facts, catalogs, and reports in
the database… no RBAC