SlideShare a Scribd company logo
FOSDEM 2011 @Brussels, Belgium




   A tale of disaster recovery
               Cfengine everyday, practices and tools




Nicolas Charles      <nch@normation.com>
Jonathan Clarke      <jcl@normation.com>

                              
About the speakers
   Nicolas Charles                  Jonathan Clarke


 Cfengine contributor          OpenLDAP commiter
 Cfengine ”Community
    Champion” (C3)


   Scala Developer                    Sysadmin
            But we get on pretty well!
                      (mostly...)



                   
Agenda

1) Configuration Management 101
2) Our choice of tool
3) A tale of disaster recovery
4) Introducing Cfengine 3
5) Why we love Cfengine 3




                  
A bit about
Configuration Management...




           
Configuration management
   What is it ?
                Configuration Management is a field of
                  management that focuses on establishing and
                  maintaining consistency of a system (..)
                  throughout its life
                Software configuration management is the task
                  of tracking and controlling changes in the
                  software




Sources:
http://en.wikipedia.org/wiki/Configuration_management
http://en.wikipedia.org/wiki/Software_configuration_management

                                    
Configuration management
   Why is it useful ?
              Control changes
              Reproduce over time and nodes
              Audit and keep history data
              Repair automaticaly




                             
Configuration Management
          Tools

     What we chose, and why




            
Our choice
   Back in mid 2009
   Needed a configuration management tool
   Criteria:
              Open source
              Multi-platform agent (including Windows)
              Resilient
              Non-disruptive




                              
Our choice: candidates


 Cfengine 3       Puppet   Chef




               
Our choice: candidates


 Cfengine 3
                  More on this
                  choice later...




               
Disaster Recovery

       An ill-fated tale
    from the recent past

        (CASE STUDY)




          
Before the disaster...
   Our company's IT infrastructure

   Small company: small requirements
              Web site, email
              Git repository, Redmine...

   Small company: small budget
              All on one hosted server



                             
Asking for trouble?
   Just one hosted server! Critical services!

   No, a ”safe” configuration:
              Redundant hardware, 3 disk RAID-5 array
              All services automatically installed and setup
                 using Configuration Management
              Backups: daily (several off-site locations)
              Several VMs to separate services




                              
A critical failure
   2 hard drives fail simultaneously


            → RAID-5 array is down
            → Almost all services fail immediately
            → ”The end of the world as we know it”
            → Need to rebuild everything NOW




                          
Recovering
   Step 1: Panic!
   Step 2: Get a new server
   Step 3: Reinstall base OS + virtualization
   Step 4: Restore VM configuration... whoops
   Step 4: Re-create the VMs manually
   Step 5: Reinstall each OS in each VM...




                      
Recovering
   Step 6: Installation Configuration Management
   Step 7: Sit back and watch all the services
     coming back online as if by magic!
   Step 8: Huh, where's my data?
   Step 9: Manually restore backups
   Step 10: Make a list of missing data...




                      
Lessons learned
1) Hard disks fail reliably
2) Restoring virtualization setups:
     ●    Backing up the config files would have helped
     ●    Need CM tools to describe the desired state!
           (Cfengine Nova does this)
3) Configuration Management should tie in to our
    backup system
4) Backups were lacking some files: always test!




                        
Wishlist and discussion
   Integrating Configuration Management tools
      and backup systems is a crucial step for CM
      to be efficient for disaster recovery
              What do others do?


   Provisioning VMs and their resources (disks,
     network) should be automated too
              Cloud providers are one solution
              What about ”plain” virtualization?



                             
A bit about Cfengine 3...
    Sources: across the Internet



              
Cfengine: History




Source:
http://verticalsysadmin.com/blog/uncategorized/relative-origins-of-cfengine-chef-and-puppet

                                      
Cfengine 3: Intro
   Configuration management software
   Written in C
   Two versions :
              Community (GPL v3)
              Nova (closed source) : Community + extra
                features
   Backed by Cfengine AS – Norway based
     company founded in 2009




                            
Cfengine 3: Features
    According to Kuleven comparative study of
      configuration management systems:
                  Very mature
                  Cross platform (*BSD, AIX, HP-UX, Linux, Mac
                    OS X, Solaris, Windows)
                  Strongly distributed
                  Based on state description and convergence
                  Very high scalabily ( > 10000 nodes )
                  Very small footprint

Source: http://distrinet.cs.kuleuven.be/software/sysconfigtools/overview

                                       
Cfengine 3: Components
   Cf-agent
              Runs on all managed hosts
              Applies configuration – this is the heart
              Can connect to cf-serverd to get policies / files
   Cf-serverd
              Distributes policies and files
              Must be run on policy server(s)
              Usually run on all hosts to enable remote runs
   Cf-monitord
              Collects statistics on all nodes

                              
Cfengine 3: Promises
   Configuration rules are called promises
              ”Promise” to be in the desired state
              Cfengine agent handles the steps to get there:
                convergence


   Promise theory is based on research done in
     the University of Oslo




                             
Cfengine 3: Usage examples
   Large companies (Facebook, AMD, …)
   Critical systems: Joint Australia Tsunami
     Warning Centre
   Personal computers
   Mobile devices: Nokia N900
   Underwater devices: army submarines
   Small and medium companies...




                     
Why we love Cfengine 3...
Sources: our experience and opinions



              
Memory usage
   Daemon consumption on managed hosts




                   
Multi-platform
   Define a configuration for all operating systems
              Windows, Linux
              Make it ”transparent” (forget about the
                complexity)
              Existing standard library handling the
                differences between each OS and distribution




                             
File editing
   Only change what you need to
              You like your distribution's defaults?
              You have various different systems already
                setup and just need to change something?

   Search for lines and replace/delete/add them
   Only change one field in a file
              /etc/passwd for example...




                              
Complex tasks
   Powerful class system to trigger promises
              Based on nodes itself
              Based on time
              Based on whatever you might imagine
   Complex workflow can be created




                             
Thank you !

        FOSDEM 2011
Configuration Management room

And those brave enough to wake up early




               

More Related Content

What's hot

Linuxcon EU : Virtualization in the Cloud featuring Xen and XCP
Linuxcon EU : Virtualization in the Cloud featuring Xen and XCPLinuxcon EU : Virtualization in the Cloud featuring Xen and XCP
Linuxcon EU : Virtualization in the Cloud featuring Xen and XCP
The Linux Foundation
 
XS Boston 2008 Memory Overcommit
XS Boston 2008 Memory OvercommitXS Boston 2008 Memory Overcommit
XS Boston 2008 Memory Overcommit
The Linux Foundation
 
V mware v-sphere-replication-overview
V mware v-sphere-replication-overviewV mware v-sphere-replication-overview
V mware v-sphere-replication-overview
Firman Indrianto
 
XS Oracle 2009 Error Detection
XS Oracle 2009 Error DetectionXS Oracle 2009 Error Detection
XS Oracle 2009 Error Detection
The Linux Foundation
 
Xen Memory Management
Xen Memory ManagementXen Memory Management
Xen Memory Management
The Linux Foundation
 
Master VMware Performance and Capacity Management
Master VMware Performance and Capacity ManagementMaster VMware Performance and Capacity Management
Master VMware Performance and Capacity Management
Iwan Rahabok
 
XenServer 5.5 - Czy można zaoszczędzić na wirtualizacji serwerów? Darmowy Xen...
XenServer 5.5 - Czy można zaoszczędzić na wirtualizacji serwerów? Darmowy Xen...XenServer 5.5 - Czy można zaoszczędzić na wirtualizacji serwerów? Darmowy Xen...
XenServer 5.5 - Czy można zaoszczędzić na wirtualizacji serwerów? Darmowy Xen...
Peter Ocasek
 
Cvc2009 Moscow Xen App5 Fp1 Fabian Kienle Final
Cvc2009 Moscow Xen App5 Fp1 Fabian Kienle FinalCvc2009 Moscow Xen App5 Fp1 Fabian Kienle Final
Cvc2009 Moscow Xen App5 Fp1 Fabian Kienle Final
Liudmila Li
 
VMworld 2013: Protection for All - VMware vSphere Replication & SRM Technical...
VMworld 2013: Protection for All - VMware vSphere Replication & SRM Technical...VMworld 2013: Protection for All - VMware vSphere Replication & SRM Technical...
VMworld 2013: Protection for All - VMware vSphere Replication & SRM Technical...
VMworld
 
Linux On V Mware ESXi
Linux On V Mware ESXiLinux On V Mware ESXi
Linux On V Mware ESXi
Masafumi Ohta
 
How to Optimize Microsoft Hyper-V Failover Cluster and Double Performance
How to Optimize Microsoft Hyper-V Failover Cluster and Double PerformanceHow to Optimize Microsoft Hyper-V Failover Cluster and Double Performance
How to Optimize Microsoft Hyper-V Failover Cluster and Double Performance
StarWind Software
 
Ibm san volume controller and ibm tivoli storage flash copy manager redp4653
Ibm san volume controller and ibm tivoli storage flash copy manager redp4653Ibm san volume controller and ibm tivoli storage flash copy manager redp4653
Ibm san volume controller and ibm tivoli storage flash copy manager redp4653
Banking at Ho Chi Minh city
 
Usenix Invited Talk
Usenix Invited TalkUsenix Invited Talk
Usenix Invited Talk
webhostingguy
 
Clustering Enhancements
Clustering EnhancementsClustering Enhancements
Clustering Enhancements
Digicomp Academy AG
 
XS Oracle 2009 Intro Slides
XS Oracle 2009 Intro SlidesXS Oracle 2009 Intro Slides
XS Oracle 2009 Intro Slides
The Linux Foundation
 
„Wielka dwójka”, czyli jak działa Citrix XenApp na Citrix XenServer u dużych ...
„Wielka dwójka”, czyli jak działa Citrix XenApp na Citrix XenServer u dużych ...„Wielka dwójka”, czyli jak działa Citrix XenApp na Citrix XenServer u dużych ...
„Wielka dwójka”, czyli jak działa Citrix XenApp na Citrix XenServer u dużych ...
Peter Ocasek
 
XS Boston 2008 Fault Tolerance
XS Boston 2008 Fault ToleranceXS Boston 2008 Fault Tolerance
XS Boston 2008 Fault Tolerance
The Linux Foundation
 
ppt
pptppt
Using the tivoli storage manager hsm client for windows redp4126
Using the tivoli storage manager hsm client for windows redp4126Using the tivoli storage manager hsm client for windows redp4126
Using the tivoli storage manager hsm client for windows redp4126
Banking at Ho Chi Minh city
 
LF Collaboration Summit: Xen Project 4 4 Features and Futures
LF Collaboration Summit: Xen Project 4 4 Features and FuturesLF Collaboration Summit: Xen Project 4 4 Features and Futures
LF Collaboration Summit: Xen Project 4 4 Features and Futures
The Linux Foundation
 

What's hot (20)

Linuxcon EU : Virtualization in the Cloud featuring Xen and XCP
Linuxcon EU : Virtualization in the Cloud featuring Xen and XCPLinuxcon EU : Virtualization in the Cloud featuring Xen and XCP
Linuxcon EU : Virtualization in the Cloud featuring Xen and XCP
 
XS Boston 2008 Memory Overcommit
XS Boston 2008 Memory OvercommitXS Boston 2008 Memory Overcommit
XS Boston 2008 Memory Overcommit
 
V mware v-sphere-replication-overview
V mware v-sphere-replication-overviewV mware v-sphere-replication-overview
V mware v-sphere-replication-overview
 
XS Oracle 2009 Error Detection
XS Oracle 2009 Error DetectionXS Oracle 2009 Error Detection
XS Oracle 2009 Error Detection
 
Xen Memory Management
Xen Memory ManagementXen Memory Management
Xen Memory Management
 
Master VMware Performance and Capacity Management
Master VMware Performance and Capacity ManagementMaster VMware Performance and Capacity Management
Master VMware Performance and Capacity Management
 
XenServer 5.5 - Czy można zaoszczędzić na wirtualizacji serwerów? Darmowy Xen...
XenServer 5.5 - Czy można zaoszczędzić na wirtualizacji serwerów? Darmowy Xen...XenServer 5.5 - Czy można zaoszczędzić na wirtualizacji serwerów? Darmowy Xen...
XenServer 5.5 - Czy można zaoszczędzić na wirtualizacji serwerów? Darmowy Xen...
 
Cvc2009 Moscow Xen App5 Fp1 Fabian Kienle Final
Cvc2009 Moscow Xen App5 Fp1 Fabian Kienle FinalCvc2009 Moscow Xen App5 Fp1 Fabian Kienle Final
Cvc2009 Moscow Xen App5 Fp1 Fabian Kienle Final
 
VMworld 2013: Protection for All - VMware vSphere Replication & SRM Technical...
VMworld 2013: Protection for All - VMware vSphere Replication & SRM Technical...VMworld 2013: Protection for All - VMware vSphere Replication & SRM Technical...
VMworld 2013: Protection for All - VMware vSphere Replication & SRM Technical...
 
Linux On V Mware ESXi
Linux On V Mware ESXiLinux On V Mware ESXi
Linux On V Mware ESXi
 
How to Optimize Microsoft Hyper-V Failover Cluster and Double Performance
How to Optimize Microsoft Hyper-V Failover Cluster and Double PerformanceHow to Optimize Microsoft Hyper-V Failover Cluster and Double Performance
How to Optimize Microsoft Hyper-V Failover Cluster and Double Performance
 
Ibm san volume controller and ibm tivoli storage flash copy manager redp4653
Ibm san volume controller and ibm tivoli storage flash copy manager redp4653Ibm san volume controller and ibm tivoli storage flash copy manager redp4653
Ibm san volume controller and ibm tivoli storage flash copy manager redp4653
 
Usenix Invited Talk
Usenix Invited TalkUsenix Invited Talk
Usenix Invited Talk
 
Clustering Enhancements
Clustering EnhancementsClustering Enhancements
Clustering Enhancements
 
XS Oracle 2009 Intro Slides
XS Oracle 2009 Intro SlidesXS Oracle 2009 Intro Slides
XS Oracle 2009 Intro Slides
 
„Wielka dwójka”, czyli jak działa Citrix XenApp na Citrix XenServer u dużych ...
„Wielka dwójka”, czyli jak działa Citrix XenApp na Citrix XenServer u dużych ...„Wielka dwójka”, czyli jak działa Citrix XenApp na Citrix XenServer u dużych ...
„Wielka dwójka”, czyli jak działa Citrix XenApp na Citrix XenServer u dużych ...
 
XS Boston 2008 Fault Tolerance
XS Boston 2008 Fault ToleranceXS Boston 2008 Fault Tolerance
XS Boston 2008 Fault Tolerance
 
ppt
pptppt
ppt
 
Using the tivoli storage manager hsm client for windows redp4126
Using the tivoli storage manager hsm client for windows redp4126Using the tivoli storage manager hsm client for windows redp4126
Using the tivoli storage manager hsm client for windows redp4126
 
LF Collaboration Summit: Xen Project 4 4 Features and Futures
LF Collaboration Summit: Xen Project 4 4 Features and FuturesLF Collaboration Summit: Xen Project 4 4 Features and Futures
LF Collaboration Summit: Xen Project 4 4 Features and Futures
 

Viewers also liked

Volunteer Abroad in India - 16 Years and Counting
Volunteer Abroad in India - 16 Years and CountingVolunteer Abroad in India - 16 Years and Counting
Volunteer Abroad in India - 16 Years and Counting
Cross-Cultural Solutions
 
QL-IjmFT6Ub
QL-IjmFT6UbQL-IjmFT6Ub
QL-IjmFT6Ub
Rene Larson
 
Fall in Love with Russia, CCS Webinar Presentation
Fall in Love with Russia, CCS Webinar PresentationFall in Love with Russia, CCS Webinar Presentation
Fall in Love with Russia, CCS Webinar Presentation
Cross-Cultural Solutions
 
Volunteer in Russica: Learn From CCS Alumni - CCS Webinar Presentation
Volunteer in Russica: Learn From CCS Alumni - CCS Webinar PresentationVolunteer in Russica: Learn From CCS Alumni - CCS Webinar Presentation
Volunteer in Russica: Learn From CCS Alumni - CCS Webinar Presentation
Cross-Cultural Solutions
 
LSC - Synchronizing identities @ Loadays 2010
LSC - Synchronizing identities @ Loadays 2010LSC - Synchronizing identities @ Loadays 2010
LSC - Synchronizing identities @ Loadays 2010
Jonathan Clarke
 
Sharing automation - why we need a language like ncf for this (Ignite @ devop...
Sharing automation - why we need a language like ncf for this (Ignite @ devop...Sharing automation - why we need a language like ncf for this (Ignite @ devop...
Sharing automation - why we need a language like ncf for this (Ignite @ devop...
Jonathan Clarke
 
Why Volunteer Abroad? A Look at FAQs - CCS Webinar Presentation
Why Volunteer Abroad? A Look at FAQs - CCS Webinar PresentationWhy Volunteer Abroad? A Look at FAQs - CCS Webinar Presentation
Why Volunteer Abroad? A Look at FAQs - CCS Webinar Presentation
Cross-Cultural Solutions
 
Rudder 3.0 and beyond
Rudder 3.0 and beyondRudder 3.0 and beyond
Rudder 3.0 and beyond
Jonathan Clarke
 
Configuration management: automating and rationalizing server setup with CFEn...
Configuration management: automating and rationalizing server setup with CFEn...Configuration management: automating and rationalizing server setup with CFEn...
Configuration management: automating and rationalizing server setup with CFEn...
Jonathan Clarke
 

Viewers also liked (9)

Volunteer Abroad in India - 16 Years and Counting
Volunteer Abroad in India - 16 Years and CountingVolunteer Abroad in India - 16 Years and Counting
Volunteer Abroad in India - 16 Years and Counting
 
QL-IjmFT6Ub
QL-IjmFT6UbQL-IjmFT6Ub
QL-IjmFT6Ub
 
Fall in Love with Russia, CCS Webinar Presentation
Fall in Love with Russia, CCS Webinar PresentationFall in Love with Russia, CCS Webinar Presentation
Fall in Love with Russia, CCS Webinar Presentation
 
Volunteer in Russica: Learn From CCS Alumni - CCS Webinar Presentation
Volunteer in Russica: Learn From CCS Alumni - CCS Webinar PresentationVolunteer in Russica: Learn From CCS Alumni - CCS Webinar Presentation
Volunteer in Russica: Learn From CCS Alumni - CCS Webinar Presentation
 
LSC - Synchronizing identities @ Loadays 2010
LSC - Synchronizing identities @ Loadays 2010LSC - Synchronizing identities @ Loadays 2010
LSC - Synchronizing identities @ Loadays 2010
 
Sharing automation - why we need a language like ncf for this (Ignite @ devop...
Sharing automation - why we need a language like ncf for this (Ignite @ devop...Sharing automation - why we need a language like ncf for this (Ignite @ devop...
Sharing automation - why we need a language like ncf for this (Ignite @ devop...
 
Why Volunteer Abroad? A Look at FAQs - CCS Webinar Presentation
Why Volunteer Abroad? A Look at FAQs - CCS Webinar PresentationWhy Volunteer Abroad? A Look at FAQs - CCS Webinar Presentation
Why Volunteer Abroad? A Look at FAQs - CCS Webinar Presentation
 
Rudder 3.0 and beyond
Rudder 3.0 and beyondRudder 3.0 and beyond
Rudder 3.0 and beyond
 
Configuration management: automating and rationalizing server setup with CFEn...
Configuration management: automating and rationalizing server setup with CFEn...Configuration management: automating and rationalizing server setup with CFEn...
Configuration management: automating and rationalizing server setup with CFEn...
 

Similar to A tale of Disaster Recovery (Cfengine everyday, practices and tools)

Configuration management 101 - A tale of disaster recovery using CFEngine 3
Configuration management 101 - A tale of disaster recovery using CFEngine 3Configuration management 101 - A tale of disaster recovery using CFEngine 3
Configuration management 101 - A tale of disaster recovery using CFEngine 3
RUDDER
 
Configuration management: automating and rationalizing server setup with CFEn...
Configuration management: automating and rationalizing server setup with CFEn...Configuration management: automating and rationalizing server setup with CFEn...
Configuration management: automating and rationalizing server setup with CFEn...
RUDDER
 
Cfengine 2 Overview
Cfengine 2 OverviewCfengine 2 Overview
Cfengine 2 Overview
Scott Lackey
 
ICALEPCS 2011: Testing Environments using Virtualization
ICALEPCS 2011: Testing Environments using VirtualizationICALEPCS 2011: Testing Environments using Virtualization
ICALEPCS 2011: Testing Environments using Virtualization
Omer Khalid
 
LOAD BALANCING OF APPLICATIONS USING XEN HYPERVISOR
LOAD BALANCING OF APPLICATIONS  USING XEN HYPERVISORLOAD BALANCING OF APPLICATIONS  USING XEN HYPERVISOR
LOAD BALANCING OF APPLICATIONS USING XEN HYPERVISOR
Vanika Kapoor
 
Getting Started With CFEngine - Updated Version
Getting Started With CFEngine - Updated VersionGetting Started With CFEngine - Updated Version
Getting Started With CFEngine - Updated Version
CFEngine
 
Configuration
ConfigurationConfiguration
Configuration
Tomer Paz
 
A Xen Case Study
A Xen Case StudyA Xen Case Study
A Xen Case Study
Kris Buytaert
 
OpenQrm
OpenQrmOpenQrm
OpenQrm
Kris Buytaert
 
Em library
Em libraryEm library
Em library
shady999
 
Private Cloud Academy: Backup and DPM 2010
Private Cloud Academy: Backup and DPM 2010Private Cloud Academy: Backup and DPM 2010
Private Cloud Academy: Backup and DPM 2010
Aidan Finn
 
Presentation 1 open source tools in continuous integration environment v1.0
Presentation 1   open source tools in continuous integration environment v1.0Presentation 1   open source tools in continuous integration environment v1.0
Presentation 1 open source tools in continuous integration environment v1.0
Jasmine Conseil
 
Using openQRM to Manage Virtual Machines
Using openQRM to Manage Virtual MachinesUsing openQRM to Manage Virtual Machines
Using openQRM to Manage Virtual Machines
Kris Buytaert
 
Orchestrated Android-Style System Upgrades for Embedded Linux
Orchestrated Android-Style System Upgrades for Embedded LinuxOrchestrated Android-Style System Upgrades for Embedded Linux
Orchestrated Android-Style System Upgrades for Embedded Linux
Kynetics
 
Orchestrated Android-Style System Upgrades for Embedded Linux
Orchestrated Android-Style System Upgrades for Embedded LinuxOrchestrated Android-Style System Upgrades for Embedded Linux
Orchestrated Android-Style System Upgrades for Embedded Linux
NicolaLaGloria
 
OSCamp Kubernetes 2024 | Zero-Touch OS-Infrastruktur für Container und Kubern...
OSCamp Kubernetes 2024 | Zero-Touch OS-Infrastruktur für Container und Kubern...OSCamp Kubernetes 2024 | Zero-Touch OS-Infrastruktur für Container und Kubern...
OSCamp Kubernetes 2024 | Zero-Touch OS-Infrastruktur für Container und Kubern...
NETWAYS
 
J+s
J+sJ+s
J+s
happyuk
 
Os
OsOs
Cloud Application Security: Lessons Learned
Cloud Application Security: Lessons LearnedCloud Application Security: Lessons Learned
Cloud Application Security: Lessons Learned
Jason Chan
 
Part 2 improving your software development v1.0
Part 2   improving your software development v1.0Part 2   improving your software development v1.0
Part 2 improving your software development v1.0
Jasmine Conseil
 

Similar to A tale of Disaster Recovery (Cfengine everyday, practices and tools) (20)

Configuration management 101 - A tale of disaster recovery using CFEngine 3
Configuration management 101 - A tale of disaster recovery using CFEngine 3Configuration management 101 - A tale of disaster recovery using CFEngine 3
Configuration management 101 - A tale of disaster recovery using CFEngine 3
 
Configuration management: automating and rationalizing server setup with CFEn...
Configuration management: automating and rationalizing server setup with CFEn...Configuration management: automating and rationalizing server setup with CFEn...
Configuration management: automating and rationalizing server setup with CFEn...
 
Cfengine 2 Overview
Cfengine 2 OverviewCfengine 2 Overview
Cfengine 2 Overview
 
ICALEPCS 2011: Testing Environments using Virtualization
ICALEPCS 2011: Testing Environments using VirtualizationICALEPCS 2011: Testing Environments using Virtualization
ICALEPCS 2011: Testing Environments using Virtualization
 
LOAD BALANCING OF APPLICATIONS USING XEN HYPERVISOR
LOAD BALANCING OF APPLICATIONS  USING XEN HYPERVISORLOAD BALANCING OF APPLICATIONS  USING XEN HYPERVISOR
LOAD BALANCING OF APPLICATIONS USING XEN HYPERVISOR
 
Getting Started With CFEngine - Updated Version
Getting Started With CFEngine - Updated VersionGetting Started With CFEngine - Updated Version
Getting Started With CFEngine - Updated Version
 
Configuration
ConfigurationConfiguration
Configuration
 
A Xen Case Study
A Xen Case StudyA Xen Case Study
A Xen Case Study
 
OpenQrm
OpenQrmOpenQrm
OpenQrm
 
Em library
Em libraryEm library
Em library
 
Private Cloud Academy: Backup and DPM 2010
Private Cloud Academy: Backup and DPM 2010Private Cloud Academy: Backup and DPM 2010
Private Cloud Academy: Backup and DPM 2010
 
Presentation 1 open source tools in continuous integration environment v1.0
Presentation 1   open source tools in continuous integration environment v1.0Presentation 1   open source tools in continuous integration environment v1.0
Presentation 1 open source tools in continuous integration environment v1.0
 
Using openQRM to Manage Virtual Machines
Using openQRM to Manage Virtual MachinesUsing openQRM to Manage Virtual Machines
Using openQRM to Manage Virtual Machines
 
Orchestrated Android-Style System Upgrades for Embedded Linux
Orchestrated Android-Style System Upgrades for Embedded LinuxOrchestrated Android-Style System Upgrades for Embedded Linux
Orchestrated Android-Style System Upgrades for Embedded Linux
 
Orchestrated Android-Style System Upgrades for Embedded Linux
Orchestrated Android-Style System Upgrades for Embedded LinuxOrchestrated Android-Style System Upgrades for Embedded Linux
Orchestrated Android-Style System Upgrades for Embedded Linux
 
OSCamp Kubernetes 2024 | Zero-Touch OS-Infrastruktur für Container und Kubern...
OSCamp Kubernetes 2024 | Zero-Touch OS-Infrastruktur für Container und Kubern...OSCamp Kubernetes 2024 | Zero-Touch OS-Infrastruktur für Container und Kubern...
OSCamp Kubernetes 2024 | Zero-Touch OS-Infrastruktur für Container und Kubern...
 
J+s
J+sJ+s
J+s
 
Os
OsOs
Os
 
Cloud Application Security: Lessons Learned
Cloud Application Security: Lessons LearnedCloud Application Security: Lessons Learned
Cloud Application Security: Lessons Learned
 
Part 2 improving your software development v1.0
Part 2   improving your software development v1.0Part 2   improving your software development v1.0
Part 2 improving your software development v1.0
 

More from Jonathan Clarke

Interfacing infrastructure-as-code with non-expert users
Interfacing infrastructure-as-code with non-expert usersInterfacing infrastructure-as-code with non-expert users
Interfacing infrastructure-as-code with non-expert users
Jonathan Clarke
 
What is new in CFEngine 3.6
What is new in CFEngine 3.6What is new in CFEngine 3.6
What is new in CFEngine 3.6
Jonathan Clarke
 
Automating security policies (compliance) with Rudder
Automating security policies (compliance) with RudderAutomating security policies (compliance) with Rudder
Automating security policies (compliance) with Rudder
Jonathan Clarke
 
OpenLDAP - Astuces pour en faire l'annuaire d'entreprise idéal
OpenLDAP - Astuces pour en faire l'annuaire d'entreprise idéalOpenLDAP - Astuces pour en faire l'annuaire d'entreprise idéal
OpenLDAP - Astuces pour en faire l'annuaire d'entreprise idéal
Jonathan Clarke
 
LDAP Synchronization Connector presentation at LDAPCon 2009
LDAP Synchronization Connector presentation at LDAPCon 2009LDAP Synchronization Connector presentation at LDAPCon 2009
LDAP Synchronization Connector presentation at LDAPCon 2009
Jonathan Clarke
 
LDAP Synchronization Connector (LSC)
LDAP Synchronization Connector (LSC)LDAP Synchronization Connector (LSC)
LDAP Synchronization Connector (LSC)
Jonathan Clarke
 

More from Jonathan Clarke (6)

Interfacing infrastructure-as-code with non-expert users
Interfacing infrastructure-as-code with non-expert usersInterfacing infrastructure-as-code with non-expert users
Interfacing infrastructure-as-code with non-expert users
 
What is new in CFEngine 3.6
What is new in CFEngine 3.6What is new in CFEngine 3.6
What is new in CFEngine 3.6
 
Automating security policies (compliance) with Rudder
Automating security policies (compliance) with RudderAutomating security policies (compliance) with Rudder
Automating security policies (compliance) with Rudder
 
OpenLDAP - Astuces pour en faire l'annuaire d'entreprise idéal
OpenLDAP - Astuces pour en faire l'annuaire d'entreprise idéalOpenLDAP - Astuces pour en faire l'annuaire d'entreprise idéal
OpenLDAP - Astuces pour en faire l'annuaire d'entreprise idéal
 
LDAP Synchronization Connector presentation at LDAPCon 2009
LDAP Synchronization Connector presentation at LDAPCon 2009LDAP Synchronization Connector presentation at LDAPCon 2009
LDAP Synchronization Connector presentation at LDAPCon 2009
 
LDAP Synchronization Connector (LSC)
LDAP Synchronization Connector (LSC)LDAP Synchronization Connector (LSC)
LDAP Synchronization Connector (LSC)
 

Recently uploaded

Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
Neo4j
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
Ajin Abraham
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Neo4j
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
Safe Software
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
Javier Junquera
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Pitangent Analytics & Technology Solutions Pvt. Ltd
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 

Recently uploaded (20)

Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 

A tale of Disaster Recovery (Cfengine everyday, practices and tools)

  • 1. FOSDEM 2011 @Brussels, Belgium A tale of disaster recovery Cfengine everyday, practices and tools Nicolas Charles <nch@normation.com> Jonathan Clarke <jcl@normation.com>    
  • 2. About the speakers Nicolas Charles Jonathan Clarke Cfengine contributor OpenLDAP commiter Cfengine ”Community Champion” (C3) Scala Developer Sysadmin But we get on pretty well! (mostly...)    
  • 3. Agenda 1) Configuration Management 101 2) Our choice of tool 3) A tale of disaster recovery 4) Introducing Cfengine 3 5) Why we love Cfengine 3    
  • 4. A bit about Configuration Management...    
  • 5. Configuration management  What is it ?  Configuration Management is a field of management that focuses on establishing and maintaining consistency of a system (..) throughout its life  Software configuration management is the task of tracking and controlling changes in the software Sources: http://en.wikipedia.org/wiki/Configuration_management http://en.wikipedia.org/wiki/Software_configuration_management    
  • 6. Configuration management  Why is it useful ?  Control changes  Reproduce over time and nodes  Audit and keep history data  Repair automaticaly    
  • 7. Configuration Management Tools What we chose, and why    
  • 8. Our choice  Back in mid 2009  Needed a configuration management tool  Criteria:  Open source  Multi-platform agent (including Windows)  Resilient  Non-disruptive    
  • 9. Our choice: candidates Cfengine 3 Puppet Chef    
  • 10. Our choice: candidates Cfengine 3 More on this choice later...    
  • 11. Disaster Recovery An ill-fated tale from the recent past (CASE STUDY)    
  • 12. Before the disaster...  Our company's IT infrastructure  Small company: small requirements  Web site, email  Git repository, Redmine...  Small company: small budget  All on one hosted server    
  • 13. Asking for trouble?  Just one hosted server! Critical services!  No, a ”safe” configuration:  Redundant hardware, 3 disk RAID-5 array  All services automatically installed and setup using Configuration Management  Backups: daily (several off-site locations)  Several VMs to separate services    
  • 14. A critical failure  2 hard drives fail simultaneously → RAID-5 array is down → Almost all services fail immediately → ”The end of the world as we know it” → Need to rebuild everything NOW    
  • 15. Recovering  Step 1: Panic!  Step 2: Get a new server  Step 3: Reinstall base OS + virtualization  Step 4: Restore VM configuration... whoops  Step 4: Re-create the VMs manually  Step 5: Reinstall each OS in each VM...    
  • 16. Recovering  Step 6: Installation Configuration Management  Step 7: Sit back and watch all the services coming back online as if by magic!  Step 8: Huh, where's my data?  Step 9: Manually restore backups  Step 10: Make a list of missing data...    
  • 17. Lessons learned 1) Hard disks fail reliably 2) Restoring virtualization setups: ● Backing up the config files would have helped ● Need CM tools to describe the desired state! (Cfengine Nova does this) 3) Configuration Management should tie in to our backup system 4) Backups were lacking some files: always test!    
  • 18. Wishlist and discussion  Integrating Configuration Management tools and backup systems is a crucial step for CM to be efficient for disaster recovery  What do others do?  Provisioning VMs and their resources (disks, network) should be automated too  Cloud providers are one solution  What about ”plain” virtualization?    
  • 19. A bit about Cfengine 3... Sources: across the Internet    
  • 21. Cfengine 3: Intro  Configuration management software  Written in C  Two versions :  Community (GPL v3)  Nova (closed source) : Community + extra features  Backed by Cfengine AS – Norway based company founded in 2009    
  • 22. Cfengine 3: Features  According to Kuleven comparative study of configuration management systems:  Very mature  Cross platform (*BSD, AIX, HP-UX, Linux, Mac OS X, Solaris, Windows)  Strongly distributed  Based on state description and convergence  Very high scalabily ( > 10000 nodes )  Very small footprint Source: http://distrinet.cs.kuleuven.be/software/sysconfigtools/overview    
  • 23. Cfengine 3: Components  Cf-agent  Runs on all managed hosts  Applies configuration – this is the heart  Can connect to cf-serverd to get policies / files  Cf-serverd  Distributes policies and files  Must be run on policy server(s)  Usually run on all hosts to enable remote runs  Cf-monitord  Collects statistics on all nodes    
  • 24. Cfengine 3: Promises  Configuration rules are called promises  ”Promise” to be in the desired state  Cfengine agent handles the steps to get there: convergence  Promise theory is based on research done in the University of Oslo    
  • 25. Cfengine 3: Usage examples  Large companies (Facebook, AMD, …)  Critical systems: Joint Australia Tsunami Warning Centre  Personal computers  Mobile devices: Nokia N900  Underwater devices: army submarines  Small and medium companies...    
  • 26. Why we love Cfengine 3... Sources: our experience and opinions    
  • 27. Memory usage  Daemon consumption on managed hosts    
  • 28. Multi-platform  Define a configuration for all operating systems  Windows, Linux  Make it ”transparent” (forget about the complexity)  Existing standard library handling the differences between each OS and distribution    
  • 29. File editing  Only change what you need to  You like your distribution's defaults?  You have various different systems already setup and just need to change something?  Search for lines and replace/delete/add them  Only change one field in a file  /etc/passwd for example...    
  • 30. Complex tasks  Powerful class system to trigger promises  Based on nodes itself  Based on time  Based on whatever you might imagine  Complex workflow can be created    
  • 31. Thank you ! FOSDEM 2011 Configuration Management room And those brave enough to wake up early