Your SlideShare is downloading. ×
Designing Puppet: Roles/Profiles Pattern
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Designing Puppet: Roles/Profiles Pattern

29,924
views

Published on

"Designing Puppet: Roles/Profiles Pattern" by Craig Dunn of Puppet Labs, at Puppet Camp Stockholm 2013.

"Designing Puppet: Roles/Profiles Pattern" by Craig Dunn of Puppet Labs, at Puppet Camp Stockholm 2013.

Published in: Technology

0 Comments
54 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
29,924
On Slideshare
0
From Embeds
0
Number of Embeds
18
Actions
Shares
0
Downloads
402
Comments
0
Likes
54
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Designing Puppet: Roles / Profiles Design Pattern Puppet Camp Stockholm, Feb 2013Thursday, 7 February 13
  • 2. Hello • Craig Dunn • Puppet user since 2008 as an IT contractor • Started with 0.24 • Joined Puppet Labs in June 2012 • @crayfishX • Freenode IRC: crayfishxThursday, 7 February 13
  • 3. Agenda • How people typically design Puppet • Real-world case study • Thinking about components • Designing Puppet for your users • Node classification • Data separationThursday, 7 February 13
  • 4. Background • Originally a blog post written in May 2012 • Advocated by many Puppet Labs Engineers • Based on a real world solution • Several community members have adopted with successThursday, 7 February 13
  • 5. Designing Puppet • You write awesome modules • You classify them to your nodeThursday, 7 February 13
  • 6. Designing Puppet Node Classification ModulesThursday, 7 February 13
  • 7. Down the road... • Your infrastructure grows • Business requirements will change • Your Puppet code feels bulky and high maintenence • There will always be edge cases eventually • You decide it needs refactoringThursday, 7 February 13
  • 8. Danger Signs • Resources being declared in two modules • You don’t know where your implementation “fits” • Lot’s of logic at a node level • Repetition and duplication • The if statement is your go-to-guyThursday, 7 February 13
  • 9. Write good modules • Should manage only it’s own resources • Should be granular • Should be portableThursday, 7 February 13
  • 10. Thinking beyond the module.... • Puppet is a code base • How do I design an effective framework • Gluing everything togetherThursday, 7 February 13
  • 11. Node-level logic • Risks duplication and repetition • No guarantee of consistency • TMI!Thursday, 7 February 13
  • 12. Node-level logic node basil { class { ‘apache’: version => ‘latest’, } class { ‘motd’: } class { ‘ssh’: } class { ‘users’: default_shell => ‘/bin/false’, } Class[‘ssh’] -> Class[‘users’] }Thursday, 7 February 13
  • 13. Node-level logic • What happens when I have 1000 nodes • Or 10,000 nodes!! • That’s a lot of code! • So where should implement this?Thursday, 7 February 13
  • 14. Designing Puppet • Provide business logic to classification • Provide an abstraction layer for implementation of components • Make code adaptable to complex requirements • Reduce node-level logic • Reduce functionality overlapThursday, 7 February 13
  • 15. What is the worse thing that is going to happen to your Puppet code?Thursday, 7 February 13
  • 16. Business requirementsThursday, 7 February 13
  • 17. Business logic does not align with technologyThursday, 7 February 13
  • 18. Case study • Real world problem • Solved through designThursday, 7 February 13
  • 19. Case study “We have 3 applications we need to deploy using Puppet”Thursday, 7 February 13
  • 20. The business view Application X Application Y Application ZThursday, 7 February 13
  • 21. Go forth and Puppetize!Thursday, 7 February 13
  • 22. Go forth and Puppetize! And we jumped right in...Thursday, 7 February 13
  • 23. Things got painfulThursday, 7 February 13
  • 24. Problems • These applications aren’t that different • They seem to share a whole bunch of similarities • Implementation differed on different environments and locations • Writing 3 separate modules creates conflicts and duplicationThursday, 7 February 13
  • 25. Our code was hackyThursday, 7 February 13
  • 26. We are trying to code business logic.Thursday, 7 February 13
  • 27. Stop thinking about what it looks like • Break everything down into components • Granularity is the key • Think about what it actually isThursday, 7 February 13
  • 28. What we realised • Each application stack is a collection of a subset of the same Java apps implemented in different waysThursday, 7 February 13
  • 29. The business view Application X Application Y Application ZThursday, 7 February 13
  • 30. The technical reality Application X ApplicationApplication Z YThursday, 7 February 13
  • 31. We only have one application Implemented many different waysThursday, 7 February 13
  • 32. So we had an idea! • Reduce each Java sub application into granular Puppet modules • Create a code layer responsible for implementation • Let’s call them profilesThursday, 7 February 13
  • 33. class profiles::x { include tomcat include mysql include componenta include componentb componentb::resource { ‘name’: ensure => present, } } class profiles::y { include tomcat include mysql include componenta include componentc include componentd } class profiles::z { include tomcat include mysql include componenta include componentb include componentd include dependancy Class[‘dependancy’] -> Class[‘componentd’] }Thursday, 7 February 13
  • 34. Use inheritance for abstraction within profiles class profiles::application { include tomcat include mysql include componenta } class profiles::application::x inherits profiles::application { include componentb componentb::resource { ‘name’: ensure => present, } } class profiles::application::y inherits profiles::application { include componentc include componentd } class profiles::application::z inherits profiles::application { include componentb include componentd include dependancy Class[‘dependancy’] -> Class[‘componentd’] }Thursday, 7 February 13
  • 35. Profiles and Components ResourcesThursday, 7 February 13
  • 36. Profiles and Components Components: Resource modelling ResourcesThursday, 7 February 13
  • 37. Profiles and Components Profiles : Implementation Components: Resource modelling ResourcesThursday, 7 February 13
  • 38. In reality it was worseThursday, 7 February 13
  • 39. In reality it was worse • 2 different deployment types made up of over 15 server types eachThursday, 7 February 13
  • 40. In reality it was worse • 2 different deployment types made up of over 15 server types each • 10+ locationsThursday, 7 February 13
  • 41. In reality it was worse • 2 different deployment types made up of over 15 server types each • 10+ locations • 4 environment typesThursday, 7 February 13
  • 42. In reality it was worse • 2 different deployment types made up of over 15 server types each • 10+ locations • 4 environment types • Every installation was an edge case!Thursday, 7 February 13
  • 43. In reality it was worse • 2 different deployment types made up of over 15 server types each • 10+ locations • 4 environment types • Every installation was an edge case! • My slides weren’t big enough.Thursday, 7 February 13
  • 44. Lessons learned • Granularity is good • Don’t assume business logic will directly translate to technology • Abstraction is awesome.... but that’s nothing new....Thursday, 7 February 13
  • 45. Abstraction is a core principle of coding • Functions are abstracted by methods • Methods abstracted by classes and modules • They are abstracted with libraries • Puppet is code!Thursday, 7 February 13
  • 46. Puppet is all about abstraction • Data is abstracted by Hiera • Providers are abstracted by types • Resources are abstracted by classes • Classes are abstracted by modulesThursday, 7 February 13
  • 47. Puppet is all about abstraction • Data is abstracted by Hiera • Providers are abstracted by types • Resources are abstracted by classes • Classes are abstracted by modules • Modules are abstracted by profilesThursday, 7 February 13
  • 48. Focussing on Abstraction • We’ve turned business logic into a technology stack • Can we translate that back into business logic? • Why would we even want to do that?Thursday, 7 February 13
  • 49. UAT Cluster node Our example configuration model: include security include users include ntp include ssh::server include customapp include tomcat::server class { ‘jenkins’: require => Class[‘tomcat::server’], } include mysql database { ‘apptest’: ensure => present, }Thursday, 7 February 13
  • 50. Think about the users Meet John, Susan and Bill.Thursday, 7 February 13
  • 51. John is a Sysadmin • Wants to ensure all servers have kernel hardening, NTP and SSH Server installed • Wants to manage what packages, services, files and other resources • Is responsible for maintaining all the components of a UAT cluster serverThursday, 7 February 13
  • 52. Susan is an application specialist • Cares that a UAT Cluster node requires MySQL Server, Tomcat Server and Jenkins server installed.Thursday, 7 February 13
  • 53. Bill is an IT manager • Bill cares that the server is a UAT Cluster nodeThursday, 7 February 13
  • 54. What do they care about? • John cares about modelling all resources • Susan cares about the technology stack • Bill cares about the business logicThursday, 7 February 13
  • 55. In Puppet • Resource modelling is done in component modules • The technology stack is defined in profiles • Where do we represent the business logic for Bill?Thursday, 7 February 13
  • 56. Introducing Roles • Represent business logic, not technology • Define a set of technology stacks (profiles) that make up the logical role • Allow the business to manage how the infrastructure looks without defining what it isThursday, 7 February 13
  • 57. A node can only have one role • A role can include as many profiles as required to define itself • If a node requires two roles, it has by definition become a new roleThursday, 7 February 13
  • 58. A node can only have one role • A role can include as many profiles as required to define itself • If a node requires two roles, it has by definition become a new role • Something couldn’t be a lion and a kangaroo at the same time!Thursday, 7 February 13
  • 59. It would be a LingarooThursday, 7 February 13
  • 60. Roles • One-to-one to nodes • One-to-many to profiles • Only implement profilesThursday, 7 February 13
  • 61. Example role class role::uat_server { include profiles::base include profiles::customapp include profiles::test_tools }Thursday, 7 February 13
  • 62. Classification • Node classification simply assigns roles to nodes • Roles expose profilesThursday, 7 February 13
  • 63. Classification node ‘craig.puppetlabs.vm’ { include roles::uat_server }Thursday, 7 February 13
  • 64. ClassificationThursday, 7 February 13
  • 65. The Stack ResourcesThursday, 7 February 13
  • 66. The Stack Components: Resource modelling ResourcesThursday, 7 February 13
  • 67. The Stack Profiles : Implementation Components: Resource modelling ResourcesThursday, 7 February 13
  • 68. The Stack Roles : Business Logic Profiles : Implementation Components: Resource modelling ResourcesThursday, 7 February 13
  • 69. Terminology • Profiles and Roles are Puppet modules • Components are Puppet modules responsible for modelling resources • Everything is a moduleThursday, 7 February 13
  • 70. Naming conventions • Components should be named after what they manage (apache, ssh, mysql) • Profiles should be named after the logical stack they implement (database, bastion, email) • Roles should be named in business logic convention (uat_server, web_cluster, application, archive)Thursday, 7 February 13
  • 71. Hiera Overview Let’s talk about data!Thursday, 7 February 13
  • 72. Managing infrastructure DevThursday, 7 February 13
  • 73. Managing infrastructure Dev QAThursday, 7 February 13
  • 74. Managing infrastructure Dev QA ProductionThursday, 7 February 13
  • 75. Managing infrastructure Dev QA DC1 ProductionThursday, 7 February 13
  • 76. Managing infrastructure Dev QA DC1 DC2 DC3 ProductionThursday, 7 February 13
  • 77. Managing data in Puppet is hard.Thursday, 7 February 13
  • 78. Without Hiera? if ( $::environment == ‘dev’ ) { $ntpserver = ‘192.168.2.1’ } else { if ( $::fqdn == ‘host4.mycorp.com’) { $ntpserver = ‘127.0.0.1’ } else { $ntpserver = ‘213.21.6.4’ } }Thursday, 7 February 13
  • 79. With Hiera? $ntpserver = hiera(‘ntpserver’)Thursday, 7 February 13
  • 80. Hierarchical lookups • Hiera uses facter facts to determine a hierarchy • Top down hierarchy for overriding configuration values based on roles, environments, locations.... or anything else • And do this without any coding!Thursday, 7 February 13
  • 81. Separation of data from code • Puppet modules without hard-coded data are easily shared and more re-usable • Infrastructure configuration can be managed without needing to edit Puppet codeThursday, 7 February 13
  • 82. Pluggable Backends • Source data from multiple locations • Data source is abstracted from codeThursday, 7 February 13
  • 83. Pluggable Backends • Source data from multiple locations • Data source is abstracted from code • hiera-gpg • hiera-redis • hiera-http • hiera-json • hiera-mysql • hiera-zookeeperThursday, 7 February 13
  • 84. Data Separation • Use Hiera to abstract your data from your code • Components and profiles can source data from HieraThursday, 7 February 13
  • 85. Profiles and Hiera • Use Hiera to model your data • Use profiles to model your implementationThursday, 7 February 13
  • 86. The Stack Roles : Business Logic Profiles : Implementation Components: Resource modelling ResourcesThursday, 7 February 13
  • 87. The Stack Roles : Business Logic Profiles : Implementation Hiera: Data Components: Resource modelling ResourcesThursday, 7 February 13
  • 88. Classification • Assigning classes to a node • You can classify within Puppet code (site.pp) • You can use an External Node Classifier (ENC)Thursday, 7 February 13
  • 89. Leveraging an ENC • You can classify your nodes however you want • Puppet Dashboard • Enterprise Console • Foreman • Site.pp • Custom scriptThursday, 7 February 13
  • 90. Leveraging an ENC • An ENC should classify a node to it’s role • Nothing elseThursday, 7 February 13
  • 91. The Stack Roles : Business Logic Profiles : Implementation Hiera: Data Components: Resource modelling ResourcesThursday, 7 February 13
  • 92. The Stack Roles : Business Logic Classifier Profiles : Implementation Hiera: Data Components: Resource modelling ResourcesThursday, 7 February 13
  • 93. Key benefits • Reduced node-level logic to a role. • Gain the ability to be flexible with implementation • Business logic improves managability by non-Puppet users • Edge cases are now easy to solveThursday, 7 February 13
  • 94. Enough Preaching!Thursday, 7 February 13
  • 95. This is not the way to design Puppet... It’s a way.Thursday, 7 February 13
  • 96. Can I implement this design without roles?Thursday, 7 February 13
  • 97. Can I implement this design without roles? • Yes. • You lose the layer of abstraction that exposes business logicThursday, 7 February 13
  • 98. Can my roles be defined in my ENC?Thursday, 7 February 13
  • 99. Can my roles be defined in my ENC? • Yes. • Keeping it in code makes it versionableThursday, 7 February 13
  • 100. Can’t I just use Hiera to define profiles?Thursday, 7 February 13
  • 101. Can’t I just use Hiera to define profiles? • Technically yes. • You lose the flexibility to implement code logic in profiles and it may become restrictive • You could possibly use: https://github.com/ ripienaar/hiera-puppet-nodesThursday, 7 February 13
  • 102. The fundamental concepts....Thursday, 7 February 13
  • 103. The fundamental concepts.... • Abstraction, abstraction, abstractionThursday, 7 February 13
  • 104. The fundamental concepts.... • Abstraction, abstraction, abstraction • Decoupling business logic, implementation and resource modelling.Thursday, 7 February 13
  • 105. The fundamental concepts.... • Abstraction, abstraction, abstraction • Decoupling business logic, implementation and resource modelling. • Separating data and codeThursday, 7 February 13
  • 106. The fundamental concepts.... • Abstraction, abstraction, abstraction • Decoupling business logic, implementation and resource modelling. • Separating data and code • Reducing node-level complexityThursday, 7 February 13
  • 107. Other Resources • Adrien Thebos’ excellent blog post http:// sysadvent.blogspot.co.uk/2012/12/day-13-configuration-management-as- legos.html • My original blog post 2012/05/239/ http://www.craigdunn.org/ • Module Structure Redux by R.I.Pienaar http:// www.devco.net/archives/2012/12/13/simple-puppet-module-structure- redux.phpThursday, 7 February 13
  • 108. Thank you. Questions? • Follow me at @crayfishX • Bug me on Freenode: crayfishx Enjoy the rest of Puppet Camp! In memory of Giles Constant, who spent many nights debating Puppet design patterns with me over copious amounts of beer and helped me on my journey of discovery learning how to implement Puppet properly. R.I.PThursday, 7 February 13

×