Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
THE EVOLVING DESIGN PATTERNS OF PUPPET ENTERPRISE
John Painter Jon Spinks 
Solutions Architect Solutions Architect 
www.linkedin.com/in/painterj www.linkedin.com/in/jonspin...
Sourced Group architect and 
deploy a globally distributed 
multi-master puppet solution 
for a large investment bank 
Feb...
CUSTOMER REQUIREMENTS 
• Provide a single point of control for a globally 
distributed IT environment 
• Reduce overall co...
DESIGN PATTERN 1 
Global Multi Master
CENTRAL MGMT. SYDNEY 
Puppeteer 
LDAP 
ENC 
Puppet 
Master 
Puppet 
Console 
Puppet 
Master 
NEW YORK 
LDAP 
ENC 
Puppet 
...
Follow-the-Sun Code Release 
ASIA: 6pm - 8am Europe: 6pm - 8am USA: 6pm - 8am 
Client Client 
Puppet Master 
Client Client...
AUDITING GOVERNANCE SIMPLIFIED 
CONTROL 
Control and report on the point of truth
KEY TAKEAWAYS 
• LDAP is a great distributed ENC 
• Puppet Reports can be large en masse 
• A Puppet Master can used to ma...
Sourced Group architect and 
deploy a globally distributed 
multi-master puppet solution 
for a large investment bank 
Feb...
Sourced are the first to onboard 
an Asia Pacific 
telecommunications company 
onto AWS, powered by Puppet 
Enterprise 
Fe...
CUSTOMER REQUIREMENTS 
• Provide a single point of control for instance 
provisioning 
• Provide infrastructure orchestrat...
EVERYTHING IS 
PROGRAMATIC 
STANDARDISATION OF 
INFRASTRUCTURE
DESIGN PATTERN 2 
Puppet Managing AWS Compute
Puppet CloudPack 
EC2 only support circa 2012
Public IP Public IP 
Compute Compute 
Compute Compute 
Public IP Public IP 
EC2 
Customer 
IP Whitelisting
Customer Compute Compute 
Virtual Private Cloud! 
Private IP Space 
Compute Compute 
VPN / Direct 
Connect 
Public IP 
VPC
Bootstrap Agent 
Classify Node 
Application Deployed 
Create Instance
KEY TAKEAWAYS 
• Puppet Masters can be used for OS instance 
delivery 
• CloudPack supports EC2 and VPC
Sourced are the first to onboard 
an Asia Pacific 
telecommunications company 
onto AWS, powered by Puppet 
Enterprise 
Fe...
“Using Puppet Enterprise 
Edition as Heterogeneous 
Cloud Glue” 
PuppetConf 2012
CLOUDPACK UPLIFT 
• Uplifted the existing Puppet CloudPack 
to support AWS VPC 
• Left Puppet Enterprise in the middle of ...
“Using Puppet Enterprise 
Edition as Heterogeneous 
Cloud Glue” 
PuppetConf 2012
Sourced implement a 
masterless Puppet Enterprise 
Edition solution for a financial 
services organisation 
March 2013
CUSTOMER REQUIREMENTS 
• Harness transient compute while enforcing 
compliance 
• Provide OS and application state transpa...
STANDARDISATION AND 
CONSOLIDATION
DESIGN PATTERN 3 
Masterless Puppet with AWS Integration
CLOUD COMPUTING IS 
FUNDAMENTALLY DIFFERENT
V1.0 V1.0 
myapp.com
V1.0 V1.0 
myapp.com 
V2.0 V2.0
myapp.com 
V1.0 V1.0 V2.0 V2.0
WEBAPP V. 2 
TEST 
BAKE 
LAUNCH 
TEARDOWN 
WEBAPP V. 1 
TEST 
BAKE 
LAUNCH 
TEARDOWN 
WEBAPP DEV -V. 1 
TEST 
BAKE 
LAUNCH...
UAT 
Production 
APP V1.1 
APP V1.0 
V1.1 
Features 
V2.0 
V1.1 
V2.0
Infrastructure, deployment, and configuration 
have standardised 
! 
It was not just servers that were transient, it was 
...
“18 months really sucks” 
–NIGEL KERSTEN - PUPPETCAMP SYDNEY 2014
“18 months really sucks” 
- EVERY SINGLE PRODUCT MANAGER. EVER.
Weekly/Monthly -> Hourly
Load Time 
$ 
$ 
$ $ 
$ 
$
PROBLEM SPACE 
• Puppet Masters were designed to manage long lived 
OS instances 
• Transient compute doesn’t really need ...
MASTERLESS COMPONENTS 
• CloudFormation for orchestration 
• AWS EC2 Instance Tags provide node definition 
• Facter is ex...
Version Control 
CI/CD 
AWS S3 
Instance Instance Instance 
Instance 
Centralised Logging
Configuration Managent 
Application Configuration 
Governance and Compliance
Configuration Managent 
Application Configuration 
Governance and Compliance
KEY TAKEAWAYS 
• Cloud Computing is fundamentally different 
• Harnessing transient compute is extremely 
valuable 
• Losi...
DESIGN PATTERN 4 
Puppet Master and Transient Compute
DATA BUILDS 
TRANSPARENCY 
Getting the Puppet Master Back in the Loop
AWS COMPUTE TYPES 
Transient 
Compute 
Stateful! 
Compute
PUPPET AWS AUTOSCALING BROKER
Autoscale BROKER Simple Queue Service (SQS) 
Instance Instance 
SNS Topic 
Simple Notification Service (SNS) 
Autoscaling ...
KEY TAKEAWAYS 
• Using a Puppet Master as a point of control for all 
compute builds transparency 
• The Puppet Master fra...
A LOOK FORWARD 
Heterogeneous Puppet Governance
Application 
Runtime 
Operating System 
Hardware
Application 
Runtime 
Operating System 
Hardware
Rather than making the OS a good candidate for 
the application, we are now tailoring the 
applications to be a good citiz...
Pivotal CF 
Docker 
Azure Pack 
OpenShift 
Examples of higher-level application delivery frameworks that can run on 
AWS
THE FRAMEWORKS STILL 
NEED TO BE GOVERNED
HYBRID CLOUD 
On Premise Managed 
Stateful 
Compute 
Transient 
Compute
GOVERNANCE 
GOVERNANCE 
GOVERNANCE 
GOVERNANCE 
GOVERNANCE 
GOVERNANCE
In an ideal world with true vendor arbitrage we 
would be able to deliver an application to AWS, 
OpenShift on Rackspace a...
Puppet + Autoscale Broker + Global Distribution + Multi Cloud 
APP APP APP APP 
AWS - Sydney 
Puppet Master 
APP DELIVERY ...
KEY TAKEAWAYS 
• Through the adoption of PaaS the bar of 
commoditisation continues to rise 
• Governance and compliance a...
SOURCEDGROUP.COM
Upcoming SlideShare
Loading in …5
×

The Evolving Design Patterns of Puppet Enterprise - PuppetConf 2014

1,683 views

Published on

The Evolving Design Patterns of Puppet Enterprise - Jonathan Spinks, Sourced Group & John Painter, Sourced Group

Published in: Technology
  • Be the first to comment

The Evolving Design Patterns of Puppet Enterprise - PuppetConf 2014

  1. 1. THE EVOLVING DESIGN PATTERNS OF PUPPET ENTERPRISE
  2. 2. John Painter Jon Spinks Solutions Architect Solutions Architect www.linkedin.com/in/painterj www.linkedin.com/in/jonspinks sourcedgroup.com
  3. 3. Sourced Group architect and deploy a globally distributed multi-master puppet solution for a large investment bank February 2011
  4. 4. CUSTOMER REQUIREMENTS • Provide a single point of control for a globally distributed IT environment • Reduce overall configuration drift in the existing fleet • Provide a change control process to manage code updates
  5. 5. DESIGN PATTERN 1 Global Multi Master
  6. 6. CENTRAL MGMT. SYDNEY Puppeteer LDAP ENC Puppet Master Puppet Console Puppet Master NEW YORK LDAP ENC Puppet Master Puppet Console Puppet Master LDAP ENC Master HONG KONG LDAP ENC Puppet Master Puppet Console Puppet Master LONDON LDAP ENC Puppet Master Puppet Console Puppet Master
  7. 7. Follow-the-Sun Code Release ASIA: 6pm - 8am Europe: 6pm - 8am USA: 6pm - 8am Client Client Puppet Master Client Client Puppet Master Client Client Puppet Master Global risk management, simplified releases
  8. 8. AUDITING GOVERNANCE SIMPLIFIED CONTROL Control and report on the point of truth
  9. 9. KEY TAKEAWAYS • LDAP is a great distributed ENC • Puppet Reports can be large en masse • A Puppet Master can used to manage other Puppet Masters • Version control is critical as a basis for a change control process
  10. 10. Sourced Group architect and deploy a globally distributed multi-master puppet solution for a large investment bank February 2011
  11. 11. Sourced are the first to onboard an Asia Pacific telecommunications company onto AWS, powered by Puppet Enterprise February 2012
  12. 12. CUSTOMER REQUIREMENTS • Provide a single point of control for instance provisioning • Provide infrastructure orchestration • Maintain portability of applications to the traditional data centre
  13. 13. EVERYTHING IS PROGRAMATIC STANDARDISATION OF INFRASTRUCTURE
  14. 14. DESIGN PATTERN 2 Puppet Managing AWS Compute
  15. 15. Puppet CloudPack EC2 only support circa 2012
  16. 16. Public IP Public IP Compute Compute Compute Compute Public IP Public IP EC2 Customer IP Whitelisting
  17. 17. Customer Compute Compute Virtual Private Cloud! Private IP Space Compute Compute VPN / Direct Connect Public IP VPC
  18. 18. Bootstrap Agent Classify Node Application Deployed Create Instance
  19. 19. KEY TAKEAWAYS • Puppet Masters can be used for OS instance delivery • CloudPack supports EC2 and VPC
  20. 20. Sourced are the first to onboard an Asia Pacific telecommunications company onto AWS, powered by Puppet Enterprise February 2012
  21. 21. “Using Puppet Enterprise Edition as Heterogeneous Cloud Glue” PuppetConf 2012
  22. 22. CLOUDPACK UPLIFT • Uplifted the existing Puppet CloudPack to support AWS VPC • Left Puppet Enterprise in the middle of the provisioning and lifecycle management • Supported Multi-cloud provisioning • http://www.youtube.com/watch? v=mwiBjJZWraA
  23. 23. “Using Puppet Enterprise Edition as Heterogeneous Cloud Glue” PuppetConf 2012
  24. 24. Sourced implement a masterless Puppet Enterprise Edition solution for a financial services organisation March 2013
  25. 25. CUSTOMER REQUIREMENTS • Harness transient compute while enforcing compliance • Provide OS and application state transparency
  26. 26. STANDARDISATION AND CONSOLIDATION
  27. 27. DESIGN PATTERN 3 Masterless Puppet with AWS Integration
  28. 28. CLOUD COMPUTING IS FUNDAMENTALLY DIFFERENT
  29. 29. V1.0 V1.0 myapp.com
  30. 30. V1.0 V1.0 myapp.com V2.0 V2.0
  31. 31. myapp.com V1.0 V1.0 V2.0 V2.0
  32. 32. WEBAPP V. 2 TEST BAKE LAUNCH TEARDOWN WEBAPP V. 1 TEST BAKE LAUNCH TEARDOWN WEBAPP DEV -V. 1 TEST BAKE LAUNCH TEARDOWN UAT Production
  33. 33. UAT Production APP V1.1 APP V1.0 V1.1 Features V2.0 V1.1 V2.0
  34. 34. Infrastructure, deployment, and configuration have standardised ! It was not just servers that were transient, it was entire application stacks ! Cloud computing has significantly changed the application lifecycle
  35. 35. “18 months really sucks” –NIGEL KERSTEN - PUPPETCAMP SYDNEY 2014
  36. 36. “18 months really sucks” - EVERY SINGLE PRODUCT MANAGER. EVER.
  37. 37. Weekly/Monthly -> Hourly
  38. 38. Load Time $ $ $ $ $ $
  39. 39. PROBLEM SPACE • Puppet Masters were designed to manage long lived OS instances • Transient compute doesn’t really need a dashboard • Puppet Console accuracy was impacted by stale instance data • Requirement to simplify the Auto Scale process
  40. 40. MASTERLESS COMPONENTS • CloudFormation for orchestration • AWS EC2 Instance Tags provide node definition • Facter is extended to understand AWS Instance Metadata • Manifests and modules are bundled and centrally managed • Centralised logging to provide state data
  41. 41. Version Control CI/CD AWS S3 Instance Instance Instance Instance Centralised Logging
  42. 42. Configuration Managent Application Configuration Governance and Compliance
  43. 43. Configuration Managent Application Configuration Governance and Compliance
  44. 44. KEY TAKEAWAYS • Cloud Computing is fundamentally different • Harnessing transient compute is extremely valuable • Losing the Puppet Master reduces overall visibility when considering long lived and transient instances as a whole
  45. 45. DESIGN PATTERN 4 Puppet Master and Transient Compute
  46. 46. DATA BUILDS TRANSPARENCY Getting the Puppet Master Back in the Loop
  47. 47. AWS COMPUTE TYPES Transient Compute Stateful! Compute
  48. 48. PUPPET AWS AUTOSCALING BROKER
  49. 49. Autoscale BROKER Simple Queue Service (SQS) Instance Instance SNS Topic Simple Notification Service (SNS) Autoscaling Group Instance Instance
  50. 50. KEY TAKEAWAYS • Using a Puppet Master as a point of control for all compute builds transparency • The Puppet Master framework is very extensible • PuppetDB is very valuable • Leverage AWS Auto Scale orchestration features
  51. 51. A LOOK FORWARD Heterogeneous Puppet Governance
  52. 52. Application Runtime Operating System Hardware
  53. 53. Application Runtime Operating System Hardware
  54. 54. Rather than making the OS a good candidate for the application, we are now tailoring the applications to be a good citizen of a standardised compute platform.
  55. 55. Pivotal CF Docker Azure Pack OpenShift Examples of higher-level application delivery frameworks that can run on AWS
  56. 56. THE FRAMEWORKS STILL NEED TO BE GOVERNED
  57. 57. HYBRID CLOUD On Premise Managed Stateful Compute Transient Compute
  58. 58. GOVERNANCE GOVERNANCE GOVERNANCE GOVERNANCE GOVERNANCE GOVERNANCE
  59. 59. In an ideal world with true vendor arbitrage we would be able to deliver an application to AWS, OpenShift on Rackspace and Docker running on managed VMware using only CI/CD tooling
  60. 60. Puppet + Autoscale Broker + Global Distribution + Multi Cloud APP APP APP APP AWS - Sydney Puppet Master APP DELIVERY FRAMEWORK APP DELIVERY FRAMEWORK APP DELIVERY FRAMEWORK AWS - USA Puppet Master APP DELIVERY FRAMEWORK APP DELIVERY FRAMEWORK APP DELIVERY FRAMEWORK <CLOUD> - Singapore Puppet Master APP DELIVERY FRAMEWORK APP DELIVERY FRAMEWORK APP DELIVERY FRAMEWORK Runtime Your Datacenter Puppet Master APP DELIVERY FRAMEWORK APP DELIVERY FRAMEWORK APP DELIVERY FRAMEWORK
  61. 61. KEY TAKEAWAYS • Through the adoption of PaaS the bar of commoditisation continues to rise • Governance and compliance are still critical • Application delivery performance continues to increase
  62. 62. SOURCEDGROUP.COM

×