Automating it management with Puppet + ServiceNow
•Download as PPTX, PDF•
0 likes•565 views
As the leading IT Service Management and IT Operations Management platform in the marketplace, ServiceNow is used by many organizations to address everything from self service IT requests to Change, Incident and Problem Management. The strength of the platform is in the workflows and processes that are built around the shared data model, represented in the CMDB. This provides the ‘single source of truth’ for the organization. Puppet Enterprise is a leading automation platform focused on the IT Configuration Management and Compliance space. Puppet Enterprise has a unique perspective on the state of systems being managed, constantly being updated and kept accurate as part of the regular Puppet operation. Puppet Enterprise is the automation engine ensuring that the environment stays consistent and in compliance. In this webinar, we will explore how to maximize the value of both solutions, with Puppet Enterprise automating the actions required to drive a change, and ServiceNow governing the process around that change, from definition to approval. We will introduce and demonstrate several published integration points between the two solutions, in the areas of Self-Service Infrastructure, Enriched Change Management and Automated Incident Registration.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Automating it management with Puppet + ServiceNow
Similar to Automating it management with Puppet + ServiceNow (20)
More from Puppet
More from Puppet (20)
Recently uploaded
Recently uploaded (20)
Automating it management with Puppet + ServiceNow
- 1. Automating IT Management with Puppet + ServiceNow Josef Singer Principal Systems Engineer - Puppet NorthEast
- 2. Housekeeping ● Please submit questions in the Q&A chat box. We will address as many as we have time for at the end of the webinar. ● Technical difficulties? Let us know via the Q&A chat and we can help. ● This webinar will be recorded and shared in the next few days via email.
- 4. The right approach for your challenges. Task, model-based and event-driven automation using agentless and agent-based technologies Puppet Enterprise Platform Intelligence Automation Orchestration ModelTask Event Forge Certified Content Community Content Custom Modules & Tasks 3rd Party Custom Content Integrations Automation
- 5. Intelligence Automation Orchestration Seamlessly integration into core IT systems. Extend automation across IT via APIs and/or custom user interfaces. API Collaboration Continuous Delivery and Release Automation Data Aggregation & Monitoring Security Cloud Provisioning Service Management Puppet Enterprise Platform Custom Modules & Tasks 3rd Party Custom Content Forge Certified Content Community Content
- 6. Platform & Infrastructure Teams Config Management, Compliance & Impact Analysis VP I&O IT Ops, Audit & InfoSec Teams Remediation, Patch Management & Audit CISO Application Development Teams VP of Apps Application Provisioning & Orchestration for DevOps IT Ops Teams & Cloud Ops Self-service Automation & Infrastructure Provisioning Puppet Enterprise Platform Teams and Uses Custom Modules & Tasks Forge Intelligence Orchestration Automation Integrations
- 7. 7 Provide standardized and consistent physical and virtual infrastructure, resulting in fewer security and compliance issues Risk mitigation Enable faster deployment and configuration of infrastructure in response to changing stakeholder demands Agility, Innovation & productivity gains Drive efficient configuration management and provide a flexible framework for delivering and managing infrastructure Cost-efficiencies The Value of Puppet Enterprise Platform Puppet makes infrastructure actionable, scalable and intelligent
- 9. ServiceNow is the smarter way to workflow™ Widely used by IT organizations to manage CMDB, change requests, ticketing and self-service.
- 10. ServiceNow provides a shared data model for ITSM Replace silos of disconnected tools & databases with a central, integrated & connected system
- 11. ServiceNow ITSM consists of 5 components Matching to their respective processes as described in ITIL Incident Management Problem Management Change Management Request Management (Service Catalog) CMDB
- 12. ServiceNow and Puppet are often used in parallel Catering to different, but adjacent, aspects of IT operations
- 13. Now, you can finally connect them together Enable bi-directional data sharing between ServiceNow and Puppet Enterprise
- 14. Puppet integrates with ServiceNow in 4 areas Let developers control aspects of their own systems without sacrificing compliance, security, or operational predictability. Self-Service Infrastructure Reduce the risk of change by enriching change requests with impact analysis details and letting ServiceNow control approvals of Puppet changes. Enriched Change Management Reduce the time and effort required to maintain an accurate drift remediation log. Automated Incident Registration Get accurate, up-to-date information about your CMDB assets in ServiceNow, without having to perform frequent discovery runs. Up-to-date Asset Management
- 16. Self-Service Infrastructure Reduce the risk of change by enriching change requests with impact analysis details and letting ServiceNow control approvals of Puppet changes Enriched Change Management Reduce the time and effort required to maintain an accurate drift remediation log Automated Incident Registration Get accurate, up-to-date information about your CMDB assets in ServiceNow, without having to perform frequent discovery runs Up-to-date Asset Management Let developers control aspects of their own systems without sacrificing compliance, security, or operational predictability Self-Service Infrastructure
- 17. Integration: Self-Service Infrastructure WHY ● Teaching the entire company to use Puppet for making changes is unrealistic. ● In order for everyone to easily leverage Puppet automation, a better way to interact with Puppet is needed. ?
- 18. Integration: Self-Service Infrastructure WHAT ● Let teams control their own systems without needing any Puppet skills. ● Expose control of specific aspects of Puppet automation directly from the ServiceNow user interface. ● Leverage ServiceNow workflows to streamline common changes.
- 19. Integration: Self-Service Infrastructure HOW ● Puppet reads the fields for a system from a ServiceNow table of choice, and provides the information as facts for that node. ● Use custom fields in ServiceNow to automate any use case, by writing Puppet logic that uses this data. ● Full node classification is possible as well, for even higher levels of flexibility. { "authenticated" : "remote", "certname" : "server1.puppet.com", "domain" : "puppet.com", "extensions" : { }, "external" : { "servicenow" : { "category" : "Hardware", "classification" : "Production", "name" : "server1.puppet.com", "os" : “CentOS”, "os_version" : "7.7.1908", "puppet_classes" : { "role::dbserver" : { } }, "puppet_environment" : "production", "sys_class_name" : "Server", "u_enforced_packages" : "{"openssl":"present", "redis":"absent"}", } } } trusted
- 20. • Being able to do something like adding a few extra packages through ServiceNow, is a game-changer for us. It lets us expose the power and flexibility of Puppet to users who do not have Puppet expertise • Having the ServiceNow CMDB data directly available as facts is super useful for when we need to drive Puppet behavior based on details that are stored in ServiceNow “This is going to make our lives so much easier” Cloud Infrastructure Engineer at a major bank in London, UK
- 21. Exposing ServiceNow CI data as Trusted Facts Data is freshly retrieved from ServiceNow before each Puppet agent run Configuration Item, with extra fields added as needed: - Name: srv1.company.com - Manufacturer: Amazon EC2 - Enforced Packages: { nano: present, vim: present } PE enforces state with info from ServiceNow srv1.company.com Package[nano] => present Package[vim] => present Puppet Enterprise PE retrieves State info for node
- 22. Using ServiceNow as a Node Classifier for Puppet Fully control Puppet code assignments from ServiceNow, including parameters for classes Configuration Item, with extra fields added as needed: - Name: srv1.company.com - Manufacturer: Amazon EC2 - Puppet Environment: production - Puppet Classes: role::dbserver{} PE enforces state based on classification from ServiceNow srv1.company.com environment: production classes => role::dbserver{} Puppet Enterprise PE retrieves classification info for node
- 23. Demo
- 25. Enriched Change Management Let developers control aspects of their own systems without sacrificing compliance, security, or operational predictability. Self-Service Infrastructure Reduce the time and effort required to maintain an accurate drift remediation log. Automated Incident Registration Get accurate, up-to-date information about your CMDB assets in ServiceNow, without having to perform frequent discovery runs. Up-to-date Asset Management Reduce the risk of change by enriching change requests with impact analysis details and letting ServiceNow control approvals of Puppet changes. Enriched Change Management
- 26. Integration: Enriched Change Management WHY ● When you have: ○ Puppet for change execution ○ ServiceNow for change workflow connecting the two is the obvious choice to get the most out of DevOps. ● Reduce manual effort and ensure full registration of the change impact. DevOps
- 27. Integration: Enriched Change Management WHAT ● Automatically generate ServiceNow change requests for proposed Puppet code changes ● Automatically populate change requests with details from Puppet’s Impact Analysis result. ● Automatically deploy changes when the change request is approved.
- 28. Integration: Enriched Change Management HOW ● Integrates CD4PE with ServiceNow ● Interacts with the ServiceNow Change Management API to create change requests, associate affected systems and populate relevant details. ● Comes with a Business Rule for ServiceNow to orchestrate the automated deployment of approved Puppet changes.
- 29. • It can be challenging to know exactly what the impact of a proposed change will be to the larger environment. Will it affect multiple applications? Multiple systems? • ServiceNow provides the workflow process around Change Management, while Puppet with CD4PE automates the implementation of the change. “Using Impact Analysis to de- risk the Change approvals process and completely changes the way we work” Director of Cloud Architecture at a major health insurance provider in the U.S.
- 30. Automated change requests from CD4PE Delegate control to ServiceNow for approving production changes Change Request: - Name: CHG0030023 - Risk and Impact: <Impact Analysis info> - Affected CIs: srv3.company.com - srv5.company.com CD4PE creates Change Request and populates info from Impact Analysis Admin proposes Puppet code change, triggering CD4PE Git CD4PE Approval workflow Upon approval, ServiceNow interacts with CD4PE to deploy the change
- 31. Demo
- 33. Automated Incident Registration Let developers control aspects of their own systems without sacrificing compliance, security, or operational predictability. Self-Service Infrastructure Reduce the risk of change by enriching change requests with impact analysis details and letting ServiceNow control approvals of Puppet changes. Enriched Change Management Get accurate, up-to-date information about your CMDB assets in ServiceNow, without having to perform frequent discovery runs. Up-to-date Asset Management Reduce the time and effort required to maintain an accurate drift remediation log. Automated Incident Registration
- 34. Integration: Automated Incident Registration WHY ● When system configuration that drifted out of compliance is corrected, this information should be registered in ServiceNow. ● Ideally, you want custom business logic to determine when an incident should be created ● Doing all of this manually would not be feasible at scale.
- 35. Integration: Automated Incident Registration WHAT ● Automatically forward relevant details to ServiceNow when Puppet corrects a system that drifted out of compliance. ● Either create incidents directly, or publish events to ServiceNow Event Management to enable custom logic for when incidents should be created.
- 36. Integration: Automated Incident Registration HOW ● Puppet agent run reports are scanned for corrective changes and failures ● When changes or failures are detected, relevant details are forwarded to the ServiceNow API ● This can either be done as events, enabling custom logic, or directly as regular incidents.
- 37. Registering Incidents from Puppet Agent runs Automatically create & close incidents based on corrective changes made by Puppet Incident: - Name: INC0010483 - Configuration Item: srv1.company.com - Description: <info on config corrected by Puppet> Node submits change report after run srv1.company.com Puppet Enterprise PE creates & closes incident
- 38. Demo
- 40. Up-to-date Asset Management Let developers control aspects of their own systems without sacrificing compliance, security, or operational predictability. Self-Service Infrastructure Reduce the risk of change by enriching change requests with impact analysis details and letting ServiceNow control approvals of Puppet changes. Enriched Change Management Reduce the time and effort required to maintain an accurate drift remediation log. Automated Incident Registration Get accurate, up-to-date information about your CMDB assets in ServiceNow, without having to perform frequent discovery runs. Up-to-date Asset Management
- 41. Integration: Up-to-date Asset Management WHY ● Without Puppet, you need ServiceNow Discovery to keep the details of systems in the CMDB up-to-date ● Such discovery runs are known to have an unwanted stability impact on production systems. ● It is more efficient to update the details in the CMDB from Puppet’s database directly. on the roadmap
- 42. Integration: Up-to-date Asset Management WHAT ● Inventory data from the Puppet database is periodically gathered and uploaded to ServiceNow. ● A Puppet app for ServiceNow processes the staged data and updates the CMDB as necessary. ● Focus ServiceNow Discovery usage to detecting new/rogue systems only, while Puppet keeps information up-to- date for all known systems. on the roadmap
- 43. Integration: Up-to-date Asset Management HOW ● Puppet will periodically upload details about the systems it knows about to a holding area in ServiceNow. ● A new Puppet app for ServiceNow will then process the uploaded information and update CI details as necessary with the latest information. on the roadmap task: servicenow_assets::get_node_facts schedule: daily params: - targets: [srv1.company.com, …] - facts: [serialnumber, operatingsystem, …] ServiceNow MID Server JSON JSON JSON JSON update Puppet CMDB Sync (ServiceNow Marketplace app)
- 44. • Runs are agentless, so you have to manage lots of credentials • Discovery runs negatively affect the performance & stability of our production systems • Puppet CMDB update sync would significantly reduce the need for discovery runs just for keeping CMDB information up to date “ServiceNow Discovery has been the bane of my existence” Configuration Manager at a major bank in Columbus, Ohio
- 45. Update ServiceNow CMDB from Puppet facts Automatically update CI records with Puppet captured data Configuration item: - Name: srv1.company.com - Manufacturer: Amazon EC2 - Model ID: t3a.medium - Serial number: ec2c60a0-2e4b-230 - Operating System: CentOS - OS Version: 7.6.1810 Node submits facts during agent runs srv1.company.com Puppet Enterprise PE periodically uploads facts about known nodes: - bios_vendor - serialnumber - operatingsystem - operatingsystemrelease ServiceNow periodically processes the received fact upload data and updates information in the CMDB
- 46. ServiceNow App high level architecture CMDB CI: - Name: srv1.company.com - Manufacturer: Amazon EC2 - Model ID: t3a.medium - Serial number: ec2c60a0-2e4b-230 - Operating System: CentOS - OS Version: 7.6.1810 PE Orchestrator API Endpoint: /v1/command/task task: servicenow_tasks::get_node_facts Params: - targets: [srv1.company.com, …] -facts:[serialnumber, operationsystem, …] Puppet Connector App: - Mapping: certname <-> Name bios_vendor <-> Manufacturer serialnumber <-> Serial Number operatingsystem <-> Operating System operatingsystemrelease <-> OS Version Node Facts: - certname: srv1.company.com - serialnumber: ec2c60a0-2e4b-230 - operatingsystem: centos - operatingsystemrelease: 7.6.1810 - bios_vendor: Amazon EC2 PDB ETL
- 47. The 4 integrations of Puppet and ServiceNow Type: Puppet Module Available: Now Where: Puppet Forge Name: servicenow_cmdb_integration Self-Service Infrastructure Type: Puppet Module Available: Now Where: Puppet Forge Name: servicenow_change_requests Enriched Change Management Type: Puppet Module Available: Now Where: Puppet Forge Name: servicenow_reporting_integration Automated Incident Registration Type: ServiceNow App Available: TBD Where: ServiceNow Marketplace Name: TBD Up-to-date Asset Management
- 48. Q & A
- 49. Thank You
Editor's Notes
- First a very brief introduction to the world of Puppet. You are attending Puppetize, so presumably you know something about it, but let’s frame it a specific way for this conversation.
- The first layer of Enterprise features provides Automation content. Whether it’s task-based actions, infrastructure-as-code or event-driven automation, we provide access to a vast collection of community and certified content to accelerate your efforts to automate the systems you have today. More and more 3rd parties are providing Puppet automation content directly as well, and of course you have the ability to create your own.
- The orchestration layer provides API access to all of Puppet’s features. This gives you powerful ways of extending automation to and from your other investments. We provide a number of prebuilt integrations with systems like ServiceNow, VMware, Splunk and Tenable, to ensure you see immediate value from using these products together.
- The breadth of control & insight that the Puppet Enterprise platform now provides makes it a useful resource for teams reporting to many different Lines of Business. The Platform and Infrastructure teams use Puppet for configuration management and Compliance. The IT Operations and Provisioning teams use Puppet for Self-Service automation, and the CISO uses Puppet to ensure the infrastructure remains fully patched, in line with compliance standards and clear of vulnerabilities.
- As you can see, the Puppet Enterprise platform makes infrastructure actionable, scalable and intelligent. We help you improve agility and productivity by faster deployments and better control over infrastructure configuration We help you boost efficiency by reducing provisioning time and providing a single framework for all automation activity Finally we reduce the risk of change by standardizing systems and improving consistency, resulting in less operational disruption or security issues.
- Now lets do a brief introduction to the world of ServiceNow. I suspect some of you know quite a bit about it, but let’s get some basic ITSM terminology and process out the way.
- You’re probably well aware of what ServiceNow is, but just in case you’re not: ServiceNow is one of the leading IT Service Management & IT Operations Management platforms on the market. It is one of the first all-cloud Platform-as-a-Service solutions in this space, with competitors being mostly on-premise or partner-hosted solutions.
- Good ITSM platforms give you a shared data model to replace lots of individual tools, spreadsheets, databases, email processes, etc. Everyone can work from a single source of truth, and every process can be governed by a single tool.
- The core ITSM offering of ServiceNow is made up of these 5 components: A CMDB to track all your IT assets Request Management to enable self-service to users via a Service Catalog Change Management to document & approve planned changes Incident Management to document & track service disruptions Problem Management to track known issues These processes, that stem from ITIL, are closely related to activities that Puppet performs in your infrastructure. Note that we can also send Events to ServiceNow where ultimately those may end up as Incidents. A way to look at it is this: where Puppet takes action to execute a change, the process around it -from definition to approval- is governed by ServiceNow.
- Therefore we can think of ServiceNow and Puppet as opposite sides of the same coin. The vast majority of Puppet’s customers have ServiceNow in place, working mostly in parallel. That also means that some information is duplicated across both platforms, and some level of manual effort is involved to get information of one platform into the other. Over time, several of our customers have built their own integrations to make this more efficient; but more generally supported integrations have been a long-standing desire from our customers.
- Well, today I’m happy to announce that the wait is over! We have been working hard to connect ServiceNow and Puppet in several different ways, making it easier to share data between both platforms. We worked with our customers to determine the specific areas where bi-directional data sharing & automation is needed, and developed integrations that cater to those needs.
- We are going to look at 3 brand-new integrations that are available for you today, and we are announcing a 4th integration that we plan to deliver in the future. Available now are: An integration that allows you to use ServiceNow as a self-service frontend for controlling Puppet automation .This may eventually just be called the node classifier. ServiceNow publishes a note on how to do this. Its pages and pages long and it’s quite complex. What you will see is a much simpler approach An integration that connects planned Puppet changes to the ITIL change management process in ServiceNow An integration that sends change events to ServiceNow for correlation & analysis, generating incidents when needed Planned for later is an integration to update the ServiceNow CMDB with data from Puppet, we will come back to that at the end. Looking at asset management, writing it as ServiceNow actions, ensuring that it stays supportable
- So let’s start with Self Service.
- There’s a very good reason for why you’d want an integration like this: not everybody in the company knows how to use Puppet to automate changes. There’s probably a fairly limited amount of people in your company today that directly work with Puppet. Others would likely benefit from using Puppet as well, but expecting everyone to get properly trained to do so is unrealistic. So we need to make it far easier to leverage Puppet, which is what this integration does. Customers would rather use the ServiceNow Integration Hub and have catalog items that they can control. These would be Spokes, some standard offerings, e.g. reboot, patch, install, etc.
- In a nutshell, the integration allows you to use ServiceNow as a friendly user interface to control either all or parts of Puppet automation. For the most user friendly experience, create custom fields on your CMDB table for anything you’d like to give users direct control over, for example: Additional packages that can be installed at will OS kernel settings Application tuning parameters This data can then be easily acted upon in your Puppet automation. You can allow direct changes to the fields provided, or make them read-only and use ServiceNow workflows to allow controlled modifications only.
- The way the integration works, is that we read the fields for a system directly from ServiceNow, and exposes the information as a fact. That makes the content directly usable in your Puppet code, as input for your automation. You remain in complete control over what happens. For the utmost level of self service, you can create a Puppet Environment and a Puppet Classes field in ServiceNow, turning it into a fully fledged node classifier. The integration provides built-in logic to parse those 2 specific fields, converting the data into proper Puppet classes and Hiera data automatically. However, direct control for users over this option would require a bit of Puppet knowledge, so direct control might not always be the best approach there. A good alternative is to have these fields be read-only for users, and use ServiceNow workflows to facilitate controlled modifications to the classification data.
- This group presented earlier today in the Puppetize EMEA session. Their key benefit was allowing end users to add packages easily and safely.
- Note: Requires PE 2019.3 or higher, since that’s where trusted external fact capability was added.
- Next, let’s take a look at Change Management.
- For most of our customers, Puppet is their primary change execution platform. For many of those customers, ServiceNow is their primary change registration & approval platform. Naturally this should be one, interconnected, solution. That’s exactly what this integration provides. This one is all about the handoffs between one platform to the other and vice versa.
- When you propose a Puppet code change, this integration automatically generates a ServiceNow Change Request from the Puppet Impact Analysis report. Once the Change Request has been approved and reaches the Implement stage, the integration lets ServiceNow orchestrate the promotion of the change into production. This integration eliminates a number of manual steps in the change process, enabling greater efficiency, improved change documentation and faster cycle times.
- This integration is built in the Continuous Delivery for Puppet Enterprise add-on, and uses its Impact Analysis feature to generate relevant details that need to go into the ServiceNow Change Request. The integration interacts with the ServiceNow Change Management API to create the Change Request and associate affected systems to the change. The integration also provides a Business Rule for ServiceNow to automate the promotion of the Puppet code into production. It can even approve deployments for protected environments in CD4PE, for an additional layer of security.
- Next, let’s take a look at registering incidents from change events.
- While the previous integration deals with planned changes, this integration is for compliance drift corrections: changes that were made to bring a system back to the correct configuration after it drifted out of compliance. Since there is no way to know when this happens, the Change Management process is not suited for this scenario. Instead, this should be documented as an event and incident should be created if the circumstances warrant it. This is definitely best left for robots to do, instead of humans.
- With this integration, Puppet forwards summary data of Puppet run results to ServiceNow. This can be in the form of basic incidents (when a corrective changed occured), or -and this is better- you can send the information as events to ServiceNow Event Management. In the latter case, you can create your own Event Rules to correlate events and create consolidated Incidents when certain thresholds are crossed. This way, there will be 1 incident for the same change happening on multiple systems.
- The integration comes in the form of a Puppet report processor, that automatically analyses incoming Puppet run reports for changes and failures. In its basic mode, it will create an incident whenever a corrective change or failure occurs on a system. In its advanced mode, it will send events to ServiceNow Event Management so that you can more finely control what happens to these events, when alerts are generated and if that should result in the creation of an Incident.
- Finally, I have one more integration to announce. This integration is not yet ready, but planned for the future. It is intended to provide a better way to keep your CMDB up-to-date, without having to resort to frequent discovery runs.
- The biggest struggle of maintaining a CMDB is to keep the information up-to-date. Outdated CMDBs have been a problem for many organizations, and have given rise to discovery & inventory add-ons like ServiceNow Discovery. However such tools often run agentlessly and are known to cause a performance impact or stability problems on the systems they interrogate. As a result, customer don’t like running these tools frequently. By contrast, Puppet already knows everything about the systems it manages, and stores that information in its own database. Updating the CMDB in ServiceNow directly from Puppet’s database, would be a lot more efficient compared to agentless discovery tools.
- Puppet automatically collects information about the systems it manages, as part of its normal enforcement runs: Software Inventory Standard facts about the hardware and operating system Custom facts you create yourself The integration will allow you to select which data you’d like to send to ServiceNow for updating existing CI’s. This reduces the scope for ServiceNow Discovery to just finding new or rogue systems on the network.
- On the Puppet Enterprise side, a Task will periodically gather the latest information from the Puppet database, and upload it to a staging location on a ServiceNow MID server. On the ServiceNow side, a Puppet App (from the ServiceNow Marketplace) periodically processes the uploaded data and updates fields in the CMDB as necessary.
- The three integration I discussed first, are available right now as modules on the Puppet Forge. They come with clear instructions on how to setup each integration. The CMDB update integration will be delivered as an app in the ServiceNow Marketplace, probably somewhere in 2021. Stay tuned for more info on that as we move forward.