DevOps Powered By Splunk
•
2 likes•636 views
DevOps is powering the computing environments of tomorrow. When properly configured, the Splunk platform allows us to gain real-time visibility into the velocity, quality, and business impact of DevOps-driven application delivery across all roles, departments, process, and systems. Splunk can be used by DevOps practitioners to provide continuous integration/deployment and the real-time feedback to help the organisation with their operational intelligence. Join us for an exciting talk about Splunk’s current approach to DevOps, and for examples of how Splunk is being used by customers today to transform DevOps initiatives.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to DevOps Powered By Splunk
Similar to DevOps Powered By Splunk (20)
More from Splunk
More from Splunk (20)
Recently uploaded
Recently uploaded (20)
DevOps Powered By Splunk
- 1. © 2017 SPLUNK INC. DevOps Powered By Splunk Domnick Eger | Global DevOps Practitioner, Splunk MAY 11, 2017 | LONDON
- 2. © 2017 SPLUNK INC. During the course of this presentation, we may make forward-looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release. Splunk, Splunk>, Listen to Your Data, The Engine for Machine Data, Splunk Cloud, Splunk Light and SPL are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners. © 2017 Splunk Inc. All rights reserved. Safe Harbor Statement
- 3. © 2017 SPLUNK INC. 1. Introduction to DevOps 2. Commit, Build, Test, Run, Repeat 3. DevOps In Operations 4. DevOps for Developers 5. DevOps Powered by Splunk In This Session
- 4. © 2017 SPLUNK INC. Introductions in DevOps
- 5. © 2017 SPLUNK INC. Evolution of DevOps in the Work Place Waterfall ITIL Scrum Agile LiveOps BizOps 12 + 6+ 3+ 1+ JIT Real Time
- 6. © 2017 SPLUNK INC. Why is DevOps so Important ? SERVER, STORAGE, NETWORKING VIRTUALIZATION INFRASTRUCTURE APPLICATIONS PACKAGED APPLICATIONS CUSTOM APPLICATIONS CLOUD SERVICES Machine Learning Batch & Real-time Collection Heavy Event Collection Hybrid Environment Developer Platform Robust SDK Active Response Security Operations & ITOA UBA
- 7. © 2017 SPLUNK INC. DevOps is a growing demographic.
- 8. © 2017 SPLUNK INC. Growing at a Global Scale
- 9. © 2017 SPLUNK INC. Entering all Markets
- 10. © 2017 SPLUNK INC.© 2016 SPLUNK INC. CONFIDENTIAL. INTERNAL USE ONLY. Where are you in your Journey ?
- 11. © 2017 SPLUNK INC. Commit, Build, Test, Run, Repeat
- 12. © 2017 SPLUNK INC.© 2016 SPLUNK INC. CONFIDENTIAL. INTERNAL USE ONLY. Traditional Services – Report, Break, Fix, Repeat HEC Jolokia APILogs Splunk Apps Splunk Platform Capture Application Errors Numbers of Call Application Response Time Capture Transaction Calls API SDK WireUF Session ID -> Transaction Calls -> JMX Process -> DB Calls Open Create Issue Closed In Progress Reopen Triage Issue Patch IssueResolve Issue Rinse and Repeat… Over.. and Over…
- 13. © 2017 SPLUNK INC. DevOps in Operations
- 14. © 2017 SPLUNK INC. A Platform Approach for Operations Network InfrastructureLayer Packet, Payload, Traffic, Utilization, Perf Storage Utilization, Capacity, Performance Server Performance, Usage, Dependency ApplicationLayer User Experience Usage, Response Time, Failed Interactions Byte Code Instrumentation Usage, Experience, Performance, Quality Business Performance Corporate Data, Intake, Output, Throughput Splunk Approach: ▶ Single repository for ALL data ▶ Data in original raw format ▶ Machine learning ▶ Simplified architecture ▶ Fewer resources to manage ▶ Collaborative approach MACHINE DATA
- 15. © 2017 SPLUNK INC.© 2016 SPLUNK INC. CONFIDENTIAL. INTERNAL USE ONLY. DevOps in Operations DevOps Team Master BranchSprint Epic Tasks Story Production Project Management Project Management OS Image Automation Service Discovery Change Request Build Pipeline Operation Team ▶ - Traditional Operations - ▶ Operations provides support to the internal and external customers. ▶ Operations focuses on Infrastructure, Monitoring, Architecture, Planning Maintenance, and Support. ▶ - DevOps in Operations – ▶ Answering the who, what, when, how, and adding magic to figuring out issues in the environment. ▶ Maintain and Build the Tool stack to support a full CI/CD methodology.
- 16. © 2017 SPLUNK INC. Index and Analyze Data Across Your Technology Stack Splunk Add-Ons, APIs and Apps Integrate With Other Tools You Already Have App Performance Monitoring Operations and Service Desks Server, Storage, Network Virtualization, Containers Operating Systems + Databases Custom Applications Business Applications Cloud Services Mobile Applications Web Intelligence Stream No rigid schemas – add in data from any other source. API SDKs UI DB Connect
- 17. © 2017 SPLUNK INC. DevOps in Development
- 18. © 2017 SPLUNK INC.© 2016 SPLUNK INC. CONFIDENTIAL. INTERNAL USE ONLY. DevOps in Development DevOps Team Master Branch Branch Build Sprint Epic Tasks Story Production Staging Development Environment Project Management ▶ - Traditional Development- ▶ Developers create software and process to interact with systems and people. ▶ Developers follow multiple types of development framework that include prototyping, waterfall, agile and rapid. ▶ - DevOps in Developers– ▶ Developer in a full Agile methodology are able to move and shift work to accommodate projects. Backlogs can be a saving grace or a nightmare. ▶ Developers also in a Agile workflow can create stories that represent and track their efforts.
- 19. © 2017 SPLUNK INC.© 2016 SPLUNK INC. CONFIDENTIAL. INTERNAL USE ONLY. Build Automation - Example
- 20. © 2017 SPLUNK INC.© 2016 SPLUNK INC. CONFIDENTIAL. INTERNAL USE ONLY. Breaking the Silo’s of Traditional IT Open Resolved In Progress Reopened DevOps Team Create Issue Chat Ops Incident / Change Closed ▶ In 2008 Agile software development started to gain traction and the birth of DevOps began. ▶ With the rise of AWS in 2010 the convergence of compute, storage and network became a the next generation of on demand datacenters environments. ▶ With the advent of AWS many other cloud providers are creating ways for groups to deliver software in a near real-time fashion and giving consumers a way to scale endlessly.
- 21. © 2017 SPLUNK INC. DevOps Powered by Splunk
- 22. © 2017 SPLUNK INC.© 2017 SPLUNK INC. Splunk Demo
- 23. © 2017 SPLUNK INC. Build Chain Automation Jenkins Splunk Bitbucket Commit Developer DeployBuild Node.js Metrics
- 24. © 2017 SPLUNK INC. Run & Monitor Node.js Splunk Browser /Mobile APM User Interactions Metrics & Alerts CDN Cache Metrics Static Content Auth0 Auth0
- 25. © 2017 SPLUNK INC. 25 API SDKs UI Other Tools Escalation/ Collaboration No rigid schemas – add in data from any other source. Visibility Across the Dev Lifecycle Plan Code Build Test/QA Stage Release Config Monitor
- 26. © 2017 SPLUNK INC. ▶ Ingest data once – single source of truth across teams ▶ Analyze machine data across entire stack ▶ Integrate data from other management tools ▶ Connect machine data to business services ▶ Identify root cause of problems quickly ▶ Apply best practices in analytics to predict changes in reliability and service usage Reliability Requires a Platform Approach Data Fabric OTHER TEAMS PRODUCT MANAGERS/ BUSINESS OWNERS DEVOPS, SRE PERF MANAGER APP MANAGERS/ OPERATIONS DEVELOPERS
- 27. © 2017 SPLUNK INC. Complex Technology Stacks Make Reliability More Difficult End Users Networking/ Load-balancing Web Servers App Servers Legacy Systems Messaging Databases Security Virtualization, Containers, Servers, Storage Java, .NET, PHP, etc.
- 28. © 2017 SPLUNK INC. curl -k https://<host>:8088/services/collector -H 'Authorization: Splunk <token>' -d '{"event":"Hello Event Collector"}' Applications IoT Devices Agentless, direct data onboarding via a standard API HTTP Event Collector – Agentless Fast Insight Scales to Millions of Events/Second 28
- 29. © 2017 SPLUNK INC. Splunk IT Service Intelligence ▶ Visualize entire tech stack – bare metal through business layer ▶ View the entire ecosystem with customized views for execs ▶ Apply context to events to prioritize investigation based on impact Dynamic Service Model Machine Learning ▶ Adaptive threshold automation to minimize false alerts ▶ Behavior anomaly alerts to proactively address issues ▶ Automatic correlation of data into intelligence, mitigating SME dependency ▶ Accelerators minimize SPL coding ▶ Trend aggregation to enable rapid visualization ▶ Multi KPI Alerts for proactive irregularity identification Search-Based KPIs ▶ Time Series Index ▶ Schema on Read ▶ Handle any and all data Platform for Operational Intelligence
- 30. © 2017 SPLUNK INC. Solution Architecture DATA SOURCES SOLUTIONS Cloud Servers Open Source Database APM Network Hypervisor Wire Data MobileStorage Applications Service Monitoring Entity Monitoring Event Intelligence PLATFORM Automation Tools (THIRD PARTY) Service Management Tools (THIRD PARTY) TOOLS & APIs LOGS TroubleshootingMonitoring Platform for Operational Intelligence
- 31. © 2017 SPLUNK INC. ▶ Gain real-time insight into application performance and customer experience ▶ Attain visibility into cloud services ▶ Deliver immediate insights from streaming network ▶ Network-based packet capture does not require DBA or other admin tools and doesn’t affect performance Gaining Transaction Insight From Your Network Splunk Stream
- 32. © 2017 SPLUNK INC. ▶ Immediate visibility to mobile app crashes ▶ Insight into mobile app use – MAU/DAU, device usage, network insight ▶ Transaction performance insights ▶ Correlate mobile with other data types for complete insight Gaining Insight on Your Mobile Apps
- 33. © 2017 SPLUNK INC. ▶ Accelerate your AWS deployment through better visibility into usage and user behavior ▶ Gain increased visibility into AWS resource utilization ▶ Supports wide range of AWS data sources Gaining Insight on Your AWS Hosted Apps AWS CloudTrail AWS Config AWS Billing AWS CloudWatch AWS CloudFront AWS ELB Logs AWS S3 Access Logs Other AWS VPC Flow Logs
- 34. © 2017 SPLUNK INC. ▶ Accelerated code delivery from 3 months to 20 minutes ▶ Reduced infrastructure and hardware costs by migrating to the cloud ▶ Gained visibility and able to perform analysis from metrics: • What users were doing, how long was it taking ▶ Eliminated other tools Faster Development, Better Troubleshooting
- 35. © 2017 SPLUNK INC. 35 Enable Data-Driven Continuous Delivery -Alison Perkins, Senior Systems Engineer “Dump all the logs into Splunk, and it starts looking like one big system, instead of a bazillion teeny ones that hate each other.” Key Customer Benefits • Quickly validate and troubleshoot code pushes to production • Ensure that new code does not negatively impact performance or user experience • Reduced one application’s error rate by 2 orders of magnitude in a matter of weeks Change to look more consistent with other slides
- 36. © 2017 SPLUNK INC. SEPT 25-28, 2017 Walter E. Washington Convention Center Washington, D.C. .conf2017 The 8th Annual Splunk Conference conf.splunk.com You will receive an email after registration opens with a link to save over $450 on the full conference rate. You’ll have 30 days to take advantage of this special promotional rate! SAVE OVER $450
- 37. © 2017 SPLUNK INC.© 2017 SPLUNK INC. THANK YOU
Editor's Notes
- Insert names, and SL Date and location. Summary: Welcome, we’d like to learn more about our audience Key Points: Welcome! And intros Ask “How many are active Splunk users?” How many of you also have APM tools? You’ll enjoy seeing what you can do with those tools when you integrate them w/ Splunk
- Summary: You’ll learn more about 1) why you’ll want a platform approach, 2) Splunk’s approach and relevant technology. Key Points: - This is not a ”how to” session but it does provide an overview of relevant Splunk technologies - We’ll cover the need for a platform approach and how Splunk is at the center of a complete APM approach - We don’t just discuss, we will also show. We’ll have three demos throughout our hour together
- Spacer Slide – move through this quickly…
- Spacer Slide – move through this quickly…
- Spacer Slide – move through this quickly…
- Summary: Splunk platform approach – same data, many lenses Key Points - Like what you heard in the keynote this morning… - Collect, index and analyze across data sources - Those data sources aren’t just the parts of your stack, but also the tools you already have that’s collecting data
- Summary- you can also bring data from your management tools in via an app and add-on too! Key points - Liberate your management data – pull it into Splunk - Accelearate ttriage efforts Three important things to remember: If a logo you have doesn't show up here, Splunk still doesn't’t limit you – you can always index data from that technology – Splunk extensions simply help you accelerate the process. We provide a full featured REST API and a variety of SDKs that help you build your own custom apps for technologies and insights custom to your business. This is to help you create a specific interface to your data in special format and development languages your team is used to. 3. To accelerate that effort even more, we introduced an Add-On Builder, which some of our team uses internally when they need to onboard new source types quickly and persistently
- Spacer Slide – move through this quickly…
- Spacer Slide – move through this quickly…
- Lets looks at a monitoring and troubleshooting scenario with Splunk
- Splunk can provide insight across the entire application delivery lifecycle. Developers can search and visualize data from entire build pipeline and production environments without needing to access production machines.
- Summary: The alternative to random acts of tools – taking a platform approach, where the data coming directly from your environmnent and the tools that manage them can be indexed and analyzed by people who need that data Key points - Collect data once, use it in multiple use cases with multiple stakeholders - Liberate your data – make it available to people so they can do their job
- Summary – Complex technology stacks makes monitoring problems and triage more difficult than ever Key Points Technology Stack – more points of failure Need to transition to “there’s a problem” to “there’s the problem” ASAP
- HTTP Event Collector is easy way to send data to Splunk Enterprise. Notably, the EC enables you to send data over HTTP/ HTTPS directly to Splunk Enterprise from your application. The EC was developed with application developers in mind, so that all it takes is a few lines of code added to an app for the app to send data. Also, the EC is token-based, so you never need to hard-code your Splunk Enterprise credentials in your app or supporting files. HTTP Event Collector provides a new way for developers to send application logging and metrics directly to Splunk Enterprise via HTTP in a highly efficient, scalable and secure manner
- What makes Splunk ITSI different is not only all the cool visualizations that you just saw in the premium solution, but more importantly, the platform that it was built on top of. Just about every CIO or Ops Executive we talk to is frustrated with Manual Integration within and across tools and Correlation issues with their current Service Management and Monitoring Solutions. The number of tools they’ve had to buy, deploy, administer, and attempt to integrate just don’t live up to their original promises. An impact of this lack of integration and correlation is the customer’s difficulty meeting or accurately measuring their SLAs. One way that Splunk differs from existing approaches is that it is a Universal Machine Data Platform which allows you to reliably collect, index, prepare and store data from tens of thousands of sources, in real time -- any type, any format, any location with no pre-defined schema. We are data driven. We take in all the data. Splunk is also in network latent real time and can leverage historical data as well. To avoid the problems associated with adding or changing Alerts, Splunk delivers Schema on the Fly to provide for rapid creation of alerts from either KPIs or raw data to adapt to business needs quickly. Splunk applies structure at search time, making it easy to search, visualize and analyze your data without any knowledge of the underlying structure. No DBA is required! We also use machine learning to baseline normal operations, detect anomalous behavior to drive meaningful actions, and enable highly correlated searches to create meaningful “alerts” off your KPIs, not ours. And, you get the information from the data that you need when you need it. With Splunk, you can ask any question of the data any time! Splunk’s powerful platform helps you to realize faster time to value as it leverages all of the data, allows you to answer any questions of the data and empowers the greatest data fidelity With existing Event Driven solutions, our customers tell us that getting true Service Intelligence is a challenge. Today, Service Owners tell us that they determine Service Health through summarized events that have limited retention time. The business impact here surrounds the time and expense in identifying root cause and fixing the problem To address this, Splunk ITSI delivers a 360 degree view of service health from one place. We call this Full Fidelity Service Health. We allow for adaptable and flexible definitions of service health. Customers can now move seamlessly from Business Service Reports to Remediation, all while providing complete historical context. Our solution remains adaptable and yet still maintains complete historical context. Want to visualize and measure what was happening 10 minutes ago?… an hour ago?… Not a problem. This unique differentiation enables Splunk ITSI to deliver a seamless, connected experience from reporting through to remediation. The ability to leverage Deep Dive Incident Reviews, delivers event, metrics and KPIs – including ad hoc, on the fly searches – you can see and correlate complex interactions easily. And like we just discussed, with full access to historical data, you can compare any two time ranges for all data sets side by side to quickly understand what’s ‘normal’ for that Service by minute, hour, day or week regardless of size or scale. Every day we hear from customers that change is a constant and the Legacy Service Management solutions struggle with keeping up. With Legacy Solutions, Service Definitions come from Legacy CMDBs that come with questionable data quality. We also hear that it is hard to create new KPIs to keep everything relevant to the Business. The impact that we hear from Service Owners is that the business perceives IT as being inefficient. So what else does Splunk ITSI do here that is different? Search Based KPIs deliver a flexible way to impose schema only at retrieval, without a pre-defined schema or hard coded collectors. Often the business may need to see new KPIs or change existing ones. You can easily write, manage and change both services and KPIs so that you can best align business and technology priorities. An example of this in action comes from one of our Beta customers, Fiserve. With Splunk ITSI, Fiserve was able to generate 1000s of KPIs in a manner of weeks. They were able to easily write, manage and change both services and KPIs. Splunk runs on-prem, in the Cloud or in hybrid environments while collecting data from all the newest technologies. Our visualizations and analytics are one-of-a-kind. They can be personalized, meaningful, and contextual. Better visualizations and analytics provide and enable IT with actionable insights. Every one can look at the data in the manner that is most relevant to them.
- Summary: In addtion to a platform, we have solutions that use that platform for faster TTV and expanded value Key Points Platform – collect, index, analyze data Solution – IT Service Intelligence – enables improved event analytics, service monitoring – you’ll see more later!
- Summary: Some apps are built by Splunk – Splunk Stream allows you to use network packet contents as a data source for Splunk Key Points - Can’t always install a forwarder, sometime the network is the ONLY source we have - It’s not just about packet routing, there is valuable application insight inside those packets
- Summary: Mobile Apps have additional requirements – Splunk Mobile Intelligence give you insight on usage, performance and problems with the mobile app Key Points: - Splunk MINT – visibility to crash stack traces, usage (what OS/HW are people using for your mobile apps), transaction performance - SDK based – developers initialize the SDK in their apps
- Summary: If your apps are on AWS, you can get even more insight thanks to AWS data sources + Splunk Key Points - Get insight into resource usage and AWS config on Splunk - Cloudwatch, CloudTrail and 15 other AWS-based data sources are included - AWS app for Splunk is FREE – give it a try and plug in your AWS admin credentials
- Summary: FamilySearch uses Splunk for insight that helps both operations and developer teams Key Points Founded over 100 years ago, FamilySearch International is the largest genealogy organization in the world, hosting, maintaining and sharing genealogical records at FamilySearch.org and through over 4,600 family history centers in 132 countries. FamilySearch needed a way to move to a continuous delivery model, manage its all-in migration to Amazon Web Services (AWS) and immediately troubleshoot website errors. Since beginning its effort, the organization has seen benefits including: Challenges Wanted to increase update release frequency Needed to monitor and immediately detect changes to website to move to a DevOps model Issues with troubleshooting and keeping website stable Other Benegits 900 deploys per day Gained back 12 developers due to efficiency
- Challenges: No single place to access and visualize machine data Manual diagnosing and searching through data generated by servers and applications To retrieve information, sysadmins have to ssh into production machines before sending off to developers to grep through the logs With Splunk: Quickly validate and troubleshoot code pushes to production Ensure that new code does not negatively impact performance or user experience Reduced one application’s error rate by 2 orders of magnitude in a matter of weeks
- And of course, your biggest education opportunity this year is .conf2017 which will be held right back here in Washington, DC on September 25 – 28. I know you have heard a lot about .conf2017 today but don’t forget that by attending SplunkLive! today we are extending you a discount of over $450. You will be able to register with a unique link that will be sent in the post SplunkLive! emails to go out next week.