Uploaded bySplunk
The Church of Jesus Christ of Latter-day Saints Customer Presentation
This document summarizes the Church of Latter-day Saints' use of Splunk for log aggregation, monitoring, and reporting across their IT platforms. It describes how Splunk helped consolidate over 2,100 log forwarders, provided dashboards for visibility, and enabled self-service troubleshooting. Through automating Splunk deployment on their Cloud Foundry platform, they increased adoption of logging and monitoring services amongst internal application teams.
The Church of Jesus Christ of Latter-day Saints Customer Presentation
- 1.
- 2.
- 3. 3 About Us The Church of Jesus Christ of La?er-day Saints • 15.6 Million+ Members globally • 30M visitors/month to main customer-facing websites • Widely known for: • Missionary • Family • Humanitarian • Genealogical
- 4. 4 Pla[orm as a Service Team • Services many church Departments • Helps Streamline department service delivery • Automates the configuraHon of their environment • Provide Architectural pla[orms/design pa?erns • Favorite Splunk T-Shirt Slogan: Because Ninjas are too busy Note: We are not the Splunk / Monitoring team, we are users of their tools
- 5. 5 Before Splunk • Manually Processes for TroubleshooHng • Log and Event CorrelaHon • Inconsistent Logging Formats • Manual VM Setup for Logging • No Historical Trending Data • More outages due to lack of visibility • Complexity of MulHple alerHng systems “In the past we had to manually configure VMs, runAmes and logs so it took a lot longer for that process to happen .”
- 6. 6 Splunk AdopHon at LDS • Splunk Enterprise, ~ 1TB, 4TB licenses • Architecture: – 2100 forwarders – 8 indexers – 6 search heads
- 7. 7 Using Splunk for… • Consolidated Logs • Monitoring and ReporHng • Dashboards / Visibility • As a PlaForm Team, we want sure users can do this easily.
- 8.
- 9.
- 10.
- 11.
- 12. 12 How our automaHon increases adopHon • We have about 1200-1300 app instances on our Pla[orm • 1-2 new app instances per day • ~600 app instances use Splunk • Why is this important ? • Our users easily have self-service Logging, Dashboarding, and Metrics available • Value: Faster Hme to repair, restore services, and focus on your service
- 13.
- 14. 14 Monitoring CloudFoundry • CloudFoundry itself must be monitored • Hundreds of VMs • Many components • Problem • Too many tools • Cross metric/logging analysis and alerHng • Lack of self service • Dashboard • Customer transparency • Load balancer data
- 15. 15 Splunk Does It All Past Present Log AggregaHon Splunk Splunk Metric Storage Oracle DB Splunk AlerHng Nagios & Splunk Splunk Dashboard Custom Web UI Splunk Management Email another Team Self Service Cross System Analysis Good luck Spunk AcHve Monitoring (Smoke Tests) Anthill Splunk
- 16.
- 17. 17 Pla[orm Visibility (extending self service) • Pla[orm monitoring data available to customers • Self service troubleshooHng • Help correlate system with app data
- 18.
- 19. 19 Splunking Ahead…. • Consider using IT Service Intelligence or ITSI • Move integraHon to new HTTP Event collector • Send metrics to Heavy Forwarder instead of Indexers • ConHnue to grow our suite of pla[orm alerts
- 20. 20 Splunk Words to the Wise…. • Splunk is far more than a logging tool. If you’re only using it for logging, you’re missing some key features – dashboards, alerHng, metrics, unified problem analysis, etc. • Unlike the many enterprise alerHng and monitoring tools, Splunk is very self service oriented, making it so easy to use. • Even educaHon is self service: online API documentaHon, Q&A, and reference docs.
- 21.
Editor's Notes
- #9 Changed the world from a VM world to an App and Service World No VM Configuration No Runtime Configuration No Middleware Configuration No Mismatches of VMs, Software patches, installs No out of date Security Patches No Inconsistencies on Deploys across technologies Same for Java, NodeJS, Static Assets, Docker Provide all this AND Insulate your application from future hosting and cloud provider differences?
- #11 Changed the world from a VM world to an App and Service World No VM Configuration No Runtime Configuration No Middleware Configuration No Mismatches of VMs, Software patches, installs No out of date Security Patches No Inconsistencies on Deploys across technologies Same for Java, NodeJS, Static Assets, Docker Provide all this AND Insulate your application from future hosting and cloud provider differences?
- #15 Cloud Foundry is a great platform to host applications. Our users love that we make it easy for them to put application data into splunk so they can monitor their applications using splunk. But, Cloud Foundry itself is also an application. A very important one. If cloud foundry goes down that could impact the availability of running in cloud foundry so it is very important that Cloud Foundry is also monitored. Here are some of the problems we had with our original monitoring solution. We went through an extensive evaluation process with the different enterprise monitoring tools available to us. We soon discovered that Splunk isn’t only good at aggregating log data it is also excellent at injecting metrics and alerting, it is completely self service, you can create dashboards for us and our users.
- #19 If app is going slow is it because of the app or some other app running on the same machine? Would be great to add VMWare Host data to this graph