• Save
SplunkLive! Chicago April 2013 - CME Group

SplunkLive! Chicago April 2013 - CME Group






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • This can serve as an agenda slide…I’ll walk you through how Splunk helped us initially to simply find the data we needed to do our jobThen become proactiveTo deliver a better customer experienceand now Splunk has helped us gain internal credibility (and additional work) by delivering dashboards that showcase the data driving our business.
  • Other vendors, LogLogic, SenSage and Snareserver offer a lot of canned reports and forms, but to truly understand our environment—and get up and running quickly, Splunk was the best answer.
  • #ALL INDEXERS[serverClass:all_indexer]filterType=blacklistblacklist.0=*whitelist.0=x.x.x.x

SplunkLive! Chicago April 2013 - CME Group SplunkLive! Chicago April 2013 - CME Group Presentation Transcript

  • Copyright © 2012 Splunk, Inc.Bob Beard, CME Group
  • • Offering widest range of benchmark futures and options products available on any exchange, covering all major asset classes• Interest rates, equities, FX, commodities, and alternative investments such as weather and real estate• Joint venture owning 90% of Dow Jones Indexes• Our customers include brokerage firms, banks, hedge funds, pension funds• We monitor network infrastructure and the artifacts our apps generate 2
  • About Bob BeardUsing Splunk for the past 6 yearsDirector, Network EngineeringDesign and implement monitoring solutions for applications andnetworking for the ExchangeTeam responsible for Monitoring for fault tolerance andperformanceServed in various management and engineering roles for 20+ years 3
  • Before and After Splunk Problem • No solid log collection platform • Multiple Monitoring solutions • All visualizations or Analytics required custom programming Results • Search functionality allows for quick and easy isolation • Single log monitoring infrastructure for all IT and Executive staff 4
  • Splunk for Monitoring 5
  • Moving From Reactive to Proactive with Splunk Proactive Visibility for Management and Clients Statistical Reporting Proactive Monitoring and Alerting Forensic Investigation Reactive
  • Why Splunk?Real-time Monitoring / Immediate Statistical Proactive Response Feedback Real-time Enhanced Customer Dashboards Service/ Experience
  • Our Splunk Architecture• Our Splunk • 3 Data centers • 56 Indexers • 2 Search heads for ad hoc searches • 2 Search heads for Real time searches • 1 Search head for saved searches and alerts • 2500 Forwarders+• 1 TB per day
  • Real-time Analytics• Moving from reacting to proactive—avoiding downtime before it happens• Our apps teams don’t log in a standard way• Troubleshooting across lots of apps and log types very time consuming• Research took too long and was often incomplete
  • Real-time Dashboards Across Multiple Departments• Each team sees specific statistics/ • NOC dashboards/ reports/ searches in • Operations Center real time • Customer Service• Role-based access limits access to • Various specific indexes/ data Development Teams• People have direct access to the data they need• 300+ folks using Splunk Even senior management can log in and get value
  • Real-time Reactions• Tried months and several homegrown solutions to surface real time insight—with Splunk it was working in 1 week• Threshold-based alerting supports proactive customer engagement • Thresholds based on exchange activity • Could also indicate application problem• Lookup function makes it easy to correlate various alerts
  • Real-time Improvements• Match engine dashboards show Key Performance Indicators• Developers can see how changes they make to match engines affect performance in real time in a parallel environment
  • Splunk Adaptation 13
  • Getting “De-Used” to your Database• Anyone can search in Splunk• No need to learn a query language• Splunk encourages exploration which helps lead to other discoveries• Knowing sourcetypes just makes parsing through the data easier “The speed of finding answers in Splunk is amazing. I’m fascinated by how quickly it returns results from across our entire data set.”
  • AHA!Took me two days from nothing to a workingenvironmentThe ability to correlate the log typesThe ability to keep improving parsing over timeBeing able to pull reports for upper level management inminutes vs. taking hours to produce a single monthlyreport. 15
  • Deployment GotchasHave to restart the indexersSeparate search heads for real timeGood index planning for delegation of access 16
  • Looking ForwardIncrease number of search heads to 8Indexer replicationConnection poolingUpgrade to Splunk 5Search heads pooling 17
  • Thank You!