Manager - DevOps Tools/Monitoring
A world-class publisher of news and business information.
We are newspapers, newswires, websites, apps, newsletters, magazines, proprietary databases,
conferences and more.
Our premier brands include The Wall Street Journal, Dow Jones Newswires, Factiva, Barron's,
MarketWatch, and All Things D.
We used an agent-based proprietary log monitoring
solution that was . . .
Difficult to update & upgrade
Inflexible (very limited filtering options)
Provided limited visibility into data
Unable to handle ever-increasing load from applications
Identify problems quickly
Monitor logs in real-time
Handle non-standard file behaviors and formats
Accommodate multiple inputs (e.g. SNMP trap, syslog)
Integrate alarms into centralized alert console
Provide longer retention
Keep pace with new business initiatives
Initial Success Measurements
Reduced Mean Time To Resolve (MTTR)
Primary Metrics for all of Operations and the reason we invest in
Instrumentation – reduce the length of service impacting events by knowing
about them as early as possible
Increased % of Incident Detection
We want to know about service impacting events before the user does
Why We Chose Splunk
It addressed all our requirements and pain points, and provided a way
for internal and external groups to view their data.
Easy to spin up
Reliable & easily scalable
Easy to upgrade
Handles massive data volume
Flexible filtering options
Offers deep visibility
Problems are easily identified
Total Prod Indexers: 30
Total Forwarders: @10,000
Daily Log Volume: 150GB/day
Indexer Deployment &
Agent Deployment Head
Internal Splunk Customers
Infrastructure and Operations
Custom dashboards for number of alerts by server, source, product, business, etc.
Multiple Splunk apps used (e.g. Exchange, AWS, VMWare, etc.)
Create their own custom dashboard tracking application anomalies and patterns
Threat identification – FireEye app
Network performance and utilization – Multiple Cisco apps
Customer insight – in-house WSJ.com Realtime app and globe
Additional uses for Splunk are actively pursued.
Business leadership sees opportunity & expresses interest.
Outside groups recognize value beyond original purpose
Adoption quickly spreads through initial user community
> > > Initial implementation < < <
Promote wider adoption of Splunk
Continuous education via internal workshops & training
Promote application logging standards
Enhance lifecycle & administration
Expand into AWS and use configuration automation where possible
Further centralize application logs
Show value of apps and plug-ins
Leverage newly available features/capabilities of Splunk6, Splunk Storm,
and Splunk Cloud