This document discusses Bill Lindsay's role at MetLife and how Splunk has helped MetLife's operations. It describes how MetLife used to have a complex, siloed IT landscape that required large, specialized teams to resolve issues. MetLife implemented Splunk in 2011 to aggregate logs, helping consolidate visibility and making it easier to diagnose problems across different systems. Using Splunk reduced resolution times and costs, needing fewer engineers and less time to handle incidents. Splunk helped shift support load to lower-cost resources by empowering teams to resolve more issues themselves.

1. Bill Lindsay
Service Design Specialist

2. 2
About MetLifeAbout MetLife
• MetLife, Inc. is a leading global provider of
insurance, annuities and employee benefit
programs
• Through its subsidiaries and affiliates, MetLife
holds leading market positions in the United
States, Japan, Latin America, Asia, Europe, the
Middle East and Africa
• MetLife is #1 within the Life and Health
Insurance category on the FORTUNE Magazine
“World’s Most Admired Companies” survey
list

3. 3
About Me
Bill Lindsay – Technology since 1979.
First code deploy.

4. 4
Stuff Happens
“A mistake is ok, as long as we fix it once. ”
- Jim McPeak – Bradford National Computer Services. (BONY)
Continuous Improvement - 0 defects.
Splunk takes this and scales it across the Enterprise.

5. 5

6. 6
MetLife and Me
• Joined 2005 as Software Engineer (J2EE, .NET, Mainframe).
• 2008 - Newly formed 24x7 Global Support Model.
Level 3 Support (SWAT) 8 Person Team
Escalations Outages Root Cause Analysis
Service Restoration Remediation Risk Assessments
Process
Reengineering
Best Practices

7. 7
Global Support Model
Centralized Production Management (Americas, Corporate)
Level 1 Level 2 Level 3
Global Partners Senior Incident
Mgrs. (FTE)
Subject Matter
Experts
Cognizant, iGate Full Time
Employees
Application Teams
SWAT
Administrators
Engineers
Software Vendors
Economies of
Scale
Shift Left

8. 8
Shift Left
Centralized Production Management (Americas, Corporate)
Level 1 Level 2 Level 3
Move the ability to see a problem, resolve an issue, restore service to the Left.
Empower the most cost effective resources
Has 800+ Applications.
Critical Applications are:
• Customer/Partner Facing.
• Revenue Generating.
• Have Contractual Liability.

9. 9
MetLife and Splunk
2011 Innovation
Campaign – Common
Log Aggregation
2012 Implemented by
IT Risk and Channels
& Integration
2014 Established as
Shared Service
2015 Working
towards wider
adoptance.
Operations is biggest Use
Case and Benefactor.

10. 10
Before Splunk: Complex Siloed Landscape
• Topology
• Operating Systems.
• Application Containers.
• Databases.
• User Interfaces.
• Consoles.
• Frameworks.
• Hardware.
• Protocols.
• Legacy Applications.
Impact to Operations
Level 1 Level 2 Level 3
1. Expertise required to support the application.
2. You need to know what you are looking for, where
to look and who to talk to.
3. Do they know what you’re talking about?

11. Time? Money?
Business? Brand?
Cost of
Silos
Case Study >

12. 12
Case Study : Annuities Client Acquisition
Problem: Application Failing during user session
intermittently.
Root Cause: Application normally runs Active-Active on
Primary and Secondary data centers. Secondary site
brought out of service due to environmental issues.
Someone rebooted box and secondary site started taking
traffic. Users started losing their sessions.
Hooking logs into a framework. (Apache, IIS, Siteminder,
App Logs) Windows Event Logs.
Incident Before Splunk After Splunk
No. of FTE 5 1
No. of NON-FTE 5 2
Duration(hrs) 12 2
Cost of FTE $6,000 $200
Cost of NON-FTE $2,400 $160
Billed $8,400 $360
Avg incidents per Day 2
Avg incidents per Week 12
Billed Weekly $100,800 $4,320
Billed Annually $5,040,000 $216,000
These are not accounting numbers. They purely represent scale based on
full commitment. The numbers before Splunk are most accurate. The
others are projections.

13. Why Splunk?
• Brings everything into one place
• Understand the problem across the
whole application, not on a specific
machine or a specific technology
• More cost effective problem
resolutions – smaller teams and less
reliance on senior resources.
• Apply big data concepts to all that
supplemental information that aids
in determining the health of your
application or enterprise.
13

14. Residual Advantages of Splunk
1
Availability
Resource
Allocation
OnBoarding
Cross
Training
Analytics

15. 1
.conf2014
MGM Grand – Las Vegas