Copyright	©	2016	Splunk	Inc.	
T-Mobile	
Nathan	Haynie		
Principal	Engineer,
Operational Intelligence
2	
About	T-Mobile	
•  Founded	in	1999	
•  Headquartered	in	Bellevue,	WA	
•  Fastest	growing	wireless	company	in	America	
•  1-2+	million	new	customers	every	quarter,	12	
consecuNve	quarters.	
•  45,000	employees	
•  Recognized	“Best	Place	to	Work”	year	aUer	year
3	
About	Me	
•  Business	&	op	intelligence	-	12	years	
•  Splunk	customer	-	7	years	
•  With	T-Mobile	-	2	years	
•  Product	owner	for	Splunk
4	
EvoluNon	of	Splunk	@	T-Mobile	
Gen	1	
Gen	2	
Gen	3	
+	extend	service	assurance	tool	suite	
+	increased	capacity	
+	high-profile	use	cases	
+	increased	capacity	
+	Nered	service	
+	geo	capabiliNes	
+	mission-criNcal	posture	
+	widely	diverse	use	cases	
+	systemaNc	integraNon		
				with	other	tools	
+	development	to	expand	
				self-serve	capabiliNes
5
Splunk	as	a	Service
7	
Splunk	Service	Portal	(Gen	2)	
  Service	Request	instrucNons	
  Access	Request	instrucNons	
  Training	links	and	informaNon	
  Support	informaNon	
  Forwarder	install	instrucNons	with	pre-configured	packages	
–  packages	are	configured	for	different	networks	and	deployment	servers
8	
Splunk	Service	Catalog	Items	
  Self-serve	search	interface	
  Self-serve	report,	dashboard,	app	
development	
  Self-serve	REST	API	interface	
  Self-serve	simple	alerNng	
  Self-serve	pre-configured	
forwarder	packages	
  Managed	data	onboarding	
  Managed	report/dashboard/apps	
  Managed	advanced	alerNng	
  Managed	system	integraNon	w/	
3rd	party	tools	
  Advanced	UI	component	
development	
  Complex	Event	Processing	
(inbound	mediaNon	layer)
9	
Common	QuesNons	
  What	is	Splunk?	
  How	is	Splunk	different	than	<tool>?	
  Why	are	we	replacing	<tool>	with	Splunk?	
  Why	do	we	need	Splunk	AND	<tool>?
10	
Key	AcNviNes	of	any	Intelligence	PracNce	
T-Mobile	Internal	Use	Only	 10	5/4/16	
Instrument	 Collect	 Process	 Store	 Model	
Publish/	
Consume	
Splunk	>	
Viz	Tools	
TradiNonal	ETL	 SQL-DB	 Cubes	 ReporNng
11	
Reference	framework	for	deeper	conversaNons
Splunk	and	ITSM
13	
Service	Providers	>	Services
14	
AcNviNes	>	Processes	>	Service	Deliberables
15	
Example	Service	Provider:	IT
16	
Example:	Incident	Management	AcNviNes	
NoNce	
•  Certain	acNviNes	are	more	data-centric	than	others	
•  Data-centric	acNviNes	are	where	Splunk	makes	biggest	impact
17	
Strategy	Maps	to	show	Value	Streams
18	
How	event	data	flows	(representaNve	sample)	
Event	
Mgmt
Fault	
Mgmt
Incident	
Mgmt
Problem	
Mgmt
Change	
Mgmt
Performance
Mgmt
Capacity	
Mgmt
Service	Level	
Mgmt
Logs Polling Receiver
Event	
Mgmt
Cost	
Mgmt
Demand	
Mgmt
Security
PCI
19	
SERVICE	DESIGN	
		Service	Catalog	Mgmt	
		Service	Level	Mgmt	
		Availability	Mgmt	
		Capacity	Mgmt	
SERVICE	TRANSITION	
		Change	Mgmt	
		Release	&	Deployment	
		Service	TesNng	&	ValidaNon	
		ConfiguraNon	Mgmt		
SERVICE	OPERATIONS	
		Incident	Mgmt	
		Problem	Mgmt	
		Event/Fault	Mgmt	
	
SERVICE	STRATEGY	
		Demand	Mgmt	
		Financial	&	Cost	Mgmt	
CONTINUAL	SERVICE	IMPROVEMENT	
		Service	Measurement	&	ReporNng	
		Service	Improvement
20	
-  Over-reliance	on	tribal	knowledge		
-  Burned	out	heroes	holding	up	the	world	
-  Different	people,	same	process,	same	results	
-  Tracking	in	place	to	provide	numbers	
-  Learning	from	numbers	to	understand	dynamics	
-  Numbers	and	process	understood	
-  Performance	metrics	acNonable	
Maturity	model	 -  Tuning	
-  AutomaNon
21	
-  Over-reliance	on	tribal	knowledge		
-  Burned	out	heroes	holding	up	the	world	
-  Different	people,	same	process,	same	results	
-  Tracking	in	place	to	provide	numbers	
-  Learning	from	numbers	to	understand	dynamics	
-  Numbers	and	process	understood	
-  Performance	metrics	acNonable	
Maturity	model	 -  Tuning	
-  AutomaNon	
Note:	As	a	process	matures,	data	and	
tools	are	needed	for	different	reasons.	
Explore,	learn,	educate,	convince	
Simplify	steps,	verify	intended	results,…	
Measure,	track,	correlate,	noOfy,…	
Analyze,	test	acOons,	predict,…	
Raise	events,	track	acOons…
22	
Maturity	model	
Tip:	PosiNon	Splunk	as	a		
significant	accelerator		
for	your	key	processes
Splunk	Program	Management
24	
Service	Reviews	
AGENDA	
  Availability	&	Performance	Review	
  Demand	Review	
  Capacity	Review	
  Roadmaps	
–  Customer	
–  Infrastructure	
–  Service	improvement	
  Q&A
25	
Program	Maturity	(sample	assessment	report)	
Ad-hoc	 Repeatable	 Measured	 Managed	 OpOmizing	
Incident	Mgmt	
Problem	Mgmt	
Change	Mgmt	
Config	Mgmt	
Request	Fulfill.	
Project	Mgmt	
Service	Improv.	
Q2	<-	we	are	here	 intermediate	goal	 Q1	 target	goal	
Q3	
Q3	
Q3	
Q3	
Q3	
Q3	
Q3	Q2	
Q2	
Q2	
Q2	
Q2
26	
Sophisticated	
capabilities
Enterprise	
driven
Silo	driven
Basic	
capabilities
Bottom	up
approach
A
B
Top	down	
approach
A.	Top	down	approach
				-	More	collaborative	&	correlation,	wider	audience
				-	Can	tend	to	suffer	from	paralysis	by	analysis
B.	Bottom	up	approach
				-	Relative	quick	time	to	value
				-	Right	hand	vs.	left	hand	building	train	wrecks	
						may	never	overcome	or	expensive	to	rework
Balanced	iteraNve	approach
27	
Sophisticated	
capabilities
Enterprise	
driven
Silo	driven
Basic	
capabilities
Bottom	up
approach
A
B
C
Top	down	
approach
A.	Top	down	approach
				-	More	collaborative	&	correlation,	wider	audience
				-	Can	tend	to	suffer	from	paralysis	by	analysis
B.	Bottom	up	approach
				-	Relative	quick	time	to	value
				-	Right	hand	vs.	left	hand	building	train	wrecks	
						may	never	overcome	or	expensive	to	rework
C.	Balanced	iterative	approach
				-	Best	of	both	approaches
							-	Establish	top-down	framework
							-	Work	on	highly	targeted	needs	of	silos	to	
										provide	immediate	value	in	each	iteration		
										working	within	framework
							-	Extend	framework	incrementally	as	required
				-	Requires	strong	vision	and	technical	voice	to	
							maintain	reasonable	balance	throughout.
Balanced	iteraNve	approach
Next	Local	Searle	Splunk	Users	Group	
hrp://www.meetup.com/Splunk-Searle/		
	
Date:	June	2nd	
Time:	5:30	
ITSI	in	AcOon,	Andy	Koo,	T-Mobile	
The	Purple	Cafe	
430	106th	Avenue	NE	
Bellevue,	WA	98004
Thank	You

T-Mobile Customer Presentation

Editor's Notes

  • #19 The same event data can be used in many different contexts Purpose-built tools often collect just enough event-based info to support it’s use cases but often ignores others A good event management strategy will consolidate event management where possible for cross-functional processes and correlation Splunk can be the de-facto event management, correlation, and dissemination tool though it is wise to compliment it with other tools for specific use cases.