SlideShare a Scribd company logo
1 of 52
Download to read offline
Benchmark	&	Metrics	
Yuta	Imai
Agenda	
1.  Metrics	
2.  Benchmark
Cita:ons	
•  This	slide	deck	is	based	on	the	stories	what	
Robert	Barnes	told	us	at	his	AWS	:me.	
hCps://www.youtube.com/watch?v=jffB30FRmlY
Why	benchmark?	
•  How	long	will	the	current	configura:on	be	adequate?	
•  Will	this	plaSorm	provide	adequate	performance,	now	and	in	the	
future?	
•  For	a	specific	workload,	how	does	one	plaSorm	compare	to	
another?		
•  What	configura:on	will	it	take	to	meet	current	needs?	
•  What	size	instance	will	provide	the	best	cost/performance	for	my	
applica:on?	
•  Are	the	changes	being	made	to	a	system	going	to	have	the	
intended	impact	on	the	system?
Agenda	
1.  Metrics	
2.  Benchmark
Metrics	
•  To	measure/benchmark	system	performance	
or	business,	what	to	monitor	is	so	important.	
•  Does	that	metrics	describe	your	challenge	
well?	
•  Is	that	metrics	difficult	to	hack?
Business?
Sample	case1:		
Metrics	to	monitor	the	business	
•  If	you	want	to	monitor	how	the	business	is	
going	on,	which	metrics	do	you	monitor??	
hCp://www.slideshare.net/TokorotenNakayama/dau-21559783
Customer	Experience?
Sample	case2:	
Metrics	to	monitor	customer	experience	
•  If	you	want	to	monitor	how	good	is	the	
customer	experience,	which	metrics	do	you	
monitor??
Percen:le
Percen:le	
•  Amazon	heavily	relies	on	“Percen:le”.	
•  Percen:le:	
– Describes	user/customer	experience	directly.	
	 99.9%	=	42ms
Percen:le	
•  Amazon	heavily	relies	on	“Percen:le”.	
•  Percen:le:	
– Describes	user/customer	experience	directly.	
	
samples=1,000	
It	means	999	queries	has	been	finished	in	42ms.	
99.9%	=	42ms
Percen:le	
•  If	you	pick	average	for	your	SLA,	it	does	not	
describe	customer’s	experience.	
99.9%	=	42ms	
Average=29ms	
In	such	standard	distribu:on,	
Average	might	be	OK	but…
Percen:le	
99.9%	
=46ms	
99.5%	
=44ms	
•  Even	if	such	form	of	histogram,	percen:le	can	
properly	describe	customer	experience.	
99%	
=41ms
Percen:le	
99.9%	=	50ms	
Average=31ms	
•  If	you	pick	average,	it	does	not	describe	
customer’s	experience.	
In	such	distribu:on,	
Average	does	not	work	well
Percen:le	
99.9%	
=45ms	
99.5%	
=42ms	
•  Percen:le	is	good	for	SLA	decision	in	business	
because	it	well	describes	customer’s	
experience.	
99%	
=40ms
Percen:le	
99.9%	
=45ms	
99.5%	
=42ms	
•  Percen:le	is	good	for	SLA	decision	in	business	
because	it	well	describes	customer’s	
experience.	
99%	
=40ms
Percen:le	
99.9%	
=45ms	
99.5%	
=42ms	
•  Percen:le	is	good	for	SLA	decision	in	business	
because	it	well	describes	customer’s	
experience.	
99%	
=40ms	
OK,	let’s	set	business	SLA	to	
40ms	in	99.9%
99.9%	
=45ms	
99.5%	
=42ms	
99%	
=40ms	
99.9%	
=40ms	
If	you	want	to	provide	40ms	or	lower	
latencies	in	99.9%	of	query…	
	
Then	you	will	have	to	move	
distribu:on	lel.	
AS-IS	
TO-BE
Percen:le	
•  Percen:le	is	also	good	for	service	level	
monitoring.	
4/1	
99.9%	=	42ms
Percen:le	
•  Percen:le	is	also	good	for	service	level	
monitoring.	
4/1	
99.9%	=	42ms	
4/7	
99.9%	=	44ms
Percen:le	
•  Percen:le	is	also	good	for	service	level	
monitoring.	
4/1	
99.9%	=	42ms	
4/7	
99.9%	=	44ms	
4/14	
99.9%	=	46ms
Percen:le	
•  Percen:le	is	also	good	for	service	level	
monitoring.	
4/1	
99.9%	=	42ms	
4/7	
99.9%	=	44ms	
4/14	
99.9%	=	46ms	
Throughput	increased?	
Data	volume	increased?	
	
Let’s	start	inves:ga:on.
Metrics:	Summary	
•  Choose	metrics	well	describe	your	challenge.	
•  Choose	NOT	hack-able	metrics!
Agenda	
1.  Metrics	
2.  Benchmark
The	Benchmark	Lifecycle	
Test	Design	
Test	
Analysis	
Measure	
against	goal	
Report	
Test	
Configura:on	
Start	with	a	Goal	
Carefully	
control	
changes	
Test	
Execu:on	
Run	a	series	of	
controlled	
experiments	
Design	your	
workload	
Build	
Environment	
Generate	
Load
The	Benchmark	Lifecycle	
Test	Design	
Test	
Analysis	
Measure	
against	goal	
Report	
Test	
Configura:on	
Start	with	a	Goal	
Carefully	
control	
changes	
Test	
Execu:on	
Run	a	series	of	
controlled	
experiments	
Design	your	
workload	
Build	
Environment	
Generate	
Load
First…	
•  What	is	“OK”?	
– “Faster”	means	“Infinite”.	
•  Choose	your	benchmark.	
– Your	applica:on	is	the	best	benchmark	tool.
Ensure	your	design	works	if	scale	changes	by	10X	or	
20X	but	the	right	solu:on	for	X	olen	not	op:mal	for	
100X	
	
Jeff	Dean,	Google	
The	hints	for	define	“OK”
Sacrificial	Architecture	
	
Essen:ally	it	means	accep:ng	now	that	in	a	few	years	:me	
you’ll	(hopefully)	need	to	throw	away	what	you’re	currently	
building.	
	
Mar:n	Fowler	
The	hints	for	define	“OK”
Set	performance	targets	
Target:	Achieve	adequate	performance	
•  If	no	target	exists	
–  Use	current	performance	
–  Run	experiments	to	define	baseline	
–  Copy	from	someone	else	
–  Guess	
•  Why	set	performance	targets?	
–  To	know	when	you	are	done	
–  Target	met	or	:me	to	rewrite…
Example:	Set	performance	targets	
Total	users:	10,000,000	
Request	rate:	1,000	RPS	
Peak	rate:	5,000	RPS	
Concurrent	users:	10,000	
Peak	users:	50,000	
	
Transac'on	 Mix	
ra'o	
95%
(msec)	
New	user	sign-up	 5%	 1500	
Sign-in	 25%	 1250	
Catalog	search	 50%	 1000	
Order	item	 10%	 1500	
Check	order	status	 10%	 1000
Choose	your	workloads	
•  Select	features	
–  Most	important	
–  Most	popular	
–  Highest	complaints	
–  “Worst”	performing	
•  Define	the	workload	mix	
–  Ra:o	of	features	
–  Typical	“uesrs”	and	what	they	do	
–  Popula:on	and	distribu:on	of	users	
•  Random(even	distribu:on)	
•  Hotspots
3	ways	to	use	benchmark	
1.  Run	a	benchmark	using	your	exis:ng	
applica:on	and	workloads	
2.  Run	a	standard	benchmark	
3.  Use	published	benchmark	results
1.	Use	your	exis:ng	applica:on	
•  Choose	which	part	of	the	applica:on	
•  Determine	how	to	generate	load	
•  Decide	how	to	measure	and	what	metrics	
•  Design	how	reports	get	generated
2.	Run	a	standard	benchmark	
•  Is	the	test	relevant	to	your	requirements?	
•  How	does	the	test	map	to	your	applica:on?	
•  Be	aware	of	most	of	them	are	micro-bench.
When	you	cant’	use	your	applica:on,	standard	
benchmarks	can	help	
•  Standard	benchmarks	s:ll	leave	work	to	be	done:	
–  Tuning	needed	
–  Automa:on	and	test	execu:on	
–  How	are	they	test	results	relevant?	
–  How	is	this	test	implementa:on	relevant?	
•  Examples	and	:ps	referencing	standard	benchmarks	
are	not	endorsements	of	these	benchmarks		
2.	Run	a	standard	benchmark
3.	Use	published	benchmark	results	
•  What	is	being	measured?	
•  Why	is	it	being	measured?	
•  How	is	it	being	measured?	
•  How	closely	does	this	benchmark	resemble	my	
results?	
•  How	accurate	are	the	reports	and	cita:ons?	
•  Are	the	results	repeatable?
Tip:	The	4	Rs	
•  Relevant	
–  the	best	test	is	based	on	your	applica:on	
•  Recent	
–  Out	of	date	results	are	rarely	useful	
•  Repeatable	
–  Is	there	enough	informa:on	to	repeat	test?	
•  Reliable	
–  Do	you	trust	the	tools,	the	publisher	and	the	results?
The	Benchmark	Lifecycle	
Test	Design	
Test	
Analysis	
Measure	
against	goal	
Report	
Test	
Configura:on	
Start	with	a	Goal	
Carefully	
control	
changes	
Test	
Execu:on	
Run	a	series	of	
controlled	
experiments	
Design	your	
workload	
Build	
Environment	
Generate	
Load
How	to	generate	load	
•  Humans(Don’t	use	human,	if	you	want	repeatable	and	
reproducible	one)	
–  “Record/Playback”	traffic	
–  Volunteers	
–  Mechanical	Turk	
•  Synthe:c	load	
–  Open	source	
–  Commercial	
•  SOASTA,	Neustar,	Gomez,	Keynote	
–  Write	your	own…
How	to	measure	
•  Load	generator	metrics	
•  Applica:on	metrics(end	to	end)	
•  Add	instrumenta:on	
•  Stopwatch	
•  Use	log	files	
–  Note	that	emiung	lot	of	log	will	introduce	another	
workload.
Tips:	End-to-end	tes:ng	
•  You	need	to	understand	and	trust	the	tests	
–  Some:mes	tools(clients)	have	boClenecks	
•  Use	realis:c	data	
–  Scale	
–  Distribu:on	
•  Use	ramp-up,	steady-state,	and	ramp-down	
•  Choose	reasonable	test	dura:on	
–  Use	scale	down	environment	for	longer	test.	For	something	like	Like	
SLA	proof	tests.	
•  Run	mul:ple	tests	and	calculate	variability
Finding	boClenecks	
•  Search	metrics	and	and	logs	for	clues	
•  If	there	aren’t	any,	add	instrumenta:on	
•  Isolate	and	individually	test	services	and	infrastructure	
•  Test	“categories”	
–  Business	logic	
–  Presenta:on	
–  Compute	
–  Memory	
–  Disk	I/O	
–  Network	
–  Database	
–  Other	services
Cloud:	the	good	tool	for	benchmark	
•  Benchmark	is	not	easy	because	building	up	
and	tearing	down	test	configura:ons	can	be	
very	labor	intensive	
•  Benchmarking	in	cloud	is	fast	with	parallel	
execu:on,	affordable(pay	as	you	go),	scalable	
and	can	be	automated!
The	Benchmark	Lifecycle	
Test	Design	
Test	
Analysis	
Measure	
against	goal	
Report	
Test	
Configura:on	
Start	with	a	Goal	
Carefully	
control	
changes	
Test	
Execu:on	
Run	a	series	of	
controlled	
experiments	
Design	your	
workload	
Build	
Environment	
Generate	
Load
In	my	experience	
•  I	had	to	run	Sysbench	to	find	CPU/Memory/IO	
performances	are	consistent	in	each	Amazon	
EC2	instance	type.	
•  I	spun	up	60	instances	for	each	instance	type	
and	ran	Sysbench….	
•  Of	cource	automa:cally.
To	automate	perf	tests…	
Result_Value1	 Result_Value2	 Result_Value3	 Result_Value4	 Result_Value5	
Condi:on1	
Condi:on2	
Condi:on3	
Condi:on4	
Condi:on5	
•  Create	output/report	format	first.	
•  Then	write	a	script	to	run	tests	like…
Automate	end-to-end	
foreach	my	$pram	(@condi:ons){	
	write_report(run_ec2(	
	 	$param{instance_type},	
	 	$param{image_id},	
	 	$param{script_to_run}	
	));	
}
API	
Gateway	
Slack	
Lambda	
ECS	
Lambda	 S3	
Aurora	
Outgoing	Webhook	
-  cluster	name	
-  #	of	tasks	
-  commands	
RunTasks	
-  cluster	name	
-  #	of	tasks	
-  commands	as	environment	variables	
-  output	loca:on	
Output	STDOUT	as	file	
Spin	up	containers	and	run	tasks	
Incoming	Webhook	
-  Read	file	from	S3	and	emit	it	to	Slack	
Automated	distributed	Sysbench	to	Amazon	Aurora
Benchmark:	Summary	
•  Goal?	
•  Workload?	
•  Load	generator?	Environment?	
•  Make	the	list	of	all	of	tests	
•  Run(and	automate!)

More Related Content

What's hot

Infosys – Quality Management Software Manufacturing | White Paper
Infosys – Quality Management Software Manufacturing | White PaperInfosys – Quality Management Software Manufacturing | White Paper
Infosys – Quality Management Software Manufacturing | White PaperInfosys
 
Chapter 8 strategic evaluation and control
Chapter 8 strategic evaluation and controlChapter 8 strategic evaluation and control
Chapter 8 strategic evaluation and controlRoshan Pant
 
Business Performance Management
Business Performance ManagementBusiness Performance Management
Business Performance ManagementSeta Wicaksana
 
Using the balanced scorecard as a strategic management
Using the balanced scorecard as a strategic managementUsing the balanced scorecard as a strategic management
Using the balanced scorecard as a strategic managementTahia
 
Introduction of Global Strategic Management
Introduction of Global Strategic ManagementIntroduction of Global Strategic Management
Introduction of Global Strategic ManagementSamita Mahapatra
 
Strategic Leadership and Management
Strategic Leadership and ManagementStrategic Leadership and Management
Strategic Leadership and ManagementCharles Cotter, PhD
 
Strategic+management+2
Strategic+management+2Strategic+management+2
Strategic+management+2Sampath
 
The Balance Scorecard
The Balance ScorecardThe Balance Scorecard
The Balance ScorecardPreet Gill
 
Strategy Review, Evaluation, and Control
Strategy Review, Evaluation, and ControlStrategy Review, Evaluation, and Control
Strategy Review, Evaluation, and ControlNoel Buensuceso
 
The vrio framework in English
The vrio framework in EnglishThe vrio framework in English
The vrio framework in EnglishYesica Adicondro
 
Learning Organization Governance for Top Performers
Learning Organization Governance for Top PerformersLearning Organization Governance for Top Performers
Learning Organization Governance for Top PerformersCorpU
 

What's hot (20)

Infosys – Quality Management Software Manufacturing | White Paper
Infosys – Quality Management Software Manufacturing | White PaperInfosys – Quality Management Software Manufacturing | White Paper
Infosys – Quality Management Software Manufacturing | White Paper
 
How ERP works
How ERP worksHow ERP works
How ERP works
 
Chapter 8 strategic evaluation and control
Chapter 8 strategic evaluation and controlChapter 8 strategic evaluation and control
Chapter 8 strategic evaluation and control
 
Core Competencies
Core CompetenciesCore Competencies
Core Competencies
 
Business Performance Management
Business Performance ManagementBusiness Performance Management
Business Performance Management
 
Using the balanced scorecard as a strategic management
Using the balanced scorecard as a strategic managementUsing the balanced scorecard as a strategic management
Using the balanced scorecard as a strategic management
 
Introduction of Global Strategic Management
Introduction of Global Strategic ManagementIntroduction of Global Strategic Management
Introduction of Global Strategic Management
 
Turnaround Strategies
Turnaround StrategiesTurnaround Strategies
Turnaround Strategies
 
Amazon
AmazonAmazon
Amazon
 
Alibaba group
Alibaba groupAlibaba group
Alibaba group
 
CSf presentation 5
CSf presentation 5CSf presentation 5
CSf presentation 5
 
Strategic Leadership and Management
Strategic Leadership and ManagementStrategic Leadership and Management
Strategic Leadership and Management
 
Strategic management ppt
Strategic management pptStrategic management ppt
Strategic management ppt
 
Strategic+management+2
Strategic+management+2Strategic+management+2
Strategic+management+2
 
Amazon business model
Amazon business modelAmazon business model
Amazon business model
 
The Balance Scorecard
The Balance ScorecardThe Balance Scorecard
The Balance Scorecard
 
Strategy Review, Evaluation, and Control
Strategy Review, Evaluation, and ControlStrategy Review, Evaluation, and Control
Strategy Review, Evaluation, and Control
 
The vrio framework in English
The vrio framework in EnglishThe vrio framework in English
The vrio framework in English
 
Strategic Analysis
Strategic AnalysisStrategic Analysis
Strategic Analysis
 
Learning Organization Governance for Top Performers
Learning Organization Governance for Top PerformersLearning Organization Governance for Top Performers
Learning Organization Governance for Top Performers
 

Viewers also liked

Dynamic Resource Allocation in Apache Spark
Dynamic Resource Allocation in Apache SparkDynamic Resource Allocation in Apache Spark
Dynamic Resource Allocation in Apache SparkYuta Imai
 
Spark at Scale
Spark at ScaleSpark at Scale
Spark at ScaleYuta Imai
 
Deep Learning On Apache Spark
Deep Learning On Apache SparkDeep Learning On Apache Spark
Deep Learning On Apache SparkYuta Imai
 
Global Gaming On AWS
Global Gaming On AWSGlobal Gaming On AWS
Global Gaming On AWSYuta Imai
 
Hadoop in adtech
Hadoop in adtechHadoop in adtech
Hadoop in adtechYuta Imai
 
Hadoop最新事情とHortonworks Data Platform
Hadoop最新事情とHortonworks Data PlatformHadoop最新事情とHortonworks Data Platform
Hadoop最新事情とHortonworks Data PlatformYuta Imai
 
Hadoop/Spark セルフサービス系の事例まとめ
Hadoop/Spark セルフサービス系の事例まとめHadoop/Spark セルフサービス系の事例まとめ
Hadoop/Spark セルフサービス系の事例まとめYuta Imai
 
Apache Hiveの今とこれから - 2016
Apache Hiveの今とこれから - 2016Apache Hiveの今とこれから - 2016
Apache Hiveの今とこれから - 2016Yuta Imai
 
Apache ambari
Apache ambariApache ambari
Apache ambariYuta Imai
 
IoTアプリケーションで利用するApache NiFi
IoTアプリケーションで利用するApache NiFiIoTアプリケーションで利用するApache NiFi
IoTアプリケーションで利用するApache NiFiYuta Imai
 
Hadoop and Kerberos
Hadoop and KerberosHadoop and Kerberos
Hadoop and KerberosYuta Imai
 
OLAP options on Hadoop
OLAP options on HadoopOLAP options on Hadoop
OLAP options on HadoopYuta Imai
 
HDP2.5 Updates
HDP2.5 UpdatesHDP2.5 Updates
HDP2.5 UpdatesYuta Imai
 
TPC TC And TPC-Energy Slide Deck 5.4.09
TPC TC And TPC-Energy Slide Deck 5.4.09TPC TC And TPC-Energy Slide Deck 5.4.09
TPC TC And TPC-Energy Slide Deck 5.4.09forrestcarman
 
Hive - Apache hadoop Bigdata training by Desing Pathshala
Hive - Apache hadoop Bigdata training by Desing PathshalaHive - Apache hadoop Bigdata training by Desing Pathshala
Hive - Apache hadoop Bigdata training by Desing PathshalaDesing Pathshala
 
Hadoop Summit Amsterdam 2014: Capacity Planning In Multi-tenant Hadoop Deploy...
Hadoop Summit Amsterdam 2014: Capacity Planning In Multi-tenant Hadoop Deploy...Hadoop Summit Amsterdam 2014: Capacity Planning In Multi-tenant Hadoop Deploy...
Hadoop Summit Amsterdam 2014: Capacity Planning In Multi-tenant Hadoop Deploy...Sumeet Singh
 
Scaling Spark Workloads on YARN - Boulder/Denver July 2015
Scaling Spark Workloads on YARN - Boulder/Denver July 2015Scaling Spark Workloads on YARN - Boulder/Denver July 2015
Scaling Spark Workloads on YARN - Boulder/Denver July 2015Mac Moore
 
Dynamic Allocation in Spark
Dynamic Allocation in SparkDynamic Allocation in Spark
Dynamic Allocation in SparkDatabricks
 
Capacity Management and BigData/Hadoop - Hitchhiker's guide for the Capacity ...
Capacity Management and BigData/Hadoop - Hitchhiker's guide for the Capacity ...Capacity Management and BigData/Hadoop - Hitchhiker's guide for the Capacity ...
Capacity Management and BigData/Hadoop - Hitchhiker's guide for the Capacity ...Renato Bonomini
 

Viewers also liked (20)

Dynamic Resource Allocation in Apache Spark
Dynamic Resource Allocation in Apache SparkDynamic Resource Allocation in Apache Spark
Dynamic Resource Allocation in Apache Spark
 
Spark at Scale
Spark at ScaleSpark at Scale
Spark at Scale
 
Deep Learning On Apache Spark
Deep Learning On Apache SparkDeep Learning On Apache Spark
Deep Learning On Apache Spark
 
Global Gaming On AWS
Global Gaming On AWSGlobal Gaming On AWS
Global Gaming On AWS
 
Hadoop in adtech
Hadoop in adtechHadoop in adtech
Hadoop in adtech
 
Hadoop最新事情とHortonworks Data Platform
Hadoop最新事情とHortonworks Data PlatformHadoop最新事情とHortonworks Data Platform
Hadoop最新事情とHortonworks Data Platform
 
Hadoop/Spark セルフサービス系の事例まとめ
Hadoop/Spark セルフサービス系の事例まとめHadoop/Spark セルフサービス系の事例まとめ
Hadoop/Spark セルフサービス系の事例まとめ
 
Apache Hiveの今とこれから - 2016
Apache Hiveの今とこれから - 2016Apache Hiveの今とこれから - 2016
Apache Hiveの今とこれから - 2016
 
Apache ambari
Apache ambariApache ambari
Apache ambari
 
IoTアプリケーションで利用するApache NiFi
IoTアプリケーションで利用するApache NiFiIoTアプリケーションで利用するApache NiFi
IoTアプリケーションで利用するApache NiFi
 
Hadoop and Kerberos
Hadoop and KerberosHadoop and Kerberos
Hadoop and Kerberos
 
OLAP options on Hadoop
OLAP options on HadoopOLAP options on Hadoop
OLAP options on Hadoop
 
HDP2.5 Updates
HDP2.5 UpdatesHDP2.5 Updates
HDP2.5 Updates
 
Benchmarking
BenchmarkingBenchmarking
Benchmarking
 
TPC TC And TPC-Energy Slide Deck 5.4.09
TPC TC And TPC-Energy Slide Deck 5.4.09TPC TC And TPC-Energy Slide Deck 5.4.09
TPC TC And TPC-Energy Slide Deck 5.4.09
 
Hive - Apache hadoop Bigdata training by Desing Pathshala
Hive - Apache hadoop Bigdata training by Desing PathshalaHive - Apache hadoop Bigdata training by Desing Pathshala
Hive - Apache hadoop Bigdata training by Desing Pathshala
 
Hadoop Summit Amsterdam 2014: Capacity Planning In Multi-tenant Hadoop Deploy...
Hadoop Summit Amsterdam 2014: Capacity Planning In Multi-tenant Hadoop Deploy...Hadoop Summit Amsterdam 2014: Capacity Planning In Multi-tenant Hadoop Deploy...
Hadoop Summit Amsterdam 2014: Capacity Planning In Multi-tenant Hadoop Deploy...
 
Scaling Spark Workloads on YARN - Boulder/Denver July 2015
Scaling Spark Workloads on YARN - Boulder/Denver July 2015Scaling Spark Workloads on YARN - Boulder/Denver July 2015
Scaling Spark Workloads on YARN - Boulder/Denver July 2015
 
Dynamic Allocation in Spark
Dynamic Allocation in SparkDynamic Allocation in Spark
Dynamic Allocation in Spark
 
Capacity Management and BigData/Hadoop - Hitchhiker's guide for the Capacity ...
Capacity Management and BigData/Hadoop - Hitchhiker's guide for the Capacity ...Capacity Management and BigData/Hadoop - Hitchhiker's guide for the Capacity ...
Capacity Management and BigData/Hadoop - Hitchhiker's guide for the Capacity ...
 

Similar to Benchmark and Metrics

Measuring Business Analyst Impact
Measuring Business Analyst ImpactMeasuring Business Analyst Impact
Measuring Business Analyst ImpactASPE, Inc.
 
Porfolio Management in TFS 2013
Porfolio Management in TFS 2013Porfolio Management in TFS 2013
Porfolio Management in TFS 2013Gian Maria Ricci
 
Continuous Performance Testing: The New Standard
Continuous Performance Testing: The New StandardContinuous Performance Testing: The New Standard
Continuous Performance Testing: The New StandardTechWell
 
Facility Management Metrics That Matter
Facility Management Metrics That MatterFacility Management Metrics That Matter
Facility Management Metrics That MatterRobert Lambe, CFM
 
Awesome CMS! Implementing Configuration Management to Maximise Value #LEADit
Awesome CMS! Implementing Configuration Management to Maximise Value #LEADitAwesome CMS! Implementing Configuration Management to Maximise Value #LEADit
Awesome CMS! Implementing Configuration Management to Maximise Value #LEADitAwesome CMS
 
The Good, The Bad, and The Metrics
 The Good, The Bad, and The Metrics The Good, The Bad, and The Metrics
The Good, The Bad, and The MetricsTeamQualityPro
 
Simplifying the Complexity of Salesforce CPQ: Tips & Best Practices
Simplifying the Complexity of Salesforce CPQ: Tips & Best PracticesSimplifying the Complexity of Salesforce CPQ: Tips & Best Practices
Simplifying the Complexity of Salesforce CPQ: Tips & Best Practicespanayaofficial
 
Agile for product owners v12
Agile for product owners  v12Agile for product owners  v12
Agile for product owners v12Ravi Tadwalkar
 
Critical steps in Determining Your Value Stream Management Solution
Critical steps in Determining Your Value Stream Management SolutionCritical steps in Determining Your Value Stream Management Solution
Critical steps in Determining Your Value Stream Management SolutionDevOps.com
 
10 Best Practices for Magento Maintenance and Support
10 Best Practices for Magento Maintenance and Support10 Best Practices for Magento Maintenance and Support
10 Best Practices for Magento Maintenance and SupportAPPSeCONNECT
 
MVP Process Automation Showdown by Chris Edwards, Jennifer Lee, Michael Gill ...
MVP Process Automation Showdown by Chris Edwards, Jennifer Lee, Michael Gill ...MVP Process Automation Showdown by Chris Edwards, Jennifer Lee, Michael Gill ...
MVP Process Automation Showdown by Chris Edwards, Jennifer Lee, Michael Gill ...Salesforce Admins
 
Launching Successful Applications
Launching Successful ApplicationsLaunching Successful Applications
Launching Successful ApplicationsChris Chew
 
Improving Speed to Market in E-commerce
Improving Speed to Market in E-commerceImproving Speed to Market in E-commerce
Improving Speed to Market in E-commerceCognizant
 
story points v2
story points v2story points v2
story points v2Jane Yip
 
Top 10 Agile Metrics
Top 10 Agile MetricsTop 10 Agile Metrics
Top 10 Agile MetricsXBOSoft
 
Agile overview class for scrum masters
Agile overview class for scrum mastersAgile overview class for scrum masters
Agile overview class for scrum mastersBhawani N Prasad
 

Similar to Benchmark and Metrics (20)

Measuring Business Analyst Impact
Measuring Business Analyst ImpactMeasuring Business Analyst Impact
Measuring Business Analyst Impact
 
PQF Overview
PQF OverviewPQF Overview
PQF Overview
 
Porfolio Management in TFS 2013
Porfolio Management in TFS 2013Porfolio Management in TFS 2013
Porfolio Management in TFS 2013
 
Continuous Performance Testing: The New Standard
Continuous Performance Testing: The New StandardContinuous Performance Testing: The New Standard
Continuous Performance Testing: The New Standard
 
Facility Management Metrics That Matter
Facility Management Metrics That MatterFacility Management Metrics That Matter
Facility Management Metrics That Matter
 
Benchmarking
BenchmarkingBenchmarking
Benchmarking
 
Awesome CMS! Implementing Configuration Management to Maximise Value #LEADit
Awesome CMS! Implementing Configuration Management to Maximise Value #LEADitAwesome CMS! Implementing Configuration Management to Maximise Value #LEADit
Awesome CMS! Implementing Configuration Management to Maximise Value #LEADit
 
The Good, The Bad, and The Metrics
 The Good, The Bad, and The Metrics The Good, The Bad, and The Metrics
The Good, The Bad, and The Metrics
 
Simplifying the Complexity of Salesforce CPQ: Tips & Best Practices
Simplifying the Complexity of Salesforce CPQ: Tips & Best PracticesSimplifying the Complexity of Salesforce CPQ: Tips & Best Practices
Simplifying the Complexity of Salesforce CPQ: Tips & Best Practices
 
Telecom testing
Telecom testingTelecom testing
Telecom testing
 
Agile for product owners v12
Agile for product owners  v12Agile for product owners  v12
Agile for product owners v12
 
Critical steps in Determining Your Value Stream Management Solution
Critical steps in Determining Your Value Stream Management SolutionCritical steps in Determining Your Value Stream Management Solution
Critical steps in Determining Your Value Stream Management Solution
 
Magento maintenance
Magento maintenanceMagento maintenance
Magento maintenance
 
10 Best Practices for Magento Maintenance and Support
10 Best Practices for Magento Maintenance and Support10 Best Practices for Magento Maintenance and Support
10 Best Practices for Magento Maintenance and Support
 
MVP Process Automation Showdown by Chris Edwards, Jennifer Lee, Michael Gill ...
MVP Process Automation Showdown by Chris Edwards, Jennifer Lee, Michael Gill ...MVP Process Automation Showdown by Chris Edwards, Jennifer Lee, Michael Gill ...
MVP Process Automation Showdown by Chris Edwards, Jennifer Lee, Michael Gill ...
 
Launching Successful Applications
Launching Successful ApplicationsLaunching Successful Applications
Launching Successful Applications
 
Improving Speed to Market in E-commerce
Improving Speed to Market in E-commerceImproving Speed to Market in E-commerce
Improving Speed to Market in E-commerce
 
story points v2
story points v2story points v2
story points v2
 
Top 10 Agile Metrics
Top 10 Agile MetricsTop 10 Agile Metrics
Top 10 Agile Metrics
 
Agile overview class for scrum masters
Agile overview class for scrum mastersAgile overview class for scrum masters
Agile overview class for scrum masters
 

More from Yuta Imai

Node-RED on device to Apache NiFi on cloud, via SORACOM Canal, with no Internet
Node-RED on device to Apache NiFi on cloud, via SORACOM Canal, with no InternetNode-RED on device to Apache NiFi on cloud, via SORACOM Canal, with no Internet
Node-RED on device to Apache NiFi on cloud, via SORACOM Canal, with no InternetYuta Imai
 
Spark Streaming + Amazon Kinesis
Spark Streaming + Amazon KinesisSpark Streaming + Amazon Kinesis
Spark Streaming + Amazon KinesisYuta Imai
 
オンラインゲームの仕組みと工夫
オンラインゲームの仕組みと工夫オンラインゲームの仕組みと工夫
オンラインゲームの仕組みと工夫Yuta Imai
 
Amazon Machine Learning
Amazon Machine LearningAmazon Machine Learning
Amazon Machine LearningYuta Imai
 
Digital marketing on AWS
Digital marketing on AWSDigital marketing on AWS
Digital marketing on AWSYuta Imai
 
EC2のストレージどう使う? -Instance Storageを理解して高速IOを上手に活用!-
EC2のストレージどう使う? -Instance Storageを理解して高速IOを上手に活用!-EC2のストレージどう使う? -Instance Storageを理解して高速IOを上手に活用!-
EC2のストレージどう使う? -Instance Storageを理解して高速IOを上手に活用!-Yuta Imai
 
クラウドネイティブなアーキテクチャでサクサク解析
クラウドネイティブなアーキテクチャでサクサク解析クラウドネイティブなアーキテクチャでサクサク解析
クラウドネイティブなアーキテクチャでサクサク解析Yuta Imai
 
CloudFront経由でのCORS利用
CloudFront経由でのCORS利用CloudFront経由でのCORS利用
CloudFront経由でのCORS利用Yuta Imai
 

More from Yuta Imai (8)

Node-RED on device to Apache NiFi on cloud, via SORACOM Canal, with no Internet
Node-RED on device to Apache NiFi on cloud, via SORACOM Canal, with no InternetNode-RED on device to Apache NiFi on cloud, via SORACOM Canal, with no Internet
Node-RED on device to Apache NiFi on cloud, via SORACOM Canal, with no Internet
 
Spark Streaming + Amazon Kinesis
Spark Streaming + Amazon KinesisSpark Streaming + Amazon Kinesis
Spark Streaming + Amazon Kinesis
 
オンラインゲームの仕組みと工夫
オンラインゲームの仕組みと工夫オンラインゲームの仕組みと工夫
オンラインゲームの仕組みと工夫
 
Amazon Machine Learning
Amazon Machine LearningAmazon Machine Learning
Amazon Machine Learning
 
Digital marketing on AWS
Digital marketing on AWSDigital marketing on AWS
Digital marketing on AWS
 
EC2のストレージどう使う? -Instance Storageを理解して高速IOを上手に活用!-
EC2のストレージどう使う? -Instance Storageを理解して高速IOを上手に活用!-EC2のストレージどう使う? -Instance Storageを理解して高速IOを上手に活用!-
EC2のストレージどう使う? -Instance Storageを理解して高速IOを上手に活用!-
 
クラウドネイティブなアーキテクチャでサクサク解析
クラウドネイティブなアーキテクチャでサクサク解析クラウドネイティブなアーキテクチャでサクサク解析
クラウドネイティブなアーキテクチャでサクサク解析
 
CloudFront経由でのCORS利用
CloudFront経由でのCORS利用CloudFront経由でのCORS利用
CloudFront経由でのCORS利用
 

Recently uploaded

Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Servicegiselly40
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 

Recently uploaded (20)

Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 

Benchmark and Metrics