A review of when and how to build a scorecard for key metrics such as the output Y for improvements that were piloted or implemented.

1. Section & Lesson #:
Pre-Requisite Lessons:
Complex Tools + Clear Teaching = Powerful Results
Building a Scorecard
Six Sigma-Control – Lesson 2
A review of when and how to build a scorecard for key metrics such as the
output Y for improvements that were piloted or implemented.
Six Sigma-Improve #09 – Piloting Solutions: Build the Pilot Plan
Copyright © 2011-2019 by Matthew J. Hansen. All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted by any means
(electronic, mechanical, photographic, photocopying, recording or otherwise) without prior permission in writing by the author and/or publisher.

2. Scorecards Defined
o What is a project scorecard?
• A report that measures the critical metric(s) for piloted and/or implemented improvements.
 It usually includes only one or two metrics – typically the output (Y) of the process being improved.
 It often includes the targets (e.g., goals or LSL/USL) for what’s being measured.
 It generally tracks the metric over time (trending) to monitor its progress.
o How is a scorecard different from a dashboard?
• A dashboard is generally a snapshot report of multiple metrics at a higher level of the business.
 Below is a general comparison between a scorecard and dashboard:
Copyright © 2011-2019 by Matthew J. Hansen. All Rights Reserved. No part of this publication may be
reproduced, stored in a retrieval system, or transmitted by any means (electronic, mechanical, photographic,
photocopying, recording or otherwise) without prior permission in writing by the author and/or publisher.
2
Characteristic Dashboard Scorecard
# of Metrics Reported 4 or more 1 to 3
Method to Display Metric Snapshot Trending
Organizational Level High (executive level) Low (manager or process level)
Inclusion of targets? Sometimes Yes
30
35
40
45
50
55
1/1/2011
1/2/2011
1/3/2011
1/4/2011
1/5/2011
1/6/2011
1/7/2011
1/8/2011
1/9/2011
1/10/2011
1/11/2011
1/12/2011
1/13/2011
1/14/2011
1/15/2011
Avg Speed
AvgSpeed
Target
Dashboard: Scorecard:
A dashboard will
snapshot multiple
metrics such as speed…
…a scorecard may
trend only one metric
like average speed.
FuelE
F
Temp
C
H
MPH0
10
20
30
40 50 60
70
80
90

3. How to Build a Scorecard
o How do you build a scorecard?
• The general process for building a scorecard can be defined as follows:
o Is a scorecard required when piloting/implementing improvements?
• No, but there are some potential risks to consider when not using a scorecard:
 Inaccuracy – a scorecard standardizes how the critical metrics are measured and reported. Without it,
some team members may pull the wrong metrics and make wrong conclusions.
 Miscommunication – a scorecard unifies the team’s understanding of the progress and status of the
improvements being measured and their relationship to the defined targets.
 Duplicate Reporting – a scorecard lets the team depend on one resource to create the scorecard rather
than each person duplicating work by attempting to pull the critical metrics themselves.
• If the above risks are low, then a scorecard may not be necessary
 For example, if there is a small team and the critical metrics are already standardized and separately
reported from a single, reliable source, then a scorecard may not be necessary.
Copyright © 2011-2019 by Matthew J. Hansen. All Rights Reserved. No part of this publication may be
reproduced, stored in a retrieval system, or transmitted by any means (electronic, mechanical, photographic,
photocopying, recording or otherwise) without prior permission in writing by the author and/or publisher.
3
1. Identify
critical metrics
and targets
2. Identify
trending
duration
3. Build
scorecard
4. Acquire
team approval
5. Define
maintenance
process
INPUT to Scorecard PROCESS of building Scorecard OUTPUT from Scorecard

4. Building a Scorecard: Steps 1 to 3
o INPUT to Scorecard
• #1 – Identify critical metrics and targets
 The critical metrics should at least include the primary output (Y) that the improvements are targeting.
– Other critical metrics can be included, but there shouldn’t be too many (aim for no more than 3 metrics).
 The metric targets are the business goals for the metrics.
– These will generally be reflected as a lower or upper specification limit (a.k.a., LSL or USL).
– If the business has goals defined for the critical metrics, then be sure those are included. If the project has a different set
of goals for these metrics, then it’s OK to include both the business and project targets as long as they don’t conflict.
• #2 – Identify trending duration
 Basically, this answers how far back you want to track the metrics on the scorecard.
– A good rule of thumb is to include about 8 to 12 periods of metrics on the report.
– The periods will depend on the frequency of the reported metrics. For example…
» For metrics reported monthly, include about 12 months worth (i.e., one year’s worth)
» For metrics reported weekly, include about 12 weeks (i.e., about 3 months worth)
– It’s acceptable to include more than 12 periods if the type of metrics allows for reporting more than 12.
» Just as with a dashboard, the goal is to limit the scorecard to one page.
o PROCESS of building Scorecard
• #3 – Build scorecard
 Believe it or not, this is the easy part. It’s a matter of designing and formatting the metrics.
– Acquiring the data to include in a scorecard is generally the hardest part.
 The scorecard should at a minimum include a visual graphic
– This is typically a line graph reflecting the actual metric(s) and respective targets.
– The detailed data can be included if space allows, but the scorecard will more often be driven by the embedded charts.
 Build a draft layout of the scorecard including dummy data for the team to approve the design.
 Validate the data reflected in the scorecard as you build it.
– Since the scorecard is generally derived by referencing detail from another source, be sure it’s pulling that data correctly.
Copyright © 2011-2019 by Matthew J. Hansen. All Rights Reserved. No part of this publication may be
reproduced, stored in a retrieval system, or transmitted by any means (electronic, mechanical, photographic,
photocopying, recording or otherwise) without prior permission in writing by the author and/or publisher.
4

5. Building a Scorecard: Steps 4 & 5
o PROCESS of building Scorecard (continued)
• #4 – Acquire team approval
 As noted previously, the team should approve the initial draft design of the scorecard.
 In addition, the team must agree on the final scorecard design once real data is included.
 Be certain the entire team agrees or you may find yourself doing a lot of re-designing.
o OUTPUT from Scorecard
• #5 – Define Maintenance Process
 Now having a completed and approved scorecard, you need answer the following:
– How long will the scorecard need to be maintained?
– Who will perform the on-going creation of the scorecard?
– Who will be responsible for communicating the scorecard content and any future design changes?
 Typically, the customer is the one who will maintain the scorecard.
– It’s not common for the project leader (e.g., a Black Belt) to perform this work; it’s usually the Process Owner or a SME.
 Before handing-off the scorecard for someone else to create/manage, ensure it is fully documented.
– The process for creating the scorecard should be provided in simple, yet detailed steps.
– This implies that the process should be simplified as much as possible (or risk being called on frequently for assistance).
» Use simple formulas and references in the scorecard to the source data.
» Use formatting that allows the user to easily import the new data and automatically update the scorecard.
» Use macros and pivot tables/charts that can compile multiple or complex steps.
Copyright © 2011-2019 by Matthew J. Hansen. All Rights Reserved. No part of this publication may be
reproduced, stored in a retrieval system, or transmitted by any means (electronic, mechanical, photographic,
photocopying, recording or otherwise) without prior permission in writing by the author and/or publisher.
5

6. Scorecard Example
o Below is a fictitious scorecard used for tracking quality and efficiency in Billing:
Copyright © 2011-2019 by Matthew J. Hansen. All Rights Reserved. No part of this publication may be
reproduced, stored in a retrieval system, or transmitted by any means (electronic, mechanical, photographic,
photocopying, recording or otherwise) without prior permission in writing by the author and/or publisher.
6
0.00
0.50
1.00
1.50
2.00
2.50
3.00
96.00%
96.50%
97.00%
97.50%
98.00%
98.50%
99.00%
99.50%
100.00%
Jan
2012
Feb
2012
Mar
2012
Apr
2012
May
2012
Jun
2012
Jul
2012
Aug
2012
Sep
2012
Oct
2012
Nov
2012
Dec
2012
CycleTime(CT)inDays
Quality&EfficiencyRates
2012 Billing Performance(with trending)
Billing Quality
Billing Efficiency
Billing CT (days)
Jan 2012 Feb 2012 Mar 2012 Apr 2012 May 2012 Jun 2012 Jul 2012 Aug 2012 Sep 2012 Oct 2012 Nov 2012 Dec 2012
Bills Processed 4,258,654 4,471,587 4,248,007 4,460,408 4,371,200 4,458,624 4,681,555 4,634,739 4,542,044 4,678,306 4,771,872 4,485,560
Effectiveness:
Billing Errors 68915 74578 78855 70882 67116 69181 65828 73969 80969 73794 69132 64099
Billing Quality 98.38% 98.33% 98.14% 98.41% 98.46% 98.45% 98.59% 98.40% 98.22% 98.42% 98.55% 98.57%
Efficiency:
Billing Delays 15645 29638 19546 18025 24794 17948 11264 16625 20615 12675 18649 11088
Billing Efficiency 99.63% 99.34% 99.54% 99.60% 99.43% 99.60% 99.76% 99.64% 99.55% 99.73% 99.61% 99.75%
Billing CT (days) 2.30 2.30 2.40 2.30 2.20 2.20 2.00 2.10 2.00 1.90 2.00 1.90
Metric
This scorecard has only a
few key metrics that are
trended over time.
Visually displaying the metrics is
always ideal. Only plot critical
metrics and use creativity (like
coloring) to simplify it.

7. Practical Application
o Identify at least 2 different projects or initiatives at your organization that involved the
implementation of a significant process change and answer the following questions:
• Was a scorecard used for tracking the critical metrics related to the process change?
 If not, then why not?
– What problems (if any) were encountered that could have been avoided if a scorecard was used and closely monitored?
– Were there any delays for finding any critical failures that would’ve been discovered in a scorecard?
 If a scorecard was used, then what problems were avoided or mitigated due to using a scorecard?
– In what ways (if any) did the scorecard not help avoid or mitigate any problems? What can be done differently next time?
Copyright © 2011-2019 by Matthew J. Hansen. All Rights Reserved. No part of this publication may be
reproduced, stored in a retrieval system, or transmitted by any means (electronic, mechanical, photographic,
photocopying, recording or otherwise) without prior permission in writing by the author and/or publisher.
7