Innovation Benefits Realization for Industrial Research (Part-8)

Technology Innovation Management
Framework for Industrial Research
Part-8
Dr. Iain Sanders

January 2005

WHAT R&D METRICS?

Platform 3:
(Stages VIII, Part 3)

Integrating Technology Innovation with Business Function – Part I: Laying the Foundation

PLATFORM 3: VIII (3)

What are our R&D Metrics?

 R&D Metrics
 Set standards for measuring new concept R&D performance
and evaluating progress / outcomes achieved against
milestones / targets set.

6

Top 39 Ranking R&D metrics
11.5 Strategic Alignment 5.5 Development Cycle Time 3 Product Quality and Reliability

11.5 a. Corporate & business unit 5.5 a. Market cycle time 3 a. Customer Evaluation
11.5 b. Goal coverage 5.5 b. Project management cycle 3 b. Reliability/Defects
time
10 Financial Return 5.5 Customer Satisfaction 3 Market Share
10 a. New Sales Ratio 5 Number of Ways Technology is 2.3 Development Pipeline Milestone
Exploited Achieved
10 b. Cost Savings Ratio 5 Projects with Business/ 2 Goal Clarity
Marketing Approval
10 c. R&D Yield 5 a. Project has approval 1.8 Comparative Manufacturing Cost
10 d. R&D Return 5 b. Ratio of projects 1.8 Gross Margin
9 Projected Value of the R&D 3.3 Quality of Personnel 1.8 Use of Cross-Functional Teams
Pipeline
9 a. Projected Sales 3.3 a. Internal Customer Rating 1.8 Rating of Technology Features
and Benefits
9 b. Projected Income 3.3 b. External Customer Rating 1.8 Response Time to Competitors
moves
7.8 Distribution of Technology 3.3 c. External Recognition 1.5 Comparative Technology
Investment Investment
6.8 Use of Project Milestone/ 3.3 Efficiency of Internal Technical 1.5 Employee Morale
Stage-Gate Process Processes
7


Strategic Alignment Metric (11.5)

 Definition
 This R&D metric assess the degree of alignment of an R&D project or an
R&D portfolio with the strategic intents of the company or a division of the
company. The strategic intents are often the corporate goals embodied in
its business plan.

 Why it is used
 This metric is used to gauge the degree of relevance of the R&D program
to the corporate goals. The strategic intents of a company may change
more rapidly than the R&D program can respond to those changes
creating various degrees of misalignment. The misalignment can be with
regard to work area, long term versus short term needs, or degree of risk.

8



 How to use metric
 There are a number of ways this metric can be applied, including both prospective and
retrospective views. It can be applied by R&D management, general management, or by both
working in partnership. An alignment index would be assigned to each project; a linear scale
of 1-5, for example, would work well. these scores may also have value when considering
relative merit of individual projects in the portfolio. A composite score for the entire portfolio
would then be determined. This could be a weighted average reflecting sizes of projects with
regard to technical head count, project budget or some other appropriate weighting factor.
Once a baseline for alignment has been established, R&D management can then decide if
and how this index should migrate to greater or lesser degrees of alignment through
modifications of the portfolio. Applied to individual projects, there could be a cut-off point for
the alignment parameter below which projects are not supported.

 Consideration must be given to the degree of alignment desired. Although the desired state
in may cases is toward greater degrees of alignment, you can envision situations where that
may not be the case. A research organization charged with taking the company in new
directions may not want its project portfolio highly aligned with the current business plan. A
more visionary business plan may capture new directions as well as current businesses, but
many organizations find that some degree of decoupling is desirable.

9



 Options
 Both retrospective and prospective views are options of this metric. The
retrospective view entails applying the metric to an existing portfolio of
R&D projects to determine the degree of the portfolio with corporate goals.

 The prospective view is to apply the metric to a proposed project or slate
of projects. If the R&D organization is trying to increase its alignment
index, then the management will be less likely to initiate projects that move
the composite score in the wrong direction. Similarly, an alignment index
cut-off may be instituted. Projects falling below some minimum value of
alignment would not be supported.

10


Financial Return (10)
 New Sales Ratio (a)
 The New Sales Ratio is the % of current sales originating from new products.
There are two sub-definitions that are required. What is a new product? and, How
old is new?

 The most frequently used and simplest definition of a new product is any new SKU
(inventory code: Stock Keeping Unit) that has required R&D support to implement.
This avoids counting new SKUs which are only packaging changes or other
modifications made easily by marketing or manufacturing. When is a new product
old is different for each business and technology. In rapidly changing and evolving
fields, such as electronic chips and software, new might only be one year, but
certainly not more than three. Three years is more likely the norm for businesses
that are a mix of fashion and formulation, such as cosmetics & toiletries. And, for
more intense capital and industrial products, three to seven is a more likely range
to select a number that is right for you.

11


 New Cost Savings Ratio (b)
The Cost Savings Ratio is the % reduction in cost of goods or cost of operations
(including depreciation charges) that are realized in a year to year comparison that
have originated from technology changes that are new. Again, the same issues must be
resolved as sub-definitions as required for the New Sales Ratio. In other words, what is
being attributed to R&D and how long is new?

 Since SKUs are not used to catalogue changes made in operations an alternative must
be found that will work for your firm. A simple solution can often be created based on
the capital approval process. Most companies require a specific approval for all
individual capital projects. These can be coded and tracked for R&D involvement and
for realized cost savings. However, it is recognized that the accounting involved in
examining the impacts on cost savings may be more difficult than that for new sales. It
is therefore a more common metric only in those cases which are more significantly
impacted by cost savings than by new product sales. And, since new technology for
operations or manufacturing has a different useful life than a new product per se, it
must be tracked for a different length of time that is industry specific. In some cases, it
may be very linked to the product life in other cases, it may go on much longer. In any
case, it is not likely that 'new' will reach beyond 7-10 years.

12


 R&D Yield (c)
 R&D Yield is the contribution of R&D to current financial performance. It is a metric
that is composed of definitions from New Sales Ratio and Cost Savings Ratio, plus
an evaluation of gross profit from the new sales.

 It is the annual combined financial benefit that is derived from the annual gross
profit of new products and the annual cost savings of new processes. This is the
current contribution that the company receives that is associated with its past
stream of R&D investments, i.e. the part of the 'bottom line' that is relatively 'new'
and derived from R&D.

13


 R&D Return (d)
 R&D Return is the relative ROI measure that relates to R&D. It is composed of the
R&D Yield divided by the annual investment in R&D. Hopefully, this is a large
number that is proportional to the risks and variances that are part of R&D.

14


 Advantages and Limitations
 The advantages of these financial metrics are that they relate directly to the financial
benefits to the company, they are quantitative and they are comparable to metrics that
can be used in different parts of the same firm or between firms. They capture the
degree to which R&D is truly making a financial contribution to the value of the
enterprise. They answer the question: What have you [R&D] done for me [the
Business] lately? However, they only represent the tip of a process that takes place
over a number of years and that involves other functions besides R&D. This means that
the numbers reflected by these metrics are associated with activities that are in the
past. These metrics are lagging indicators. They are a nice track record, but they may
not be reflecting accurately a current level of effectiveness.

15


 How to Use the Metric
 The metrics should be tracked at least on a once a year basis. Because of measurement and definition
problems, a baseline of two years or more of historical data is needed before accurate judgments can be
made about trends and ratio efficiencies.

 The metrics should be examined carefully for consistency with business strategies and the results required
vs. the investments in R&D. In situations where the metrics, requirements and available resources are not
in balance, there will be a difficulty in executing the overall business and technology strategies. One or the
other must be shifted, and variations in how R&D is conducted need to be examined.

 If the Financial Return metrics are being maintained or going up: the corporation has the likely raw material
to extend a technology-based or innovation-based growth program; the investors have the possibility of an
extended stream of positive returns from the accumulation of financial pay-offs from technology-based
innovations; and the R&D units enjoy the likelihood of consistent funding to reinvest in various aspects of
technology application for the near term and base building for the future.

 The key words here are likely or possible. Positive Financial Returns are a necessary but not sufficient
condition for growth. It is also only a measure of the moment, whether it is looking to the past or to the
future. And, any downward movements will predict the difficulties the business will have in achieving solid
gains against the competition. These indicators are crucial to assessing the total returns from R&D
investments, whether enough is being spent on R&D, and what is the likely future value of the company
from a technology perspective.

16



 Options and Variations
 There is always something new and innovative that is contributing to revenue or profit.
These financial return metrics are intended to capture the new portion of these changes
in the business that are related to R&D. They require definitions of what is to be
considered new and for how long. These elements can then be tracked separately or
together, in ratio or absolute form, by themselves as benefits or as an investment return
vs R&D. The options and variations fall into place based on each company's views of
these items.

 The most common variations are based on the length of time that is new. the most
frequent categories are three years, five years and seven years. Another variation is to
use these same metrics in a prospective, future mode.

17


Projected Value of the R&D Pipeline (9)

 Projected Sales (a)
 Projected Sales is the calculated sum of future sales from
current R&D projects. This metric may be expressed in
absolute terms or a % of future sales.

 Definitions must be provided for how this is to be evaluated.
Normally, it is the forward side of the newness range, i.e. if
new products are those introduced within five years, projected
sales will be calculated for five years after commercialization.
A probability of attainment is usually figured into this metric.

18



 Projected Income (b)
 Projected Income is the income stream associated with the
Projected Sales. Similar definitions apply. This metric may be
expressed as an absolute number or as a fraction of net
income.

19



 This metric provides an ongoing anticipation of the expected
results from R&D. Because it is projected, it provides an
evaluation of the benefits that are being created with today's
R&D investments.

 The limitations are due to the intrinsic difficulties of obtaining
estimates about the likely commercial benefits if the
technology is successful.

20


 How to use the Metric
 This is perhaps the single most important and least used metrics. It is the singular
indication of the future business that is to be developed as a result of successful
outcomes from R&D. Are the sales impacts large enough, are there enough new
products, is the timing of elements in the pipeline adequate, are the overall returns
related to R&D adequate? Are these metrics on a year to year basis showing
constancy, increase or decline?

 This metric provides ongoing guidance to the company regarding the future gain to
be expected in the business due to R&D. This should be used as a check that both
the strategy and the resource allocations are correct.

 If this metric is staying constant or increasing, particularly with respect to the R&D
resources, then the effectiveness of R&D is being maintained or increasing.

 If, alternatively, this metric is declining, then further diagnostics should be
examined to understand the reasons and to take corrective action.
21



 Common options are few because this is an underused
metric. One is to look at absolute sales or net income over a
five year horizon on the presumption that some projects will
finish and be productive within the next two-three years and
that others will impact a bit later, i.e. in the fourth and fifth
year. A related option is not to adjust the commercial impact
by any probabilities.

22


Distribution of Technology
Investment (7.8)
 Definition
 This metric provides a means of assessing how well an R&D
program is protecting the technology investment and technical
position of the company. It forces consideration of how the technical
assets should be distributed, setting directions for modifying the R&D
portfolio.

 The portfolio of an R&D organization may not be protecting the
strategic interests of the company for any number of reasons such as
skill set mismatch, slow response to changes in the company‟s
mission and markets, and a rapidly changing competitive
environment. There can be an over-emphasis on certain business
units and products. This metric causes the management to examine
how well the R&D effort is protecting and expanding the technical
position of the company in areas of greatest importance.
23


Investment (7.8)
 This metric is applied by first determining how the technology investment should be distributed. As an
example, consider a company with six business units. The R&D portfolio can be distributed among these
six business according to a number of models. Six examples of distribution models are listed below:

 The revenue that each business generates.
 The opportunity market share (potential market growth).
 The impact that technology can make in the different business units.
 Competitive Impact (Base, Key, Pacing)
 Some combination of the above distribution models.
 The profitability that each business demonstrates.

 These considerations often involve the concept of the technological basis of competition, that is how does
technology provide a sustainable competitive advantage in a particular product or market. The discussion
could also consider the distribution of the competitive impact of the company‟s technology investment by
categorizing them as Base, Key or Pacing . Base technologies are essential to the business but widely
exploited by competitors. Key technologies are highly differentiating to the company‟s current products.
Pacing technologies are new technologies where the competitive impact is less certain but likely to be
high. All three of the impact categories require protection of the competitive position, but the distribution of
resources among the three categories may vary based on the company‟s business plans.

24


Investment (7.8)

 How to Use the Metric (continued)
 Once a distribution model has been agreed upon, the portfolio is measured against that model. The
distribution could be with regard to number of R&D projects in each segment of the distribution, head count
devoted to each segment, or expected value of projects in each segment. Modifications are made to the
portfolio to move the distribution toward the desired state.

25


Investment (7.8)
 Options
 This Both prospective and retrospective views are supported by this
metric. In the retrospective view, the R&D projects are categorized
according to the corporate investments or markets they are intended
to protect or create, or the competitive impact that they offer. This
does not have to address the entire R&D effort of the company, since
the metric can be applied to any subset of the portfolio. The projects
are appropriately weighted to reflect their size and cost. A distribution
of the R&D efforts supporting each of the categories is determined.
This current state distribution is then compared with the desired
state. The degree of misappropriation can then be quantified as the
fraction or percentage of the R&D effort that is improperly distributed.

 The prospective view for this metric involves consideration of how a
proposed project shifts the distribution of technology investment
toward or away from the desired state.
26


Use of Project Milestones /
Stage-Gate Process (6.8)
 Definition
 Percent of projects in the total portfolio going through a
defined project management system with defined milestones.

 Percent of R&D expenditure on projects using a defined
project management system with defined milestones.

27


 Percent Project management systems including milestones can provide a way of reducing
cycle time and providing R&D and business management with a sense of the health of
projects. These systems also can improve the linkage between R&D, marketing and business
management by enabling coordination with manufacturing and marketing to hit key windows.
When these systems include documentation through status reports, they can be used to
promote organizational learning. Driving behaviors which increase the numerical value of this
metric should therefore improve both the linkage of R&D to the business and improve the
effectiveness of R&D.

 When used with a formal stage-gate process this metric provides a measure of compliance
with that system. Since companies will generally use a defined project management system
and establish milestones in the later phases of innovation, this metric may also be an
indicator of the distribution of projects in the innovation pipeline (see metric: "Distribution of
Technology Investment" ).

28


 Advantages and Limitations (continued)
 The metric may be limited by the difficulty of counting projects outside the project
management system. Further since project management systems may not be appropriate
early in the innovation process, the ideal value for this metric will depend on the firm's
desired balance of early and late stage projects. For short term projects such as minor
product or process variations, use of formal project management systems and this metric
may create unnecessary red tape and potential delays.

29


 How to Use the metric
 As a concurrent metric, the total number of projects with defined (written) project plans
including definite milestone dates can be divided by the total number of identifiable R&D
projects to calculate the metric.

 When used with the appropriate accounting system this metric can be calculated from the
cost of projects divided by the total R&D cost. In this case the budgeted projects should be
audited to determine compliance with requirements for plans and milestones.

 The metric can be used as a concurrent metric (a snapshot of the current R&D activities) or
as a retrospective measure to determine how many R&D projects used a defined process. It
should be equally suitable for service and manufacturing companies.

30


Development Cycle Time (5.5)
 Market Cycle Time
 This metric measures the elapsed time from identification of a customer
product need until commercial sales commence.

 Project Management Cycle Time
 This metric measures the elapsed time from establishment of a discrete
project to address an identified customer product need until commercial
sales commence.

 For both metrics described, the end point can be time when
manufacturing feasibility is established for those cases where
no commercialization occurs. Compare to historical values and
benchmark vs. competition, if possible. Group by categories of
projects (e.g. major new product, minor product variation, etc.)
Can also be used to track milestone attainment rate for firms
using a stage gate management process.
31


 Advantages and Limitations (Market Cycle Time)
 The advantages of this measure is that it is quantitative and can be used to measure the
entire process or various parts of the process if stage gates are examined. The process can
be analyzed to determine what parts are driving the overall cycle time so that improvements
to the process can be made.

 The limitations of this metric could include R&D's position that it does not adequately
influence the process until after a need has been more defined. Another limitation is that a
strong documentation system is helpful to make the cycle time metric as accurate as
possible. An additional limitation is that defining the commencement of sales as the end of
the cycle does not account for post start-up issues such as efficiency, waste, % of
manufactured product within specification, etc.; this could lead to focusing on shortening the
cycle time at the expense of later, non-measured parts of the cycle. One must also keep in
mind that for breakthrough or paradigm shifting projects,, cycle time measurements. The
advantage of this metric is that it supports having the clarity of when a project is initiated
based on approvals, assignment of resources, start of spending, etc. The metric can be used
to measure the entire process or various parts of the process if stage gates are examined.
The process can be analyzed to determine what parts are driving the overall cycle time so
that improvements to the process can be made.

32


 How to Use this Metric
 Both metrics should preferably be used in combination with a project reporting system that
can track the project initiation date (based on approval and assignment of resources), the
length of time in each stage gate of the innovation process, and date of sales
commencement. For the first metric, the initiation date could be the date the customer need
was determined (i.e. marketing request or date of customer research results). Cycle times for
different types of projects (new products, cost savings, product improvements, etc.) should
be compared to help predict and manage resource allocation. Cycle times for different
divisions could also be compared (with caution) to identify practices driving lower cycle time
to adapt where possible.

 Variations could exist regarding looking at cycle times for only certain parts of the process for
which R&D feels it has most control or influence. Cycle time could be extended past the
commencement of sales based on what is important to the organization and R&D's
involvement, i.e. when target efficiency is achieved, target manufacturing cost, % of product
within spec, etc.

33


Customer Satisfaction (5.5)

 Definition
 The customer satisfaction metric has two variations:

1. Measures of external (end-customer) satisfaction. These may be
such metrics as ratings of quality of technical personnel or
technical capabilities, or technology benefits within products or
processes.
2. Internal customer satisfaction. Since the immediate customer of
R&D is normally the businesses within the corporation that R&D
serves, measures such as customer satisfaction in
engineering, marketing, or manufacturing may be appropriate.
Typical metrics might include on-time technology
delivery, competitiveness or appropriateness of the technology
solutions delivered, and overall satisfaction with the R&D track
record of technological support.

34


 There are few disadvantages to good customer satisfaction metrics. In
the case of external, or end-customer satisfaction metrics, one
complication may be that the entire innovation cycle is under review by
the end customer. A bad grade by the customer, while a valid rating of
the corporate innovation process, may not be merely an indictment of the
R&D operation, but a judgment of the overall product development
process within the company, involving manufacturing and product
engineering, market forecasts, consumer needs and attitudes, and
competence of corporate management. One the other hand, a well-
thought-out customer satisfaction metric (or set of metrics) for the
internal or immediate customer within the corporation -- normally the
corporate businesses and their various R&D-related organizations -- may
be the key diagnostic to indicate that R&D processes are lacking and
need adjustment or redesign. The R&D organization itself is probably
better served with well thought-out internal metrics than with external
metrics that complicate the diagnostic process for R&D when problems
are indicated.
35


 External customer satisfaction metrics. These metrics will normally be marketing-
related or implemented. One variation would be to use a marketing survey, in
which various aspects of technology benefits are rated on a five -point scale

 There are two dimensions for internal customer satisfaction metrics: strategic and
tactical.

 Strategic metrics deal specifically with whether the R&D function is meeting the
strategic needs of the customer. The review process might involve matching
technology and product roadmaps in a joint meeting, in which technology timing
mismatches are resolved. Information can be exchanged; technology previews by
R&D to alert the businesses to possible market-creating or market share
increasing discontinuities, and the businesses to share future market window and
product definitions with R&D. Various metrics can be used, including the five-point
rating system mentioned above, or a metric which highlights number or percent of
mismatches between product and technology roadmaps.

36


 Tactical metrics deal with whether specific projects are meeting the goal or delivery
requirements of the internal customer base. For technology projects in the latter
stages of development (near to or entering product development), regular project
reviews with the intended customer(s) is important. At quarterly or perhaps
semiannual reviews, customer and R&D representatives join in a review of project
progress. A useful metric in this case is a report card which each customer
representative is required to fill out in the review meeting. This is a very simple
questionnaire which has 3-5 survey questions on project progress, and suitability
of both the project and the technological approach to satisfy the customer(s)
needs. Typical questions might be:

 Does this project meet your product technology needs?
 Do project milestone dates meet your market window?
 Have any strategy changes on your part not been addressed?
 What is the overall project score (typical scale 1-5)?
 As a related metric, trend analyses can be made both by project and organizationally.

37


 One variation on the customer satisfaction process is to establish steering teams
to address technology, business and market issues and provide guidance to the
R&D organization on strategic issues. For external issues, the teams might consist
of focus groups that meet periodically, or focus teams that convene groups which
are diverse either geographically, ethnically, or with respect to age group, for
example. The focus might be on functional needs that technology capabilities
address. The metric would be meetings held, or issues addressed and settled.
Another metric might be problems surfaced to be addressed and reported on by
the R&D team.

 Internal issues might be addressed by customer teams composed of
manufacturing, engineering, marketing, financial, and related personnel (including
even external consultants) who provide guidance and assist in forming cross-
functional project teams. Appropriate metrics are meetings held, issues settled, or
problems surfaced. An ongoing metric can also keep track of the % of problems
addressed and resolved versus those surfaced.

38


Number of Ways Technology is
Exploited (5)
 Definition
 This metric assess the number of ways a technical asset can
bring value to the corporation.

 This metric is applied to gauge project attractiveness, or to
understand the value of a technical asset already developed.
It is generally agreed that a larger number of potential
uses, both within the company's current markets and in
markets not yet developed, makes a technical asset more
valuable. The metric is a bit arbitrary and can be misleading in
cases where there are few, though very large and/or lucrative
commercializations.

39


Exploited (5)
 How to Apply the Metric
 This metric is applied by taking an existing or potential
technical asset, such as a project to develop a new type of
lower cost, light weight composite material, and conducting a
thoughtful analysis of how many ways this asset can be
exploited commercially. The count could consider:

 Number of business units in the corporation that could make
use of the asset
 Number of markets the company serves that could be
impacted by this technology
 Total number of markets served by the corporation and other
companies where the technology may have an impact.
 Number of products that could utilize the technical asset.
40


Exploited (5)
 How to Apply the Metric (continued)
 This Used in this way, the metric is a single numerical value. A larger number of
potential uses means that the corporation is not depending on a single or small
number of products to succeed in order for the technical asset to deliver value. The
risks associated with the exploitation of the technical asset are spread over a
larger number of potential uses. A larger number also provides greater opportunity
for unforeseen benefits, like taking the company into new markets and new
products.

 Using the low cost, light weight composites as an example, the primary market for
the company may be the automotive market, with four different auto parts that
could use the strong, light weight tubes produced by the new process. In
addition, there could be a market for the technology in the aeronautical
industry, served by another business unit of the company. The third exploitation
could be in high performance bicycle frames, a market that is new to the company.

41


Exploited (5)
 Options
 An option in the implementation of this metric can involve adjusting the number for
relative importance of the commercializations, or keeping sub-metrics of the
number of exploitations with certain value ranges as demonstrated with the
continuing example of the light weight composites.

42


Projects with Business /
Marketing Approval (5)

 Definition
 Percent of projects in the total R&D portfolio with explicit
business unit and or corporate business management sign-off.

 The intent of this metric is to provide an indicator of the degree
of alignment with business and corporate strategy and tactics.
The metric is closely related in some corporate structures to
metric: "percent Funding by the Business“.

43


 Advantages: Several studies have suggested that close alignment of R&D to marketing and
to business and corporate strategies increases the odds of success for new products and
processes. Thus actions which drive this metric to higher values can be expected to improve
the amount of R&D spent on successful projects and the predictability of the outcome from
R&D efforts.

 Limitations: The metric will be valuable to the extent business/marketing management and
R&D management jointly develop strategy and plans. Use of the metric to drive R&D without
such teamwork will likely lead to short term projects and suboptimal use of R&D resources. In
those companies where R&D is corporately funded, business/marketing management may
also be tempted to give approval to projects in their market segments to insure that they
receive "their share" of R&D resources. Finally if the corporation uses a formal innovation
process which requires business/marketing approval at some stage, the metric runs the risk
of becoming a measure of compliance with use of the process or a measure of the percent of
project past the approval stage.

44


 Explicit approval may be sought at any point in the innovation process.
Seeking approval early in the innovation process probably provides
maximum value. One form of approval is the provision of a sales forecast
from marketing management for each new product and agreement to
commercialize if the product meets technical requirements in a timely
manner.

 The level of approval from the marketing/business management and the
point where approval should be sought should be explicitly defined if the
firm uses a formal innovation process. If not, the level should be
commensurate with the amount of R&D resources and commercialization
resources which will be required.
45


 For projects having broad corporate strategic value, approval of a director
of corporate planning or director of corporate business development might
be an appropriate substitute for the business/marketing management
approval. For corporations where out-licensing of technology is a major
thrust, approval of a director or vice president of licensing may be an
appropriate substitution.

46


Quality of Personnel (3.3)
 Definition
 This is a measurement of the skills and ability of the R&D staff to execute strategic programs.

 Internal Customer Ratings (a)
 Internal customers rate the quality of the R&D staff on their ability to execute programs.
Measures such as percentage of mileposts met versus project plans, novelty of
concepts, patentability of concepts, and competitive advantage of the technology are
parameters that can be considered.

 External Customer Ratings (b)
 External customers rate the quality of the R&D staff of their ability to meet customer
expectations. Problem solving, novelty of approach, responsiveness, knowledge of
customer's operations are parameters that can be considered.

 External Recognition (c)
 Publications in refereed or industry trade journals, external presentations, citations in the
literature, invited lectures and patents are parameters to be considered.

47



 The internal and external customer ratings measure the ability of R&D to
meet customer expectations and contribute to the growth of the
corporation or enhance competitiveness. They are largely objective
measures that can be tied to tangible value. The value of external
recognition via patents, publications and presentations is more difficult to
measure objectively. The numbers of different subjects covered by public
disclosures should be evaluated rather than the total number of all
disclosures. It is easy to become subverted to a self servicing metric if only
numbers are considered. Maintenance of technology as trade secrets
must be considered in this evaluation.

48



 Internal customer surveys can be conducted using a 1 to 4 scale for rating. A 1
represents below standard execution on a given project. The causes for this poor
performance have to be determined since they could arise from inadequate
skills, poor judgment, lack of responsiveness, poor planning, etc. The causes may
not be related to the quality of the personnel but poor management practices.
Having multiple internal customers (marketing, manufacturing, sales, etc.), conduct
the evaluation is a form of 360o review. Superior performance by reaching targets
ahead of schedule, lower than expected costs, developing a significant competitive
advantage, etc., should be rated as a 4. In establishing this survey
system, agreement should be reached on the different levels of performance. The
survey should be applied to different projects with the same population of the
internal customers as raters. This rating should be conducted on a regular basis
and over time trends will emerge.

49



 External customer surveys should be conducted using the same 1 to 4 scale. A 1 rating
would indicate that the customer was not satisfied with the parameter being
measured, while a 4 would indicate that expectations were exceeded. Key parameters
should be selected beforehand and could include timeliness of response, knowledge of
products, knowledge of customer s operations and knowledge of customer needs. The
parameters will vary by industry. Several levels of the customer s organization should
be sampled such as plant operators, first-line supervisors and management. A simple
postcard type of survey instrument mailed after customer contact can be used. A
database can be developed over time and trends will emerge.

 The subjects covered in public disclosure should be tabulated and compared to
strategic technology goals. A subjective 1 to 4 rating system can be created by R&D
management to determine fit with the goals. A rate of 1 corresponds to a poor fit and a
need to enhance skills, while a 4 implies that all areas are being addressed.

50



 Service and consumer product companies may find the external customer survey
to be a valuable tool in assessing the effectiveness of their R&D organization. One
may also measure the quality by the number and type of external awards from
recognized organizations (ACS, AIChE, IRI ...).

51


Efficiency of Internal Technical
Processes (3.3)
 Definition
 This set of metrics seeks to provide a measure of both the efficiency and effectiveness of the
operation of R&D processes within the firm.

 Project Assessment
 The total cost of all commercially successful projects divided by the number of commercially
successful projects. (Useful when tracked over time with similar projects with similar scopes).

 The ratio of actual to projected costs (and timing) for all projects.

 Percentage of costs devoted to commercially successful projects.

 Portfolio Assessment
 The total R&D budget divided by the number of projects with commercial output. Subdivide
by projects of similar type (technical service, short term, long term) and used in conjunction
with project value assessment.

52


Processes (3.3)
 This set of metrics needs to be adapted to the needs of each firm -- considering
the goals, objectives, and priorities for the firm. Assessments need to be made for
individual projects (stage-gate , PACE, or similar processes), and for the collection
of projects (Portfolios).

53


Processes (3.3)

 Each firm will need to set the metrics relative to its specific goals and
objectives. An assessment of the selection termination and
management of projects can be made using the following four stages
for “Technical projects: Selection, termination and project
management”.

54


Processes (3.3)
 Stage 1 Technical projects:
Selection, termination and project management
A. favors short term projects
B. politically driven selection
C. no project monitoring or pre-project planning
D. little inter-functional participation in project teams
E. erratic turnover of team staffing
F. project leader roles not defined
G. no training for project leaders
H. unclear charters for project teams

55


Processes (3.3)
 Stage 2 Technical projects: Selection, termination
and project management
A. mix of short and medium-term projects
B. no inter-product-line analysis
C. priorities set erratically
D. project tracking
E. some inter-functional participation but not all key
functions represented.
F. formal release process for new products
G. some project team stability but conflicts over work
H. priorities
I. project leaders given only minimum guidance or training
56


Processes (3.3)
A. selection based on multiple inputs from internal and external
sources
B. balance of short-, medium- and long-term projects
C. risk analysis incorporated at key phases
D. projects still schedule driven
E. Inter-functional teams wherever needed
F. clear allocation of project and functional responsibilities
G. training for project leaders

57


Processes (3.3)
A. clear links between selection criteria and business and product-line
strategy
B. disciplined process for project termination
C. cross-functional planning and execution
D. continual improvement- postmortems, quality measures of both project
process and product
E. performance
F. projects are milestone driven
G. differentiated project management procedures for different types of
projects
H. scheduling and capacity planning avoid resource
contention by competing projects
58


Product Quality and Reliability (3)

 Definition

 Customer or Consumer Evaluation (a)
 Relative quality and reliability compared to competitive
products through evaluation by customers or consumers.

 Reliability/Defect Rate Assessment (b)
 Fraction of a firm's output, either by individual product or by
sum of all products, that meets or exceeds the established
quality standards.

59



 The advantage of this metric is that product benefits resulting from
R&D activities are directly evaluated by the customer or consumer.
Comparison with the competitors' products is usually the basis for
evaluation. Limitations of the metric chiefly are related to the
reliability and accuracy of the survey techniques chosen as
appropriate for the industry, though firms usually gain confidence in
their preferred methods through repeated use and incremental
improvement.

 Similar to the preceding discussion, the advantage to this metric is
that the direct benefit from R&D activities can be obtained through
specific measurements made by the firm.

60


 For the Customer or Consumer Evaluation metric, each firm will generally have a
preferred technique for directly or indirectly obtaining data showing how well the firm's
products perform in comparison to competitive products. Data for the Reliability/Defect
Rate.

 Assessment result from internal quality measurements. When both metrics are
utilized, the impact of product quality improvements on customer satisfaction should be
demonstrated. Product Quality and Reliability metrics are retrospective, showing the
results of past technology or product introduction to the market.

 This metric fits well with the trend toward greater input to R&D planning from customers
and consumers and with the "Quality" protocols that have adopted by firms of all types.
Though basically a retrospective measure, product needs that arise during the data
collection can be used for prospective purposes.
61


Market Share (3)
 Definition
 Firm (or business unit) market share in various product
categories measured as appropriate for the industry or
category, expressed as a percentage of the total market.

 This metric is meant to reflect value creation for the firm and
the value of the firm's technology. Similar to Gross Profit
Margin metrics, caution must be taken in the interpretation of
Market Share data from the perspective of measurement of
the contribution of R&D activities to the whole. There can be
many confounding factors in a market share
determination, such as the size and quality of the marketing
effort, the competitive response, the relative state of the
economy, etc.
62


Market Share (3)
 Changes in Market Share should be assessed at least annually to
determine the rate of progress or decline. The expectation is that
improvement in a firm's technologies and products will result in a
greater share of the market. Market Share is a retrospective
metric, showing the results of past technology or product introduction
to the market. Since competitors' market share data is usually also
available, this is a metric that can be used as a benchmark with the
competition.

 As an indication of threats or opportunities, share data in markets
related to a firm's products can be followed. This "Related Market
Share" metric can serve as a component of a strategy to anticipate
the potential application of similar technology into a firm's
marketplace.

63


Development Pipeline Milestones
Achieved (2.3)
 Definition
 Development Pipeline Milestones Achieved is a metric which is useful in
grading the effectiveness of management and planning of each R&D
project. It may also be used as an indicator of performance problems on a
given project which can be used to initiate diagnostic and recovery
procedures to give each project the best possible chance of success.
There are two possible variations of the metric which may be used:
1. Percent of project milestones achieved -- the percent, by project or overall by
sub-organization or laboratory, of all project milestones completed on schedule
or within some acceptable time window (90 days, for example) of the forecast
date. Trend studies can then be established for organizational performance
based on analysis of the data collected by quarter or for whatever other time
period is appropriate for the particular industry group.
2. Performance level at each milestone -- on a project basis, percent of all
expected objectives met at the milestone date by which they are forecast to be
completed.

64


Achieved (2.3)
 The primary advantages of this metric are for project diagnostics and as an
indicator of the overall planning and management health of the organization.
There are several possible limitations. First, a consistent management system
must be in place to assure that the variability of the number/quality/difficulty of
the milestones does not cause random fluctuations in the metric. Secondly, the
calibration of the metric is important. Since both the stage of the R&D project and
the industry of the business involved can greatly affect the trend analyses, the
metric user must be careful to identify what the real danger signals are in terms
of management and planning deficiencies, and what really constitutes an
indicator of project problems for this metric to be useful.

65


Achieved (2.3)
 Percent of project milestones achieved -- a simple approach is to tabulate
milestones met on schedule, 1-90 days late (which may constitute on schedule in
some cases), 90-180 days late, and so forth. The tabulation can be done
quarterly, and the trend information publicized to the organization to focus
attention on performance issues.

 Performance level at each milestone -- This is more useful as a project
diagnostic. One approach is to require project managers to use this metric to
summarize performance quarterly or semiannually at an operations review and
be prepared to cover diagnostic or recovery steps if a given project shows trends
over two or more time periods of failing to meet an acceptable performance level
(which will depend on the R&D discipline, the industry, and the stage of the
innovation process).

66


Achieved (2.3)
 One problem with using milestone achievement as a key metric is that
pressure is exerted to make all the milestones. With strong pressure to meet
milestone dates, the project manager may be tempted to populate the
project plan with easy milestones, to assure that his milestone hit rate is
good. It is important therefore to make sure that milestones are realistic and
aggressive. Especially in the later stages of the innovation cycle, in product
design and productization or manufacturing process development, it is
important to assure that the business customer, marketing
representative, etc. approve the project milestones to assure that the product
development cycle meets the market window requirements of the business.
In this case, a much closer tracking of milestone achievement may be
necessary than in the earlier stages of the innovation cycle (for
example, monthly or in some cases even more frequently).

67


Achieved (2.3)
 Options and Variations (continued)
 Another problem with using the milestone approach is that early in the innovation
cycle (the R part of R&D), milestones are often hard to define and even harder to
forecast. With the heavy level of uncertainty in a long-term research project, the
definition of a milestone may be vague, or even made in terms such as define or
develop concept. In these circumstances, project managers feel especially
uneasy in setting milestone dates for which they become accountable. An
approach here is to realize that (1) exact dates are less important, since product
or market need windows probably have not been established (and in some
cases, market needs or even product existence!), and (2) many of these projects
which are early in the innovation cycle will certainly fail, since in the research
phase, many more ideas are explored than have a positive outcome. In this
context, using milestones as a performance metric may not be useful. If
used, the metric should probably be relaxed in some way. For example, instead
of setting a goal of completing 100% of milestones, perhaps a goal of 50, 60, or
80% may be set. Another possibility is to declare any milestone made within 90
or 120 days of the forecast date to be on time. The specific approach taken will
depend on the industry, the research area, and the stage in the innovation cycle.
68


Goal Clarity (2)
 Definition
 This metric uses an interval rating scale assessing the extent to
which project performance objectives are clearly defined and
understood by all participants on the project team.

 While this metric may provide a semi-quantitative assessment there
is a degree of subjectivity for an individual to assess the extent to
which he/she really understands objectives and roles on the team.

 How to use the metric
 A member of the team would provide to all members of the team a
survey form to assess understanding of the project objectives and
commitments. This can be done using some interval rating scheme.
It would be wise to conduct this survey several times through the life
of the project to really assess the level of understanding.

69


Goal Clarity (2)
 A four stage assessment can be constructed with the following as the
highest stage for individual team members:

a) clear understanding of the expected product of team effort
b) personal belief that a team is the right way to develop/achieve the expected
product
c) understanding whether the team is an implementation, recommendation and/or
informational team
d) understanding the team's operational ground rules and end-result boundaries
e) belief that the team has all the appropriate
knowledge,functions, diversity, levels and locations, so that the expected
product of the team effort can be achieved using the minimum number of
people?
f) high personal commitment to achieving the objectives
g) clear understanding of personal role
h) acceptance of the recognized team leader
i) capability to draw on additional resources to keep the core team to a minimum
70


Comparative Manufacturing Cost (1.8)

 Definition
 Benchmarked manufacturing cost data vs. competition for
same type of unit cost.

 It is extremely useful and in some companies paramount to
know how R&D is helping to provide an advantaged cost
position to the operations. This metric reflects the
quantification of that goal. Unfortunately, cost accounting and
even further comparative cost accounting can be extremely
difficult.

71


Comparative Manufacturing Cost (1.8)

 While most firms have very accurate manufacturing cost data
for themselves, the generation of accurate manufacturing cost
data for competition is considerably more difficult.
Therefore, when using this measurement, there should be an
estimate made of the variance of the competitive estimates.

 This metric is intended to be based on unit process
comparisons. There can be many options created that are
aggregates of production processes, but these simplifications
can be misleading. Therefore caution is urged.

72


Gross Margin (1.8)

 Definition
 Gross Profit as a percentage of sales, where gross profit
equals net sales minus cost of goods sold (product costs plus
direct manufacturing costs).

 To some extent, gross profit margin reflects value of the firm's
technology assets and the value created by R&D.
However, raw material, production, and distribution costs also
directly affect the firm's gross profit margin. Each firm should
attempt to understand the correlation of Gross Profit Margin to
R&D.

73


Gross Margin (1.8)

 Value assessment should be based on change in gross profit
margin from period to period. (Periods should be appropriate
to an industry and may be in excess of one year.) Changes in
Gross Profit Margin in relationship to changes in values of
other metrics (e.g., financial return , technology transferred to
manufacturing, sales protected by proprietary position) should
be followed in an attempt to uncouple the contribution of R&D
from other factors. This is a retrospective metric that can be
used as a benchmark with the competition, if gross margin
data from competitive firms are available.

74


Use of Cross-Functional Teams (1.8)
 Number of Cross-functional Teams
 Current management philosophy suggests that the use of cross-functional
teams will improve the effectiveness and efficiency of the R&D Process and
will help R&D to be integrated with the business. The number of such teams
can be counted if they are established on a formal basis.

 Evaluation of the use of Cross-functional Teams
 A firm may rate its practice of using cross-functional teams by using a
subjective scale ranging from 1 to 4. Firms operating at Level 1 are
characterized by the existence of strong organizational boundaries, lack of
cross-functional involvement in R&D projects, and no cross-functional team
structure. Firms operating at Level 4 are characterized by a well developed
and supported team structure which effectively places all R&D work in
cross-functional teams responsible for the entire project rather than
functional silos responsible for parts. This metric is one of many relating to
how well the firm conducts the Practice of the R&D Process. It also relates
to how well R&D is integrated with the business.

75


Use of Cross-Functional Teams (1.8)
 This metric will be important if the use of cross-functional teams contributes to
the effectiveness of the R&D function in the firm, given its peculiar situation. This
is usually thought to be the case, but there may be cases where other factors
are more dominant.

 Evaluation of the use of teams can be used to diagnose problems with the
organization or the R&D process and to plan for improvement.

 How to use the Metric
 A simple count of teams is rarely as valuable as an assessment of the how the
firm uses such teams. This can be accomplished using the rating scale
suggested above. Input for the evaluation should be gathered from a broad
cross-section of the firm. The evaluation is important for planning improvements.

 Trends across time are probably more valuable than benchmarking.

76


Rating of Technology Features
and Benefits (1.8)
 Metrics for Product Features & Benefits:

 Metric 1: Competitive Technical Performance of Product (Project
Metric)
 Comparison of technical performance of a product in those dimensions
where the customer is likely to perceive a benefit. This may be used in a
prospective sense to appraise the value of a feature or in a retrospective
sense to register the value of a benefit which the market has recognized.

 Examples include:
 The use level required to achieve a needed result in the customer
application.
 The yield strength of a high-performance alloy
 The measured softness provided by a textile softener.
 The measured UV resistance of an external architectural coating.
77


and Benefits (1.8)

 Metric 2: Customer rating of Products (Business Segment or
Firm)
 Customer rating using a scale of 1-5 of the technology benefits that is
perceived in a firm‟s products. This can be compared to the rating for
the best competitor, usually in the form of a ratio. This is an
aggregate subjective measure for a business segment or for the firm.

 Metric 3: Economic Value of Products (Project, Business
Segment, or Firm)
 This metric is the price differential per unit obtained by virtue of the
technology feature minus the cost of providing the feature. The
differential can be multiplied by the volume to assess the total benefit
to the firm.
78


and Benefits (1.8)

 Metric 4: Market Share Evaluation (Business Segment or
Firm)
 If differential pricing does not occur, the advantages of superior
product technology can appear as differential market share. In this
case, the relative market share (the firm‟s share divided by the
largest share) can be used as a surrogate for the value of technology
embodied in the products.

79


and Benefits (1.8)
 Metrics for Process Features & Benefits:

 Metric 5: Competitive Technical Performance of Process
(Project Metric)
 Comparison of technical performance of a product in those
dimensions which are important to manufacturing cost or product
performance. This may be used in a prospective sense to appraise
the expected value of a new or improved process or in a
retrospective sense to register the demonstrated value.

 Examples include:
 Manpower requirements
 Efficiencies of raw material conversion
 By-product or co-product costs or values
 Consistency, controllability, and other such quality parameters.
80


and Benefits (1.8)
 Metrics for Process Features & Benefits:

 Metric 6: Economic Value of Processes (Project, Business
Segment, or Firm)
 The differential in profitability (versus the target or competitor)
attributable to new or improved process technology.

 Metric 7: Profitability Evaluation of Processes (Business
Segment or Firm)
 In the same way that Market Share is a surrogate measure of
product performance in those business areas where the basis of
competition is product performance, overall profitability in a business
segment of the firm is a measure of process performance, in those
business areas where the basis of competition is cost and/or quality.

81


and Benefits (1.8)
 Product Metrics 1 through 4 attempt to assign value to technology used in
products. However, differential value or market share are normally the result
of many different factors. These metrics can give some indications if factors
are carefully sifted, but may be misleading if the analysis is superficial.
Objective measurement of product performance and customer ratings relative
to competitors are the most accurate measures of product technology. But
note that comparison of features that the market has not recognized as
benefits may be self-serving and deceptive. Competitive rankings of
measured product performance and or customer ratings may be averaged
over market segments or over the firm to obtain average values.

 Process Metrics 5 through 7 attempt to assign value to new or improved
process technology resulting from R&D. However, economic value and
profitability are normally the result of many different factors. Objective
measurement of process performance is the most accurate measure of
process technology.

82


and Benefits (1.8)
 How to Use the Metric:
 Metric 1
 Define the key parameters which measure features the customer is likely
to perceive as benefits. Measure product performance as accurately as
methods allow. Compare to the same measurements of competitive
products. Rank performance versus best competitors.

 Metric 2
 Ask customers to rate the technology attributes of a product line
relative to solving their problems. This requires a carefully constructed
survey instrument.

 Metric 3
 This metric is valid for products which are differentiated by
performance. It is not applicable to commodity products.
83


and Benefits (1.8)
 How to Use the Metric:
 Metric 4 (See Definition)

 Metric 5
 Define the key parameters which measure or impact cost or quality. Measure
process performance as accurately as methods allow. Compare to the same
measurements of competitive processes. Rank performance versus best
competitors.


 NOTE: These metrics may be used retrospectively to measure the output of R&D over the past period
and prospectively to set targets for future accomplishments.
84


Response Time to Competitors
Moves (1.8)
 Definition
 This metric measures the ability of the firm to respond to new technical innovations
introduced by competitors. Depending upon corporate strategy, it could be the time
required to match or exceed the competitive offering.

 This metric is an indicator of technical leadership in a given field. The technical leader
will not spend a significant amount of time matching competitive innovations, while a
follower will be more reactive than proactive. The utilization of resources for this
function as a percentage of total resources should be tracked over time. An increase in
this percentage would indicate that new technology programs are not as effective as
desired and technical position relative to the competition is eroding. This metric also
measures the flexibility and creativity of the firm to change priorities to meet competitive
challenges. A strong market intelligence function is required to identify competitive
entries at an early stage in the introduction and to assess the technical merits of the
offering so that appropriate responses can be made. The technical merits of
competitive offerings must be critically assessed to differentiate from market
repositioning of existing technology.
85


Moves (1.8)
 The time between the introduction of a competitive offering and the internal
development of a comparable or superior offering can be measured and
compared to product development times for similar products. A rating system of 1
to 4 can be used.

1. Organization slow to recognize significant competitive offering in the marketplace; slow
to launch program to respond; unable to get their offering into the marketplace in
acceptable time to be combatant. Slow to recognize impact of technical innovations.
2. Organization recognizes need for competitive offering; has trouble in launching program
to develop counter-offering and gets offering to market barely in time to have impact.
3. Organization responds to competitive offering and develops counter offering to maintain
relative position.
4. Organization anticipates potential for competitive offering and has counter offering into
the marketplace with superior product allowing a gain in market/competitive position.
Very fast to recognize impact of any hints of technical innovations.

86


Moves (1.8)
 The percentage of resources used in matching competitive moves could be
measured over time. An increase in spending in this function should raise
questions regarding core research and development efforts for organizations who
strive for technical leadership.

 The importance of this metric will depend upon the technology strategy of the
firm. Firms pursuing technical leadership will be very interested in minimizing
response time. The importance may vary across different strategic segments in
the same firm.

87


Comparative Technology
Investment (1.5)
 Definition
 This measures the current annual expenditure for R&D staff and capital compared to
the best competitor and/or the industry average.

 This metric measures the rate of current activity in developing the technology of interest
with the intent of predicting whether the firm is expected to gain or lose ground in the
technology. It should be kept separately for the KEY and PACING technologies most
critical to the strategy. Retrospectively it measures the efficiency of the investment in
meeting new product and technology development goals.

 This metric should be as quantitative as possible, but in some industries it may be
necessary to make estimates as to the size of the development effort of the best
competitor and the industry average. Analyses must be based on a comparison of
similar functions. For example, some organizations include sales support as part of
their report for technology expenditures. The components of the expenditures under
study must be understood in making the comparison.
88


Comparative Technology
Investment (1.5)
 Information on a firm s overall technology expenditures are available in the firm s
annual reports or industry publications. These can be used for comparison to
internal overall investment. Information by industry is available in industry
publications and from organizations such as IRI (IRI/CIMS survey).

 Rationing the firm‟s current investment in technology versus the best competitor
and industry averages provides an insight into the efficiency of the technology
investment. Performance exceeding expectations in value creation goals at
competitive investment rates indicates an efficient organization while sub-standard
performance raises concern about the quality of the investment. Smaller firms may
require an investment higher than industry norms to maintain a competitive
position to offset critical mass issues.

89


Employee Morale (1.5)

Definition
 This metric takes quantitative ratings of key aspects of employee
satisfaction and morale as shown by direct employee survey. It is
recognized that employees may feel good and have high morale, yet
produce nothing of value for the firm -- the real question is are they
motivated and committed to create and innovate profitably?

Advantages and Limitations
 Extensive surveys are time consuming and expensive to conduct and
must be conducted with sufficient frequency to establish base lines and
understand real trends. One must understand also that technical
populations tend to have certain biases in such surveys. Many employees
feel "surveyed-out".
90



How to Use this Metric
 The typical survey uses five point scales for agreement (strongly
agree, agree, neither agree or disagree, disagree, strongly disagree) for
importance (extremely important, very important, somewhat important, of
little importance, not at all important) and for performance (very
good, good, fair, poor, very poor) in answering sets of questions related to
work environment, feelings about the company, ratings of the
company, ratings of the organization/work location/work group, feelings
about the individual's job, and general satisfaction. Opportunities are given
for comments. Such extensive surveys are most often conducted by third
parties to maintain confidentiality.

91



 How to Use this Metric (continued)
 Four tested questions of job satisfaction are:

1. If a good friend was interested in a job like yours for your firm, what
would you tell the friend?
2. All in all, how satisfied are you with your present job?
3. Knowing what you know now, if you had to decide all over
again, would you take your current job?
4. 3.4 How satisfied are you with the overall employee- employer
relations at your firm?

92



 One firm which uses e-mail extensively, every 9-14 months
conducts a broad "pulse" survey and asks employees to
provide two ratings (using 0-10 point scales with descriptors)
one rating the employees work climate/environment and the
other rating personal feelings about the work itself. Employees
spend less than 4 minutes to reply by e-mail, or to be
anonymous by fax or to a voice mailbox. Employees often
write additional comments that give information sought in the
more extensive surveys. Confidentiality is assured and rapid
feedback (within 2 weeks) of survey results to participants
maintains a high level of interest and participation.

93


Metrics for the Stage Gate Model
 Definition of the Innovation Process
 The term "innovation process" refers to the overall process in a company for
conceiving and developing ideas and concepts into profitable products and
processes for the company business(es). It is one of the 5 to 7 overall high-
level business processes within the corporation. In order to
maintain, diagnose, and improve the process, both effectiveness and
efficiency metrics are required to support decision-making.

 Note that in the context discussed herein, the assumption is made that the
innovation model will be technology-based. That is, innovation is founded in
the efforts of the R&D organization, and that the overall function of the
process is the embodiment of technology in the products of the business of
the corporation. Progress through the full innovation process will involve the
efforts of all corporate resources, including
marketing, engineering, manufacturing, and service and support functions.

94


 Relevance of Innovation Process Model to Metric Choice
 Often stakeholders in the innovation process -- R&D management, financial
management, and corporate officers -- need to utilize metrics which supply specific
decision support information about corporate capabilities in innovation. These
metrics are overall, or outcome metrics, which provide a grade or rating of the
overall result of the innovation capability of the corporation or business in question.
Such metrics might relate to value creation by the process, effectiveness of the
overall process in terms of new product revenue or profit versus total R&D
investment, or the ratio of new product revenue or profit to total company or
business revenue or profit.

 However, these metrics are often long-term and retrospective in nature, and while
they serve to evaluate the overall process as a whole (and generally, but not
always, on a lagging basis), they may offer little, if any, insight into the current or
future state of the corporate innovation process or its many and diverse activities.

95


 Relevance of Innovation Process Model to Metric Choice (continued)
 To be useful for the on-going management and evolution of corporate innovation, a
diverse set of metrics must be available which offer insight into current condition
and future vision state of the process and many of its constituent parts. For many
such metrics to be useful, they must be not simply outcome-oriented, but process-
oriented, and specifically diagnostic of discrete sub-processes within the overall
innovation process.

 Each in-process metric must be related to a process model which ties it to the
portion of the process of interest, from early idea or technology exploration through
proof of concept to technology development to productization. The following
process description is a relatively simple process model and defines the various
sub-processes within that model in order to provide anchors for useful metrics
within the innovation process.

96


 Overall Description of the Stage Gate Model for the Innovation Process
Used in this Metrics Guide
 There are a number of possible representations of the corporate innovation process. One
simple yet fairly effective representation is the so-called "stage gate model".

 The model in Stage Gate divides the overall innovation process into four stages, from
concept exploration or "ideation" to the process of productization or commercialization. This
division into four sections is arbitrary. An argument can be made for a three-stage model, or a
more complex model of multiple sections and sub-sections. The four-stage model is chosen
primarily for convenience, and to match a number of corporate models and those defined by
scholars in the area of business processes. The shape of the model recognizes several
realities of modern business.

 For example, the process funnel narrows as the process proceeds from exploratory research
or concept exploration through proof-of-concept to technology development and
commercialization, illustrating pictorially the selective filtration that occurs in the process.
There are typically many more ideas and concepts that are explored than are developed into
significant technology capabilities in a business, and fewer still that emerge finally into
finished products. In each stage of the process, the candidates dwindle, until only the most
promising are brought to full production.
97


 Overall Description of the Stage Gate Model for the Innovation Process
Used in this Metrics Guide (continued)
 Similarly, the four gates, labeled A, B, C, and D, stand for the tests or decision-making
activities that are exercised between the stages in the innovation process. Economic reality
imposes a limit on the total level of technology and product development that each company
can support. A second reality is that as each decision gate is passed, the resources and
funding required to carry a given project to the next stage increases dramatically; a good rule
of thumb is a 10X expenditure increase at each gate. These corporate "facts of life" impose a
stringent set of conditions on the suitability of R&D programs which approach a gate. Each
gate requires a set of metrics which ensure that only those programs most suitable to meet
corporate business needs pass into the next sub-process.

 The following sections describe the decision gates and stage processes shown in the stage
gate model. Underlined terms link to appropriate metrics for the gate or process under
discussion.

98


 Description of the Major Sub-process Divisions in the Innovation
Process Model
 As noted in the introduction to the process model, the division of the innovation process
model is arbitrary, but in general it seeks to approximate reality. The four divisions cover the
very early technology or idea exploration phase of innovation (when possibilities are
defined), a proof-of- concept phase when the mapping of ideas into the realities of the
business world occurs, the development, and finally commercialization. Each of the separate
phases, or sub-processes, is covered in the following sections, together with the entry gate
which defines conditions for admission to that sub-process. Refer to the stage gate model in
the references that follow.

99



Gate A
Gate B
Gate C
Stage I Gate D
Stage II

Stage III Stage IV

100



101


Stage-Gate Model Description
 Process Entry Gate (A) and Exploratory Concept Sub-process (I)
 Gate A is the entry not only to Process I but to the entire innovation process. The purpose of
Process I is to explore new ideas and concepts and set in motion as many promising "seed"
projects as possible. The cost of research and investigation is small here. In companies
where exploratory concepts consist of exploring product ideas and concepts, a single worker
may have a project or even several in work simultaneously. Where true basic research is
involved, it is most often at the University level, with industry participating through
grants, contracts, or research agreements. In either case, there are typically many avenues
being explored and no valid idea or concept is neglected.

 The main consideration at Gate A is whether the idea or concept is strategically appropriate
(at this stage, the alignment to corporate business goals may be ephemeral in some
cases), and whether the expertise available to address the concept or idea is adequate.
Metrics at Gate A should address these issues.

 The purpose of Sub-process I is the validation of concepts or physical principles. Metrics for
this process should simply address the validity of results and whether or not basic principles
are established. Competency metrics may also be valuable to support assessments of
required resources to execute projects.

102

Innovation Benefits Realization for Industrial Research (Part-8)

Innovation Benefits Realization for Industrial Research (Part-8)

Recommended

Recommended

More Related Content

What's hot

What's hot (10)

Viewers also liked

Viewers also liked (20)

Similar to Innovation Benefits Realization for Industrial Research (Part-8)

Similar to Innovation Benefits Realization for Industrial Research (Part-8) (20)

More from Iain Sanders

More from Iain Sanders (18)

Recently uploaded

Recently uploaded (20)

Innovation Benefits Realization for Industrial Research (Part-8)