Innovation Benefits Realization for Industrial Research (Part-8)


Published on

Published in: Business, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Innovation Benefits Realization for Industrial Research (Part-8)

  1. 1. Technology Innovation Management Framework for Industrial Research Part-8 Dr. Iain Sanders January 2005
  2. 2. 2
  3. 3. 3
  4. 4. 4
  5. 5. WHAT R&D METRICS? Platform 3: (Stages VIII, Part 3)Integrating Technology Innovation with Business Function – Part I: Laying the Foundation
  6. 6. PLATFORM 3: VIII (3) What are our R&D Metrics?  R&D Metrics  Set standards for measuring new concept R&D performance and evaluating progress / outcomes achieved against milestones / targets set. 6
  7. 7. PLATFORM 3: VIII (3) Top 39 Ranking R&D metrics11.5 Strategic Alignment 5.5 Development Cycle Time 3 Product Quality and Reliability11.5 a. Corporate & business unit 5.5 a. Market cycle time 3 a. Customer Evaluation11.5 b. Goal coverage 5.5 b. Project management cycle 3 b. Reliability/Defects time10 Financial Return 5.5 Customer Satisfaction 3 Market Share10 a. New Sales Ratio 5 Number of Ways Technology is 2.3 Development Pipeline Milestone Exploited Achieved10 b. Cost Savings Ratio 5 Projects with Business/ 2 Goal Clarity Marketing Approval10 c. R&D Yield 5 a. Project has approval 1.8 Comparative Manufacturing Cost10 d. R&D Return 5 b. Ratio of projects 1.8 Gross Margin 9 Projected Value of the R&D 3.3 Quality of Personnel 1.8 Use of Cross-Functional Teams Pipeline 9 a. Projected Sales 3.3 a. Internal Customer Rating 1.8 Rating of Technology Features and Benefits 9 b. Projected Income 3.3 b. External Customer Rating 1.8 Response Time to Competitors moves 7.8 Distribution of Technology 3.3 c. External Recognition 1.5 Comparative Technology Investment Investment 6.8 Use of Project Milestone/ 3.3 Efficiency of Internal Technical 1.5 Employee Morale Stage-Gate Process Processes 7
  8. 8. PLATFORM 3: VIII (3) Strategic Alignment Metric (11.5) Definition  This R&D metric assess the degree of alignment of an R&D project or an R&D portfolio with the strategic intents of the company or a division of the company. The strategic intents are often the corporate goals embodied in its business plan. Why it is used  This metric is used to gauge the degree of relevance of the R&D program to the corporate goals. The strategic intents of a company may change more rapidly than the R&D program can respond to those changes creating various degrees of misalignment. The misalignment can be with regard to work area, long term versus short term needs, or degree of risk. 8
  9. 9. PLATFORM 3: VIII (3) Strategic Alignment Metric (11.5) How to use metric  There are a number of ways this metric can be applied, including both prospective and retrospective views. It can be applied by R&D management, general management, or by both working in partnership. An alignment index would be assigned to each project; a linear scale of 1-5, for example, would work well. these scores may also have value when considering relative merit of individual projects in the portfolio. A composite score for the entire portfolio would then be determined. This could be a weighted average reflecting sizes of projects with regard to technical head count, project budget or some other appropriate weighting factor. Once a baseline for alignment has been established, R&D management can then decide if and how this index should migrate to greater or lesser degrees of alignment through modifications of the portfolio. Applied to individual projects, there could be a cut-off point for the alignment parameter below which projects are not supported.  Consideration must be given to the degree of alignment desired. Although the desired state in may cases is toward greater degrees of alignment, you can envision situations where that may not be the case. A research organization charged with taking the company in new directions may not want its project portfolio highly aligned with the current business plan. A more visionary business plan may capture new directions as well as current businesses, but many organizations find that some degree of decoupling is desirable. 9
  10. 10. PLATFORM 3: VIII (3) Strategic Alignment Metric (11.5) Options  Both retrospective and prospective views are options of this metric. The retrospective view entails applying the metric to an existing portfolio of R&D projects to determine the degree of the portfolio with corporate goals.  The prospective view is to apply the metric to a proposed project or slate of projects. If the R&D organization is trying to increase its alignment index, then the management will be less likely to initiate projects that move the composite score in the wrong direction. Similarly, an alignment index cut-off may be instituted. Projects falling below some minimum value of alignment would not be supported. 10
  11. 11. PLATFORM 3: VIII (3) Financial Return (10) New Sales Ratio (a)  The New Sales Ratio is the % of current sales originating from new products. There are two sub-definitions that are required. What is a new product? and, How old is new?  The most frequently used and simplest definition of a new product is any new SKU (inventory code: Stock Keeping Unit) that has required R&D support to implement. This avoids counting new SKUs which are only packaging changes or other modifications made easily by marketing or manufacturing. When is a new product old is different for each business and technology. In rapidly changing and evolving fields, such as electronic chips and software, new might only be one year, but certainly not more than three. Three years is more likely the norm for businesses that are a mix of fashion and formulation, such as cosmetics & toiletries. And, for more intense capital and industrial products, three to seven is a more likely range to select a number that is right for you. 11
  12. 12. PLATFORM 3: VIII (3) Financial Return (10) New Cost Savings Ratio (b) The Cost Savings Ratio is the % reduction in cost of goods or cost of operations (including depreciation charges) that are realized in a year to year comparison that have originated from technology changes that are new. Again, the same issues must be resolved as sub-definitions as required for the New Sales Ratio. In other words, what is being attributed to R&D and how long is new?  Since SKUs are not used to catalogue changes made in operations an alternative must be found that will work for your firm. A simple solution can often be created based on the capital approval process. Most companies require a specific approval for all individual capital projects. These can be coded and tracked for R&D involvement and for realized cost savings. However, it is recognized that the accounting involved in examining the impacts on cost savings may be more difficult than that for new sales. It is therefore a more common metric only in those cases which are more significantly impacted by cost savings than by new product sales. And, since new technology for operations or manufacturing has a different useful life than a new product per se, it must be tracked for a different length of time that is industry specific. In some cases, it may be very linked to the product life in other cases, it may go on much longer. In any case, it is not likely that new will reach beyond 7-10 years. 12
  13. 13. PLATFORM 3: VIII (3) Financial Return (10) R&D Yield (c)  R&D Yield is the contribution of R&D to current financial performance. It is a metric that is composed of definitions from New Sales Ratio and Cost Savings Ratio, plus an evaluation of gross profit from the new sales.  It is the annual combined financial benefit that is derived from the annual gross profit of new products and the annual cost savings of new processes. This is the current contribution that the company receives that is associated with its past stream of R&D investments, i.e. the part of the bottom line that is relatively new and derived from R&D. 13
  14. 14. PLATFORM 3: VIII (3) Financial Return (10) R&D Return (d)  R&D Return is the relative ROI measure that relates to R&D. It is composed of the R&D Yield divided by the annual investment in R&D. Hopefully, this is a large number that is proportional to the risks and variances that are part of R&D. 14
  15. 15. PLATFORM 3: VIII (3) Financial Return (10) Advantages and Limitations  The advantages of these financial metrics are that they relate directly to the financial benefits to the company, they are quantitative and they are comparable to metrics that can be used in different parts of the same firm or between firms. They capture the degree to which R&D is truly making a financial contribution to the value of the enterprise. They answer the question: What have you [R&D] done for me [the Business] lately? However, they only represent the tip of a process that takes place over a number of years and that involves other functions besides R&D. This means that the numbers reflected by these metrics are associated with activities that are in the past. These metrics are lagging indicators. They are a nice track record, but they may not be reflecting accurately a current level of effectiveness. 15
  16. 16. PLATFORM 3: VIII (3) Financial Return (10) How to Use the Metric  The metrics should be tracked at least on a once a year basis. Because of measurement and definition problems, a baseline of two years or more of historical data is needed before accurate judgments can be made about trends and ratio efficiencies.  The metrics should be examined carefully for consistency with business strategies and the results required vs. the investments in R&D. In situations where the metrics, requirements and available resources are not in balance, there will be a difficulty in executing the overall business and technology strategies. One or the other must be shifted, and variations in how R&D is conducted need to be examined.  If the Financial Return metrics are being maintained or going up: the corporation has the likely raw material to extend a technology-based or innovation-based growth program; the investors have the possibility of an extended stream of positive returns from the accumulation of financial pay-offs from technology-based innovations; and the R&D units enjoy the likelihood of consistent funding to reinvest in various aspects of technology application for the near term and base building for the future.  The key words here are likely or possible. Positive Financial Returns are a necessary but not sufficient condition for growth. It is also only a measure of the moment, whether it is looking to the past or to the future. And, any downward movements will predict the difficulties the business will have in achieving solid gains against the competition. These indicators are crucial to assessing the total returns from R&D investments, whether enough is being spent on R&D, and what is the likely future value of the company from a technology perspective. 16
  17. 17. PLATFORM 3: VIII (3) Financial Return (10) Options and Variations  There is always something new and innovative that is contributing to revenue or profit. These financial return metrics are intended to capture the new portion of these changes in the business that are related to R&D. They require definitions of what is to be considered new and for how long. These elements can then be tracked separately or together, in ratio or absolute form, by themselves as benefits or as an investment return vs R&D. The options and variations fall into place based on each companys views of these items.  The most common variations are based on the length of time that is new. the most frequent categories are three years, five years and seven years. Another variation is to use these same metrics in a prospective, future mode. 17
  18. 18. PLATFORM 3: VIII (3)Projected Value of the R&D Pipeline (9)  Projected Sales (a)  Projected Sales is the calculated sum of future sales from current R&D projects. This metric may be expressed in absolute terms or a % of future sales.  Definitions must be provided for how this is to be evaluated. Normally, it is the forward side of the newness range, i.e. if new products are those introduced within five years, projected sales will be calculated for five years after commercialization. A probability of attainment is usually figured into this metric. 18
  19. 19. PLATFORM 3: VIII (3)Projected Value of the R&D Pipeline (9)  Projected Income (b)  Projected Income is the income stream associated with the Projected Sales. Similar definitions apply. This metric may be expressed as an absolute number or as a fraction of net income. 19
  20. 20. PLATFORM 3: VIII (3)Projected Value of the R&D Pipeline (9)  Advantages and Limitations  This metric provides an ongoing anticipation of the expected results from R&D. Because it is projected, it provides an evaluation of the benefits that are being created with todays R&D investments.  The limitations are due to the intrinsic difficulties of obtaining estimates about the likely commercial benefits if the technology is successful. 20
  21. 21. PLATFORM 3: VIII (3)Projected Value of the R&D Pipeline (9) How to use the Metric  This is perhaps the single most important and least used metrics. It is the singular indication of the future business that is to be developed as a result of successful outcomes from R&D. Are the sales impacts large enough, are there enough new products, is the timing of elements in the pipeline adequate, are the overall returns related to R&D adequate? Are these metrics on a year to year basis showing constancy, increase or decline?  This metric provides ongoing guidance to the company regarding the future gain to be expected in the business due to R&D. This should be used as a check that both the strategy and the resource allocations are correct.  If this metric is staying constant or increasing, particularly with respect to the R&D resources, then the effectiveness of R&D is being maintained or increasing.  If, alternatively, this metric is declining, then further diagnostics should be examined to understand the reasons and to take corrective action. 21
  22. 22. PLATFORM 3: VIII (3)Projected Value of the R&D Pipeline (9)  Options and Variations  Common options are few because this is an underused metric. One is to look at absolute sales or net income over a five year horizon on the presumption that some projects will finish and be productive within the next two-three years and that others will impact a bit later, i.e. in the fourth and fifth year. A related option is not to adjust the commercial impact by any probabilities. 22
  23. 23. PLATFORM 3: VIII (3) Distribution of Technology Investment (7.8)  Definition  This metric provides a means of assessing how well an R&D program is protecting the technology investment and technical position of the company. It forces consideration of how the technical assets should be distributed, setting directions for modifying the R&D portfolio.  Advantages and Limitations  The portfolio of an R&D organization may not be protecting the strategic interests of the company for any number of reasons such as skill set mismatch, slow response to changes in the company‟s mission and markets, and a rapidly changing competitive environment. There can be an over-emphasis on certain business units and products. This metric causes the management to examine how well the R&D effort is protecting and expanding the technical position of the company in areas of greatest importance. 23
  24. 24. PLATFORM 3: VIII (3) Distribution of Technology Investment (7.8) How to Use the Metric  This metric is applied by first determining how the technology investment should be distributed. As an example, consider a company with six business units. The R&D portfolio can be distributed among these six business according to a number of models. Six examples of distribution models are listed below:  The revenue that each business generates.  The opportunity market share (potential market growth).  The impact that technology can make in the different business units.  Competitive Impact (Base, Key, Pacing)  Some combination of the above distribution models.  The profitability that each business demonstrates.  These considerations often involve the concept of the technological basis of competition, that is how does technology provide a sustainable competitive advantage in a particular product or market. The discussion could also consider the distribution of the competitive impact of the company‟s technology investment by categorizing them as Base, Key or Pacing . Base technologies are essential to the business but widely exploited by competitors. Key technologies are highly differentiating to the company‟s current products. Pacing technologies are new technologies where the competitive impact is less certain but likely to be high. All three of the impact categories require protection of the competitive position, but the distribution of resources among the three categories may vary based on the company‟s business plans. 24
  25. 25. PLATFORM 3: VIII (3) Distribution of Technology Investment (7.8) How to Use the Metric (continued)  Once a distribution model has been agreed upon, the portfolio is measured against that model. The distribution could be with regard to number of R&D projects in each segment of the distribution, head count devoted to each segment, or expected value of projects in each segment. Modifications are made to the portfolio to move the distribution toward the desired state. 25
  26. 26. PLATFORM 3: VIII (3) Distribution of Technology Investment (7.8)  Options  This Both prospective and retrospective views are supported by this metric. In the retrospective view, the R&D projects are categorized according to the corporate investments or markets they are intended to protect or create, or the competitive impact that they offer. This does not have to address the entire R&D effort of the company, since the metric can be applied to any subset of the portfolio. The projects are appropriately weighted to reflect their size and cost. A distribution of the R&D efforts supporting each of the categories is determined. This current state distribution is then compared with the desired state. The degree of misappropriation can then be quantified as the fraction or percentage of the R&D effort that is improperly distributed.  The prospective view for this metric involves consideration of how a proposed project shifts the distribution of technology investment toward or away from the desired state. 26
  27. 27. PLATFORM 3: VIII (3) Use of Project Milestones / Stage-Gate Process (6.8)  Definition  Percent of projects in the total portfolio going through a defined project management system with defined milestones.  Percent of R&D expenditure on projects using a defined project management system with defined milestones. 27
  28. 28. PLATFORM 3: VIII (3) Use of Project Milestones / Stage-Gate Process (6.8) Advantages and Limitations  Percent Project management systems including milestones can provide a way of reducing cycle time and providing R&D and business management with a sense of the health of projects. These systems also can improve the linkage between R&D, marketing and business management by enabling coordination with manufacturing and marketing to hit key windows. When these systems include documentation through status reports, they can be used to promote organizational learning. Driving behaviors which increase the numerical value of this metric should therefore improve both the linkage of R&D to the business and improve the effectiveness of R&D.  When used with a formal stage-gate process this metric provides a measure of compliance with that system. Since companies will generally use a defined project management system and establish milestones in the later phases of innovation, this metric may also be an indicator of the distribution of projects in the innovation pipeline (see metric: "Distribution of Technology Investment" ). 28
  29. 29. PLATFORM 3: VIII (3) Use of Project Milestones / Stage-Gate Process (6.8) Advantages and Limitations (continued)  The metric may be limited by the difficulty of counting projects outside the project management system. Further since project management systems may not be appropriate early in the innovation process, the ideal value for this metric will depend on the firms desired balance of early and late stage projects. For short term projects such as minor product or process variations, use of formal project management systems and this metric may create unnecessary red tape and potential delays. 29
  30. 30. PLATFORM 3: VIII (3) Use of Project Milestones / Stage-Gate Process (6.8) How to Use the metric  As a concurrent metric, the total number of projects with defined (written) project plans including definite milestone dates can be divided by the total number of identifiable R&D projects to calculate the metric.  When used with the appropriate accounting system this metric can be calculated from the cost of projects divided by the total R&D cost. In this case the budgeted projects should be audited to determine compliance with requirements for plans and milestones. Options and Variations  The metric can be used as a concurrent metric (a snapshot of the current R&D activities) or as a retrospective measure to determine how many R&D projects used a defined process. It should be equally suitable for service and manufacturing companies. 30
  31. 31. PLATFORM 3: VIII (3) Development Cycle Time (5.5)  Market Cycle Time  This metric measures the elapsed time from identification of a customer product need until commercial sales commence.  Project Management Cycle Time  This metric measures the elapsed time from establishment of a discrete project to address an identified customer product need until commercial sales commence.  For both metrics described, the end point can be time when manufacturing feasibility is established for those cases where no commercialization occurs. Compare to historical values and benchmark vs. competition, if possible. Group by categories of projects (e.g. major new product, minor product variation, etc.) Can also be used to track milestone attainment rate for firms using a stage gate management process. 31
  32. 32. PLATFORM 3: VIII (3) Development Cycle Time (5.5) Advantages and Limitations (Market Cycle Time)  The advantages of this measure is that it is quantitative and can be used to measure the entire process or various parts of the process if stage gates are examined. The process can be analyzed to determine what parts are driving the overall cycle time so that improvements to the process can be made.  The limitations of this metric could include R&Ds position that it does not adequately influence the process until after a need has been more defined. Another limitation is that a strong documentation system is helpful to make the cycle time metric as accurate as possible. An additional limitation is that defining the commencement of sales as the end of the cycle does not account for post start-up issues such as efficiency, waste, % of manufactured product within specification, etc.; this could lead to focusing on shortening the cycle time at the expense of later, non-measured parts of the cycle. One must also keep in mind that for breakthrough or paradigm shifting projects,, cycle time measurements. The advantage of this metric is that it supports having the clarity of when a project is initiated based on approvals, assignment of resources, start of spending, etc. The metric can be used to measure the entire process or various parts of the process if stage gates are examined. The process can be analyzed to determine what parts are driving the overall cycle time so that improvements to the process can be made. 32
  33. 33. PLATFORM 3: VIII (3) Development Cycle Time (5.5) How to Use this Metric  Both metrics should preferably be used in combination with a project reporting system that can track the project initiation date (based on approval and assignment of resources), the length of time in each stage gate of the innovation process, and date of sales commencement. For the first metric, the initiation date could be the date the customer need was determined (i.e. marketing request or date of customer research results). Cycle times for different types of projects (new products, cost savings, product improvements, etc.) should be compared to help predict and manage resource allocation. Cycle times for different divisions could also be compared (with caution) to identify practices driving lower cycle time to adapt where possible. Options and Variations  Variations could exist regarding looking at cycle times for only certain parts of the process for which R&D feels it has most control or influence. Cycle time could be extended past the commencement of sales based on what is important to the organization and R&Ds involvement, i.e. when target efficiency is achieved, target manufacturing cost, % of product within spec, etc. 33
  34. 34. PLATFORM 3: VIII (3) Customer Satisfaction (5.5)  Definition  The customer satisfaction metric has two variations: 1. Measures of external (end-customer) satisfaction. These may be such metrics as ratings of quality of technical personnel or technical capabilities, or technology benefits within products or processes. 2. Internal customer satisfaction. Since the immediate customer of R&D is normally the businesses within the corporation that R&D serves, measures such as customer satisfaction in engineering, marketing, or manufacturing may be appropriate. Typical metrics might include on-time technology delivery, competitiveness or appropriateness of the technology solutions delivered, and overall satisfaction with the R&D track record of technological support. 34
  35. 35. PLATFORM 3: VIII (3) Customer Satisfaction (5.5)  Advantages and Limitations  There are few disadvantages to good customer satisfaction metrics. In the case of external, or end-customer satisfaction metrics, one complication may be that the entire innovation cycle is under review by the end customer. A bad grade by the customer, while a valid rating of the corporate innovation process, may not be merely an indictment of the R&D operation, but a judgment of the overall product development process within the company, involving manufacturing and product engineering, market forecasts, consumer needs and attitudes, and competence of corporate management. One the other hand, a well- thought-out customer satisfaction metric (or set of metrics) for the internal or immediate customer within the corporation -- normally the corporate businesses and their various R&D-related organizations -- may be the key diagnostic to indicate that R&D processes are lacking and need adjustment or redesign. The R&D organization itself is probably better served with well thought-out internal metrics than with external metrics that complicate the diagnostic process for R&D when problems are indicated. 35
  36. 36. PLATFORM 3: VIII (3) Customer Satisfaction (5.5) How to Use the Metric  External customer satisfaction metrics. These metrics will normally be marketing- related or implemented. One variation would be to use a marketing survey, in which various aspects of technology benefits are rated on a five -point scale  There are two dimensions for internal customer satisfaction metrics: strategic and tactical.  Strategic metrics deal specifically with whether the R&D function is meeting the strategic needs of the customer. The review process might involve matching technology and product roadmaps in a joint meeting, in which technology timing mismatches are resolved. Information can be exchanged; technology previews by R&D to alert the businesses to possible market-creating or market share increasing discontinuities, and the businesses to share future market window and product definitions with R&D. Various metrics can be used, including the five-point rating system mentioned above, or a metric which highlights number or percent of mismatches between product and technology roadmaps. 36
  37. 37. PLATFORM 3: VIII (3) Customer Satisfaction (5.5) How to Use the Metric (continued)  Tactical metrics deal with whether specific projects are meeting the goal or delivery requirements of the internal customer base. For technology projects in the latter stages of development (near to or entering product development), regular project reviews with the intended customer(s) is important. At quarterly or perhaps semiannual reviews, customer and R&D representatives join in a review of project progress. A useful metric in this case is a report card which each customer representative is required to fill out in the review meeting. This is a very simple questionnaire which has 3-5 survey questions on project progress, and suitability of both the project and the technological approach to satisfy the customer(s) needs. Typical questions might be:  Does this project meet your product technology needs?  Do project milestone dates meet your market window?  Have any strategy changes on your part not been addressed?  What is the overall project score (typical scale 1-5)?  As a related metric, trend analyses can be made both by project and organizationally. 37
  38. 38. PLATFORM 3: VIII (3) Customer Satisfaction (5.5) Options and Variations  One variation on the customer satisfaction process is to establish steering teams to address technology, business and market issues and provide guidance to the R&D organization on strategic issues. For external issues, the teams might consist of focus groups that meet periodically, or focus teams that convene groups which are diverse either geographically, ethnically, or with respect to age group, for example. The focus might be on functional needs that technology capabilities address. The metric would be meetings held, or issues addressed and settled. Another metric might be problems surfaced to be addressed and reported on by the R&D team.  Internal issues might be addressed by customer teams composed of manufacturing, engineering, marketing, financial, and related personnel (including even external consultants) who provide guidance and assist in forming cross- functional project teams. Appropriate metrics are meetings held, issues settled, or problems surfaced. An ongoing metric can also keep track of the % of problems addressed and resolved versus those surfaced. 38
  39. 39. PLATFORM 3: VIII (3) Number of Ways Technology is Exploited (5)  Definition  This metric assess the number of ways a technical asset can bring value to the corporation.  Advantages and Limitations  This metric is applied to gauge project attractiveness, or to understand the value of a technical asset already developed. It is generally agreed that a larger number of potential uses, both within the companys current markets and in markets not yet developed, makes a technical asset more valuable. The metric is a bit arbitrary and can be misleading in cases where there are few, though very large and/or lucrative commercializations. 39
  40. 40. PLATFORM 3: VIII (3) Number of Ways Technology is Exploited (5)  How to Apply the Metric  This metric is applied by taking an existing or potential technical asset, such as a project to develop a new type of lower cost, light weight composite material, and conducting a thoughtful analysis of how many ways this asset can be exploited commercially. The count could consider:  Number of business units in the corporation that could make use of the asset  Number of markets the company serves that could be impacted by this technology  Total number of markets served by the corporation and other companies where the technology may have an impact.  Number of products that could utilize the technical asset. 40
  41. 41. PLATFORM 3: VIII (3) Number of Ways Technology is Exploited (5) How to Apply the Metric (continued)  This Used in this way, the metric is a single numerical value. A larger number of potential uses means that the corporation is not depending on a single or small number of products to succeed in order for the technical asset to deliver value. The risks associated with the exploitation of the technical asset are spread over a larger number of potential uses. A larger number also provides greater opportunity for unforeseen benefits, like taking the company into new markets and new products.  Using the low cost, light weight composites as an example, the primary market for the company may be the automotive market, with four different auto parts that could use the strong, light weight tubes produced by the new process. In addition, there could be a market for the technology in the aeronautical industry, served by another business unit of the company. The third exploitation could be in high performance bicycle frames, a market that is new to the company. 41
  42. 42. PLATFORM 3: VIII (3) Number of Ways Technology is Exploited (5) Options  An option in the implementation of this metric can involve adjusting the number for relative importance of the commercializations, or keeping sub-metrics of the number of exploitations with certain value ranges as demonstrated with the continuing example of the light weight composites. 42
  43. 43. PLATFORM 3: VIII (3) Projects with Business / Marketing Approval (5)  Definition  Percent of projects in the total R&D portfolio with explicit business unit and or corporate business management sign-off.  The intent of this metric is to provide an indicator of the degree of alignment with business and corporate strategy and tactics. The metric is closely related in some corporate structures to metric: "percent Funding by the Business“. 43
  44. 44. PLATFORM 3: VIII (3) Projects with Business / Marketing Approval (5) Advantages and Limitations  Advantages: Several studies have suggested that close alignment of R&D to marketing and to business and corporate strategies increases the odds of success for new products and processes. Thus actions which drive this metric to higher values can be expected to improve the amount of R&D spent on successful projects and the predictability of the outcome from R&D efforts.  Limitations: The metric will be valuable to the extent business/marketing management and R&D management jointly develop strategy and plans. Use of the metric to drive R&D without such teamwork will likely lead to short term projects and suboptimal use of R&D resources. In those companies where R&D is corporately funded, business/marketing management may also be tempted to give approval to projects in their market segments to insure that they receive "their share" of R&D resources. Finally if the corporation uses a formal innovation process which requires business/marketing approval at some stage, the metric runs the risk of becoming a measure of compliance with use of the process or a measure of the percent of project past the approval stage. 44
  45. 45. PLATFORM 3: VIII (3) Projects with Business / Marketing Approval (5) How to Use the Metric  Explicit approval may be sought at any point in the innovation process. Seeking approval early in the innovation process probably provides maximum value. One form of approval is the provision of a sales forecast from marketing management for each new product and agreement to commercialize if the product meets technical requirements in a timely manner.  The level of approval from the marketing/business management and the point where approval should be sought should be explicitly defined if the firm uses a formal innovation process. If not, the level should be commensurate with the amount of R&D resources and commercialization resources which will be required. 45
  46. 46. PLATFORM 3: VIII (3) Projects with Business / Marketing Approval (5) Options and Variations  For projects having broad corporate strategic value, approval of a director of corporate planning or director of corporate business development might be an appropriate substitute for the business/marketing management approval. For corporations where out-licensing of technology is a major thrust, approval of a director or vice president of licensing may be an appropriate substitution. 46
  47. 47. PLATFORM 3: VIII (3) Quality of Personnel (3.3) Definition  This is a measurement of the skills and ability of the R&D staff to execute strategic programs. Internal Customer Ratings (a)  Internal customers rate the quality of the R&D staff on their ability to execute programs. Measures such as percentage of mileposts met versus project plans, novelty of concepts, patentability of concepts, and competitive advantage of the technology are parameters that can be considered. External Customer Ratings (b)  External customers rate the quality of the R&D staff of their ability to meet customer expectations. Problem solving, novelty of approach, responsiveness, knowledge of customers operations are parameters that can be considered. External Recognition (c)  Publications in refereed or industry trade journals, external presentations, citations in the literature, invited lectures and patents are parameters to be considered. 47
  48. 48. PLATFORM 3: VIII (3) Quality of Personnel (3.3) Advantages and Limitations  The internal and external customer ratings measure the ability of R&D to meet customer expectations and contribute to the growth of the corporation or enhance competitiveness. They are largely objective measures that can be tied to tangible value. The value of external recognition via patents, publications and presentations is more difficult to measure objectively. The numbers of different subjects covered by public disclosures should be evaluated rather than the total number of all disclosures. It is easy to become subverted to a self servicing metric if only numbers are considered. Maintenance of technology as trade secrets must be considered in this evaluation. 48
  49. 49. PLATFORM 3: VIII (3) Quality of Personnel (3.3) How to Use the Metric  Internal customer surveys can be conducted using a 1 to 4 scale for rating. A 1 represents below standard execution on a given project. The causes for this poor performance have to be determined since they could arise from inadequate skills, poor judgment, lack of responsiveness, poor planning, etc. The causes may not be related to the quality of the personnel but poor management practices. Having multiple internal customers (marketing, manufacturing, sales, etc.), conduct the evaluation is a form of 360o review. Superior performance by reaching targets ahead of schedule, lower than expected costs, developing a significant competitive advantage, etc., should be rated as a 4. In establishing this survey system, agreement should be reached on the different levels of performance. The survey should be applied to different projects with the same population of the internal customers as raters. This rating should be conducted on a regular basis and over time trends will emerge. 49
  50. 50. PLATFORM 3: VIII (3) Quality of Personnel (3.3) How to Use the Metric (continued)  External customer surveys should be conducted using the same 1 to 4 scale. A 1 rating would indicate that the customer was not satisfied with the parameter being measured, while a 4 would indicate that expectations were exceeded. Key parameters should be selected beforehand and could include timeliness of response, knowledge of products, knowledge of customer s operations and knowledge of customer needs. The parameters will vary by industry. Several levels of the customer s organization should be sampled such as plant operators, first-line supervisors and management. A simple postcard type of survey instrument mailed after customer contact can be used. A database can be developed over time and trends will emerge.  The subjects covered in public disclosure should be tabulated and compared to strategic technology goals. A subjective 1 to 4 rating system can be created by R&D management to determine fit with the goals. A rate of 1 corresponds to a poor fit and a need to enhance skills, while a 4 implies that all areas are being addressed. 50
  51. 51. PLATFORM 3: VIII (3) Quality of Personnel (3.3) Options and Variations  Service and consumer product companies may find the external customer survey to be a valuable tool in assessing the effectiveness of their R&D organization. One may also measure the quality by the number and type of external awards from recognized organizations (ACS, AIChE, IRI ...). 51
  52. 52. PLATFORM 3: VIII (3) Efficiency of Internal Technical Processes (3.3) Definition  This set of metrics seeks to provide a measure of both the efficiency and effectiveness of the operation of R&D processes within the firm. Project Assessment  The total cost of all commercially successful projects divided by the number of commercially successful projects. (Useful when tracked over time with similar projects with similar scopes).  The ratio of actual to projected costs (and timing) for all projects.  Percentage of costs devoted to commercially successful projects. Portfolio Assessment  The total R&D budget divided by the number of projects with commercial output. Subdivide by projects of similar type (technical service, short term, long term) and used in conjunction with project value assessment. 52
  53. 53. PLATFORM 3: VIII (3) Efficiency of Internal Technical Processes (3.3) Advantages and Limitations  This set of metrics needs to be adapted to the needs of each firm -- considering the goals, objectives, and priorities for the firm. Assessments need to be made for individual projects (stage-gate , PACE, or similar processes), and for the collection of projects (Portfolios). 53
  54. 54. PLATFORM 3: VIII (3) Efficiency of Internal Technical Processes (3.3)  Options and Variations  Each firm will need to set the metrics relative to its specific goals and objectives. An assessment of the selection termination and management of projects can be made using the following four stages for “Technical projects: Selection, termination and project management”. 54
  55. 55. PLATFORM 3: VIII (3) Efficiency of Internal Technical Processes (3.3)  Stage 1 Technical projects: Selection, termination and project management A. favors short term projects B. politically driven selection C. no project monitoring or pre-project planning D. little inter-functional participation in project teams E. erratic turnover of team staffing F. project leader roles not defined G. no training for project leaders H. unclear charters for project teams 55
  56. 56. PLATFORM 3: VIII (3) Efficiency of Internal Technical Processes (3.3)  Stage 2 Technical projects: Selection, termination and project management A. mix of short and medium-term projects B. no inter-product-line analysis C. priorities set erratically D. project tracking E. some inter-functional participation but not all key functions represented. F. formal release process for new products G. some project team stability but conflicts over work H. priorities I. project leaders given only minimum guidance or training 56
  57. 57. PLATFORM 3: VIII (3) Efficiency of Internal Technical Processes (3.3)  Stage 3 Technical projects: Selection, termination and project management A. selection based on multiple inputs from internal and external sources B. balance of short-, medium- and long-term projects C. risk analysis incorporated at key phases D. projects still schedule driven E. Inter-functional teams wherever needed F. clear allocation of project and functional responsibilities G. training for project leaders 57
  58. 58. PLATFORM 3: VIII (3) Efficiency of Internal Technical Processes (3.3)  Stage 4 Technical projects: Selection, termination and project management A. clear links between selection criteria and business and product-line strategy B. disciplined process for project termination C. cross-functional planning and execution D. continual improvement- postmortems, quality measures of both project process and product E. performance F. projects are milestone driven G. differentiated project management procedures for different types of projects H. scheduling and capacity planning avoid resource contention by competing projects 58
  59. 59. PLATFORM 3: VIII (3) Product Quality and Reliability (3)  Definition  Customer or Consumer Evaluation (a)  Relative quality and reliability compared to competitive products through evaluation by customers or consumers.  Reliability/Defect Rate Assessment (b)  Fraction of a firms output, either by individual product or by sum of all products, that meets or exceeds the established quality standards. 59
  60. 60. PLATFORM 3: VIII (3) Product Quality and Reliability (3)  Advantages and Limitations  The advantage of this metric is that product benefits resulting from R&D activities are directly evaluated by the customer or consumer. Comparison with the competitors products is usually the basis for evaluation. Limitations of the metric chiefly are related to the reliability and accuracy of the survey techniques chosen as appropriate for the industry, though firms usually gain confidence in their preferred methods through repeated use and incremental improvement.  Similar to the preceding discussion, the advantage to this metric is that the direct benefit from R&D activities can be obtained through specific measurements made by the firm. 60
  61. 61. PLATFORM 3: VIII (3) Product Quality and Reliability (3) How to Use the Metric  For the Customer or Consumer Evaluation metric, each firm will generally have a preferred technique for directly or indirectly obtaining data showing how well the firms products perform in comparison to competitive products. Data for the Reliability/Defect Rate.  Assessment result from internal quality measurements. When both metrics are utilized, the impact of product quality improvements on customer satisfaction should be demonstrated. Product Quality and Reliability metrics are retrospective, showing the results of past technology or product introduction to the market. Options and Variations  This metric fits well with the trend toward greater input to R&D planning from customers and consumers and with the "Quality" protocols that have adopted by firms of all types. Though basically a retrospective measure, product needs that arise during the data collection can be used for prospective purposes. 61
  62. 62. PLATFORM 3: VIII (3) Market Share (3)  Definition  Firm (or business unit) market share in various product categories measured as appropriate for the industry or category, expressed as a percentage of the total market.  Advantages and Limitations  This metric is meant to reflect value creation for the firm and the value of the firms technology. Similar to Gross Profit Margin metrics, caution must be taken in the interpretation of Market Share data from the perspective of measurement of the contribution of R&D activities to the whole. There can be many confounding factors in a market share determination, such as the size and quality of the marketing effort, the competitive response, the relative state of the economy, etc. 62
  63. 63. PLATFORM 3: VIII (3) Market Share (3)  How to Use the Metric  Changes in Market Share should be assessed at least annually to determine the rate of progress or decline. The expectation is that improvement in a firms technologies and products will result in a greater share of the market. Market Share is a retrospective metric, showing the results of past technology or product introduction to the market. Since competitors market share data is usually also available, this is a metric that can be used as a benchmark with the competition.  Options and Variations  As an indication of threats or opportunities, share data in markets related to a firms products can be followed. This "Related Market Share" metric can serve as a component of a strategy to anticipate the potential application of similar technology into a firms marketplace. 63
  64. 64. PLATFORM 3: VIII (3) Development Pipeline Milestones Achieved (2.3)  Definition  Development Pipeline Milestones Achieved is a metric which is useful in grading the effectiveness of management and planning of each R&D project. It may also be used as an indicator of performance problems on a given project which can be used to initiate diagnostic and recovery procedures to give each project the best possible chance of success. There are two possible variations of the metric which may be used: 1. Percent of project milestones achieved -- the percent, by project or overall by sub-organization or laboratory, of all project milestones completed on schedule or within some acceptable time window (90 days, for example) of the forecast date. Trend studies can then be established for organizational performance based on analysis of the data collected by quarter or for whatever other time period is appropriate for the particular industry group. 2. Performance level at each milestone -- on a project basis, percent of all expected objectives met at the milestone date by which they are forecast to be completed. 64
  65. 65. PLATFORM 3: VIII (3) Development Pipeline Milestones Achieved (2.3)  Advantages and Limitations  The primary advantages of this metric are for project diagnostics and as an indicator of the overall planning and management health of the organization. There are several possible limitations. First, a consistent management system must be in place to assure that the variability of the number/quality/difficulty of the milestones does not cause random fluctuations in the metric. Secondly, the calibration of the metric is important. Since both the stage of the R&D project and the industry of the business involved can greatly affect the trend analyses, the metric user must be careful to identify what the real danger signals are in terms of management and planning deficiencies, and what really constitutes an indicator of project problems for this metric to be useful. 65
  66. 66. PLATFORM 3: VIII (3) Development Pipeline Milestones Achieved (2.3)  How to Use the Metric  Percent of project milestones achieved -- a simple approach is to tabulate milestones met on schedule, 1-90 days late (which may constitute on schedule in some cases), 90-180 days late, and so forth. The tabulation can be done quarterly, and the trend information publicized to the organization to focus attention on performance issues.  Performance level at each milestone -- This is more useful as a project diagnostic. One approach is to require project managers to use this metric to summarize performance quarterly or semiannually at an operations review and be prepared to cover diagnostic or recovery steps if a given project shows trends over two or more time periods of failing to meet an acceptable performance level (which will depend on the R&D discipline, the industry, and the stage of the innovation process). 66
  67. 67. PLATFORM 3: VIII (3) Development Pipeline Milestones Achieved (2.3)  Options and Variations  One problem with using milestone achievement as a key metric is that pressure is exerted to make all the milestones. With strong pressure to meet milestone dates, the project manager may be tempted to populate the project plan with easy milestones, to assure that his milestone hit rate is good. It is important therefore to make sure that milestones are realistic and aggressive. Especially in the later stages of the innovation cycle, in product design and productization or manufacturing process development, it is important to assure that the business customer, marketing representative, etc. approve the project milestones to assure that the product development cycle meets the market window requirements of the business. In this case, a much closer tracking of milestone achievement may be necessary than in the earlier stages of the innovation cycle (for example, monthly or in some cases even more frequently). 67
  68. 68. PLATFORM 3: VIII (3) Development Pipeline Milestones Achieved (2.3)  Options and Variations (continued)  Another problem with using the milestone approach is that early in the innovation cycle (the R part of R&D), milestones are often hard to define and even harder to forecast. With the heavy level of uncertainty in a long-term research project, the definition of a milestone may be vague, or even made in terms such as define or develop concept. In these circumstances, project managers feel especially uneasy in setting milestone dates for which they become accountable. An approach here is to realize that (1) exact dates are less important, since product or market need windows probably have not been established (and in some cases, market needs or even product existence!), and (2) many of these projects which are early in the innovation cycle will certainly fail, since in the research phase, many more ideas are explored than have a positive outcome. In this context, using milestones as a performance metric may not be useful. If used, the metric should probably be relaxed in some way. For example, instead of setting a goal of completing 100% of milestones, perhaps a goal of 50, 60, or 80% may be set. Another possibility is to declare any milestone made within 90 or 120 days of the forecast date to be on time. The specific approach taken will depend on the industry, the research area, and the stage in the innovation cycle. 68
  69. 69. PLATFORM 3: VIII (3) Goal Clarity (2)  Definition  This metric uses an interval rating scale assessing the extent to which project performance objectives are clearly defined and understood by all participants on the project team.  Advantages and Limitations  While this metric may provide a semi-quantitative assessment there is a degree of subjectivity for an individual to assess the extent to which he/she really understands objectives and roles on the team.  How to use the metric  A member of the team would provide to all members of the team a survey form to assess understanding of the project objectives and commitments. This can be done using some interval rating scheme. It would be wise to conduct this survey several times through the life of the project to really assess the level of understanding. 69
  70. 70. PLATFORM 3: VIII (3) Goal Clarity (2)  Options and Variations  A four stage assessment can be constructed with the following as the highest stage for individual team members: a) clear understanding of the expected product of team effort b) personal belief that a team is the right way to develop/achieve the expected product c) understanding whether the team is an implementation, recommendation and/or informational team d) understanding the teams operational ground rules and end-result boundaries e) belief that the team has all the appropriate knowledge,functions, diversity, levels and locations, so that the expected product of the team effort can be achieved using the minimum number of people? f) high personal commitment to achieving the objectives g) clear understanding of personal role h) acceptance of the recognized team leader i) capability to draw on additional resources to keep the core team to a minimum 70
  71. 71. PLATFORM 3: VIII (3)Comparative Manufacturing Cost (1.8)  Definition  Benchmarked manufacturing cost data vs. competition for same type of unit cost.  Advantages and Limitations  It is extremely useful and in some companies paramount to know how R&D is helping to provide an advantaged cost position to the operations. This metric reflects the quantification of that goal. Unfortunately, cost accounting and even further comparative cost accounting can be extremely difficult. 71
  72. 72. PLATFORM 3: VIII (3)Comparative Manufacturing Cost (1.8)  How to Use the Metric  While most firms have very accurate manufacturing cost data for themselves, the generation of accurate manufacturing cost data for competition is considerably more difficult. Therefore, when using this measurement, there should be an estimate made of the variance of the competitive estimates.  Options and Variations  This metric is intended to be based on unit process comparisons. There can be many options created that are aggregates of production processes, but these simplifications can be misleading. Therefore caution is urged. 72
  73. 73. PLATFORM 3: VIII (3) Gross Margin (1.8)  Definition  Gross Profit as a percentage of sales, where gross profit equals net sales minus cost of goods sold (product costs plus direct manufacturing costs).  Advantages and Limitations  To some extent, gross profit margin reflects value of the firms technology assets and the value created by R&D. However, raw material, production, and distribution costs also directly affect the firms gross profit margin. Each firm should attempt to understand the correlation of Gross Profit Margin to R&D. 73
  74. 74. PLATFORM 3: VIII (3) Gross Margin (1.8)  How to Use the Metric  Value assessment should be based on change in gross profit margin from period to period. (Periods should be appropriate to an industry and may be in excess of one year.) Changes in Gross Profit Margin in relationship to changes in values of other metrics (e.g., financial return , technology transferred to manufacturing, sales protected by proprietary position) should be followed in an attempt to uncouple the contribution of R&D from other factors. This is a retrospective metric that can be used as a benchmark with the competition, if gross margin data from competitive firms are available. 74
  75. 75. PLATFORM 3: VIII (3) Use of Cross-Functional Teams (1.8)  Number of Cross-functional Teams  Current management philosophy suggests that the use of cross-functional teams will improve the effectiveness and efficiency of the R&D Process and will help R&D to be integrated with the business. The number of such teams can be counted if they are established on a formal basis.  Evaluation of the use of Cross-functional Teams  A firm may rate its practice of using cross-functional teams by using a subjective scale ranging from 1 to 4. Firms operating at Level 1 are characterized by the existence of strong organizational boundaries, lack of cross-functional involvement in R&D projects, and no cross-functional team structure. Firms operating at Level 4 are characterized by a well developed and supported team structure which effectively places all R&D work in cross-functional teams responsible for the entire project rather than functional silos responsible for parts. This metric is one of many relating to how well the firm conducts the Practice of the R&D Process. It also relates to how well R&D is integrated with the business. 75
  76. 76. PLATFORM 3: VIII (3) Use of Cross-Functional Teams (1.8)  Advantages and Limitations  This metric will be important if the use of cross-functional teams contributes to the effectiveness of the R&D function in the firm, given its peculiar situation. This is usually thought to be the case, but there may be cases where other factors are more dominant.  Evaluation of the use of teams can be used to diagnose problems with the organization or the R&D process and to plan for improvement.  How to use the Metric  A simple count of teams is rarely as valuable as an assessment of the how the firm uses such teams. This can be accomplished using the rating scale suggested above. Input for the evaluation should be gathered from a broad cross-section of the firm. The evaluation is important for planning improvements.  Trends across time are probably more valuable than benchmarking. 76
  77. 77. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  Metrics for Product Features & Benefits:  Metric 1: Competitive Technical Performance of Product (Project Metric)  Comparison of technical performance of a product in those dimensions where the customer is likely to perceive a benefit. This may be used in a prospective sense to appraise the value of a feature or in a retrospective sense to register the value of a benefit which the market has recognized.  Examples include:  The use level required to achieve a needed result in the customer application.  The yield strength of a high-performance alloy  The measured softness provided by a textile softener.  The measured UV resistance of an external architectural coating. 77
  78. 78. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  Metrics for Product Features & Benefits:  Metric 2: Customer rating of Products (Business Segment or Firm)  Customer rating using a scale of 1-5 of the technology benefits that is perceived in a firm‟s products. This can be compared to the rating for the best competitor, usually in the form of a ratio. This is an aggregate subjective measure for a business segment or for the firm.  Metric 3: Economic Value of Products (Project, Business Segment, or Firm)  This metric is the price differential per unit obtained by virtue of the technology feature minus the cost of providing the feature. The differential can be multiplied by the volume to assess the total benefit to the firm. 78
  79. 79. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  Metrics for Product Features & Benefits:  Metric 4: Market Share Evaluation (Business Segment or Firm)  If differential pricing does not occur, the advantages of superior product technology can appear as differential market share. In this case, the relative market share (the firm‟s share divided by the largest share) can be used as a surrogate for the value of technology embodied in the products. 79
  80. 80. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  Metrics for Process Features & Benefits:  Metric 5: Competitive Technical Performance of Process (Project Metric)  Comparison of technical performance of a product in those dimensions which are important to manufacturing cost or product performance. This may be used in a prospective sense to appraise the expected value of a new or improved process or in a retrospective sense to register the demonstrated value.  Examples include:  Manpower requirements  Efficiencies of raw material conversion  By-product or co-product costs or values  Consistency, controllability, and other such quality parameters. 80
  81. 81. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  Metrics for Process Features & Benefits:  Metric 6: Economic Value of Processes (Project, Business Segment, or Firm)  The differential in profitability (versus the target or competitor) attributable to new or improved process technology.  Metric 7: Profitability Evaluation of Processes (Business Segment or Firm)  In the same way that Market Share is a surrogate measure of product performance in those business areas where the basis of competition is product performance, overall profitability in a business segment of the firm is a measure of process performance, in those business areas where the basis of competition is cost and/or quality. 81
  82. 82. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  Advantages and Limitations  Product Metrics 1 through 4 attempt to assign value to technology used in products. However, differential value or market share are normally the result of many different factors. These metrics can give some indications if factors are carefully sifted, but may be misleading if the analysis is superficial. Objective measurement of product performance and customer ratings relative to competitors are the most accurate measures of product technology. But note that comparison of features that the market has not recognized as benefits may be self-serving and deceptive. Competitive rankings of measured product performance and or customer ratings may be averaged over market segments or over the firm to obtain average values.  Process Metrics 5 through 7 attempt to assign value to new or improved process technology resulting from R&D. However, economic value and profitability are normally the result of many different factors. Objective measurement of process performance is the most accurate measure of process technology. 82
  83. 83. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  How to Use the Metric:  Metric 1  Define the key parameters which measure features the customer is likely to perceive as benefits. Measure product performance as accurately as methods allow. Compare to the same measurements of competitive products. Rank performance versus best competitors.  Metric 2  Ask customers to rate the technology attributes of a product line relative to solving their problems. This requires a carefully constructed survey instrument.  Metric 3  This metric is valid for products which are differentiated by performance. It is not applicable to commodity products. 83
  84. 84. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  How to Use the Metric:  Metric 4 (See Definition)  Metric 5  Define the key parameters which measure or impact cost or quality. Measure process performance as accurately as methods allow. Compare to the same measurements of competitive processes. Rank performance versus best competitors.  Metric 6 (See Definition)  Metric 7 (See Definition)  NOTE: These metrics may be used retrospectively to measure the output of R&D over the past period and prospectively to set targets for future accomplishments. 84
  85. 85. PLATFORM 3: VIII (3) Response Time to Competitors Moves (1.8) Definition  This metric measures the ability of the firm to respond to new technical innovations introduced by competitors. Depending upon corporate strategy, it could be the time required to match or exceed the competitive offering. Advantages and Limitations  This metric is an indicator of technical leadership in a given field. The technical leader will not spend a significant amount of time matching competitive innovations, while a follower will be more reactive than proactive. The utilization of resources for this function as a percentage of total resources should be tracked over time. An increase in this percentage would indicate that new technology programs are not as effective as desired and technical position relative to the competition is eroding. This metric also measures the flexibility and creativity of the firm to change priorities to meet competitive challenges. A strong market intelligence function is required to identify competitive entries at an early stage in the introduction and to assess the technical merits of the offering so that appropriate responses can be made. The technical merits of competitive offerings must be critically assessed to differentiate from market repositioning of existing technology. 85
  86. 86. PLATFORM 3: VIII (3) Response Time to Competitors Moves (1.8) How to Use the Metric  The time between the introduction of a competitive offering and the internal development of a comparable or superior offering can be measured and compared to product development times for similar products. A rating system of 1 to 4 can be used. 1. Organization slow to recognize significant competitive offering in the marketplace; slow to launch program to respond; unable to get their offering into the marketplace in acceptable time to be combatant. Slow to recognize impact of technical innovations. 2. Organization recognizes need for competitive offering; has trouble in launching program to develop counter-offering and gets offering to market barely in time to have impact. 3. Organization responds to competitive offering and develops counter offering to maintain relative position. 4. Organization anticipates potential for competitive offering and has counter offering into the marketplace with superior product allowing a gain in market/competitive position. Very fast to recognize impact of any hints of technical innovations. 86
  87. 87. PLATFORM 3: VIII (3) Response Time to Competitors Moves (1.8) How to Use the Metric (continued)  The percentage of resources used in matching competitive moves could be measured over time. An increase in spending in this function should raise questions regarding core research and development efforts for organizations who strive for technical leadership. Options and Variations  The importance of this metric will depend upon the technology strategy of the firm. Firms pursuing technical leadership will be very interested in minimizing response time. The importance may vary across different strategic segments in the same firm. 87
  88. 88. PLATFORM 3: VIII (3) Comparative Technology Investment (1.5) Definition  This measures the current annual expenditure for R&D staff and capital compared to the best competitor and/or the industry average. Advantages and Limitations  This metric measures the rate of current activity in developing the technology of interest with the intent of predicting whether the firm is expected to gain or lose ground in the technology. It should be kept separately for the KEY and PACING technologies most critical to the strategy. Retrospectively it measures the efficiency of the investment in meeting new product and technology development goals.  This metric should be as quantitative as possible, but in some industries it may be necessary to make estimates as to the size of the development effort of the best competitor and the industry average. Analyses must be based on a comparison of similar functions. For example, some organizations include sales support as part of their report for technology expenditures. The components of the expenditures under study must be understood in making the comparison. 88
  89. 89. PLATFORM 3: VIII (3) Comparative Technology Investment (1.5) How to Use the Metric  Information on a firm s overall technology expenditures are available in the firm s annual reports or industry publications. These can be used for comparison to internal overall investment. Information by industry is available in industry publications and from organizations such as IRI (IRI/CIMS survey).  Rationing the firm‟s current investment in technology versus the best competitor and industry averages provides an insight into the efficiency of the technology investment. Performance exceeding expectations in value creation goals at competitive investment rates indicates an efficient organization while sub-standard performance raises concern about the quality of the investment. Smaller firms may require an investment higher than industry norms to maintain a competitive position to offset critical mass issues. 89
  90. 90. PLATFORM 3: VIII (3) Employee Morale (1.5)Definition  This metric takes quantitative ratings of key aspects of employee satisfaction and morale as shown by direct employee survey. It is recognized that employees may feel good and have high morale, yet produce nothing of value for the firm -- the real question is are they motivated and committed to create and innovate profitably?Advantages and Limitations  Extensive surveys are time consuming and expensive to conduct and must be conducted with sufficient frequency to establish base lines and understand real trends. One must understand also that technical populations tend to have certain biases in such surveys. Many employees feel "surveyed-out". 90
  91. 91. PLATFORM 3: VIII (3) Employee Morale (1.5)How to Use this Metric  The typical survey uses five point scales for agreement (strongly agree, agree, neither agree or disagree, disagree, strongly disagree) for importance (extremely important, very important, somewhat important, of little importance, not at all important) and for performance (very good, good, fair, poor, very poor) in answering sets of questions related to work environment, feelings about the company, ratings of the company, ratings of the organization/work location/work group, feelings about the individuals job, and general satisfaction. Opportunities are given for comments. Such extensive surveys are most often conducted by third parties to maintain confidentiality. 91
  92. 92. PLATFORM 3: VIII (3) Employee Morale (1.5) How to Use this Metric (continued)  Four tested questions of job satisfaction are: 1. If a good friend was interested in a job like yours for your firm, what would you tell the friend? 2. All in all, how satisfied are you with your present job? 3. Knowing what you know now, if you had to decide all over again, would you take your current job? 4. 3.4 How satisfied are you with the overall employee- employer relations at your firm? 92
  93. 93. PLATFORM 3: VIII (3) Employee Morale (1.5)  Options and Variations  One firm which uses e-mail extensively, every 9-14 months conducts a broad "pulse" survey and asks employees to provide two ratings (using 0-10 point scales with descriptors) one rating the employees work climate/environment and the other rating personal feelings about the work itself. Employees spend less than 4 minutes to reply by e-mail, or to be anonymous by fax or to a voice mailbox. Employees often write additional comments that give information sought in the more extensive surveys. Confidentiality is assured and rapid feedback (within 2 weeks) of survey results to participants maintains a high level of interest and participation. 93
  94. 94. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model  Definition of the Innovation Process  The term "innovation process" refers to the overall process in a company for conceiving and developing ideas and concepts into profitable products and processes for the company business(es). It is one of the 5 to 7 overall high- level business processes within the corporation. In order to maintain, diagnose, and improve the process, both effectiveness and efficiency metrics are required to support decision-making.  Note that in the context discussed herein, the assumption is made that the innovation model will be technology-based. That is, innovation is founded in the efforts of the R&D organization, and that the overall function of the process is the embodiment of technology in the products of the business of the corporation. Progress through the full innovation process will involve the efforts of all corporate resources, including marketing, engineering, manufacturing, and service and support functions. 94
  95. 95. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model Relevance of Innovation Process Model to Metric Choice  Often stakeholders in the innovation process -- R&D management, financial management, and corporate officers -- need to utilize metrics which supply specific decision support information about corporate capabilities in innovation. These metrics are overall, or outcome metrics, which provide a grade or rating of the overall result of the innovation capability of the corporation or business in question. Such metrics might relate to value creation by the process, effectiveness of the overall process in terms of new product revenue or profit versus total R&D investment, or the ratio of new product revenue or profit to total company or business revenue or profit.  However, these metrics are often long-term and retrospective in nature, and while they serve to evaluate the overall process as a whole (and generally, but not always, on a lagging basis), they may offer little, if any, insight into the current or future state of the corporate innovation process or its many and diverse activities. 95
  96. 96. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model Relevance of Innovation Process Model to Metric Choice (continued)  To be useful for the on-going management and evolution of corporate innovation, a diverse set of metrics must be available which offer insight into current condition and future vision state of the process and many of its constituent parts. For many such metrics to be useful, they must be not simply outcome-oriented, but process- oriented, and specifically diagnostic of discrete sub-processes within the overall innovation process.  Each in-process metric must be related to a process model which ties it to the portion of the process of interest, from early idea or technology exploration through proof of concept to technology development to productization. The following process description is a relatively simple process model and defines the various sub-processes within that model in order to provide anchors for useful metrics within the innovation process. 96
  97. 97. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model Overall Description of the Stage Gate Model for the Innovation Process Used in this Metrics Guide  There are a number of possible representations of the corporate innovation process. One simple yet fairly effective representation is the so-called "stage gate model".  The model in Stage Gate divides the overall innovation process into four stages, from concept exploration or "ideation" to the process of productization or commercialization. This division into four sections is arbitrary. An argument can be made for a three-stage model, or a more complex model of multiple sections and sub-sections. The four-stage model is chosen primarily for convenience, and to match a number of corporate models and those defined by scholars in the area of business processes. The shape of the model recognizes several realities of modern business.  For example, the process funnel narrows as the process proceeds from exploratory research or concept exploration through proof-of-concept to technology development and commercialization, illustrating pictorially the selective filtration that occurs in the process. There are typically many more ideas and concepts that are explored than are developed into significant technology capabilities in a business, and fewer still that emerge finally into finished products. In each stage of the process, the candidates dwindle, until only the most promising are brought to full production. 97
  98. 98. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model Overall Description of the Stage Gate Model for the Innovation Process Used in this Metrics Guide (continued)  Similarly, the four gates, labeled A, B, C, and D, stand for the tests or decision-making activities that are exercised between the stages in the innovation process. Economic reality imposes a limit on the total level of technology and product development that each company can support. A second reality is that as each decision gate is passed, the resources and funding required to carry a given project to the next stage increases dramatically; a good rule of thumb is a 10X expenditure increase at each gate. These corporate "facts of life" impose a stringent set of conditions on the suitability of R&D programs which approach a gate. Each gate requires a set of metrics which ensure that only those programs most suitable to meet corporate business needs pass into the next sub-process.  The following sections describe the decision gates and stage processes shown in the stage gate model. Underlined terms link to appropriate metrics for the gate or process under discussion. 98
  99. 99. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model Description of the Major Sub-process Divisions in the Innovation Process Model  As noted in the introduction to the process model, the division of the innovation process model is arbitrary, but in general it seeks to approximate reality. The four divisions cover the very early technology or idea exploration phase of innovation (when possibilities are defined), a proof-of- concept phase when the mapping of ideas into the realities of the business world occurs, the development, and finally commercialization. Each of the separate phases, or sub-processes, is covered in the following sections, together with the entry gate which defines conditions for admission to that sub-process. Refer to the stage gate model in the references that follow. 99
  100. 100. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model Gate A Gate B Gate C Stage I Gate D Stage II Stage III Stage IV 100