Innovation Benefits Realization for Industrial Research (Part-8)

  • 669 views
Uploaded on

 

More in: Business , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
669
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Technology Innovation Management Framework for Industrial Research Part-8 Dr. Iain Sanders January 2005
  • 2. 2
  • 3. 3
  • 4. 4
  • 5. WHAT R&D METRICS? Platform 3: (Stages VIII, Part 3)Integrating Technology Innovation with Business Function – Part I: Laying the Foundation
  • 6. PLATFORM 3: VIII (3) What are our R&D Metrics?  R&D Metrics  Set standards for measuring new concept R&D performance and evaluating progress / outcomes achieved against milestones / targets set. 6
  • 7. PLATFORM 3: VIII (3) Top 39 Ranking R&D metrics11.5 Strategic Alignment 5.5 Development Cycle Time 3 Product Quality and Reliability11.5 a. Corporate & business unit 5.5 a. Market cycle time 3 a. Customer Evaluation11.5 b. Goal coverage 5.5 b. Project management cycle 3 b. Reliability/Defects time10 Financial Return 5.5 Customer Satisfaction 3 Market Share10 a. New Sales Ratio 5 Number of Ways Technology is 2.3 Development Pipeline Milestone Exploited Achieved10 b. Cost Savings Ratio 5 Projects with Business/ 2 Goal Clarity Marketing Approval10 c. R&D Yield 5 a. Project has approval 1.8 Comparative Manufacturing Cost10 d. R&D Return 5 b. Ratio of projects 1.8 Gross Margin 9 Projected Value of the R&D 3.3 Quality of Personnel 1.8 Use of Cross-Functional Teams Pipeline 9 a. Projected Sales 3.3 a. Internal Customer Rating 1.8 Rating of Technology Features and Benefits 9 b. Projected Income 3.3 b. External Customer Rating 1.8 Response Time to Competitors moves 7.8 Distribution of Technology 3.3 c. External Recognition 1.5 Comparative Technology Investment Investment 6.8 Use of Project Milestone/ 3.3 Efficiency of Internal Technical 1.5 Employee Morale Stage-Gate Process Processes 7
  • 8. PLATFORM 3: VIII (3) Strategic Alignment Metric (11.5) Definition  This R&D metric assess the degree of alignment of an R&D project or an R&D portfolio with the strategic intents of the company or a division of the company. The strategic intents are often the corporate goals embodied in its business plan. Why it is used  This metric is used to gauge the degree of relevance of the R&D program to the corporate goals. The strategic intents of a company may change more rapidly than the R&D program can respond to those changes creating various degrees of misalignment. The misalignment can be with regard to work area, long term versus short term needs, or degree of risk. 8
  • 9. PLATFORM 3: VIII (3) Strategic Alignment Metric (11.5) How to use metric  There are a number of ways this metric can be applied, including both prospective and retrospective views. It can be applied by R&D management, general management, or by both working in partnership. An alignment index would be assigned to each project; a linear scale of 1-5, for example, would work well. these scores may also have value when considering relative merit of individual projects in the portfolio. A composite score for the entire portfolio would then be determined. This could be a weighted average reflecting sizes of projects with regard to technical head count, project budget or some other appropriate weighting factor. Once a baseline for alignment has been established, R&D management can then decide if and how this index should migrate to greater or lesser degrees of alignment through modifications of the portfolio. Applied to individual projects, there could be a cut-off point for the alignment parameter below which projects are not supported.  Consideration must be given to the degree of alignment desired. Although the desired state in may cases is toward greater degrees of alignment, you can envision situations where that may not be the case. A research organization charged with taking the company in new directions may not want its project portfolio highly aligned with the current business plan. A more visionary business plan may capture new directions as well as current businesses, but many organizations find that some degree of decoupling is desirable. 9
  • 10. PLATFORM 3: VIII (3) Strategic Alignment Metric (11.5) Options  Both retrospective and prospective views are options of this metric. The retrospective view entails applying the metric to an existing portfolio of R&D projects to determine the degree of the portfolio with corporate goals.  The prospective view is to apply the metric to a proposed project or slate of projects. If the R&D organization is trying to increase its alignment index, then the management will be less likely to initiate projects that move the composite score in the wrong direction. Similarly, an alignment index cut-off may be instituted. Projects falling below some minimum value of alignment would not be supported. 10
  • 11. PLATFORM 3: VIII (3) Financial Return (10) New Sales Ratio (a)  The New Sales Ratio is the % of current sales originating from new products. There are two sub-definitions that are required. What is a new product? and, How old is new?  The most frequently used and simplest definition of a new product is any new SKU (inventory code: Stock Keeping Unit) that has required R&D support to implement. This avoids counting new SKUs which are only packaging changes or other modifications made easily by marketing or manufacturing. When is a new product old is different for each business and technology. In rapidly changing and evolving fields, such as electronic chips and software, new might only be one year, but certainly not more than three. Three years is more likely the norm for businesses that are a mix of fashion and formulation, such as cosmetics & toiletries. And, for more intense capital and industrial products, three to seven is a more likely range to select a number that is right for you. 11
  • 12. PLATFORM 3: VIII (3) Financial Return (10) New Cost Savings Ratio (b) The Cost Savings Ratio is the % reduction in cost of goods or cost of operations (including depreciation charges) that are realized in a year to year comparison that have originated from technology changes that are new. Again, the same issues must be resolved as sub-definitions as required for the New Sales Ratio. In other words, what is being attributed to R&D and how long is new?  Since SKUs are not used to catalogue changes made in operations an alternative must be found that will work for your firm. A simple solution can often be created based on the capital approval process. Most companies require a specific approval for all individual capital projects. These can be coded and tracked for R&D involvement and for realized cost savings. However, it is recognized that the accounting involved in examining the impacts on cost savings may be more difficult than that for new sales. It is therefore a more common metric only in those cases which are more significantly impacted by cost savings than by new product sales. And, since new technology for operations or manufacturing has a different useful life than a new product per se, it must be tracked for a different length of time that is industry specific. In some cases, it may be very linked to the product life in other cases, it may go on much longer. In any case, it is not likely that new will reach beyond 7-10 years. 12
  • 13. PLATFORM 3: VIII (3) Financial Return (10) R&D Yield (c)  R&D Yield is the contribution of R&D to current financial performance. It is a metric that is composed of definitions from New Sales Ratio and Cost Savings Ratio, plus an evaluation of gross profit from the new sales.  It is the annual combined financial benefit that is derived from the annual gross profit of new products and the annual cost savings of new processes. This is the current contribution that the company receives that is associated with its past stream of R&D investments, i.e. the part of the bottom line that is relatively new and derived from R&D. 13
  • 14. PLATFORM 3: VIII (3) Financial Return (10) R&D Return (d)  R&D Return is the relative ROI measure that relates to R&D. It is composed of the R&D Yield divided by the annual investment in R&D. Hopefully, this is a large number that is proportional to the risks and variances that are part of R&D. 14
  • 15. PLATFORM 3: VIII (3) Financial Return (10) Advantages and Limitations  The advantages of these financial metrics are that they relate directly to the financial benefits to the company, they are quantitative and they are comparable to metrics that can be used in different parts of the same firm or between firms. They capture the degree to which R&D is truly making a financial contribution to the value of the enterprise. They answer the question: What have you [R&D] done for me [the Business] lately? However, they only represent the tip of a process that takes place over a number of years and that involves other functions besides R&D. This means that the numbers reflected by these metrics are associated with activities that are in the past. These metrics are lagging indicators. They are a nice track record, but they may not be reflecting accurately a current level of effectiveness. 15
  • 16. PLATFORM 3: VIII (3) Financial Return (10) How to Use the Metric  The metrics should be tracked at least on a once a year basis. Because of measurement and definition problems, a baseline of two years or more of historical data is needed before accurate judgments can be made about trends and ratio efficiencies.  The metrics should be examined carefully for consistency with business strategies and the results required vs. the investments in R&D. In situations where the metrics, requirements and available resources are not in balance, there will be a difficulty in executing the overall business and technology strategies. One or the other must be shifted, and variations in how R&D is conducted need to be examined.  If the Financial Return metrics are being maintained or going up: the corporation has the likely raw material to extend a technology-based or innovation-based growth program; the investors have the possibility of an extended stream of positive returns from the accumulation of financial pay-offs from technology-based innovations; and the R&D units enjoy the likelihood of consistent funding to reinvest in various aspects of technology application for the near term and base building for the future.  The key words here are likely or possible. Positive Financial Returns are a necessary but not sufficient condition for growth. It is also only a measure of the moment, whether it is looking to the past or to the future. And, any downward movements will predict the difficulties the business will have in achieving solid gains against the competition. These indicators are crucial to assessing the total returns from R&D investments, whether enough is being spent on R&D, and what is the likely future value of the company from a technology perspective. 16
  • 17. PLATFORM 3: VIII (3) Financial Return (10) Options and Variations  There is always something new and innovative that is contributing to revenue or profit. These financial return metrics are intended to capture the new portion of these changes in the business that are related to R&D. They require definitions of what is to be considered new and for how long. These elements can then be tracked separately or together, in ratio or absolute form, by themselves as benefits or as an investment return vs R&D. The options and variations fall into place based on each companys views of these items.  The most common variations are based on the length of time that is new. the most frequent categories are three years, five years and seven years. Another variation is to use these same metrics in a prospective, future mode. 17
  • 18. PLATFORM 3: VIII (3)Projected Value of the R&D Pipeline (9)  Projected Sales (a)  Projected Sales is the calculated sum of future sales from current R&D projects. This metric may be expressed in absolute terms or a % of future sales.  Definitions must be provided for how this is to be evaluated. Normally, it is the forward side of the newness range, i.e. if new products are those introduced within five years, projected sales will be calculated for five years after commercialization. A probability of attainment is usually figured into this metric. 18
  • 19. PLATFORM 3: VIII (3)Projected Value of the R&D Pipeline (9)  Projected Income (b)  Projected Income is the income stream associated with the Projected Sales. Similar definitions apply. This metric may be expressed as an absolute number or as a fraction of net income. 19
  • 20. PLATFORM 3: VIII (3)Projected Value of the R&D Pipeline (9)  Advantages and Limitations  This metric provides an ongoing anticipation of the expected results from R&D. Because it is projected, it provides an evaluation of the benefits that are being created with todays R&D investments.  The limitations are due to the intrinsic difficulties of obtaining estimates about the likely commercial benefits if the technology is successful. 20
  • 21. PLATFORM 3: VIII (3)Projected Value of the R&D Pipeline (9) How to use the Metric  This is perhaps the single most important and least used metrics. It is the singular indication of the future business that is to be developed as a result of successful outcomes from R&D. Are the sales impacts large enough, are there enough new products, is the timing of elements in the pipeline adequate, are the overall returns related to R&D adequate? Are these metrics on a year to year basis showing constancy, increase or decline?  This metric provides ongoing guidance to the company regarding the future gain to be expected in the business due to R&D. This should be used as a check that both the strategy and the resource allocations are correct.  If this metric is staying constant or increasing, particularly with respect to the R&D resources, then the effectiveness of R&D is being maintained or increasing.  If, alternatively, this metric is declining, then further diagnostics should be examined to understand the reasons and to take corrective action. 21
  • 22. PLATFORM 3: VIII (3)Projected Value of the R&D Pipeline (9)  Options and Variations  Common options are few because this is an underused metric. One is to look at absolute sales or net income over a five year horizon on the presumption that some projects will finish and be productive within the next two-three years and that others will impact a bit later, i.e. in the fourth and fifth year. A related option is not to adjust the commercial impact by any probabilities. 22
  • 23. PLATFORM 3: VIII (3) Distribution of Technology Investment (7.8)  Definition  This metric provides a means of assessing how well an R&D program is protecting the technology investment and technical position of the company. It forces consideration of how the technical assets should be distributed, setting directions for modifying the R&D portfolio.  Advantages and Limitations  The portfolio of an R&D organization may not be protecting the strategic interests of the company for any number of reasons such as skill set mismatch, slow response to changes in the company‟s mission and markets, and a rapidly changing competitive environment. There can be an over-emphasis on certain business units and products. This metric causes the management to examine how well the R&D effort is protecting and expanding the technical position of the company in areas of greatest importance. 23
  • 24. PLATFORM 3: VIII (3) Distribution of Technology Investment (7.8) How to Use the Metric  This metric is applied by first determining how the technology investment should be distributed. As an example, consider a company with six business units. The R&D portfolio can be distributed among these six business according to a number of models. Six examples of distribution models are listed below:  The revenue that each business generates.  The opportunity market share (potential market growth).  The impact that technology can make in the different business units.  Competitive Impact (Base, Key, Pacing)  Some combination of the above distribution models.  The profitability that each business demonstrates.  These considerations often involve the concept of the technological basis of competition, that is how does technology provide a sustainable competitive advantage in a particular product or market. The discussion could also consider the distribution of the competitive impact of the company‟s technology investment by categorizing them as Base, Key or Pacing . Base technologies are essential to the business but widely exploited by competitors. Key technologies are highly differentiating to the company‟s current products. Pacing technologies are new technologies where the competitive impact is less certain but likely to be high. All three of the impact categories require protection of the competitive position, but the distribution of resources among the three categories may vary based on the company‟s business plans. 24
  • 25. PLATFORM 3: VIII (3) Distribution of Technology Investment (7.8) How to Use the Metric (continued)  Once a distribution model has been agreed upon, the portfolio is measured against that model. The distribution could be with regard to number of R&D projects in each segment of the distribution, head count devoted to each segment, or expected value of projects in each segment. Modifications are made to the portfolio to move the distribution toward the desired state. 25
  • 26. PLATFORM 3: VIII (3) Distribution of Technology Investment (7.8)  Options  This Both prospective and retrospective views are supported by this metric. In the retrospective view, the R&D projects are categorized according to the corporate investments or markets they are intended to protect or create, or the competitive impact that they offer. This does not have to address the entire R&D effort of the company, since the metric can be applied to any subset of the portfolio. The projects are appropriately weighted to reflect their size and cost. A distribution of the R&D efforts supporting each of the categories is determined. This current state distribution is then compared with the desired state. The degree of misappropriation can then be quantified as the fraction or percentage of the R&D effort that is improperly distributed.  The prospective view for this metric involves consideration of how a proposed project shifts the distribution of technology investment toward or away from the desired state. 26
  • 27. PLATFORM 3: VIII (3) Use of Project Milestones / Stage-Gate Process (6.8)  Definition  Percent of projects in the total portfolio going through a defined project management system with defined milestones.  Percent of R&D expenditure on projects using a defined project management system with defined milestones. 27
  • 28. PLATFORM 3: VIII (3) Use of Project Milestones / Stage-Gate Process (6.8) Advantages and Limitations  Percent Project management systems including milestones can provide a way of reducing cycle time and providing R&D and business management with a sense of the health of projects. These systems also can improve the linkage between R&D, marketing and business management by enabling coordination with manufacturing and marketing to hit key windows. When these systems include documentation through status reports, they can be used to promote organizational learning. Driving behaviors which increase the numerical value of this metric should therefore improve both the linkage of R&D to the business and improve the effectiveness of R&D.  When used with a formal stage-gate process this metric provides a measure of compliance with that system. Since companies will generally use a defined project management system and establish milestones in the later phases of innovation, this metric may also be an indicator of the distribution of projects in the innovation pipeline (see metric: "Distribution of Technology Investment" ). 28
  • 29. PLATFORM 3: VIII (3) Use of Project Milestones / Stage-Gate Process (6.8) Advantages and Limitations (continued)  The metric may be limited by the difficulty of counting projects outside the project management system. Further since project management systems may not be appropriate early in the innovation process, the ideal value for this metric will depend on the firms desired balance of early and late stage projects. For short term projects such as minor product or process variations, use of formal project management systems and this metric may create unnecessary red tape and potential delays. 29
  • 30. PLATFORM 3: VIII (3) Use of Project Milestones / Stage-Gate Process (6.8) How to Use the metric  As a concurrent metric, the total number of projects with defined (written) project plans including definite milestone dates can be divided by the total number of identifiable R&D projects to calculate the metric.  When used with the appropriate accounting system this metric can be calculated from the cost of projects divided by the total R&D cost. In this case the budgeted projects should be audited to determine compliance with requirements for plans and milestones. Options and Variations  The metric can be used as a concurrent metric (a snapshot of the current R&D activities) or as a retrospective measure to determine how many R&D projects used a defined process. It should be equally suitable for service and manufacturing companies. 30
  • 31. PLATFORM 3: VIII (3) Development Cycle Time (5.5)  Market Cycle Time  This metric measures the elapsed time from identification of a customer product need until commercial sales commence.  Project Management Cycle Time  This metric measures the elapsed time from establishment of a discrete project to address an identified customer product need until commercial sales commence.  For both metrics described, the end point can be time when manufacturing feasibility is established for those cases where no commercialization occurs. Compare to historical values and benchmark vs. competition, if possible. Group by categories of projects (e.g. major new product, minor product variation, etc.) Can also be used to track milestone attainment rate for firms using a stage gate management process. 31
  • 32. PLATFORM 3: VIII (3) Development Cycle Time (5.5) Advantages and Limitations (Market Cycle Time)  The advantages of this measure is that it is quantitative and can be used to measure the entire process or various parts of the process if stage gates are examined. The process can be analyzed to determine what parts are driving the overall cycle time so that improvements to the process can be made.  The limitations of this metric could include R&Ds position that it does not adequately influence the process until after a need has been more defined. Another limitation is that a strong documentation system is helpful to make the cycle time metric as accurate as possible. An additional limitation is that defining the commencement of sales as the end of the cycle does not account for post start-up issues such as efficiency, waste, % of manufactured product within specification, etc.; this could lead to focusing on shortening the cycle time at the expense of later, non-measured parts of the cycle. One must also keep in mind that for breakthrough or paradigm shifting projects,, cycle time measurements. The advantage of this metric is that it supports having the clarity of when a project is initiated based on approvals, assignment of resources, start of spending, etc. The metric can be used to measure the entire process or various parts of the process if stage gates are examined. The process can be analyzed to determine what parts are driving the overall cycle time so that improvements to the process can be made. 32
  • 33. PLATFORM 3: VIII (3) Development Cycle Time (5.5) How to Use this Metric  Both metrics should preferably be used in combination with a project reporting system that can track the project initiation date (based on approval and assignment of resources), the length of time in each stage gate of the innovation process, and date of sales commencement. For the first metric, the initiation date could be the date the customer need was determined (i.e. marketing request or date of customer research results). Cycle times for different types of projects (new products, cost savings, product improvements, etc.) should be compared to help predict and manage resource allocation. Cycle times for different divisions could also be compared (with caution) to identify practices driving lower cycle time to adapt where possible. Options and Variations  Variations could exist regarding looking at cycle times for only certain parts of the process for which R&D feels it has most control or influence. Cycle time could be extended past the commencement of sales based on what is important to the organization and R&Ds involvement, i.e. when target efficiency is achieved, target manufacturing cost, % of product within spec, etc. 33
  • 34. PLATFORM 3: VIII (3) Customer Satisfaction (5.5)  Definition  The customer satisfaction metric has two variations: 1. Measures of external (end-customer) satisfaction. These may be such metrics as ratings of quality of technical personnel or technical capabilities, or technology benefits within products or processes. 2. Internal customer satisfaction. Since the immediate customer of R&D is normally the businesses within the corporation that R&D serves, measures such as customer satisfaction in engineering, marketing, or manufacturing may be appropriate. Typical metrics might include on-time technology delivery, competitiveness or appropriateness of the technology solutions delivered, and overall satisfaction with the R&D track record of technological support. 34
  • 35. PLATFORM 3: VIII (3) Customer Satisfaction (5.5)  Advantages and Limitations  There are few disadvantages to good customer satisfaction metrics. In the case of external, or end-customer satisfaction metrics, one complication may be that the entire innovation cycle is under review by the end customer. A bad grade by the customer, while a valid rating of the corporate innovation process, may not be merely an indictment of the R&D operation, but a judgment of the overall product development process within the company, involving manufacturing and product engineering, market forecasts, consumer needs and attitudes, and competence of corporate management. One the other hand, a well- thought-out customer satisfaction metric (or set of metrics) for the internal or immediate customer within the corporation -- normally the corporate businesses and their various R&D-related organizations -- may be the key diagnostic to indicate that R&D processes are lacking and need adjustment or redesign. The R&D organization itself is probably better served with well thought-out internal metrics than with external metrics that complicate the diagnostic process for R&D when problems are indicated. 35
  • 36. PLATFORM 3: VIII (3) Customer Satisfaction (5.5) How to Use the Metric  External customer satisfaction metrics. These metrics will normally be marketing- related or implemented. One variation would be to use a marketing survey, in which various aspects of technology benefits are rated on a five -point scale  There are two dimensions for internal customer satisfaction metrics: strategic and tactical.  Strategic metrics deal specifically with whether the R&D function is meeting the strategic needs of the customer. The review process might involve matching technology and product roadmaps in a joint meeting, in which technology timing mismatches are resolved. Information can be exchanged; technology previews by R&D to alert the businesses to possible market-creating or market share increasing discontinuities, and the businesses to share future market window and product definitions with R&D. Various metrics can be used, including the five-point rating system mentioned above, or a metric which highlights number or percent of mismatches between product and technology roadmaps. 36
  • 37. PLATFORM 3: VIII (3) Customer Satisfaction (5.5) How to Use the Metric (continued)  Tactical metrics deal with whether specific projects are meeting the goal or delivery requirements of the internal customer base. For technology projects in the latter stages of development (near to or entering product development), regular project reviews with the intended customer(s) is important. At quarterly or perhaps semiannual reviews, customer and R&D representatives join in a review of project progress. A useful metric in this case is a report card which each customer representative is required to fill out in the review meeting. This is a very simple questionnaire which has 3-5 survey questions on project progress, and suitability of both the project and the technological approach to satisfy the customer(s) needs. Typical questions might be:  Does this project meet your product technology needs?  Do project milestone dates meet your market window?  Have any strategy changes on your part not been addressed?  What is the overall project score (typical scale 1-5)?  As a related metric, trend analyses can be made both by project and organizationally. 37
  • 38. PLATFORM 3: VIII (3) Customer Satisfaction (5.5) Options and Variations  One variation on the customer satisfaction process is to establish steering teams to address technology, business and market issues and provide guidance to the R&D organization on strategic issues. For external issues, the teams might consist of focus groups that meet periodically, or focus teams that convene groups which are diverse either geographically, ethnically, or with respect to age group, for example. The focus might be on functional needs that technology capabilities address. The metric would be meetings held, or issues addressed and settled. Another metric might be problems surfaced to be addressed and reported on by the R&D team.  Internal issues might be addressed by customer teams composed of manufacturing, engineering, marketing, financial, and related personnel (including even external consultants) who provide guidance and assist in forming cross- functional project teams. Appropriate metrics are meetings held, issues settled, or problems surfaced. An ongoing metric can also keep track of the % of problems addressed and resolved versus those surfaced. 38
  • 39. PLATFORM 3: VIII (3) Number of Ways Technology is Exploited (5)  Definition  This metric assess the number of ways a technical asset can bring value to the corporation.  Advantages and Limitations  This metric is applied to gauge project attractiveness, or to understand the value of a technical asset already developed. It is generally agreed that a larger number of potential uses, both within the companys current markets and in markets not yet developed, makes a technical asset more valuable. The metric is a bit arbitrary and can be misleading in cases where there are few, though very large and/or lucrative commercializations. 39
  • 40. PLATFORM 3: VIII (3) Number of Ways Technology is Exploited (5)  How to Apply the Metric  This metric is applied by taking an existing or potential technical asset, such as a project to develop a new type of lower cost, light weight composite material, and conducting a thoughtful analysis of how many ways this asset can be exploited commercially. The count could consider:  Number of business units in the corporation that could make use of the asset  Number of markets the company serves that could be impacted by this technology  Total number of markets served by the corporation and other companies where the technology may have an impact.  Number of products that could utilize the technical asset. 40
  • 41. PLATFORM 3: VIII (3) Number of Ways Technology is Exploited (5) How to Apply the Metric (continued)  This Used in this way, the metric is a single numerical value. A larger number of potential uses means that the corporation is not depending on a single or small number of products to succeed in order for the technical asset to deliver value. The risks associated with the exploitation of the technical asset are spread over a larger number of potential uses. A larger number also provides greater opportunity for unforeseen benefits, like taking the company into new markets and new products.  Using the low cost, light weight composites as an example, the primary market for the company may be the automotive market, with four different auto parts that could use the strong, light weight tubes produced by the new process. In addition, there could be a market for the technology in the aeronautical industry, served by another business unit of the company. The third exploitation could be in high performance bicycle frames, a market that is new to the company. 41
  • 42. PLATFORM 3: VIII (3) Number of Ways Technology is Exploited (5) Options  An option in the implementation of this metric can involve adjusting the number for relative importance of the commercializations, or keeping sub-metrics of the number of exploitations with certain value ranges as demonstrated with the continuing example of the light weight composites. 42
  • 43. PLATFORM 3: VIII (3) Projects with Business / Marketing Approval (5)  Definition  Percent of projects in the total R&D portfolio with explicit business unit and or corporate business management sign-off.  The intent of this metric is to provide an indicator of the degree of alignment with business and corporate strategy and tactics. The metric is closely related in some corporate structures to metric: "percent Funding by the Business“. 43
  • 44. PLATFORM 3: VIII (3) Projects with Business / Marketing Approval (5) Advantages and Limitations  Advantages: Several studies have suggested that close alignment of R&D to marketing and to business and corporate strategies increases the odds of success for new products and processes. Thus actions which drive this metric to higher values can be expected to improve the amount of R&D spent on successful projects and the predictability of the outcome from R&D efforts.  Limitations: The metric will be valuable to the extent business/marketing management and R&D management jointly develop strategy and plans. Use of the metric to drive R&D without such teamwork will likely lead to short term projects and suboptimal use of R&D resources. In those companies where R&D is corporately funded, business/marketing management may also be tempted to give approval to projects in their market segments to insure that they receive "their share" of R&D resources. Finally if the corporation uses a formal innovation process which requires business/marketing approval at some stage, the metric runs the risk of becoming a measure of compliance with use of the process or a measure of the percent of project past the approval stage. 44
  • 45. PLATFORM 3: VIII (3) Projects with Business / Marketing Approval (5) How to Use the Metric  Explicit approval may be sought at any point in the innovation process. Seeking approval early in the innovation process probably provides maximum value. One form of approval is the provision of a sales forecast from marketing management for each new product and agreement to commercialize if the product meets technical requirements in a timely manner.  The level of approval from the marketing/business management and the point where approval should be sought should be explicitly defined if the firm uses a formal innovation process. If not, the level should be commensurate with the amount of R&D resources and commercialization resources which will be required. 45
  • 46. PLATFORM 3: VIII (3) Projects with Business / Marketing Approval (5) Options and Variations  For projects having broad corporate strategic value, approval of a director of corporate planning or director of corporate business development might be an appropriate substitute for the business/marketing management approval. For corporations where out-licensing of technology is a major thrust, approval of a director or vice president of licensing may be an appropriate substitution. 46
  • 47. PLATFORM 3: VIII (3) Quality of Personnel (3.3) Definition  This is a measurement of the skills and ability of the R&D staff to execute strategic programs. Internal Customer Ratings (a)  Internal customers rate the quality of the R&D staff on their ability to execute programs. Measures such as percentage of mileposts met versus project plans, novelty of concepts, patentability of concepts, and competitive advantage of the technology are parameters that can be considered. External Customer Ratings (b)  External customers rate the quality of the R&D staff of their ability to meet customer expectations. Problem solving, novelty of approach, responsiveness, knowledge of customers operations are parameters that can be considered. External Recognition (c)  Publications in refereed or industry trade journals, external presentations, citations in the literature, invited lectures and patents are parameters to be considered. 47
  • 48. PLATFORM 3: VIII (3) Quality of Personnel (3.3) Advantages and Limitations  The internal and external customer ratings measure the ability of R&D to meet customer expectations and contribute to the growth of the corporation or enhance competitiveness. They are largely objective measures that can be tied to tangible value. The value of external recognition via patents, publications and presentations is more difficult to measure objectively. The numbers of different subjects covered by public disclosures should be evaluated rather than the total number of all disclosures. It is easy to become subverted to a self servicing metric if only numbers are considered. Maintenance of technology as trade secrets must be considered in this evaluation. 48
  • 49. PLATFORM 3: VIII (3) Quality of Personnel (3.3) How to Use the Metric  Internal customer surveys can be conducted using a 1 to 4 scale for rating. A 1 represents below standard execution on a given project. The causes for this poor performance have to be determined since they could arise from inadequate skills, poor judgment, lack of responsiveness, poor planning, etc. The causes may not be related to the quality of the personnel but poor management practices. Having multiple internal customers (marketing, manufacturing, sales, etc.), conduct the evaluation is a form of 360o review. Superior performance by reaching targets ahead of schedule, lower than expected costs, developing a significant competitive advantage, etc., should be rated as a 4. In establishing this survey system, agreement should be reached on the different levels of performance. The survey should be applied to different projects with the same population of the internal customers as raters. This rating should be conducted on a regular basis and over time trends will emerge. 49
  • 50. PLATFORM 3: VIII (3) Quality of Personnel (3.3) How to Use the Metric (continued)  External customer surveys should be conducted using the same 1 to 4 scale. A 1 rating would indicate that the customer was not satisfied with the parameter being measured, while a 4 would indicate that expectations were exceeded. Key parameters should be selected beforehand and could include timeliness of response, knowledge of products, knowledge of customer s operations and knowledge of customer needs. The parameters will vary by industry. Several levels of the customer s organization should be sampled such as plant operators, first-line supervisors and management. A simple postcard type of survey instrument mailed after customer contact can be used. A database can be developed over time and trends will emerge.  The subjects covered in public disclosure should be tabulated and compared to strategic technology goals. A subjective 1 to 4 rating system can be created by R&D management to determine fit with the goals. A rate of 1 corresponds to a poor fit and a need to enhance skills, while a 4 implies that all areas are being addressed. 50
  • 51. PLATFORM 3: VIII (3) Quality of Personnel (3.3) Options and Variations  Service and consumer product companies may find the external customer survey to be a valuable tool in assessing the effectiveness of their R&D organization. One may also measure the quality by the number and type of external awards from recognized organizations (ACS, AIChE, IRI ...). 51
  • 52. PLATFORM 3: VIII (3) Efficiency of Internal Technical Processes (3.3) Definition  This set of metrics seeks to provide a measure of both the efficiency and effectiveness of the operation of R&D processes within the firm. Project Assessment  The total cost of all commercially successful projects divided by the number of commercially successful projects. (Useful when tracked over time with similar projects with similar scopes).  The ratio of actual to projected costs (and timing) for all projects.  Percentage of costs devoted to commercially successful projects. Portfolio Assessment  The total R&D budget divided by the number of projects with commercial output. Subdivide by projects of similar type (technical service, short term, long term) and used in conjunction with project value assessment. 52
  • 53. PLATFORM 3: VIII (3) Efficiency of Internal Technical Processes (3.3) Advantages and Limitations  This set of metrics needs to be adapted to the needs of each firm -- considering the goals, objectives, and priorities for the firm. Assessments need to be made for individual projects (stage-gate , PACE, or similar processes), and for the collection of projects (Portfolios). 53
  • 54. PLATFORM 3: VIII (3) Efficiency of Internal Technical Processes (3.3)  Options and Variations  Each firm will need to set the metrics relative to its specific goals and objectives. An assessment of the selection termination and management of projects can be made using the following four stages for “Technical projects: Selection, termination and project management”. 54
  • 55. PLATFORM 3: VIII (3) Efficiency of Internal Technical Processes (3.3)  Stage 1 Technical projects: Selection, termination and project management A. favors short term projects B. politically driven selection C. no project monitoring or pre-project planning D. little inter-functional participation in project teams E. erratic turnover of team staffing F. project leader roles not defined G. no training for project leaders H. unclear charters for project teams 55
  • 56. PLATFORM 3: VIII (3) Efficiency of Internal Technical Processes (3.3)  Stage 2 Technical projects: Selection, termination and project management A. mix of short and medium-term projects B. no inter-product-line analysis C. priorities set erratically D. project tracking E. some inter-functional participation but not all key functions represented. F. formal release process for new products G. some project team stability but conflicts over work H. priorities I. project leaders given only minimum guidance or training 56
  • 57. PLATFORM 3: VIII (3) Efficiency of Internal Technical Processes (3.3)  Stage 3 Technical projects: Selection, termination and project management A. selection based on multiple inputs from internal and external sources B. balance of short-, medium- and long-term projects C. risk analysis incorporated at key phases D. projects still schedule driven E. Inter-functional teams wherever needed F. clear allocation of project and functional responsibilities G. training for project leaders 57
  • 58. PLATFORM 3: VIII (3) Efficiency of Internal Technical Processes (3.3)  Stage 4 Technical projects: Selection, termination and project management A. clear links between selection criteria and business and product-line strategy B. disciplined process for project termination C. cross-functional planning and execution D. continual improvement- postmortems, quality measures of both project process and product E. performance F. projects are milestone driven G. differentiated project management procedures for different types of projects H. scheduling and capacity planning avoid resource contention by competing projects 58
  • 59. PLATFORM 3: VIII (3) Product Quality and Reliability (3)  Definition  Customer or Consumer Evaluation (a)  Relative quality and reliability compared to competitive products through evaluation by customers or consumers.  Reliability/Defect Rate Assessment (b)  Fraction of a firms output, either by individual product or by sum of all products, that meets or exceeds the established quality standards. 59
  • 60. PLATFORM 3: VIII (3) Product Quality and Reliability (3)  Advantages and Limitations  The advantage of this metric is that product benefits resulting from R&D activities are directly evaluated by the customer or consumer. Comparison with the competitors products is usually the basis for evaluation. Limitations of the metric chiefly are related to the reliability and accuracy of the survey techniques chosen as appropriate for the industry, though firms usually gain confidence in their preferred methods through repeated use and incremental improvement.  Similar to the preceding discussion, the advantage to this metric is that the direct benefit from R&D activities can be obtained through specific measurements made by the firm. 60
  • 61. PLATFORM 3: VIII (3) Product Quality and Reliability (3) How to Use the Metric  For the Customer or Consumer Evaluation metric, each firm will generally have a preferred technique for directly or indirectly obtaining data showing how well the firms products perform in comparison to competitive products. Data for the Reliability/Defect Rate.  Assessment result from internal quality measurements. When both metrics are utilized, the impact of product quality improvements on customer satisfaction should be demonstrated. Product Quality and Reliability metrics are retrospective, showing the results of past technology or product introduction to the market. Options and Variations  This metric fits well with the trend toward greater input to R&D planning from customers and consumers and with the "Quality" protocols that have adopted by firms of all types. Though basically a retrospective measure, product needs that arise during the data collection can be used for prospective purposes. 61
  • 62. PLATFORM 3: VIII (3) Market Share (3)  Definition  Firm (or business unit) market share in various product categories measured as appropriate for the industry or category, expressed as a percentage of the total market.  Advantages and Limitations  This metric is meant to reflect value creation for the firm and the value of the firms technology. Similar to Gross Profit Margin metrics, caution must be taken in the interpretation of Market Share data from the perspective of measurement of the contribution of R&D activities to the whole. There can be many confounding factors in a market share determination, such as the size and quality of the marketing effort, the competitive response, the relative state of the economy, etc. 62
  • 63. PLATFORM 3: VIII (3) Market Share (3)  How to Use the Metric  Changes in Market Share should be assessed at least annually to determine the rate of progress or decline. The expectation is that improvement in a firms technologies and products will result in a greater share of the market. Market Share is a retrospective metric, showing the results of past technology or product introduction to the market. Since competitors market share data is usually also available, this is a metric that can be used as a benchmark with the competition.  Options and Variations  As an indication of threats or opportunities, share data in markets related to a firms products can be followed. This "Related Market Share" metric can serve as a component of a strategy to anticipate the potential application of similar technology into a firms marketplace. 63
  • 64. PLATFORM 3: VIII (3) Development Pipeline Milestones Achieved (2.3)  Definition  Development Pipeline Milestones Achieved is a metric which is useful in grading the effectiveness of management and planning of each R&D project. It may also be used as an indicator of performance problems on a given project which can be used to initiate diagnostic and recovery procedures to give each project the best possible chance of success. There are two possible variations of the metric which may be used: 1. Percent of project milestones achieved -- the percent, by project or overall by sub-organization or laboratory, of all project milestones completed on schedule or within some acceptable time window (90 days, for example) of the forecast date. Trend studies can then be established for organizational performance based on analysis of the data collected by quarter or for whatever other time period is appropriate for the particular industry group. 2. Performance level at each milestone -- on a project basis, percent of all expected objectives met at the milestone date by which they are forecast to be completed. 64
  • 65. PLATFORM 3: VIII (3) Development Pipeline Milestones Achieved (2.3)  Advantages and Limitations  The primary advantages of this metric are for project diagnostics and as an indicator of the overall planning and management health of the organization. There are several possible limitations. First, a consistent management system must be in place to assure that the variability of the number/quality/difficulty of the milestones does not cause random fluctuations in the metric. Secondly, the calibration of the metric is important. Since both the stage of the R&D project and the industry of the business involved can greatly affect the trend analyses, the metric user must be careful to identify what the real danger signals are in terms of management and planning deficiencies, and what really constitutes an indicator of project problems for this metric to be useful. 65
  • 66. PLATFORM 3: VIII (3) Development Pipeline Milestones Achieved (2.3)  How to Use the Metric  Percent of project milestones achieved -- a simple approach is to tabulate milestones met on schedule, 1-90 days late (which may constitute on schedule in some cases), 90-180 days late, and so forth. The tabulation can be done quarterly, and the trend information publicized to the organization to focus attention on performance issues.  Performance level at each milestone -- This is more useful as a project diagnostic. One approach is to require project managers to use this metric to summarize performance quarterly or semiannually at an operations review and be prepared to cover diagnostic or recovery steps if a given project shows trends over two or more time periods of failing to meet an acceptable performance level (which will depend on the R&D discipline, the industry, and the stage of the innovation process). 66
  • 67. PLATFORM 3: VIII (3) Development Pipeline Milestones Achieved (2.3)  Options and Variations  One problem with using milestone achievement as a key metric is that pressure is exerted to make all the milestones. With strong pressure to meet milestone dates, the project manager may be tempted to populate the project plan with easy milestones, to assure that his milestone hit rate is good. It is important therefore to make sure that milestones are realistic and aggressive. Especially in the later stages of the innovation cycle, in product design and productization or manufacturing process development, it is important to assure that the business customer, marketing representative, etc. approve the project milestones to assure that the product development cycle meets the market window requirements of the business. In this case, a much closer tracking of milestone achievement may be necessary than in the earlier stages of the innovation cycle (for example, monthly or in some cases even more frequently). 67
  • 68. PLATFORM 3: VIII (3) Development Pipeline Milestones Achieved (2.3)  Options and Variations (continued)  Another problem with using the milestone approach is that early in the innovation cycle (the R part of R&D), milestones are often hard to define and even harder to forecast. With the heavy level of uncertainty in a long-term research project, the definition of a milestone may be vague, or even made in terms such as define or develop concept. In these circumstances, project managers feel especially uneasy in setting milestone dates for which they become accountable. An approach here is to realize that (1) exact dates are less important, since product or market need windows probably have not been established (and in some cases, market needs or even product existence!), and (2) many of these projects which are early in the innovation cycle will certainly fail, since in the research phase, many more ideas are explored than have a positive outcome. In this context, using milestones as a performance metric may not be useful. If used, the metric should probably be relaxed in some way. For example, instead of setting a goal of completing 100% of milestones, perhaps a goal of 50, 60, or 80% may be set. Another possibility is to declare any milestone made within 90 or 120 days of the forecast date to be on time. The specific approach taken will depend on the industry, the research area, and the stage in the innovation cycle. 68
  • 69. PLATFORM 3: VIII (3) Goal Clarity (2)  Definition  This metric uses an interval rating scale assessing the extent to which project performance objectives are clearly defined and understood by all participants on the project team.  Advantages and Limitations  While this metric may provide a semi-quantitative assessment there is a degree of subjectivity for an individual to assess the extent to which he/she really understands objectives and roles on the team.  How to use the metric  A member of the team would provide to all members of the team a survey form to assess understanding of the project objectives and commitments. This can be done using some interval rating scheme. It would be wise to conduct this survey several times through the life of the project to really assess the level of understanding. 69
  • 70. PLATFORM 3: VIII (3) Goal Clarity (2)  Options and Variations  A four stage assessment can be constructed with the following as the highest stage for individual team members: a) clear understanding of the expected product of team effort b) personal belief that a team is the right way to develop/achieve the expected product c) understanding whether the team is an implementation, recommendation and/or informational team d) understanding the teams operational ground rules and end-result boundaries e) belief that the team has all the appropriate knowledge,functions, diversity, levels and locations, so that the expected product of the team effort can be achieved using the minimum number of people? f) high personal commitment to achieving the objectives g) clear understanding of personal role h) acceptance of the recognized team leader i) capability to draw on additional resources to keep the core team to a minimum 70
  • 71. PLATFORM 3: VIII (3)Comparative Manufacturing Cost (1.8)  Definition  Benchmarked manufacturing cost data vs. competition for same type of unit cost.  Advantages and Limitations  It is extremely useful and in some companies paramount to know how R&D is helping to provide an advantaged cost position to the operations. This metric reflects the quantification of that goal. Unfortunately, cost accounting and even further comparative cost accounting can be extremely difficult. 71
  • 72. PLATFORM 3: VIII (3)Comparative Manufacturing Cost (1.8)  How to Use the Metric  While most firms have very accurate manufacturing cost data for themselves, the generation of accurate manufacturing cost data for competition is considerably more difficult. Therefore, when using this measurement, there should be an estimate made of the variance of the competitive estimates.  Options and Variations  This metric is intended to be based on unit process comparisons. There can be many options created that are aggregates of production processes, but these simplifications can be misleading. Therefore caution is urged. 72
  • 73. PLATFORM 3: VIII (3) Gross Margin (1.8)  Definition  Gross Profit as a percentage of sales, where gross profit equals net sales minus cost of goods sold (product costs plus direct manufacturing costs).  Advantages and Limitations  To some extent, gross profit margin reflects value of the firms technology assets and the value created by R&D. However, raw material, production, and distribution costs also directly affect the firms gross profit margin. Each firm should attempt to understand the correlation of Gross Profit Margin to R&D. 73
  • 74. PLATFORM 3: VIII (3) Gross Margin (1.8)  How to Use the Metric  Value assessment should be based on change in gross profit margin from period to period. (Periods should be appropriate to an industry and may be in excess of one year.) Changes in Gross Profit Margin in relationship to changes in values of other metrics (e.g., financial return , technology transferred to manufacturing, sales protected by proprietary position) should be followed in an attempt to uncouple the contribution of R&D from other factors. This is a retrospective metric that can be used as a benchmark with the competition, if gross margin data from competitive firms are available. 74
  • 75. PLATFORM 3: VIII (3) Use of Cross-Functional Teams (1.8)  Number of Cross-functional Teams  Current management philosophy suggests that the use of cross-functional teams will improve the effectiveness and efficiency of the R&D Process and will help R&D to be integrated with the business. The number of such teams can be counted if they are established on a formal basis.  Evaluation of the use of Cross-functional Teams  A firm may rate its practice of using cross-functional teams by using a subjective scale ranging from 1 to 4. Firms operating at Level 1 are characterized by the existence of strong organizational boundaries, lack of cross-functional involvement in R&D projects, and no cross-functional team structure. Firms operating at Level 4 are characterized by a well developed and supported team structure which effectively places all R&D work in cross-functional teams responsible for the entire project rather than functional silos responsible for parts. This metric is one of many relating to how well the firm conducts the Practice of the R&D Process. It also relates to how well R&D is integrated with the business. 75
  • 76. PLATFORM 3: VIII (3) Use of Cross-Functional Teams (1.8)  Advantages and Limitations  This metric will be important if the use of cross-functional teams contributes to the effectiveness of the R&D function in the firm, given its peculiar situation. This is usually thought to be the case, but there may be cases where other factors are more dominant.  Evaluation of the use of teams can be used to diagnose problems with the organization or the R&D process and to plan for improvement.  How to use the Metric  A simple count of teams is rarely as valuable as an assessment of the how the firm uses such teams. This can be accomplished using the rating scale suggested above. Input for the evaluation should be gathered from a broad cross-section of the firm. The evaluation is important for planning improvements.  Trends across time are probably more valuable than benchmarking. 76
  • 77. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  Metrics for Product Features & Benefits:  Metric 1: Competitive Technical Performance of Product (Project Metric)  Comparison of technical performance of a product in those dimensions where the customer is likely to perceive a benefit. This may be used in a prospective sense to appraise the value of a feature or in a retrospective sense to register the value of a benefit which the market has recognized.  Examples include:  The use level required to achieve a needed result in the customer application.  The yield strength of a high-performance alloy  The measured softness provided by a textile softener.  The measured UV resistance of an external architectural coating. 77
  • 78. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  Metrics for Product Features & Benefits:  Metric 2: Customer rating of Products (Business Segment or Firm)  Customer rating using a scale of 1-5 of the technology benefits that is perceived in a firm‟s products. This can be compared to the rating for the best competitor, usually in the form of a ratio. This is an aggregate subjective measure for a business segment or for the firm.  Metric 3: Economic Value of Products (Project, Business Segment, or Firm)  This metric is the price differential per unit obtained by virtue of the technology feature minus the cost of providing the feature. The differential can be multiplied by the volume to assess the total benefit to the firm. 78
  • 79. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  Metrics for Product Features & Benefits:  Metric 4: Market Share Evaluation (Business Segment or Firm)  If differential pricing does not occur, the advantages of superior product technology can appear as differential market share. In this case, the relative market share (the firm‟s share divided by the largest share) can be used as a surrogate for the value of technology embodied in the products. 79
  • 80. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  Metrics for Process Features & Benefits:  Metric 5: Competitive Technical Performance of Process (Project Metric)  Comparison of technical performance of a product in those dimensions which are important to manufacturing cost or product performance. This may be used in a prospective sense to appraise the expected value of a new or improved process or in a retrospective sense to register the demonstrated value.  Examples include:  Manpower requirements  Efficiencies of raw material conversion  By-product or co-product costs or values  Consistency, controllability, and other such quality parameters. 80
  • 81. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  Metrics for Process Features & Benefits:  Metric 6: Economic Value of Processes (Project, Business Segment, or Firm)  The differential in profitability (versus the target or competitor) attributable to new or improved process technology.  Metric 7: Profitability Evaluation of Processes (Business Segment or Firm)  In the same way that Market Share is a surrogate measure of product performance in those business areas where the basis of competition is product performance, overall profitability in a business segment of the firm is a measure of process performance, in those business areas where the basis of competition is cost and/or quality. 81
  • 82. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  Advantages and Limitations  Product Metrics 1 through 4 attempt to assign value to technology used in products. However, differential value or market share are normally the result of many different factors. These metrics can give some indications if factors are carefully sifted, but may be misleading if the analysis is superficial. Objective measurement of product performance and customer ratings relative to competitors are the most accurate measures of product technology. But note that comparison of features that the market has not recognized as benefits may be self-serving and deceptive. Competitive rankings of measured product performance and or customer ratings may be averaged over market segments or over the firm to obtain average values.  Process Metrics 5 through 7 attempt to assign value to new or improved process technology resulting from R&D. However, economic value and profitability are normally the result of many different factors. Objective measurement of process performance is the most accurate measure of process technology. 82
  • 83. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  How to Use the Metric:  Metric 1  Define the key parameters which measure features the customer is likely to perceive as benefits. Measure product performance as accurately as methods allow. Compare to the same measurements of competitive products. Rank performance versus best competitors.  Metric 2  Ask customers to rate the technology attributes of a product line relative to solving their problems. This requires a carefully constructed survey instrument.  Metric 3  This metric is valid for products which are differentiated by performance. It is not applicable to commodity products. 83
  • 84. PLATFORM 3: VIII (3) Rating of Technology Features and Benefits (1.8)  How to Use the Metric:  Metric 4 (See Definition)  Metric 5  Define the key parameters which measure or impact cost or quality. Measure process performance as accurately as methods allow. Compare to the same measurements of competitive processes. Rank performance versus best competitors.  Metric 6 (See Definition)  Metric 7 (See Definition)  NOTE: These metrics may be used retrospectively to measure the output of R&D over the past period and prospectively to set targets for future accomplishments. 84
  • 85. PLATFORM 3: VIII (3) Response Time to Competitors Moves (1.8) Definition  This metric measures the ability of the firm to respond to new technical innovations introduced by competitors. Depending upon corporate strategy, it could be the time required to match or exceed the competitive offering. Advantages and Limitations  This metric is an indicator of technical leadership in a given field. The technical leader will not spend a significant amount of time matching competitive innovations, while a follower will be more reactive than proactive. The utilization of resources for this function as a percentage of total resources should be tracked over time. An increase in this percentage would indicate that new technology programs are not as effective as desired and technical position relative to the competition is eroding. This metric also measures the flexibility and creativity of the firm to change priorities to meet competitive challenges. A strong market intelligence function is required to identify competitive entries at an early stage in the introduction and to assess the technical merits of the offering so that appropriate responses can be made. The technical merits of competitive offerings must be critically assessed to differentiate from market repositioning of existing technology. 85
  • 86. PLATFORM 3: VIII (3) Response Time to Competitors Moves (1.8) How to Use the Metric  The time between the introduction of a competitive offering and the internal development of a comparable or superior offering can be measured and compared to product development times for similar products. A rating system of 1 to 4 can be used. 1. Organization slow to recognize significant competitive offering in the marketplace; slow to launch program to respond; unable to get their offering into the marketplace in acceptable time to be combatant. Slow to recognize impact of technical innovations. 2. Organization recognizes need for competitive offering; has trouble in launching program to develop counter-offering and gets offering to market barely in time to have impact. 3. Organization responds to competitive offering and develops counter offering to maintain relative position. 4. Organization anticipates potential for competitive offering and has counter offering into the marketplace with superior product allowing a gain in market/competitive position. Very fast to recognize impact of any hints of technical innovations. 86
  • 87. PLATFORM 3: VIII (3) Response Time to Competitors Moves (1.8) How to Use the Metric (continued)  The percentage of resources used in matching competitive moves could be measured over time. An increase in spending in this function should raise questions regarding core research and development efforts for organizations who strive for technical leadership. Options and Variations  The importance of this metric will depend upon the technology strategy of the firm. Firms pursuing technical leadership will be very interested in minimizing response time. The importance may vary across different strategic segments in the same firm. 87
  • 88. PLATFORM 3: VIII (3) Comparative Technology Investment (1.5) Definition  This measures the current annual expenditure for R&D staff and capital compared to the best competitor and/or the industry average. Advantages and Limitations  This metric measures the rate of current activity in developing the technology of interest with the intent of predicting whether the firm is expected to gain or lose ground in the technology. It should be kept separately for the KEY and PACING technologies most critical to the strategy. Retrospectively it measures the efficiency of the investment in meeting new product and technology development goals.  This metric should be as quantitative as possible, but in some industries it may be necessary to make estimates as to the size of the development effort of the best competitor and the industry average. Analyses must be based on a comparison of similar functions. For example, some organizations include sales support as part of their report for technology expenditures. The components of the expenditures under study must be understood in making the comparison. 88
  • 89. PLATFORM 3: VIII (3) Comparative Technology Investment (1.5) How to Use the Metric  Information on a firm s overall technology expenditures are available in the firm s annual reports or industry publications. These can be used for comparison to internal overall investment. Information by industry is available in industry publications and from organizations such as IRI (IRI/CIMS survey).  Rationing the firm‟s current investment in technology versus the best competitor and industry averages provides an insight into the efficiency of the technology investment. Performance exceeding expectations in value creation goals at competitive investment rates indicates an efficient organization while sub-standard performance raises concern about the quality of the investment. Smaller firms may require an investment higher than industry norms to maintain a competitive position to offset critical mass issues. 89
  • 90. PLATFORM 3: VIII (3) Employee Morale (1.5)Definition  This metric takes quantitative ratings of key aspects of employee satisfaction and morale as shown by direct employee survey. It is recognized that employees may feel good and have high morale, yet produce nothing of value for the firm -- the real question is are they motivated and committed to create and innovate profitably?Advantages and Limitations  Extensive surveys are time consuming and expensive to conduct and must be conducted with sufficient frequency to establish base lines and understand real trends. One must understand also that technical populations tend to have certain biases in such surveys. Many employees feel "surveyed-out". 90
  • 91. PLATFORM 3: VIII (3) Employee Morale (1.5)How to Use this Metric  The typical survey uses five point scales for agreement (strongly agree, agree, neither agree or disagree, disagree, strongly disagree) for importance (extremely important, very important, somewhat important, of little importance, not at all important) and for performance (very good, good, fair, poor, very poor) in answering sets of questions related to work environment, feelings about the company, ratings of the company, ratings of the organization/work location/work group, feelings about the individuals job, and general satisfaction. Opportunities are given for comments. Such extensive surveys are most often conducted by third parties to maintain confidentiality. 91
  • 92. PLATFORM 3: VIII (3) Employee Morale (1.5) How to Use this Metric (continued)  Four tested questions of job satisfaction are: 1. If a good friend was interested in a job like yours for your firm, what would you tell the friend? 2. All in all, how satisfied are you with your present job? 3. Knowing what you know now, if you had to decide all over again, would you take your current job? 4. 3.4 How satisfied are you with the overall employee- employer relations at your firm? 92
  • 93. PLATFORM 3: VIII (3) Employee Morale (1.5)  Options and Variations  One firm which uses e-mail extensively, every 9-14 months conducts a broad "pulse" survey and asks employees to provide two ratings (using 0-10 point scales with descriptors) one rating the employees work climate/environment and the other rating personal feelings about the work itself. Employees spend less than 4 minutes to reply by e-mail, or to be anonymous by fax or to a voice mailbox. Employees often write additional comments that give information sought in the more extensive surveys. Confidentiality is assured and rapid feedback (within 2 weeks) of survey results to participants maintains a high level of interest and participation. 93
  • 94. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model  Definition of the Innovation Process  The term "innovation process" refers to the overall process in a company for conceiving and developing ideas and concepts into profitable products and processes for the company business(es). It is one of the 5 to 7 overall high- level business processes within the corporation. In order to maintain, diagnose, and improve the process, both effectiveness and efficiency metrics are required to support decision-making.  Note that in the context discussed herein, the assumption is made that the innovation model will be technology-based. That is, innovation is founded in the efforts of the R&D organization, and that the overall function of the process is the embodiment of technology in the products of the business of the corporation. Progress through the full innovation process will involve the efforts of all corporate resources, including marketing, engineering, manufacturing, and service and support functions. 94
  • 95. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model Relevance of Innovation Process Model to Metric Choice  Often stakeholders in the innovation process -- R&D management, financial management, and corporate officers -- need to utilize metrics which supply specific decision support information about corporate capabilities in innovation. These metrics are overall, or outcome metrics, which provide a grade or rating of the overall result of the innovation capability of the corporation or business in question. Such metrics might relate to value creation by the process, effectiveness of the overall process in terms of new product revenue or profit versus total R&D investment, or the ratio of new product revenue or profit to total company or business revenue or profit.  However, these metrics are often long-term and retrospective in nature, and while they serve to evaluate the overall process as a whole (and generally, but not always, on a lagging basis), they may offer little, if any, insight into the current or future state of the corporate innovation process or its many and diverse activities. 95
  • 96. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model Relevance of Innovation Process Model to Metric Choice (continued)  To be useful for the on-going management and evolution of corporate innovation, a diverse set of metrics must be available which offer insight into current condition and future vision state of the process and many of its constituent parts. For many such metrics to be useful, they must be not simply outcome-oriented, but process- oriented, and specifically diagnostic of discrete sub-processes within the overall innovation process.  Each in-process metric must be related to a process model which ties it to the portion of the process of interest, from early idea or technology exploration through proof of concept to technology development to productization. The following process description is a relatively simple process model and defines the various sub-processes within that model in order to provide anchors for useful metrics within the innovation process. 96
  • 97. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model Overall Description of the Stage Gate Model for the Innovation Process Used in this Metrics Guide  There are a number of possible representations of the corporate innovation process. One simple yet fairly effective representation is the so-called "stage gate model".  The model in Stage Gate divides the overall innovation process into four stages, from concept exploration or "ideation" to the process of productization or commercialization. This division into four sections is arbitrary. An argument can be made for a three-stage model, or a more complex model of multiple sections and sub-sections. The four-stage model is chosen primarily for convenience, and to match a number of corporate models and those defined by scholars in the area of business processes. The shape of the model recognizes several realities of modern business.  For example, the process funnel narrows as the process proceeds from exploratory research or concept exploration through proof-of-concept to technology development and commercialization, illustrating pictorially the selective filtration that occurs in the process. There are typically many more ideas and concepts that are explored than are developed into significant technology capabilities in a business, and fewer still that emerge finally into finished products. In each stage of the process, the candidates dwindle, until only the most promising are brought to full production. 97
  • 98. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model Overall Description of the Stage Gate Model for the Innovation Process Used in this Metrics Guide (continued)  Similarly, the four gates, labeled A, B, C, and D, stand for the tests or decision-making activities that are exercised between the stages in the innovation process. Economic reality imposes a limit on the total level of technology and product development that each company can support. A second reality is that as each decision gate is passed, the resources and funding required to carry a given project to the next stage increases dramatically; a good rule of thumb is a 10X expenditure increase at each gate. These corporate "facts of life" impose a stringent set of conditions on the suitability of R&D programs which approach a gate. Each gate requires a set of metrics which ensure that only those programs most suitable to meet corporate business needs pass into the next sub-process.  The following sections describe the decision gates and stage processes shown in the stage gate model. Underlined terms link to appropriate metrics for the gate or process under discussion. 98
  • 99. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model Description of the Major Sub-process Divisions in the Innovation Process Model  As noted in the introduction to the process model, the division of the innovation process model is arbitrary, but in general it seeks to approximate reality. The four divisions cover the very early technology or idea exploration phase of innovation (when possibilities are defined), a proof-of- concept phase when the mapping of ideas into the realities of the business world occurs, the development, and finally commercialization. Each of the separate phases, or sub-processes, is covered in the following sections, together with the entry gate which defines conditions for admission to that sub-process. Refer to the stage gate model in the references that follow. 99
  • 100. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model Gate A Gate B Gate C Stage I Gate D Stage II Stage III Stage IV 100
  • 101. PLATFORM 3: VIII (3) Metrics for the Stage Gate Model 101
  • 102. PLATFORM 3: VIII (3) Stage-Gate Model Description Process Entry Gate (A) and Exploratory Concept Sub-process (I)  Gate A is the entry not only to Process I but to the entire innovation process. The purpose of Process I is to explore new ideas and concepts and set in motion as many promising "seed" projects as possible. The cost of research and investigation is small here. In companies where exploratory concepts consist of exploring product ideas and concepts, a single worker may have a project or even several in work simultaneously. Where true basic research is involved, it is most often at the University level, with industry participating through grants, contracts, or research agreements. In either case, there are typically many avenues being explored and no valid idea or concept is neglected.  The main consideration at Gate A is whether the idea or concept is strategically appropriate (at this stage, the alignment to corporate business goals may be ephemeral in some cases), and whether the expertise available to address the concept or idea is adequate. Metrics at Gate A should address these issues.  The purpose of Sub-process I is the validation of concepts or physical principles. Metrics for this process should simply address the validity of results and whether or not basic principles are established. Competency metrics may also be valuable to support assessments of required resources to execute projects. 102
  • 103. PLATFORM 3: VIII (3) Decision Gate A  Metrics which can be used to make good decisions at this gate  Strategic alignment  Metrics assessing the  Financial return efficiency and effectiveness of  New sales ratio the decision making process  Cost savings ratio  Delayed Stage kills  R&D yield  Cycle time (Decision)  R&D return  Decision Gate processes  Distribution of technology  Gate effectiveness investment  Core technical competency  Criteria for overall business success  Supporting R&D metrics 103
  • 104. PLATFORM 3: VIII (3) R&D Process Stage I  Metrics assessing the efficiency of the Stage I process  Cycle time  Market cycle time  Project management cycle time  Quality of personnel  Internal customer rating  External customer rating  External recognition  Published works  Milestones achieved 104
  • 105. PLATFORM 3: VIII (3) Stage-Gate Model Description Conceptualization Gate (B) and Proof-of-Concept Sub-process (II)  Gate B primarily tests whether a concept is validated or some physical principle has been established. Entry to Sub-process II depends mainly on appropriate resources being able to establish proof-of-principle in a business context, and whether the concept or idea to be tested has potential application areas within the business goals of the Metrics should test these concerns. The "filtration" function at this gate is fairly strong, since although the cost of research in Sub-process II is still not great, there will be many more candidate ideas than there will be resources to address them.  In Sub-process II, the emphasis is on proof-of-principle in real business applications. Business considerations such as market window and competitive reaction begin to be important, although there will still be concern about options and possible spin-offs of the technology or concept. Project metrics for tracking milestones and execution time become more important, although time frames are still lengthy and product requirements may be vague or broad-brush. Metrics concerning resources and technical capability are also appropriate. 105
  • 106. PLATFORM 3: VIII (3) Decision Gate B  Metrics which can be used to make a good decision at this gate  Strategic alignment  Financial return  Metrics assessing the efficiency  New sales ratio and effectiveness of the  Cost savings ratio decision making process  R&D yield  Delayed Stage Kills  R&D return  Distribution of technology  Cycle time (Decision) investment  Decision Gate processes  Development pipeline milestones  Gate effectiveness achieved  Percent achieved  Performance level at each  Criteria for overall business success  Supporting R&D metrics  Core technical competency  Probability of success 106
  • 107. PLATFORM 3: VIII (3) R&D Process Stage II  Metrics assessing the efficiency of the Stage II process  Cycle time  Market cycle time  Project management cycle time  Quality of personnel  Internal customer rating  External customer rating  External recognition  Milestones achieved 107
  • 108. PLATFORM 3: VIII (3) Stage-Gate Model Description Technology Development Gate (C) and Product/Process Development Sub-process (III)  The emphasis at Gate C is in suitability for product development. Gate C is also a strong filter; projects that pass this gate will be the few that are highly promising for commercialization and meet all the requirements for profitable business products. Candidate technologies which pass this gate will have forecast long-term corporate benefit, and the projects entering Sub-process III meet all the strategic requirements of fit, alignment, and attractiveness for the business. Metrics for Gate C must address these strategic requirements, as well as the tactical issues of assuring that the candidate projects have successfully completed the requirements of Sub-process II.  In Sub-process III, the emphasis shifts to harder-edge issues, such as timing and execution to assure that market windows are met and product needs are satisfied. There is also emphasis on maintaining and extending technologies to keep a competitive edge in the marketplace. Milestones are important due to cycle time issues, and project funding must be managed more carefully due to budgets which are typically millions of dollars rather than the 100X lower investment that may be typical on a project in Sub-process I. Metrics for Sub- process III must reflect the concerns of timely, accurate execution and tight budget control. 108
  • 109. PLATFORM 3: VIII (3) Decision Gate C  Metrics which can be used to make a good decision at this gate  Strategic alignment  Metrics assessing the  Financial return efficiency and effectiveness  New sales ratio of the decision making pr  Cost savings ratio  Delayed Stage kills  R&D yield  Cycle time (Decision)  R&D return  Decision Gate processes  Customer satisfaction  Gate effectiveness  Milestone achieved  Response time to competitors moves  Criteria for overall business  Core technical competency success  Probability of success  Supporting R&D metrics 109
  • 110. PLATFORM 3: VIII (3) R&D Process Stage III  Metrics assessing the efficiency of the Stage III process  Cycle time  Market cycle time  Project management cycle time  Customer satisfaction  External  Internal  Quality of personnel  Internal customer rating  External customer rating  External recognition  Product quality and reliability  Customer evaluation  Reliability/defects  Use of cross-functional teams  Milestones achieved 110
  • 111. PLATFORM 3: VIII (3) Stage-Gate Model Description Product Launch Gate (D) and Commercialization Sub-process (IV)  Gate D is the last test prior to full product launch in most cases. Where concerns -- and associated metrics -- at the first three gates will have been primarily strategic (fit with corporate goals and competencies, strategic timing, alignment with business need), the test at Gate D is primarily whether execution in Sub-process III was timely and efficient. Concerns for entering full productization are about whether all major technical hurdles are cleared, and whether commercialization costs allow for profitable entry into the marketplace. Some strategic questions must still be addressed, including market need and timing, and metrics utilized here must address both the strategic and tactical issues.  Sub-process IV is obviously market-oriented, with careful management of commercialization and product costs, timing, and execution the key issues. Since budgets of hundreds of millions of dollars may be at stake, program and resource management are paramount, and metrics appropriate to the concerns will be chosen. 111
  • 112. PLATFORM 3: VIII (3) Decision Gate D  Metrics which can be used to make a good decision at this gate  Metrics assessing the  Strategic alignment efficiency and effectiveness  Financial return of the decision making pr  New sales ratio  Delayed Stage kills  Cost savings ratio  Cycle time (Decision)  R&D yield  Decision Gate processes  R&D return  Customer satisfaction  Gate effectiveness  Milestone achieved  Response time to competitors  Criteria for overall business moves success  Probability of success  Supporting R&D metrics 112
  • 113. PLATFORM 3: VIII (3) R&D Process Stage IV  Metrics assessing the efficiency of the Stage IV process  Quality of personnel  Internal customer rating  External customer rating  External recognition  Product quality and reliability  Customer evaluation  Reliability/defects  Technology transfer to manufacturing  Milestones achieved  Cycle time  Market cycle time  Project management cycle time  Use of cross functional teams 113
  • 114. PLATFORM 3: VIII (3) Stage-Gate Model Description Gate Efficiency  A key concern at each gate is that decisions be as efficient as possible. That is, that projects passed through each gate be good candidates to benefit the business, providing the subsequent sub-processes are navigated successfully. Since the "filtration" function of the decision process at each gate will be partially imposed by the economic constraints of a business (only so much total funding is available for the projects in each stage), even some "fit" projects will simply "miss the cut" at each gate. However, those projects which pass the fitness criteria, whether actually entering the next stage or not, represent a measure of the efficiency of the previous gate decision process. A metric (discussed in the section on stage gate metrics) may be developed in terms of the percent of successful projects in the next stage for the previous gate. 114
  • 115. PLATFORM 3: VIII (3) Overall Process Efficiency Metrics  Metrics assessing the efficiency of the overall process  Use of project milestones  Cycle time  Delayed Stage kills  Cost versus budget  Efficiency of internal technical processes  Use of cross functional teams 115
  • 116. PLATFORM 3: VIII (3) Technology Value Pyramid (TVP) 116
  • 117. PLATFORM 3: VIII (3) Technology Value Pyramid (TVP) Technology Value Pyramid consists of a hierarchy of categories of metrics as follow: 1. Value Creation : Metrics which directly demonstrate the value of R&D activities to the positioning, profitability and growth of the corporation and creation of shareholder value. 2. Integration of R&D with the Business : Metrics which indicate the degree of integration, the commitment of the business to the R&D processes and program, teamwork, and ability to exploit technology across the organization. 3. Portfolio Assessment : Metrics which communicate the total R&D program and which allow optimization of the total program for the corporations benefit. 4. Value of Technology Assets : Metrics which indicate the technology strength of the corporation and relate to the probability of success versus competitors for the opportunities which have been selected by the firm. An indicator of the potential for the creation of future value. 5. Practice of R&D: Metrics which relate to the overall efficiency of the R&D processes. 117
  • 118. PLATFORM 3: VIII (3) Technology Value Pyramid (TVP) Supplemental Explanation and Implications for R&D Management  The TVP encompasses descriptors and a potential menu of metrics of the fundamental elements of R&D and the relationships with business results in the short and long term. Not all measurements are right for all companies, or at all times, for describing or tracking the most important aspects of R&D.  This is parallel to a comprehensive financial model of a business that is composed of many more descriptors and metrics than are necessary for examining the critical factors of the moment that are needed to guide the decisions of senior managers and investors.  From an R&D perspective the critical factors of the moment are dependent on the situation of the company, the perspective required and the basic dynamics of the model. 118
  • 119. PLATFORM 3: VIII (3) Technology Value Pyramid (TVP) Top Down and Output Oriented  The model provides a top-down perspective that is output oriented. Value Creation indicators are the prime drivers of overall business returns that are derived from technology-based new products/processes, predictors of business growth and (implicitly) a critical aspect of the soundness of strategic business reviews. These indicators are used to answer critical questions, such as:  Are we spending the right amount on R&D?  Are we getting good returns on our R&D?  If the Value Creation indicators are being maintained or going up:  The corporation has the likely raw material to extend a technology- based or innovation- based growth program;  The investors have the possibility of an extended stream of positive returns from the accumulation of financial pay-offs from technology- based innovations; and  The R&D units enjoy the likelihood of consistent funding to reinvest in various aspects of technology application for the near term and base building for the future.  The key words here are likely or possible. Value Creation is a necessary but not sufficient condition for growth. It is also only a measure of the moment, whether it is looking to the past or to the future. And, any downward movements will predict the difficulties the business will have in achieving solid gains against the competition. These indicators are crucial to assessing the total returns from R&D investments, whether enough is being spent on R&D, and what is the likely future value of the company from a technology perspective. 119
  • 120. PLATFORM 3: VIII (3) Technology Value Pyramid (TVP) Transformation Strategies Drive ‘Value Creation’  Value Creation is the result of an accumulation of effort by R&D and the business to produce new ideas and to put the best ones into practice. Due to the lag effects and to all of the factors that are involved, momentum is built into these factors over time. They will change, but not rapidly. Change is caused by the drivers of Value Creation. The drivers are strategies that transform the R&D foundations of competencies, know-how, etc. into specific projects and implementation. These are the strategies that are represented by the Portfolio Assessment and the Integration with Business indicators.  When Value Creation is very positive, these strategies are most likely working well. When Value Creation is going in the wrong direction, look first to these areas as to the cause. In fact, one could look at Value Creation indicators simply as the double integral of these strategy indicators over time.  Given this importance to Value Creation, it is no surprise that companies have routinely focused in recent years on methodologies and activities dealing with Portfolio Assessment and Integration with Business issues. These are correctly perceived as the means to (the ends that) improve the future stream of results from R&D.  Portfolio Assessment indicators describe the state of the various pipelines that run through the R&D enterprise, as well as the targets that are being pursued. They provide a view of how the R&D $ are being spent in terms of the timing, risks and returns that are possible. The Portfolio Assessment indicators are a prime place to look for answers if there are problems with Value Creation, competitive or market share problems, or if there are problems with internal satisfaction. 120
  • 121. PLATFORM 3: VIII (3) Technology Value Pyramid (TVP) Transformation Strategies Drive ‘Value Creation’ (Continued)  Some of the critical management questions affected by the dynamics of these indicators include the following:  Are we allocating the R&D budget optimally (to the elements of the portfolio)?  Are we maximizing our investment yield from R&D?  Unlike Value Creation indicators which tend to have modest levels of momentum associated with them and provide significant underpinnings to the business returns, the Portfolio Assessment indicators can vary quickly and have little immediate effect on the business. Their major effects are cumulative. This dynamic often leads to short term, risk averse behaviors that over time undermine the Value Creation indicators. Maintaining an aggressive monitoring of the Portfolio Assessment indicators is extremely important to the long term support to Value Creation.  Similarly, when there are problems within the portfolio that are corrected, it is necessary to give the solution enough time to work.  If Portfolio Assessment indicators show strategy by what categories of R&D and targets are being developed, the Integration with Business indicators show strategy of how it is being done...and, consequently, with what level of quality and execution. 121
  • 122. PLATFORM 3: VIII (3) Technology Value Pyramid (TVP) Transformation Strategies Drive ‘Value Creation’ (Continued)  The Integration with Business indicators focus on process, culture, teamwork and organization. They also touch on many of the aspects of cycle time. The issues addressed by these indicators change slowly in reality and are probably the true pacing factors that are applied by the organization to the Portfolio Assessment indicators that, in turn, put limits on the Value Creation that is realizable.  These indicators are often the subject of efforts aimed at TQM, cycle-time reduction, or re- engineering. They are difficult to build up to high quality levels because of the various organizational pressures, agendas, incentives, etc. And, they are easy to degrade. The organizational stability of these indicators varies from weak to very strong and are the sum of dozens of behaviors.  When there are difficulties attributable to barriers, for example, that can be removed, then the indicators and the results can be changed relatively rapidly. However, when there are difficulties due to lack of cooperation, lack of contact with the market, lack of good competitive intelligence or with a lack of risk taking, then new attitudes and new behaviors are required. This takes time to build into the culture and the indicators and the results will be slow to change. 122
  • 123. PLATFORM 3: VIII (3) Technology Value Pyramid (TVP) Business and Technology Leadership  In all aspects, the dynamics of Integration with Business indicators depend on organizational matters versus those of the Portfolio Assessment" indicators, which depend on investment decisions. And, a consideration of organizational and investment matters quickly brings the factors of business and technology leadership into the model. Second and third order dynamics pushing for immediate change are easy to see as overlays to deeper cultural values.  It is also logical to see both the importance of these two sets of transformation strategy indicators to managers who want immediate change and the criticality of both sets to the impacts on sustained Value Creation.  Unfortunately, the underlying dynamics do not allow the conversion of management desires for immediate gratification (at low risk) into sustained profitable growth. There must be allowances made for the application of enough time to link all of the elements on an ongoing basis. Over time, all of the indicators will point to a consistent improvement in the transformation strategies and to the attendant output in Value Creation.  Conversely, monitoring the transformation strategy indicators of an otherwise healthy technology- based enterprise, will show the early warning signs of degradation to the degree possible, presuming the technology foundations are sound. 123
  • 124. PLATFORM 3: VIII (3) Technology Value Pyramid (TVP) Foundations  The foundations for the strategies represented by the Portfolio Assessment and the Integration with Business are built on the Asset Value of Technology and the Practice of R&D Processes. Some of the critical questions regarding these dynamics are:  Are we becoming more or less productive with our R&D?  Are we building a strong enough future base of competencies?  Are we getting an early warning of any declining capability?  Foundation indicators have the most momentum of any category. They are very slow to change and provide the real rate limitations to growth through technology-based innovation. However, they are also very vulnerable to neglect. They need nurturing, leadership and the execution of well focused technology strategies to become strong elements of a companys growth foundation. And, just as when weak transformation strategies lead to degradation of Value Creation, they also degrade the foundations. 124
  • 125. PLATFORM 3: VIII (3) Technology Value Pyramid (TVP) Foundations (Continued)  Aside from weak transformation strategies, the other major dynamic that affects foundations are external technological changes.  In these cases, technological paradigm shifts occur that undermine the foundations comprised of traditional technical competencies. These bring in new competitors, topple current competitors and may, in fact, redefine the structure of an industry. Although these paradigm shifts take time to develop, R&D and the business are usually both entrenched in the traditional areas, and either dont see the changes coming or insist on devaluing the importance of them until it is too late. The foundations which, if strong, took considerable time to build, must nonetheless be constantly extended and rebuilt to provide an ongoing set of options that are the logical growth paths for the businesss future. Otherwise, over time, they will significantly erode.  Thus, the dynamics of the foundation indicators are that they are slow to build, a rate limiting source of quality for the options to be taken up by the transformation strategies, and that they are relatively easy to degrade in spite of strong momentum. They are also fundamental to a strong competitive basis for ongoing strategy (based on technical core competencies), to the heart of R&D creativity, productivity and a significant contributor to cycle time reduction. The drivers of these areas are the R&D leadership. 125
  • 126. PLATFORM 3: VIII (3) Technology Value Pyramid (TVP)  Purpose  The principal use of the model is for communication and control. This includes the overall R&D program, the research program, the business technical programs and the development of external relationships for which new capabilities are the goal.  (The model is not intended to aid project evaluation except as a project would contribute to the improvement of various categories and their respective indicators.) 126
  • 127. TVP Model MetricsPLATFORM 3: VIII (3) TVP Value Creation  Financial return  Market share  New sales ratio  Cost savings ratio  Gross margins  R&D yield  R&D return  Product technology rating  Customer rating  Projected value of the R&D  Economic trade-offs pipeline  Projected sales  Sales protected by proprietary  Projected income position  Patents only  Number of ways technology is  Percent proprietary sales exploited  Technology planning  Product quality and reliability  Customer evaluation  Intellectual property  Reliability/defects management 127
  • 128. TVP Model MetricsPLATFORM 3: VIII (3) TVP Portfolio Assessment  Strategic Alignment  Gross margin  Projected value of R&D  Comparative technology pipeline investment  Projected sales  Projected income  Information Technology Use in R&D  Distribution of technology investment  Probability of success  Number of ways technology is  Technology planning exploited  Environmental management in  Market share R&D 128
  • 129. TVP Model MetricsPLATFORM 3: VIII (3) TVP Integration with Business  Strategic alignment  Product quality and reliability  Customer evaluation  Use of Project milestones  Reliability / defects  Customer satisfaction  Use of cross-functional teams  Internal  Delayed Stage kills  Number of ways technology is exploited  Decision Gate processes  Projects with business /  Technology planning marketing approval 129
  • 130. TVP Model MetricsPLATFORM 3: VIII (3) TVP Value of Technical Assets  Strategic alignment  Response time to competitors  Cycle time moves  Market cycle time  Comparative technology  Project management cycle time investment  Customer satisfaction  Customer rating of technical  External capability  Internal  Number and quality of patents  Quality of personnel  Percent useful  Internal customer rating  Value ratio  External customer rating  Percent retention  External recognition  Patents per R&D Dollar  Market share  Sales protected by proprietary  Comparative manufacturing cost position  Patents only  Gross margin  Percent proprietary sales  Rating of technology features and benefits  Core technical competency  Intellectual property management 130
  • 131. TVP Model MetricsPLATFORM 3: VIII (3) TVP Practice of R&D  Gate effectiveness  Environmental management In R&D  Defects reported  Idea generation and creativity  Delayed Stage kills  People development  Cost vs. budget  Intellectual property management  Decision Gate processes  R&D climate  Probability of success  Information Technology use in R&D 131
  • 132. COMPARE EFFICIENCY & EFFECTIVENESS Platform 3: (Stages VIII, Part 4) Integrating Technology Innovation with Business Function – Part I: Laying the Foundation
  • 133. PLATFORM 3: VIII (4) How do we Measure Performance?  Benchmarking  Conduct performance and process benchmarking studies to focus new concept R&D efforts on the most critical / valuable / useful system components. 133
  • 134. PLATFORM 3: VIII (3) Benchmarking (Definitions)  Benchmarking (Generic)  Comparisons with other organizations to identify and then implement strategies for improvement in one‟s own organization.  Identification of superior performance and/or practices and implementation of those in one‟s own organization.  Organizations undertake benchmarking exercises to:  Establish how well they need to perform to achieve „best practice‟  Go beyond the experience of staff within their own enterprise in pursuit of ambitious and innovative improvement plans.  The main categories are performance benchmarking and process benchmarking, and there are significant differences between the two. 134
  • 135. ISSUE PERFORMANCE PROCESS BENCHMARKING BENCHMARKING What is being Only compares performance. Compares process (how compared things are done) andPLATFORM 3: VIII (3) performance (how well the process is done). Degree of focus Generally compares a number Always compares one process Performance of performance indicators. These could measure the at a time. performance of just one vs. Process process, but often cover a wider spectrum of the organisation‟s Benchmarking: Benchmarking processes. The most success approach is It is not advisable to choose partners to organise a quorum of benchmarking partners before Overview of benchmarking partners as soon undertaking a thorough as the broad parameters of the analysis of your own practices the Key project have been decided. (process, procedures) - particularly if best practice is Differences the goal. Form of comparison Generally by circulating a report Best done through physical containing all partners data. visits to partners places of This data has been gathered business, although other from each partner against a forms of comparison can be predetermined set of appropriate. performance indicators. Confidentiality of The identity of the partners as a Because the identity of partners‟ identity and group is generally known, but partners is known, of the information each partner‟s data in the final benchmarking partners are being shared report can be masked by a code either not competitors or the letter for each partner. information being shared is Therefore, benchmarking not confidential to the other partners may be competitors but partner. information shared remains confidential. Outcome from Generally to decide on one or Should be to implement analysis of findings more process benchmarking improvements to the process projects or „in-house‟ process that was benchmarked. improvement projects. 135
  • 136. PLATFORM 3: VIII (3) Performance Benchmarking Different organizations compare performance against a number of agreed performance measures. Examples of performance measures are:  student enrolment efficiency - total cost per student enrolled  student enrolment effectiveness - enrolment errors/1000 enrolments  student enrolment customer satisfaction - average student elapsed time per enrolment  student contact hours per staff member  training hours per staff member  employee satisfaction indexes  customer satisfaction indexes Key advantages  useful way of identifying where one stands in relation to others (i.e. identifying the gap)  can (but may not) show where you most need to improve Disadvantages  does not usually identify the processes or practices that have led to leading or best practice performance  provides no guidance on how to improve  interpretation of results may be difficult (e.g. how much improvement is really possible) Conclusion  It is important to use process benchmarking in conjunction with performance benchmarking not only to find out how to improve but also how much one needs to improve 136
  • 137. PLATFORM 3: VIII (3) Process Benchmarking Organizations compare procedures or processes as well as performance. The main aim is to identify how other organizations achieve superior efficiency, quality, innovation or customer responsiveness in a process. Examples of processes are:  enrolment  student records  handling prospective student enquiries  fees collection  project management/timetabling  course promotion/marketing  student feedback  course evaluation  resources  sponsorship  staff selection  staff development and training Advantages  Can tell how to improve as well as how much improvement is feasible.  Differences between organizations compared- benchmarking partners are easier to interpret Disadvantage  Might choose inappropriate partners if the organization has not clearly defined what it wants to benchmark. 137
  • 138. PLATFORM 3: VIII (3) Strategic Benchmarking This examines the fundamentals of:  How a business is defined  The customer groups and needs a business serves  The resources and skills used to satisfy customers. This type of benchmarking is most relevant when an organization is reviewing its mission and strategy. For example, how do niche or specialist organizations compete against larger organizations (e.g. Bendigo Bank versus National Australia Bank). Issues to be addressed include:  Customer groups and needs served (broad versus narrow)  Resources and skills used (employees versus outsourced/partnerships and alliances)  Location (single versus multiple campuses)  How products or services are provided. 138
  • 139. PLATFORM 3: VIII (3) Informal Benchmarking This refers to the well-established practice of informally comparing one’s performance, process or practice with others. Attending conferences or workplace curriculum development workshops are part of day-to-day responsibilities of staff in the vocational education and training sector. These activities provide an opportunity to compare one’s own organization with others. While useful, these comparisons should not be confused with proper process or performance benchmarking. Only the latter have the scope and rigour to deliver effective strategies for improvement. 139
  • 140. PLATFORM 3: VIII (3) When should we benchmark?  Many organizations decide to benchmark when an:  Improvement in performance is required.  These organizations hope to achieve considerable improvement as a result of benchmarking.  Sometimes organizations undertake their first benchmarking project to learn the technique for use in the future.  In general, benchmarking should be employed when it is important to gain inspiration for specific improvements from the practices of other organizations. That is, when the ideas that are generated from within are unlikely to result in sufficient change. 140
  • 141. PLATFORM 3: VIII (3) Why Should We Benchmark?  Process Benchmarking:  Generally results in improvements that are greater than would be gained from an improvement project conducted solely within the organization  Is an effective catalyst for people who need to change. If they are included in the benchmarking project (preferably on site visits) they see for themselves what changes are possible.  Because performance benchmarking can help determine where the greatest improvements can be made. 141
  • 142. PLATFORM 3: VIII (3) Who Should We Benchmark? Ideally you should benchmark an organization that has achieved a ‘best practice’ or leading performance in a particular area. While the gap between your current performance and the best practice organization may be large, you will learn a lot from them and possibly ‘fast track’ your rate of improvement. However, you will learn much from benchmarking ‘above average’ performers. These may be found in areas such as:  Other departments, divisions or groups within your own organization  Competitors (generally only possible through an intermediary)  Other organizations in the same industry sector, e.g. those offering similar products and services, but not competing in the same markets  Any private or public sector organization (sometimes called „generic‟ or „best practice‟ benchmarking). The ultimate goal is to identify the „best-in-class‟ performers who have processes that are analogous rather than identical to you own. Wherever possible you should aim to:  Benchmark against the „best practice‟ performers (doing so will accelerate your rate of improvement)  Benchmark outside your industry (the source of most major improvements)  Use the experience of organizations in the vocational education and training sector. 142
  • 143. PLATFORM 3: VIII (3) What Should We Benchmark?  The short answer is something:  Particularly important to the success of our organization  Particularly important to your clients / customers  That clearly needs to be improved. 143
  • 144. PLATFORM 3: VIII (3) How does benchmarking fit with other initiatives and continuous improvement programs? Benchmarking can be integrated into:  Continuous improvement programs (comparisons with other organizations are important sources of ideas for improvement)  Changing workplace culture (people observe first-hand the practices of other organizations. This means they are more likely to commit to changes implemented within their own organization as a result of the benchmarking exercise.)  Strategic and business planning (benchmarking can help identify achievable goals to be included in the plan and a good strategic plan can identify areas to be benchmarked)  Annual budgeting process (benchmarking projects can identify the cost implications of improvements required)  Quality assurance systems (process benchmarking can identify improved processes to incorporate into standard operating procedures)  „Business Excellence‟ Criteria (Quality Awards assessors expect benchmarking to play an integral role in an organization‟s approach to process improvement). 144
  • 145. PLATFORM 3: VIII (3) Process Benchmarking  Stage 1: Decide What to  Stage 4: Analyze Findings Benchmark  Analyze Gaps  Make Initial Choice  Identify Best Practices  Refine Choice  Stage 5: Improve Practice  Stage 2: Plan Project  Plan Implementation  Identify Scope  Establish Team  Manage Implementation  Manage Project  Plan Consequences  Stage 3: Conduct project  Train Team  Analyze Practices  Choose Partners  Design Questionnaires 145
  • 146. PLATFORM 3: VIII (3) Performance Benchmarking  Stage 1: Decide What to  Stage 3: Conduct Project Benchmark  Design Questionnaire  Stage 2: Plan Project  Gather Data  Choose Partners  Analyze Data  Plan Comparisons  Produce Report  Manage Project  Stage 4: Analyze results  Stage 5: Determine Further Action 146
  • 147. PLATFORM 3: STAGE IXINFUSE INNOVATION NORMS & VALUES
  • 148. PLATFORM 3: IX Infuse Innovation Norms & Values  3-Step Process 1. There needs to be an expression of feelings, emotions, goals, motivators, & the like from each individual member of the group. 2. Individuals need to interpret & personally internalize them. 3. Through compromise & consensus building, each group member ultimately reaches agreement on the norms & values of the group.  Guiding Principles 1. Try to be fair. 2. Use your freedom to grow. 3. Make your own commitments & keep them. 4. Consult with other colleagues prior to any action that may adversely affect the reputation or financial stability of the company. 148
  • 149. PLATFORM 3: IX Change Management 1. Create a Sense of Urgency 2. Produce a Guiding Coalition 3. Develop Vision & Strategy 4. Communicate Change Vision 5. Empower Broad-based Action 6. Generate Short-term Wins 7. Consolidate Gains, Produce More Change 8. Anchor New Approaches in Environment 149
  • 150. PLATFORM 3: IX (Parts 1-2) Change Management 1. Create a Sense of Urgency  Market & competitive realities  Potential crises & opportunities  Influence games impacting environment  Persuasive influence 2. Produce a Guiding Coalition  Assemble & organize stakeholders for max. influence  Develop relationships with critical decision-makers  Make the group work like a cohesive team 150
  • 151. PLATFORM 3: IX (Parts 3-4) Change Management 3. Develop Vision & Strategy  Create a vision to help direct change effort  Determine leverage points for exerting influence  Develop strategies to achieve vision  Build coalitions around unifying points of agreement Build momentum 4. Communicate Change Vision  Use every vehicle to CONSTANTLY communicate vision & strategies  Frame arguments to convey essential messages with clarity & persuasion  Guiding coalition role-model actions / behaviour desired 151
  • 152. PLATFORM 3: IX (Parts 5-6) Change Management5. Empower Broad-based Action  Get rid of obstacles  Develop a roadmap  Change systems / structures undermining change vision  Benchmark  Develop an industrial / market cluster  Apply effective decision-making tools  Encourage risk taking, innovation & creativity6. Generate Short-term Wins  Visible improvements in performance / outcomes  Arrange optimum strategic partnering  Execute wins efficiently & effectively  Visibly recognize & reward people responsible 152
  • 153. PLATFORM 3: IX (Parts 7-8) Change Management7. Consolidate Gains, Produce More Change  Capitalize on influence, credibility & track record to CHANGE all system, structures, policies  Hire, promote, develop people to drive change  Reinvigorate with new projects, themes, change agents  Seek continuous improvement  Develop & apply suitable metrics8. Anchor New Approaches in Environment  Optimize all processes, systems, performance etc., behaviour, leadership, management  Articulate, model & apply (via NLP) best practice for “success”  Ensure continuity: leadership development & succession 153
  • 154. IRL TECHNOLOGY INNOVATION STRATEGY FRAMEWORK END