policyELSEVIER ResearchPolicy25 (1996) 1097-1119
Learning-before-doing in the development of new process
1098 G.P. Pisano/ Research Policy 25 (1996) 1097-1119
trial-and-error. 3 Although focused on different con-
texts, studies...
G.P. Pisano / Research Policy 25 (1996) 1097-1119 1099
invested in the project). 6 The paper is organized as
follows: Sect...
1100 G.P. Pisano/ Research Policy 25 (1996) 1097-1119
levels. ,0 In a competitive environment, however,
development goals ...
G.P. Pisano / Research Policy 25 (1996) 1097-1119 1I01
representative of the performance that can be ex-
pected in some fu...
I 102 G.P. Pisano / Research Policy 25 (1996) 1097-1119
circumvents the problem of predicting C(plXc)
based on observation...
G.P. Pisano / Research Policy 25 (1996) 1097-1119 1103
identified and are well understood, t9 For the pur-
pose of determi...
1104 G.P. Pisano / Research Policy 25 (1996) 1097-1119
periments that provide a reasonably accurate predic-
tion of expect...
G.P. Pisano / Research Policy 25 (1996) 1097-1119
Table 1
Comparison of research process and commercial production process...
1106 G.P. Pisano // Research Policy 25 (1996) 1097-1119
ing chemical processes influences problem-solving
strategies in ve...
G.P. Pisano/ Research Policy 25 (1996) 1097-1119 1107
Table 2
Number and timing of synthetic
cal projects
route change in ...
1108 G.P. Pisano/ Research Policy 25 (1996) 1097-1119
3.2.1. Biotechnological processes
The characteristics of the knowled...
G.P. Pisano / Research Policy 25 (1996) 1097-1119 1109
mechanisms by which cells synthesize proteins, the
precise sequence...
1110 G.P. Pisam~/ Research Policy 25 (1996) 1097-1119
4. Research design, data, and analysis
The data used in the analysis...
G.P.Pisano/ ResearchPolicy25 (1996)1097-1119 1111
opment was calibrated using project records (e.g.
time sheets) and estim...
1112 G.P. Pisano/ Research Policy 25 (1996) 1097-1119
Table 3
Descriptive statistics means (standard deviation in parenthe...
G.P. Pisano/ Research Policy 25 (1996) 1097-1119 I 113
in the development of biotechnology processes. If, as
1114 G.P. Pisano/ Research Policy 25 (1996) 1097-1119
~. 100000
"~ o
< - 100000
. , . . , ,
o ...
G.P.Pisano/ ResearchPolicy25 (1996)1097-1119 1115
Chemical Projects (n=13)
200000 ~ . . . . . . . . . . . . .
0 1...
1116 G.P. Pisano / Research Policy 25 (1996) 1097-1119
lems with each step were solved, and additional yield
and throughpu...
G.P. Pisano/ Research Policy 25 (1996) 1097-1119 1117
consume extensive resources. No major problems
(either anticipated o...
1118 G.P. Pisano/ Research Policy 25 (1996) 1097-1119
An interesting issue for further exploration con-
cerns how firms bu...
G.P. Pisano/ Research Policy 25 (1996) 1097-1119 1119
Utterback, J.M., 1994, Mastering the Dynamics of Innovations
Upcoming SlideShare
Loading in …5

8. pisano 1996


Published on

Published in: Business, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

8. pisano 1996

  1. 1. research policyELSEVIER ResearchPolicy25 (1996) 1097-1119 Learning-before-doing in the development of new process technology 1 Gary P. Pisano Technology and Operations Management, Harvard Business School, Morgan Hall T97, Soldiers Field, Boston, MA 02163, USA FinalversionreceivedApril 1996 Abstract The concept of learning-by-doing has featured prominently in the literature on manufacturing improvement and technological innovation. Much of the focus of this literature is on the 'doing' or 'using' that takes place in an actual commercial production or usage environment. But problem-solving can also occur long before a new product or process design is introduced into the factory through computer simulations, laboratory experiments, prototype testing, pilot production runs, and other experiments. These approaches to problem-solving are referred to in this paper as 'learning-be- fore-doing.' Although there is an extensive literature on prototyping and technical problem-solvingin R &D, there have been few attempts to determine the type of conditions under which different approaches to learning are most effective. This paper explores the impact of different learning strategies on development performance with detailed data on 23 process development projects from pharmaceuticals and biotechnology. The empirical analysis focuses specifically on how the timing of technology transfer to the factory influences development costs. The results of the analysis indicate learning-by- doing is essential for efficient development in an environmentlike biotechnology where underlying theoretical and practical knowledge is relatively thin. In contrast, the need for learning-by-doingis far lower in environmentslike chemical synthesis where underlying theoretical and practical knowledge is deep enough to enable the design of laboratory experiments that effectively model future production experience. The paper concludes with a discussion of the implication of these findings for the management of process development and learning. 1. Introduction The concept of learning-by-doing has featured prominently in the literature on manufacturing im- This paperis part of a largerstudyon processdevelopmentin pharmaceuticals. I wouldlike to thankStevenWheelwrightfor his collaboration on this project, Kim Clark, Robert Hayes, Macro lansiti, Richard Nelson, Richard Rosenbloom,Eric yon Hippel, and the anonymousreferees for helpful comments,and Sharon Rossi for excellentresearch assistance.All errors and omissions remain entirely my responsibility.The financial support of the Harvard BusinessSchool Division of Research is gratefullyac- knowledged. provement and technological innovation (e.g. Arrow, 1962; Rosenberg, 1982, yon Hippel and Tyre, 1995). There is a large body of evidence documenting the tendency of manufacturing performance to improve with cumulative production experience. 2 Similarly, learning-by-doinghas also been identified in detailed studies of the R&D process which show that techni- cal problem-solving generally involves extensive 2 Sec e.g. Wright (1936), Hirsch (1952), Stobaugh and Townsend(1975),and Lieberman(1984),Adlerand Clark(1991). 0048-7333/96/$15.00 Copyright© 1996ElsevierScienceB.V. All rightsreserved. PII S0048-7333(96)00896-7
  2. 2. 1098 G.P. Pisano/ Research Policy 25 (1996) 1097-1119 trial-and-error. 3 Although focused on different con- texts, studies of manufacturing improvement and R&D processes provide compelling evidence that learning requires some form of 'doing.' More recent work by Adler and Clark (1991) and von Hippel and Tyre (1995) has illuminated some of the critical organizational mechanisms underpinning learning- by-doing. While a picture is beginning to emerge of how and why learning-by-doing occurs, there is far less understanding about the different types of experi- ences involved in learning, and the implications of these differences for problem-solving strategies. The existing literature tends to focus on the 'doing' or 'using' that takes place in an actual commercial production or usage environment. Thus, for example, studies of the learning curve generally take as their starting point the introduction of a new product or process into the full-scale, commercial manufactur- ing environment. But 'doing' can take many other forms, including computer simulations, laboratory analyses, prototype testing, and pilot production and other experiments that occur before a new process is introduced into the factory. The term 'learning-be- fore-doing' is used in this paper to differentiate these problem-solving strategies from the more conven- tional notion of learning-by-doing that takes place after a process is transferred to a commercial produc- tion environment. Although there is an extensive literature on prototyping and technical problem-solv- ing in R&D, there have been few attempts to com- pare these methods of 'learning-before-doing' to those associated with learning-by-doing in the actual production environment. 4 Is learning-by-doing (in the factory) always necessary? Under what condi- tions and for what type of problems can learning-be- fore-doing be an effective problem-solving strategy? These are the central issues explored in this paper. The issue of appropriate learning strategies has important implications for the management of devel- opment, and, in particular, the interface between R&D and manufacturing. One of the most challeng- ing, and often most problematic, aspects of product 3 Marples (1961), Allen (1966), Wheelwright and Clark (1992). 4 von Hippel and Tyre (1995), however, do examine the condi- tions which make learning-by-doing an attractive strategy. development lies in the development and transfer of new manufacturing process technology into the plant. The literature on development is rife with examples of how technology transfer problems have led to delayed product introductions, development cost overruns, excessive product costs, and quality prob- lems. From the plant's perspective, the problem is often viewed as one of receiving the process technol- ogy too late in the development cycle and not having had a chance to 'come down the learning curve' before the start of commercial production. Earlier transfer of process technology to the plant is an approach commonly advocated by manufacturing. Indeed, in recent years, there has been a tendency in the literature on innovation to emphasize the role of the factory in development. 5 The problem can look very different from R&D's perspective, which often sees the problem as one of not having had enough time to develop a fully understood and manufac- turable process before the transfer. Lurking beneath this debate are different perspectives on the potential to learn-before-doing versus the need to learn-by- doing. Resolving such debates, and structuring the development process in the most effective manner, requires a deeper understanding of when each of these models of learning is most appropriate. This paper explores the impact of learning strate- gies on development performance with detailed data on 23 process development projects from pharma- ceuticals. This sample includes both traditional chemically synthesized drugs and drugs produced through recombinant biotechnology methods. These data were collected as part of a larger study on the impact of project strategies, organizational struc- tures, and organizational routines on process devel- opment performance in the pharmaceutical industry. This paper focuses on how the timing of technology transfer to the plant influences development costs (as measured by scientific and engineering person hours 5 Leonard-Barton (1992), for example, argues that companies should think of their factories as extensions of their R&D labora- tories and utilize them more heavily for experimentation and technical development. Her article uses the example of Chaparral Steel, a highly successful 'mini-mill,' and notes that no R&D department exists separately from production. According to the company's chief executive officer, Gordon Forward, "[E]every- body is research and development. The plant is our laboratory."
  3. 3. G.P. Pisano / Research Policy 25 (1996) 1097-1119 1099 invested in the project). 6 The paper is organized as follows: Section 2 develops a framework for analyz- ing the choice between learning-by-doing and learn- ing-before-doing problem-solving strategies. Section 3 uses the framework to highlight critical differences in the structure of the knowledge bases characteriz- ing biotechnology and chemical processes, and the implications for appropriate learning strategies in each technology. Section 4 provides an empirical analysis of the impact of learning strategies on de- velopment performance using data from the process development projects in the sample. Section 5 con- cludes the paper with a discussion of the implica- tions of the findings for the management of process development and avenues for further research. 2. Framework for process development Linkages between product and process technology differ substantially across industries, 7 and these dif- ferences influence the nature of both the technical and organizational challenges of process develop- ment. For instance, in many types of assembled goods industries (automobiles, personal computers, consumer electronics), product functionality and fea- tures are not severely constrained or heavily im- pacted by the process design. At the other end of the spectrum are industries such as pharmaceuticals, chemicals, semiconductors, and advanced materials where product and process designs are highly inter- dependent and changes in process technology can have a significant impact on product characteristics. (Conversely, minor changes in product design speci- fications can require the development of completely new process technologies.) In such industries, pro- cess development capabilities play an integral role in overall product development performance. More- over, the challenge in these contexts goes well be- yond adopting design-for-manufacturability or simul- taneous product and process engineering, s Although these may be critical first steps in achieving better integration between product and process design, they do not address the issue of how process development itself can be made more efficient. The discussion below provides a framework for understanding the drivers of process development efficiency in differ- ent kinds of environments. 2.1. Process development and experimentation Despite differences in the role of process develop- ment across industries, there are certain similarities in the nature of problem-solving involved in the development of new process technologies. The start- ing point for process development is a description of the product, or a product design. When process development starts, of course, the description may be incomplete or in a state of flux. In chemicals, this might be a written description of the molecule, a formula for the set of reactions required to synthe- size the molecule, or other data characterizing the molecule. In the model of process development pre- sented here, process developers can be viewed as starting with a set of targets for process performance. 9 These might be framed in terms of unit cost, capacity required, yields, quality levels, critical tolerances, or other operating characteristics. To simplify the exposition, let C represent the set of process performance characteristics when operated under expected commercial conditions (Xc). The performance of the process (C) is determined by choices over a set of process parameters (p). One goal of the process developer is to find a set of process parameters, p, which either optimizes C(p* ]Xc) or at least achieves satisfactory target 6 Another paper in this study has examined the impact of spending on process research activities on overall project lead times. Although both papers draw from a common conceptual framework, the papers differ substantially in terms of both the dependent variable, as well as the key independent variables of interest. 7 See e.g. Abernathy (1978), Pavitt (1984), Utterback (1994). 8 There is a growing body of literature on these and related approaches to product development. See e.g. Whimey (1988), Clark and Fujimoto (1991), Clausing (1993), Ulrich and Eppinger (1995). 9 Because of uncertainty, in practice these targets are often framed as ranges rather than discrete points (e.g. cost should be between $10 and $15 per unit).
  4. 4. 1100 G.P. Pisano/ Research Policy 25 (1996) 1097-1119 levels. ,0 In a competitive environment, however, development goals should also include finding p * as efficiently and quickly as possible. Finding p* is a technical problem solving pro- cess. i, Although their activities and nature may vary across contexts, technical problem solving processes share common characteristics. One is that problem solving is triggered by gaps between desired and actual performance (Newell and Simon, 1972; Iansiti and Clark, 1994). As noted above, in the context of process development, this gap can be framed in terms of differences between what existing process technology can achieve (e.g. yields and costs) and what is required to achieve success in the market. The second characteristic of problem solving is that it generally takes place through iterative cycles of search and selection (Frischmuth and Allen, 1969; Nelson and Winter, 1977; Nelson, 1982). With each cycle, the gap between actual and desired perfor- mance becomes progressively more narrow as tech- nical solutions are identified and tested and a subset of solutions is selected. At the heart of this learning process are experiments (both physical and concep- tual) that provide feedback about gaps between cur- rent and target performance levels. 12As a result, the quality of feedback from experiments plays a critical role in determining development performance. Experiments can take many forms and be con- ducted under a variety of conditions. While the traditional image of an experiment is a laboratory- based analysis of product samples or physical proto- types, advances in technology allow some product Io A controversial issue within the economics of R&D literature is whether it makes sense to model a firm's R&D choices in an optimization framework. This issue is well beyond the scope of the present paper. For convenience, we will use the terms like 'optimal' and 'optimizing' when describing choices. However, for the present purposes, it matters little whether firms are actually optimizing or 'satisfying.' H Viewing product and process development in terms of prob- lem-solving has a long history in the innovation and organization literature. See, for example, Allen (1966), Frischmuth and Allen (1969), Simon (1978), Clark and Fujimoto (1991), Dosi and Marengo (1993), lansiti and Clark (1994), and von Hippel (1994). 12Wheelwright and Clark (1992) suggest that the role of proto- typing is so critical to the development process that it is helpful to view development projects as a series of 'design-build-test' cy- cles. The Locus of ~per/mentation and Representativeness Representativeness ofFinalProduction Environment High Low LocusofExperimentation Full-ScaleCommercialFactory PilotPlantLocatedat ProductionSite Pilot PlantLocatedat R&DSite Laboratory ComputerAided Simulation Fig. 1. The locus of experimentation and representativeness. and process designs to be tested and analyzed using computer-aided simulation. 13 Physical experiments can also be conducted in different ways. In process development, some experiments are conducted in laboratories, others are performed in pilot plants, and still others are run in full-scale commercial plants. One way to distinguish between experimental forms is the extent to which they are conducted under conditions representative of the final use envi- ronment. An example of an experiment conducted under highly representative conditions would be test- ing a process technology in a full-scale commercial facility during normal production hours, utilizing regular production operators. At the other end of the representativeness spectrum would be small-scale laboratory tests and computer simulations (see Fig. 1). Experimental conditions affect experimental out- comes. Thus, experimental outcomes are not always 13Boeing, for example, in developing its most recent generation of wide-body aircraft, the 777, did not build a single physical prototype but instead relied completely on computer-aidod design and simulation. This practice of using computers to test product designs has been referred to as ' virtual prototyping' (see Garcia et al., 1994).
  5. 5. G.P. Pisano / Research Policy 25 (1996) 1097-1119 1I01 representative of the performance that can be ex- pected in some future operating environment. Con- sider the process developer in search of C(p* ). When a set of process parameters is tested in the laboratory under conditions XL, researchers do not actually observe C(p); instead, they observe labora- tory performance, or what might be called L( pl XL)- Note, the term 'laboratory conditions' is used broadly to encompass a broad range of venues, from com- puter simulations through pilot-scale production plants. The vector of experimental conditions might include such things as the scale of the process, the type of equipment (e.g. glass-lined versus stainless steel reactors), the skill of the operators, the cleanli- ness of the air, and other environmental variables. 14 Experimenters may have different degrees of control over different experimental conditions. For instance, a researcher can choose the type of equipment they use for an experiment, but may have little control over the air cleanliness. Laboratory experiments can be viewed as repre- sentations of the future commercial manufacturing process. For instance, a laboratory experiment on a chemical process is qualitatively very different from a commercial production environment. Yet, each element of the experiment has analogies in the fac- tory. The small glass test tubes represent the stainless steel reaction tanks of the factory; the thin glass mixing rods used to stir the reaction simulate the forces of automated steel rotators; the chemist who sets up the experiment and watches over it plays the role of both the future factory operators and the computer-based process control system. Clearly, the conditions of the laboratory experiment are quite different from the future factory (small glass test tubes vs. large stainless steel tanks, a Ph.D. chemist versus factory operators and a computer system, etc.). These differences can lead to a lack of fidelity in experimental results. For example, the chemical pro- cess may behave and perform differently in a small- scale versus a large-scale vessel or in a glass versus a stainless steel vessel. The chemists may stir the process very differently from the way it would be stirred by large, computer-controlled rotators. For these reasons, the researcher can expect that C(p'l Xc) will not be equal to L( p'[ XL). Yields may be lower; costs may be higher; different impurities might be present. Even tests in a pilot plant can be quite different than what might be expected in a full-scale plant. This is one of the reasons why companies might discover problems with a process technology only after it is transferred and used in the actual commercial environment. Given that experi- mental conditions affect experimental outcomes, a major challenge of development is to make predic- tions about C(plXc) based on observations of L(p[XL). 15 2.2. Learning-by-doing versus learning-before-doing One approach to improving the fidelity of experi- ments (i.e. reducing the gap between C(plXc) and L(plXL)) is to make test conditions as close as possible to actual operating conditions. At the ex- treme, test batches might be run in the commercial manufacturing plant, rather than in the laboratory or small-scale pilot facility. If all the conditions of the future manufacturing environment are replicated in the test (the same factory, the same equipment, the same workers, the same suppliers, etc.), XL should be equal to Xc and the test should provide a rela- tively good indicator of future performance. In real- ity, given that factory conditions change from day- to-day or even from minute to minute (e.g. people change, equipment wears, the humidity fluctuates, etc.), it is impossible to replicate exactly the future conditions even with an in-factory test. Nevertheless, the in-factory test should provide the most represen- tative experimental environment. The idea that some things can only be learned by running the process in the factory is consistent with the idea of learning- by-doing. The power of learning-by-doing as a prob- lem-solving strategy is that it leads to high fidelity experimental results. Learning-by-doing essentially a4 In some contexts, the raw materials available in small quanti- fies for research purposes are slightly different than those avail- able for use in a large-scale production environment. 15For simplicity, we will repress the added complication that one can have observational errors even in the laboratory because of instrument calibration or other factors. Thus, the true L(p) might differ from the observed L(p).
  6. 6. I 102 G.P. Pisano / Research Policy 25 (1996) 1097-1119 circumvents the problem of predicting C(plXc) based on observations of L(pIXL), by making ex- perimental conditions (X L) as close to commercial conditions (Xc) as possible. Although learning-by-doing leads to high fidelity results, it also creates other costs. Direct costs per experiment are typically higher in a factory setting than in a laboratory setting because of minimum efficient batch sizes and investments in specialized equipment. 16 Factory-based experiments also use manufacturing capacity that might be deployed to make saleable product. 17 A higher cost per experi- ment means that for any given number of iterations required to converge to the desired performance level, an experimental strategy emphasizing in-fac- tory tests will involve higher development costs. In addition, because factories are complex and some- times chaotic environments, it can be difficult to identify and control all the relevant intervening vari- ables. Slight but unavoidable batch-to-batch differ- ences in raw materials, workers, equipment, equip- ment settings, and other process parameters will reduce the signal-to-noise ratio of factory experi- ments. Furthermore, in some contexts, extremely costly and delicate high precision test and measure- ment equipment that could be used in laboratory analysis would not be feasible in a production envi- ronment. This further compromises the signal-to- noise ratio of moving experimentation from the labo- ratory to the factory. Doing may be an effective way to learn, but it can also be costly. An alternative, and potentially less costly ap- proach, is to model future manufacturing perfor- mance outside the actual production environment using computer simulations, physical laboratory ex- periments, pilot plant tests, and other forms of simu- lation. These methods can be viewed as attempts to predict process performance and to identify potential problems before the process is transferred to the factory. Thus, for instance, if a process fails com- pletely in a test tube, then this may be a good indication that this will not work in the factory either. More subtly, researchers might be able to identify impurities or sources of process variance through small-scale models which can be addressed before the process is transferred to the plant. How- ever, although simulations are generally less costly experiments than on-line tests in a factory, they also potentially suffer from the fidelity problems dis- cussed earlier. Given that most product and process development projects use the full spectrum of approaches (from laboratory simulations through 'real life' tests) it suggests that the issue is one of the appropriate balance between approaches, rather than adopting one over the other. Simulations and other forms of learning-before-doing can be used to identify and solve some types of problems. But problems for which a representative experiment can not be de- signed can only be identified and solved through learning-by-doing. The balance between approaches should be related to the structure of knowledge affecting one's ability to use laboratory experiments and other forms of simulation to predict future per- formance and identify and solve potential future problems. 2.3. Structure of knowledge and the appropriate learning strategy Over the years, there have been various attempts to characterize how knowledge varies from context to context. Going back to Polanyi (1962), several writers have distinguished between 'tacit' and 'codi- fied' knowledge. 18 While this distinction can be useful, it really only refers to the form in which knowledge is stored. As most writers on the subject have recognized, beneath this distinction is the extent to which underlying cause-effect relationships (or as Polanyi put it, the 'set of rules') have been clearly t6 For example, in the automobile case, investments in produc- tion tooling for the entire car could cost nearly $100 million. If designs change, the tooling becomes obsolete and must be re- placed. 17In chemical processes, safety issues can also raise costs of full-scale experiments. is See, for example, Teece (1976) and Kogut and Zander (1992). Winter (1987) developed an expanded taxonomy of knowledge which included additional dimensions: teachability, observable in use, complexity, and links to other components of a system.
  7. 7. G.P. Pisano / Research Policy 25 (1996) 1097-1119 1103 identified and are well understood, t9 For the pur- pose of determining the appropriate problem-solving strategies, the critical cause-effect relationships re- late to the effect of experimental conditions on ex- perimental outcomes. If problem-solvers understand how specific exper- imental conditions (e.g. scale, equipment, humidity, etc.) affect particular outcomes (e.g. process yield), they can use this knowledge to make predictions about performance under a future set of operating conditions. For instance, if researchers know from either scientific theory or experience that each order of magnitude increase in scale reduces yields by approximately 10%, a small-scale experiment can be used to make predictions about performance at a larger scale. With deeper knowledge of these cause-effect relationships, the researcher can build more complete and accurate models mapping labora- tory observations into expected future performance (under some specified set of conditions). In an ex- treme case, the researcher might have complete knowledge of all the relevant experimental condi- tions, their impact on process parameters, and the first- and second-order effects of all process parame- ters on performance. 20 Take the following simple example of a chemical experiment conducted in a 1-1 glass bottle as a simulation for a commercial process to be run in a 5000 gallon stainless steel tank. Assume for simplic- ity that the only relevant experimental conditions are the scale of the process (1 1vs. 5000 gallons) and the material composition of the reaction vessel (glass versus stainless steel), that temperature is the only relevant process parameter, and that the researcher is interested only in one performance outcome: yield. If the researcher knows all the relevant experimental conditions and process parameters and the indepen- dent and interactive relationships between tempera- ture, scale, vessel materials, and yields, an experi- ment conducted under one set of conditions (a 1-1 glass bottle) at one specific temperature point can be used to find the temperature that optimizes yield under a different set of conditions (5000 gallons, stainless steel). 21 For any sufficiently complex process, such com- plete knowledge is probably impossible to obtain. However, to learn-before-doing, the researcher only needs to have a 'global' model of how outcomes observed under some specified set of conditions map into outcomes under another set of conditions. For instance, through years of testing processes in the laboratory and observing actual performance in the factory, researchers may develop a heuristic to pre- dict future performance from laboratory experiments. Prior scientific knowledge or experience provides predictive models of the form: "If we observe E(p') in the laboratory, we can expect C'(p') in the plant." In this case, the researcher may not be fully aware of how any specific condition affects the process or its performance, but has an overall understanding of how the complete set of experimental conditions jointly influence outcomes. Heuristics may emerge about what to expect based on observations in the laboratory or the pilot plant (e.g. "multiply the yields observed in the laboratory by 0.75 in order to predict future yields in the factory"). In this case, knowledge takes the form C(plXc)= F[(L(plXL), where the functional form of F could be as simple as a constant or a more complex relationship. When developers talk about a process having a 'linear scale-up,' they mean that what is learned about the process at small scale (and under other laboratory conditions) can be extrapolated to higher scale (and other commercial manufacturing conditions). In any given technological context, opportunities to learn-before-doing should depend on the degree to which critical cause-effect relationships are de- scribed by scientific theories or heuristics based on cumulative practical experience. Where underlying basic theoretical knowledge is strong (and where there is relatively complete knowledge of the condi- tions in the future production environment), one may know enough about critical variables and their be- havior to design highly representative laboratory ex- 19The Bohn and Jaikumar (1992) 'stages of knowledge' frame- work provides a useful taxonomy for differentiating knowledge along this dimension. 20This would essentially correspond to 'Stage 8' in Bohn and Jaikumar's framework. 2t Dimensional analysis can be used to map experimental obser- vations across scales. But to do dimensional analysis, one needs to know the underlying scale effects.
  8. 8. 1104 G.P. Pisano / Research Policy 25 (1996) 1097-1119 periments that provide a reasonably accurate predic- tion of expected commercial performance. 22 Under these conditions, learning-before-doing should play a greater role in the development process (although the need for some learning-by-doing in the factory should still be present). In contrast, where theoretical or practical knowledge of scale and other experimental effects is limited, it will be virtually impossible to predict how the process tested in the laboratory will work when run under actual operating conditions. Subtle and often unknown differences between the laboratory and the factory environment could have major impacts on process performance. Much of what is learned outside the factory could be irrele- vant and unless developers have been lucky, rework of the process will likely be required to get it to operate as planned. For this type of technological regime, evaluation of the process needs to be done under conditions as close to actual operating condi- tions as possible. Transferring the process to the plant when it is still immature, and developing and refining it there (learning-by-doing) should be the more efficient approach. The proposition that the value of learning-by- doing (in terms of development performance) should be higher in environments with a less mature knowl- edge base is tested with data on 23 process develop- ment projects from the biotechnology and chemical segments of the pharmaceutical industry in Section 4. To deepen the discussion of knowledge environ- ments and to motivate the statistical analysis, the next section discusses the nature of process develop- ment and process knowledge in both chemical- and biotechnology-based pharmaceuticals. 3. The anatomy of process development in phar- maceuticals biochemical compounds used in drugs. 23 In the discovery phase of pharmaceutical R&D projects, research scientists develop crude processes for syn- thesizing relatively small amounts of the molecule under investigation. These laboratory methods of production, however, are almost always completely unsuitable for manufacturing the compound in com- mercial volumes at required cost and quality levels. Table 1 provides an illustrative comparison of an 'initial discovery process' and a 'final commercial production process' for a representative chemical entity. The two basic process technologies represented in the sample of 23 projects (chemical synthesis and biotechnoiogy) provide an excellent opportunity to examine the proposition that the nature of the knowl- edge environment influences the optimal learning strategy in development. Each of these technologies is at a very different stage of historical development. Each also lies at a different end of the spectrum in terms of depth of theoretical and practical knowl- edge. These differences stem from the respective maturities of the underlying scientific fields, the development of relevant scientific theory, and the availability of process engineering heuristics to aid in scale-up. A brief description of the knowledge base and the nature of the technical challenge facing developers in each environment is provided below. 3.1. Chemical synthesis Chemical synthesis is the traditional method for producing the active ingredient found in most drugs. Chemical process development involves determining the sequence of chemical reactions required to con- struct the desired molecule (the synthetic route), and specifying the reaction conditions, equipment de- signs, mechanical manipulations, process flows, and The study focuses on the development of produc- tion processes for therapeutically active chemical or 22 On the role of scientific knowledge in supporting efficient R&D, see Nelson (1982). 23Thus, we have excluded the process of formulating the final drug form (e.g. capsule, tablet, cream, liquid, etc.) taken by patients. For clarification, the term chemical is used to describe small molecules synthesized through traditional organic chemical methods. Biotechnology is used to describe large protein molecules produced from genetically engineered cells.
  9. 9. G.P. Pisano / Research Policy 25 (1996) 1097-1119 Table 1 Comparison of research process and commercial production process for representative chemical entity 1105 Initial discovery process Final commercial production process Number of chemical steps 25 Equipment Test tubes 1-1 flasks Batch size (output) Approximately 1 g Operators Ph.D. chemists Purity Cost (kg- ' ) Criteria for process design 1-10% Approximately $20000-$50000 Biological activity of molecule Patent issues 7 2000-4000 gallon stainless steel vessels 100-200 kg Technicians Semi-skilled plant workers 99% Approximately $3500 Cost Quality (purity) Conformance to drug and environmental protection regulations Operability operating procedures that optimize performance. Problem-solving draws largely from two bodies of knowledge: chemistry and chemical engineering. Both disciplines have characteristics that influence how problems are framed and the opportunities to use theory, small-scale experimental data, and com- puter simulations to search for commercially viable solutions. Chemistry and chemical engineering are relatively mature as both academic disciplines and as areas of industrial practice. Fundamental research in chem- istry can be traced back to the 18th century. 24 Many of the critical principles of chemical theory were discovered during the late 18th and 19th centuries. Over the course of nearly 2 centuries of basic re- search, a rich body of laws, principles, and theories has evolved about the way chemical processes work. Application of this fundamental knowledge to indus- trial production processes dates to the 18th century (the use of chemical synthesis to manufacture drugs began in the mid-19th century). Years of industrial experience, starting initially with the production of synthetic dyes, not only served to validate and refine chemical theory, but it also led to the development of knowledge about how to scale-up chemical pro- cesses. Chemical engineering began to emerge as a formal discipline in the United States in the latter part of the 19th century. Through years of cumula- tive development and scale-up experience, a large 24 On the history of the chemical industry, see Haber (1958). body of chemical engineering heuristics has evolved which is still widely used today to guide process selection, scale-up, and plant design. Thus, although chemistry and chemical engineer- ing are still active research disciplines today, both fields rest on well-established, highly articulated and formalized knowledge bases. In searching for and selecting alternative chemical processes, and in de- veloping a suitable commercial manufacturing pro- cess, developers in the chemical environment have at their disposal a wealth of scientific laws (e.g. ther- modynamics), principles, and applied models that describe the structure of relationships between differ- ent process variables (e.g. pressure, volume, temper- ature, etc.). Increasingly, this knowledge is being codified in computer models which permit rapid simulation of processes and the impact of specific process parameters and conditions on yields, cost, throughput, and capacity. The knowledge base is not just technical, but also extends to organizational issues. Through cumulative experience, pharmaceutical companies have devel- oped well-established organizational routines and standard operating procedures for quality assurance and process control, production scheduling, changeovers, maintenance, and other production ac- tivities. Many of the nuances of the production envi- ronment are well understood. Experience with these routines provides concrete starting points for devel- opment and guidance about what types of process techniques are feasible within an actual production environment. The structure of knowledge surround-
  10. 10. 1106 G.P. Pisano // Research Policy 25 (1996) 1097-1119 ing chemical processes influences problem-solving strategies in very specific ways. The selection and scale-up of a synthetic route is used as an example below. 3.2. Route search, selection, and scale-up Since the route represents the basic architecture of the process, and has a major impact on its commer- cial performance, its selection and scale-up is a critical element of the development process. Any given molecule can almost always be synthesized from a variety of starting raw materials and different combinations of chemical reactions. Route search and selection involves finding the one with the best properties for commercial manufacturing. Techni- cally feasible synthetic routes are bounded by chemi- cal and physical laws which are very well under- stood. Thus, process research chemists almost al- ways start their search for a route by generating a list of theoretically feasible alternatives. Options are de- rived from theory, from clues provided in published literature, or from examining internal databases of processes used for other molecules (with similar structural characteristics). Chemists will sometime talk about conducting 'paper experiments,' meaning that they work out a potential route on paper. Before any physical laboratory experiments are conducted, extensive modeling and analysis are car- ried out to eliminate processes with serious short- comings. Two bodies of knowledge interact at this stage: knowledge of the future manufacturing envi- ronment, and knowledge of the chemistry. Chemistry tells the researchers something about the properties of the process. How many chemical reactions are required? What raw materials are needed? What intermediate chemicals are created? What are the by-products? How much energy is needed? Knowl- edge of the manufacturing environment influences the heuristic for route selection. For instance, since total process yields decline multiplicatively with each step, a common heuristic in route selection is that a route with fewer reactions is preferable to one with more reactions. Routes are also commonly screened out if they require scarce raw materials, generate very hazardous by-products, or are likely to require extreme reaction conditions. Conceptual explorations and analysis are useful for identifying promising leads, or conversely, fatal flaws in a process. But, theory can only narrow down the options so far. While chemical theory gives researchers a good idea about which molecules bond well together, it is less helpful in predicting a reac- tion's kinetic properties (i.e. how fast the reactions will occur). Thus, the next step is to evaluate the remaining candidates in small-scale experiments to gather physical data needed for thermodynamic and kinetic studies of the process. Multiple iterations are conducted to examine the impact of variations in temperature, concentration, solvents, and other reac- tion conditions. Thus, problem-solving still requires trial-and-error, but these trials are conducted in the research laboratories. These data can then be plugged into process simulation models (based on chemical engineering heuristics and principles) to get esti- mates of throughput, capacity requirements, yields, and costs under some set of future operating condi- tions. Although scale-up is commonly viewed as a criti- cal issue in chemical projects, there are no quantity or scale variables in chemistry. That is, simply scal- ing up does not, per se, change the chemistry of a process. Scale can, however, have important second-order effects. For example, at larger scale, heat may not be uniformly distributed throughout a reaction vessel. Reactions may fail to take place in those areas where the temperature has fallen below a specific threshold, thus reducing yields or leading to impurities. The challenge of scale-up is to figure out how to operate the process at larger scale, without changing its basic chemistry. This requires having a very precise characterization of the desired chemistry of the process (otherwise, you are not sure what it is you are trying to keep from changing). If the key process variables have been identified in the labora- tory, scale effects can be reasonably well modeled using chemical engineering heuristics and dimen- sional analysis, and confirmed with pilot-scale pro- duction tests. Evidence supporting the potential to learn before doing in chemical synthesis development can be gleaned from data on the number, timing, and causes of route changes during development. Table 2 pro- vides a list of the projects (with disguised names), the number of route changes during the entire pro- cess development project, and the phase when the
  11. 11. G.P. Pisano/ Research Policy 25 (1996) 1097-1119 1107 Table 2 Number and timing of synthetic cal projects route change in synthetic chemi- Project Number of Development phase when route changes final change implemented a A B C D E F G H I J K L M Pilot Pilot Pilot Pilot Pilot Pilot Research Research Pilot Pilot Production Pilot Pilot a Definition of phases: (1) Research: phase between the start of processs R&D project and the start of the first batch of pilot production. (2) Pilot: phase between start of pilot production and beginning of transfer of process to commercial production site. (3) Production: phase during the transfer and start-up of process in commercial manufacturing facility. last change was implemented. All of the projects involved at least one route change from the original route used in the discovery of the molecule. As shown, multiple route changes were not uncommon in the sample, although the majority of projects involved only one change. Most strikingly, only one route change occurred during or after the transfer of the process to the commercial production facility. And, in this particular case, there were strong indica- tions from project participants interviewed that the underlying problems with the route (which prompted the change) were well known long before this phase. According to the senior process development chemist involved with this project from its inception through its start-up in production: The last step was the Achilles heel of the process. It didn't work in the research laboratories. It didn't work in the pilot plant, and it didn't work in com- mercial manufacturing. But each of us thought we could solve the problem once we got to larger scale - that was how we had done it on other projects. An overwhelming fraction of the changes were either finalized or implemented during the pilot pro- duction phase of process development. The main reason for this is that most organizations prefer to run several production batches before committing to or implementing a specific synthetic route change. Through the interviews, an attempt was made to determine whether the technical problems leading to the eventual route change were first identified before or after the start of pilot production. Given that the process development projects in the sample can span multiple years and involve geographically dispersed development departments, tracking and classifying problems ex post involves obvious methodological difficulties and the data must be interpreted cau- tiously. Of the ten projects involving a route change during the pilot production phase, there were only three clear cases where the problem leading to the eventual change was first discovered during pilot production. 25 In the remaining cases, a problem was initially identified at laboratory scale. Of these cases, about half appeared to be situations wherein route changes were implemented only after extensive at- tempts to rectify the problem at pilot scale through process refinements failed to achieve desired results. This discussion is not meant to imply that all potential issues can be addressed in research labora- tories, and that start-up is trivial. The impact of the subtle idiosyncrasies of the particular production en- vironment (how operators perform certain steps, the lay-out of the factory and the piping, the specific equipment already installed at the plant 26) are diffi- cult to anticipate, and thus there is still a role for learning-by-doing. The issue is one of balance. Given the well-developed state of theoretical and practical knowledge surrounding chemical processes, labora- tory and pilot scale search are relatively productive in uncovering problems and in developing the basic architecture of the process. Refining the process and adapting it to the idiosyncrasies of a particular plant are most likely to require learning-by-doing in the factory setting. 25 In some instances, the organization will transfer a process with known problems to the pilot plant only to make sufficient quantities of material to support clinical testing of the drug. 26 Pilot scale facilities generally use highly flexible configura- tions and equipment in order to accommodate a wide range of potential processes. Commercial manufacturing plants in pharma- ceuticals often use equipment that was originally deployed for a specific set of products.
  12. 12. 1108 G.P. Pisano/ Research Policy 25 (1996) 1097-1119 3.2.1. Biotechnological processes The characteristics of the knowledge base under- lying biotechnology process development are quite different from those of chemical synthesis. These differences are largely related to differences in the maturities of the two technologies. In comparison to chemical-based drugs, biotechnology is in its in- fancy. The major discovery triggering commercial R&D on therapeutic recombinant proteins was only made in 1973.27 The first commercial biotechnoiogy enterprises were founded in the mid-1970s. Although there is extensive basic scientific research in molecu- lar biology, cell biology, biochemistry, protein chem- istry, and other relevant scientific disciplines, most of this work has been geared toward the problems of finding and cloning molecules with specific thera- peutic effects. Compared with the chemical world, there has been very little basic research done on the problems of engineering larger-scale biotechnology processes. Unlike chemicals, the industry as a whole has accumulated little practical experience with pro- cess design and scale-up. The first biotechnology- based pharmaceutical to be manufactured at com- mercial scale, recombinant insulin, was approved by regulatory authorities in 1982; and since that time, only a total of about 25 biotechnoiogy-based thera- peutics have been approved for sale in the United States. Indeed, there was initially skepticism by some observers that recombinantly engineered processes could even be scaled-up. Researchers interviewed during our study generally described the develop- ment process as involving 'more art than science.' The critical differences between the structures of knowledge in chemicals and biotechnology fall into three categories: (1) differences in theoretical under- standing of the basic processes; (2) ability to pre- cisely fully characterize intermediates and final prod- ucts; and (3) knowledge of the second-order effects of scale. Each of these is discussed below. 3.2.2. Lack of theory Compared with chemical synthesis, biotechnology process technology is a knowledge regime character- 27In 1973,StanleyCohenand HerbertBoyerdiscoveredthe basic techniqueusedto geneticallyengineerbacteriacellsso that theycouldbe inducedto producespecificproteins. ized by a relatively immature theoretical base. The precise principles governing the production of re- combinant proteins are dimly understood. There are few general theories or even rules of thumb to guide process search and selection. Researchers inter- viewed in the study emphasized that 'every protein is different.' Predicting how a potential process might perform based on what worked last time is generally tenuous. One researcher interviewed in the study provided an example that describes deciding whether or not to use fetal calf blood serum in the cell culture media. Because fetal calf serum is extremely expen- sive and can make purification more complex, the general tendency is to use a 'serum-free' media. But there is no theory to dictate why and when serum might be required. He noted, For one product, we could easily switch to serum-free media. Another product - which uses the exact same host cell - will die completely when switched to serum free. You just can't tell ahead of time what is going to work. Thus, unlike chemical processes, the factors af- fecting biotechnology processes even at small scale are not well understood. This makes it difficult to predict how a change in the environment (from laboratory to the factory) may affect the process. 3.2.3. Inability to characterize intermediates and final products The protein molecules produced by biotechnology processes have enormously complex structures. Moreover, any given 'molecule' is often really part of a family of related molecules, each of which may differ very slightly from its 'relatives.' Whereas the structure of the relatively small molecules produced through chemical synthesis can be fully character- ized, most proteins can not be. As a result, it can be very difficult to trace how a small change in the process might alter the structure of the protein. Moreover, unlike chemical researchers, biotech- nology researchers can not isolate, analyze, and char- acterize each reaction in the process. Such precision is impossible in the biotechnology context where each process is composed of literally thousands of reactions, virtually all of which occur inside a host cell. Although molecular biologists understand the
  13. 13. G.P. Pisano / Research Policy 25 (1996) 1097-1119 1109 mechanisms by which cells synthesize proteins, the precise sequence of intra-cellular reactions for any given product is not well known. Thus, although researchers might observe certain problems in a pro- cess (e.g. a sudden drop in yield, a change in the form of the protein), these can not be traced to a specific reaction undertaken in the cell. As a result, it is very difficult to anticipate how a change in some process parameters (e.g. temperature) or operating conditions (e.g. scale) might influence the inner- workings of the process. 3.2.4. Lack of knowledge of second-order scale-up effects As in chemicals, scale can have second-order effects on a biotechnology process, but in the case of biotechnology these effects are not well understood. Given that 'every protein is different,' broadly appli- cable scale-up heuristics have generally not emerged. It is very difficult to predict commercial process performance from laboratory data. One researcher interviewed during the study recalled the following incident: According to research, clone #53 would express at 30-40 picograms per cell per day (pcd). But once we scaled it up, performance went down to 15 pcd. Compounding the scale-up problem is the lack of deep understanding about how the process operates even at small scale (Section 3.2.2 above). In the example above, researchers could not explain why clone 53 expressed at 30-40 pcd, let alone under- stand what factors caused it to drop to 15 pcd at larger scale. Researchers generally do not know ahead of time which environmental variables may be rele- vant, nor can they fully anticipate the impact of those which they have identified. For example, in one project in the sample, low yields after the purifica- tion step were discovered during small-scale tests. The purification column (essentially a type of filter) was becoming overloaded, and the active protein was being washed off. The reasons for this occurring were difficult to pinpoint. Removing an earlier pro- cess step to reduce the concentration of the active protein in the solution seemed to work at larger scale. However, it was later discovered that it was actually the larger purification column used and not the increased concentration of the solution that solved the problem. When the process was scaled-up even further in the commercial manufacturing setting, the yield problem re-surfaced because at that scale an even larger purification column was needed. 28 Unfortunately, because the technologies them- selves are so different, it is impossible to compare the nature and causes of problems found in biotech- nology with those discussed earlier for the chemical sample. For instance, since synthetic routes are not developed in biotechnology processes, we can not compare changes along this dimension. Nevertheless, the qualitative information on process changes and problems across the ten biotechnology projects was very consistent with the examples provided above. In virtually every biotechnology project in the sample, scale-up to both pilot and commercial production was associated with significant process problems and resulting process changes. Given the large number of technical issues associated with each scale-up and transfer, it was impossible to systematically track which problems could have been or were anticipated. However, a general theme from the interviews was that problems associated with changes in scale or production equipment were very difficult, even for an experienced organization to anticipate. Using the framework advanced earlier, we would expect the patterns of effective process development to differ between biotechnology projects and chemi- cal projects. Whereas in the chemical environment, the structure of knowledge permits many major prob- lems to be identified and solved in the laboratory and pilot plants, biotechnology is likely to require a much greater emphasis on learning-by-doing in the factory. This hypothesis is tested below. 2~The failure of pilot and commercial batches is a common thread in the interviews conducted during the study and in pub- lished accounts from firms. In a published interview discussing Genentech's early efforts to seale-up human growth hormone, company co-founder Robert Swanson noted: "Scientifically, we began at square one by taking microorganisms out of the labora- tory scale and putting them into much larger stainless steel tanks. You "know, learning how those organisms worked in that environ- ment wasn't easy. The first time we did it, they all dropped dead - they didn't like it." R. Swanson (1986), "'People Make Deci- sions as Owners," interviewed with Arthur Young High Technol- ogy Group, Biotech 86: At the Crossroads: A Survey of an Industry in Evolution (cited in McElvey, 1994).
  14. 14. 1110 G.P. Pisam~/ Research Policy 25 (1996) 1097-1119 4. Research design, data, and analysis The data used in the analysis is drawn from a larger study on process development performance in the pharmaceutical industry. Since the type of infor- mation required for the analysis is not publicly avail- able, it was necessary to gain the cooperation of pharmaceutical companies willing to participate in the study. Because these data are highly proprietary, the names of the participating firms and details of specific projects, other than aggregate statistics, can- not be disclosed. Data for the present analysis were collected from 23 process development projects. Of these, 13 projects involved the development of tradi- tional chemical processes, while ten involved pro- cesses based on new biotechnology-based technol- ogy. In total, 11 organizations participated in the study (five established drug companies, five biotech- nology firms, and one biotechnology division of an established pharmaceutical company). For each pro- ject, data were collected on the history and timing of critical project events, resources expended, and the details of approaches used to identify and solve problems. The data were obtained through a combi- nation of questionnaire surveys, in-depth interviews with key project participants, and proprietary docu- ments. Data from each of the three sources were cross-checked to ensure accuracy and follow-up in- terviews were used to resolve any discrepancies in the data. In total, the data collection process spanned 2 years, and involved close to 200 interviews with personnel from participating R&D sites and plants in the USA and Europe. The nature of the data collection process is one reason the sample size is relatively small. A second factor limiting the sample size was the population of potential projects. Each process development project in the sample was associated with the development of a new molecular entity, a relatively rare event in the pharmaceutical industry. The largest and most productive pharmaceutical firms rarely launch more than one new molecular entity in any given year, and many companies have gone several years without launching any. The situation for biotechnology is even more constraining given the relatively small number of new drugs developed using this technol- ogy. In most instances, the projects in the sample represent each company's most recent development efforts, as these were the ones for which accurate data and key project participants were most avail- able. There was no reason to suspect that these sampling methods lead to any serious bias in the sample that might influence the results. 4.1. Dependent variable The dependent variable, process development costs, is measured as the total number of scientific and engineering person-hours required for all activi- ties associated with process research, development, scale-up, and transfer into the commercial plant. 29 Process development cost is a particularly important dimension of performance in pharmaceuticals given the high uncertainty surrounding development. Most new drugs in development never make it to market. 30 The capability to carry-out individual projects with fewer resources helps to reduce the risks and total costs of development. The start of a project was marked by the initiation of research designed to find a process that might eventually be suitable for com- mercial production. This point could be identified from project records and interviews that indicated when process research personnel first became in- volved in the project. The successful validation of the process at full commercial scale was the event used to mark the end of the project. Because valida- tion has regulatory implications, pharmaceutical companies keep excellent records on when it occurs. The total number of hours invested in process devel- 29 Other potential dependent variables include lead time and manufacturing performance (e.g. yield). An analysis of the deter- minants of lead time performance is presented in Pisano (1994). An analysis of manufacturing performance was not possible be- cause consistent and comparable data on this dimension were not available across the sample of projects. Pisano (1996) analyzes the quality of the manufacturing process design on a sub-sample of projects and fmds no apparent trade-offs between development lead time, development costs, and the quality of the manufacturing process design. 30The high rate of project failures does not introduce bias into the present sample. In pharmaceuticals, development project ter- minations are almost always due to failures of the drug to show safety or efficacy in clinical trials. Since these reasons are com- pletely unrelated to the development of the process technology, they do not cause the project sample to be skewed in ways that might influence the results of this study.
  15. 15. G.P.Pisano/ ResearchPolicy25 (1996)1097-1119 1111 opment was calibrated using project records (e.g. time sheets) and estimates from personnel involved in the project. 4.2. Independent variables 4.2.1. Timing of technology transfer The main independent variable of interest is the timing of the process technology transfer to the plant. Since the total length of projects varies signifi- cantly, it is important to compare the timing of the transfer to the plant relative to the total process development lead time. The variable (TRANSFER) was constructed as follows: Elapsed months from project start to technology transfer TRANSFER = Total process development lead time (months) The lower this index, the more the organization is presumed to be relying on learning-by-doing. 4.3. Additional variables 4.3.1. Project complexity and content Because there is a fair degree of heterogeneity in the sample of projects, variables to control for these differences and their impact on development produc- tivity are included in the model. One important difference is whether the projects were biotech- nology- or chemical-based. A dummy variable (CHEMICALS) is included to indicate whether the project was chemical-based. As an additive dummy, this variable indicates whether or not there are differ- ences in the means of the productivity between the chemical and biotechnology projects. Differences in complexity should also influence the number of hours required. For chemical projects, a measure of complexity is the number of chemical reactions required to synthesize the molecule. This variable (CHEM × REACTIONS) was constructed by multiplying the chemical dummy (CHEMICALS) with the number of reactions required in the synthe- sis. To help constrain differences in complexity within the biotechnology sample, only biotechnology projects involving the development of recombinant DNA processes were included (e.g. monoclonal anti- bodies were excluded from the sample). 31 An additional control for project level differences in complexity included the scale of the process (mea- sured as the total output of the product in the first full year of commercial production). Because of the enormous range in the sample (from grams to hun- dreds of thousands of metric tons), this variable was measured logarithmically. The variable appears in the model as Log(SCALE). It was suggested in discussions with study participants that the model control for whether the drug was for an acute or chronic indication. 32 A dummy variable (ACUTE) was included in the model to indicate that the pro- cess was being developed for an acute treatment drug. 33 4.4. Descriptive statistics Table 3 presents means and standard deviations of the variables for both the full sample and the chemi- cal and biotechnology sub-samples. Regarding the strategy for technology transfer, significant differ- ences appear between chemical and biotechnology pharmaceuticals. In the average chemical project, the process technology is transferred to the commercial facility when the project is approximately 79% com- pleted (in calendar months) whereas the typical bio- technology process is transferred after 57.5% com- 31Preliminarystatisticalanalysisrevealedthat differenttypesof recombinantprocesstechnologies(e.g. bacterialcell versusmam- malian cell system) did not influence development hours in a significantway. 32Chronic diseases are those which require on-goingtreatment over long periods of time (e.g. high blood pressure). An acute condition is one which requires treatmentover a very short time (e.g. a bacterial infection).Whethera drug is beingdevelopedfor a chronic disease or an acute condition may influence process development efforts. Because they are administered for a rela- tively short period of time, acute therapies typically require less lengthy clinical trials to demonstrateboth safety and efficacy. A shorter expected clinical trial period means there is less time to complete processdevelopment.Spendingmore resourcesmay be viewed as necessaryto completethe work in the requiredtime. 33In preliminaryanalysis, other control variables pertaining to projectcontent and project strategywere included.Becausethese other variablesdid not improvethe statisticalquality or insightof the models, and since their inclusion did not impact the results, they were droppedfromfurtheranalysis.
  16. 16. 1112 G.P. Pisano/ Research Policy 25 (1996) 1097-1119 Table 3 Descriptive statistics means (standard deviation in parenthesis) Full sample Chemicals Biotechnology (n = 23) (n = 13) (n = 10) DEV.HOURS 193864 181965 209334 (197243) ( 177197) (229707) ACUTE 0.208 0.231 0.182 (0.415) (0.439) (0.405) Log(SCALE) 1.700 3.200 - 0.257 (2.120) (1.390) (0.981) TRANSFER 0.697 0.791 0.575 (0.238) (0.162) (0.272) pletion. This would suggest that biotechnology pro- jects, on average, are relying more on a development strategy of 'learning-by-doing' in the actual com- mercial production environment, whereas in chemi- cal projects more of the learning is occurring before process reaches the plant floor. One might argue that this is partially due to differences in the regulatory guidelines governing biotechnology and chemical-based pharmaceutical production. With biotechnology, food and drug ad- ministration (FDA) tends to place greater emphasis on manufacturing products for the final phase of clinical trials in commercial facilities. However, there are two factors that need to be kept in mind. First, differences in regulation influence the timing of technology transfer to the factory relative to the overall product development and regulatory filing schedule. The measure used in this analysis is the timing of technology transfer relative to the time horizon of the process development schedule. Sec- ond, there is substantial variation in the timing of technology transfer to the commercial manufacturing facilities within both the chemical and biotechnoiogy samples. In the biotechnology sample, for example, one project was transferred to the plant after only about 15% of the process development work had been completed, while there were three other pro- jects which were transferred after being approxi- mately 90% completed. The range in the chemical project sample spans from 48% to 96%. Differences within technological class of this magnitude suggest that development strategy choices, rather than strict regulatory requirements, are at work. Additional evidence that managerial choice is driving differences in the timing of technology trans- fer can be gleaned from analyzing projects involving identical or closely matched molecules in the sample. Within the current sample, three pairs of projects involved development of essentially the same molecule. Pair 'A' and Pair 'B' each involved pro- cess development for nearly identical biotech- nology-based drugs; Pair 'C' involved process devel- opment for two nearly identical synthetic chemical drugs. Assuming no significant observation errors, differences in the timing of technology transfer within identical pairings should be due largely to manage- rial choice, rather than regulatory factors or uncon- trolled differences in project complexity. Within each pair, there were relatively large differences in the percentage of the project completed prior to technol- ogy transfer to the plant. Within Pair 'A,' one project was transferred after 15% completion while the other was transferred after 60% completion. The corre- sponding figures for Pair 'B' are 25% and 57%, and for Pair 'C,' they are 66% and 88%. Although three pairs are too few for statistical analysis, they provide a useful 'reality check' against alternative explana- tions. 4.5. Regression analysis To analyze the impact of technology transfer strategy on development costs, and then to determine whether this impact differs between biotechnology and chemical process technology projects, the fol- lowing model was estimated using ordinary least squares: DEV.HOURSi = a 0 +/31CHEM i +/32CHEM x REACTIONS i + fl3ACUTEi + flaLog(Scale)i +/35TRANSFER i + fl6CHEM X TRANSFERi + ei The main coefficients of interest are /35 and /36. /35 is an estimate of the effect of the timing of technol- ogy transfer on development costs for the biotech- nology projects in the sample. /36, the coefficient on the interactive term, indicates the difference between the effect in biotechnology and the effect in chemi- cals. From the earlier discussion, /35 is expected to be positive, reflecting the need for learning-by-doing
  17. 17. G.P. Pisano/ Research Policy 25 (1996) 1097-1119 I 113 in the development of biotechnology processes. If, as hypothesized, the structure of knowledge in the chemical environment requires less learning-by-doing for effective performance, /36 should be negative. 4.6. Results Results of the regression analysis are shown in Table 4. Models 1-4 include only the variables controlling for differences in project content and complexity. Given the high degree of project hetero- geneity, the explanatory power of these content vari- ables was not surprising. However, after adjusting for differences in project content and complexity, significant development costs differential between the best and worst project in the sample was a factor of 2.70. Differences of approximately this magnitude have also been found in studies of development performance in other industries. One methodological concern with a small sample of heterogeneous pro- ject is the possibility of remaining (uncontrolled for) differences in content and complexity. This issue, of course, is not unique to this study. Given the com- plex nature of development projects, it is virtually impossible to identify all possible variables. Compar- ing development costs within the closely matched project pairs discussed earlier, however, suggests that the model has done a reasonably good job controlling for such differences. Because the projects within each pair are essentially identical, differences in project content and technical complexity are virtu- ally eliminated. The ratios of development hours consumed with each of the pairs were 5:1, 1.09:1, and 2.23:1 (thus, in the first pair, one project con- sumed five times as many resources as the other, nearly identical project). The average ratio across these pairs is 2.77, a figure very close to the 2.70 ratio estimated statistically across the entire sample. These data suggest that differences in project perfor- mance are not driven solely by content or complex- ity, and that project strategies and execution capabili- ties matter. Model 5 shows the effects of the technology transfer on total process development costs across the entire sample. The coefficient is both positive and statistically significant indicating that later trans- fers to the factory are associated with higher devel- opment costs. This result seems to confirm the idea that learning-by-doing in the plant is necessary for the efficient development of process technology. Table 4 Regression results process development hours (standard errors in parentheses) Variables Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Model 7 Constant 209.3 * * * 225.5 * * * 184.7 * * * 177.5 " * * 33.9 -65.32 -72.6 (56.7) (54.8) (43.9) (42.5) (80.7) (91.08) (86.3) CHEM - 334.7 * * - 503.6 ** * - 363.6 * * - 296.2 * " * - 429.2 ** * - 198.4 30.0 (142.4) (168.9) (137.0) (105.3) (128.7) (244.1) (192.5) CHEM × REACT 31.2 * * 30.2 * * 23.3 * * 23.1 * * 23.3 * * 24.7 * * " 24.7 * * * (12.2) (11.7) (9.3) (92.5) (8.5) (7.9) (7.7) Log(SCALE) 51.4 * 19.7 22.9 7.9 (30.4) (25.2) (23.1) (22.8) ACUTE 296.6 ** * 318.5 * * * 269.1 ** * 256.4 ** * 263.6 ** * (81.2) (75.5) (75.2) (70.1) (63.3) TRANSFER 268.4 * * 436.6 ** * 444.5 * * * (124.9) (145.3) (139.7) CHEM x TRANSFER -513.3 * * -545.1 * * (266.9) (244.1) Adj. R 2 0.17 0.24 0.54 0.55 0.62 0.67 0.68 F-value 3.30 " 3.36 * * 7.49 * * * 9.99 * * * 8.12 * * 8.46 * * * 10.67 * * * * P < 0.10, * * P<0.05, * * * P<0.01. t-tests for TRANSFER and CHEM X TRANSFER are for one .tail distributions. Coefficients and standard errors divided by 103 .
  18. 18. 1114 G.P. Pisano/ Research Policy 25 (1996) 1097-1119 300000 "1" 200000 ~. 100000 "~ o 2_ < - 100000 . , . . , , 0 oy</ o 0 o - 2 0 0 0 0 0 . . . . " , , " . . . . 2 3 4 .5 .6 7 8 9 I "liming of Technology Transfer Fig. 2. Relationship between development hours and technology transfer biotechnolgy sub-sample (n = 10). Models 6 and 7, however, provide additional insight about the impact of learning-by-doing on develop- ment costs across the two technological classes in the sample. In both models, the coefficient on TRANSFER (fls) is positive and statistically signifi- cant, indicating that a high degree of learning-by- doing in biotechnology is essential for development efficiency. As hypothesized, the coefficient on the interactive term (CHEM × TRANSFER) is negative (and statistically significant), indicating that learn- ing-by-doing is less important for development effi- ciency for chemical-based projects. 34 The negative coefficient on the interactive term indicates that development performance in chemicals does not rely as much on learning-by-doing as it does in biotechnology. It does not necessarily mean that learning-by-doing is unimportant in chemicals. The sum of r5 and r6 (which is the overall effect for the chemical sub-sample) is slightly negative in Models 6 and 7. In neither case, however, is the sum statistically significant. This indicates that a chemi- cal-based project that followed a strong learning-by- doing (i.e. early transfer to the plant) may not per- form any worse than one in which most of the development work occurred before the transfer. To gain further insights, the relationship between the technology transfer strategy and development hours is plotted separately for the chemical and biotech- nology sub-samples (Fig. 2 and Fig. 3). In these 34Given the relatively small sample size, an analysis was con- ducted to examine whether any particular outliers were driving the results. The models were re-estimated 23 times, each time with one observation excluded. In no case did the exclusion of a point lead to a significant change in the coefficients of primary interest. illustrations, development hours is adjusted for dif- ferences in project complexity. 35 The differences between these plots are quite striking. In the biotechnology sub-sample, the need for early technology transfer and learning-by-doing is quite clear. The regression for the biotechnology sub-sample is highly significant (F = 11.37, P < 0.01, R2 = 0.54). The pattem for the chemical-based project is quite different. There was no correlation between how early or late a process technology was transferred to the factory and development efficiency performance (F = 0.02, P = 0.89, R2 < 0.01). Here, the need for learning-by-doing is less clear, and there appears to be no systematic advantage of transferring the process to the plant at an early stage. In chemical synthesis, learning-by-doing is neither necessary nor sufficient for high development efficiency. More- over, although learning-before-doing is a viable problem-solving strategy in this arena, there appears to be room for a much greater diversity of ap- proaches. To provide additional insights into the role of knowledge and firm-specific competences in de- velopment performance in the chemical segment, a pair of contrasting examples is provided briefly be- low. 4.7. Case examples The lack of a systematic correlation between tim- ing of technology transfer and the development costs 32 Thus, the y-axis is the residual of the regression equation DEV.HOURSi = ot0 +/3~CHEMi +/32CHEM XREACTIONSi + fl3ACUTE i + fl4Log(Scale)+ ei.
  19. 19. G.P.Pisano/ ResearchPolicy25 (1996)1097-1119 1115 Chemical Projects (n=13) 200000 ~ . . . . . . . . . . . . . 0 150000 0 100000 "" 0 S0000 o ~- o ';' o~ o"~ -50000 0 ;> d~ -looooo 0 -150000" ;.~ -200000 " < -250000 • o -300000 " • , - , • , • , - , - , - , - , - ~ , I , .45 .5 .55 .6 .65 .7 .75 .8 .85 .9 .95 Thning of Technology Transfer Fig. 3. Relationshipbetweendevelopmenthours and technology transferchemicalsub-sample(n = 13). performance in the synthetic chemical segment sug- gests that there is room for much greater diversity of approaches in this context. Closer analysis of two contrasting cases suggests that the impact of the timing of technology transfer on development perfor- mance may be a function of the locus of capabilities within the organization. 4.7.1. Example one: the problems of early transfer Interestingly, the chemical project with the earli- est transfer to the plant had the highest development costs. While one outlier does not constitute a trend, it at least raises the question: can transfer to the plant be too early? In this case, it does appear that the lack of development prior to the transfer contributed to higher overall development costs. Although the com- pany's process research chemists became involved in the project at a very early stage of the project, and indeed helped to invent the initial route used to discover the molecule, very little subsequent work was undertaken to find alternative synthetic routes that might have been more suitable for commercial production. 36 After selecting the synthetic route, the process was then scaled-up in a research pilot plant 36Afteronly 3 monthsof additionalprocessresearch, a slightly modifiedversionof the initialsyntheticroutewas discoveredand selected for commercialproduction. This time period for route selection was extremelyshortin comparisonwith otherchemical projectsin the sample(the averagetimefor makinga commitment to a final synthetic route was 40 months, and eight of the 13 projects involvedthe developmentand pilot testing of multiple syntheticroutes). where it was used over the next year to produce batches of the drug for clinical trials. The research pilot plant was approximately half the scale of com- mercial production. Transfer of the process from the research pilot plant to the commercial manufacturing site began relatively early in the project for two reasons. First, the development organization on the production site had spare engineering capacity at the time, and thus wanted to get a head start on the project. Second, the process technology had extremely low yields (ap- proximately 1%) and would thus need to be further scaled-up in order to meet clinical supply require- ments. The team of chemical engineers responsible for the process at the plant reported that these prob- lems were largely unrelated to scale. Since the prod- uct was expected to be produced commercially in relatively small volumes, and commercial scale was only approximately two times the research pilot scale, scale-up was not a significant issue (in contrast, it was typical in the sample for commercial production to involve a ten-fold scale-up). In addition, the few scale-up problems that did occur were, according to the development group at the plant, fully expected. For example, one purification step used chromatog- raphy which was known from past experience to be difficult to scale-up. An alternative purification pro- cess was implemented for this step. Despite the lack of major scale-up issues, exten- sive work was required on each of the ten steps of the process during and after the transfer in order to increase yields, throughput times, and manufactura- bility. Many of these problems were rooted in the chemistry of the process itself and had been previ- ously identified by chemical process research labora- tories and during production. For example, two steps of the process had minor stability problems, another used a raw material that was not commercially avail- able, and another generated an intermediate that was difficult to purify. Although the plant had a very strong development organization that employed both chemists and chemical engineers, its experience had largely been in solving scale-up and process equip- ment issues. Since these problems were rooted in the chemistry of the process rather than its scale or equipment configuration, much of the development at this phase required extensive assistance from the process research group. Eventually, the major prob-
  20. 20. 1116 G.P. Pisano / Research Policy 25 (1996) 1097-1119 lems with each step were solved, and additional yield and throughput improvement were implemented in other steps of the process. Participants in the project hailed it as a case of successful collaboration between process research and manufacturing. The data indicate, however, that this collaboration may have come at a very high cost. The early transfer did not appear to surface any additional technical problems with the process. But it did cost the firm its ability to fully exploit the process research laboratory's comparative advantage in solving chemistry problems (which were known at the outset). In addition, by freezing the synthetic route very early on, the organization essentially com- mitted itself to a basic process architecture with known yield and throughput problems. 37 Extensive development resources then needed to be spent to optimize and trouble-shoot individual components of the process, when an alternative synthetic route might have avoided them from the outset. 4.7.2. Example 2: early transfer and the capability for learning-before-doing Counter to the predictions of the framework, the chemical project with the lowest development costs was among those with the earliest transfer to the plant in the chemical sub-sample. In terms of timing of technology transfer, this project's strategy was not all that different from the one described above. However, more detailed investigation revealed that at the time of transfer, the process was considerably more developed than in the example above. Further- more, this firm was somewhat unusual among the other pharmaceutical companies in the study in hav- ing strong process chemistry capabilities located at the manufacturing plant. This meant that even once the process was transferred to the plant, any further improvements of the basic process chemistry could be undertaken on-site and there was no need to send the process back to the process research labs for •rework.' As in the case described above, the company's process research chemists became involved early in the project and worked with discovery chemists in synthesizing the molecule. The initial 'discovery route' was used to make small quantities of the compound for animal tests. However, the process was complex and difficult to use even at laboratory scale, and thus it became clear almost immediately that an alternative route would be needed for com- mercial production. Thus, unlike the project de- scribed above, process research chemists continued searching for alternative synthetic routes that might be more suitable for commercial manufacturing. Drawing from experience with structurally similar molecules, and from published literature, the process research derived (over a period of approximately 10 months) an alternative route that looked, in theory, to have superior characteristics. A final commitment to this route, however, was not made until the route was tested extensively in the laboratory and its prac- ticality demonstrated in the pilot plant. Process re- searchers involved in the project noted that they continued to look at alternative processes as 'fall- backs' but after a few batches of pilot production, the organization was confident that it had an efficient process, with total yields approaching 87% (ap- proximately 99% per step). Over the next several months, the process was further tested and refined in the pilot plant where, according to a developer interviewed about this pro- ject, "95% of all problems with the process were identified." Once transferred to the commercial manufacturing facility, the plant's process chemistry group undertook further evaluations of the chemistry and explored alternative methods of synthesis, and several full-scale pilot tests were undertaken on the process. Although no better synthetic routes were discovered at this time, the plant's process chemistry group worked on ways of combining process steps (in order to improve throughput) and adapting the process to specific idiosyncrasies of the production plant and the environmental regulatory constraints under which it operated. 38 These were considered to be relatively minor process changes and did not 37 Data presented in Pisano (1996) reveals that changing syn- thetic routes, on average, lead to a 70% reduction in manufactur- ing costs, whereas optimization and improvement of any specific route lead to average cost reductions of 30%. 38For example, one step of the process needed to be modified so that a particular effluent would not be released into the air and another purification step was modified so that it would be more consistent with the equipment and procedures currently in place at the plant.
  21. 21. G.P. Pisano/ Research Policy 25 (1996) 1097-1119 1117 consume extensive resources. No major problems (either anticipated or unanticipated) were reported in scaling-up the process, transferring into production, or in achieving the desired levels of commercial performance. Two factors thus seem to distinguish this case and help to explain why early transfer did not result in higher development costs. First, the company's pro- cess development strategy focused on selecting high performing synthetic routes that would be feasible in commercial production, rather than trying to opti- mize (or fix) routes which were less than desirable. A solid process architecture laid the foundation for an earlier and smoother transfer to commercial pro- duction. Second, the plant's strong process chemistry capabilities meant that it could continue to develop, refine, and if necessary change the basic chemistry of the process even after transfer and start-up of production. 5. Conclusion The notion that manufacturing should be a seam- less extension of the R&D laboratory, a place where extensive development should occur, has gained cre- dence in recent years. The results presented show clearly that this prescription is likely to be beneficial in certain types of technological environments, but not in all. In environments like chemical pharmaceu- ticals which are characterized by deep theoretical knowledge and a significant accumulation of practi- cal experience, the role of the plant in development may not be critical because developers can anticipate and respond to manufacturing issues, without actu- ally doing their work in the plant. In emerging technologies, like biotechnology, which are typically characterized by less mature theoretical underpin- nings and less accumulated practical knowledge, the plant may be a critical venue for development. In these environments, it is simply impossible for de- velopers to anticipate and respond to manufacturing concerns without actually doing their work in the actual production environment. Thus, the role of the plant in development, and the locus of development competencies within organizations may be technol- ogy life-cycle dependent. The reader is cautioned not to misinterpret the results as implying that developers can ignore manu- facturing concerns or develop un-manufacturable processes with abandon. In all contexts, whether the technology is mature or immature, developers need to understand manufacturing concerns. The findings simply show that in certain environments, process developers may be better able to anticipate and respond to manufacturing issues, without actually performing their work in the plant. In such environ- ments, most of the learning that needs to take place can be done before the process is transferred to the plant. The results highlight some important implications for both development and manufacturing strategy. One relates to vertical integration and the role of in-house manufacturing. In recent years, there has been a tendency for firms, particularly young organi- zations in emerging technology industries, to utilize outside partners or contractors for manufacturing. This appears to be consistent with the idea of focus- ing on one's 'core competencies.' However, the results presented here suggest that such a manufac- turing strategy could be costly. It is precisely in these emerging technologies that the idiosyncrasies of pro- cess technologies are not fully understood, and re- quire development activities to be closely linked to the factory. A second manufacturing strategy issue concerns capacity and facilities planning. In an environment like biotechnology where learning-by-doing is abso- lutely essential to high development productivity, physical plant and equipment need to be installed long before required to support commercial produc- tion requirements and some excess commercial man- ufacturing capacity may need to be tolerated to support development in the factory. While such a strategy adds to both development costs and risks, it may also yield relatively high returns in an environ- ment like biotechnology. Using multi-purpose or flexible manufacturing plants may help to reduce the capital risks required for early technology transfer. In contrast, in contexts where late transfer to the plant is not costly, there is less risk of using product- specific equipment since the firms can wait longer to design and install it. 39 39 In the study, we found some evidence that flexible facilities entailed significantly higher capital and operating costs than plants designed to produce a specific product.
  22. 22. 1118 G.P. Pisano/ Research Policy 25 (1996) 1097-1119 An interesting issue for further exploration con- cerns how firms build their scientific knowledge bases about process technologies. Clearly, in some industries, publicly diffused scientific research plays an important role. However, as suggested in the case examples, the capability to learn before doing may be firm-specific. Even within biotechnology, there were differences in the extent to which individual firms were building scientific knowledge bases for future process development. An interesting dynamic may be at work. To have the capability to develop and simulate processes outside the factory, an R & D laboratory needs deep scientific knowledge about the technology as well as detailed knowledge about how the factory actually operates. Few production envi- ronments are stable. As factories operate, learning takes place and the environment changes. To main- tain the capability to simulate processes and to antic- ipate production needs, feedback from production back into R&D is critical. If this proposition is true, it suggests that the value of close contact between R&D and manufacturing is realized in terms of the longer-term enrichment of the firm's R&D knowl- edge base. Clearly, this issue merits further research as it has significant implications, not only for how organizational capabilities evolve, but also for the management of development projects and processes. References Abemathy, W.J., 1978, The Productivity Dilemma (The Johns Hopkins University Press, Baltimore, MD). Adler, P. and K. Clark, 1991, Behind the learning curve, Manage- ment Science 37(3), 267-281. Allen, T.J., 1966, Studies of the problem-solving process in engineering design, IEEE Transactions on Engineering Man- agement EM-13(2), 72-83. Arrow, K., 1962, The economic implications of learning by doing, Review of Economic Studies 29, 166-170. Bohn, R. and R. Jaikumar, 1992, The structure of technological knowledge in manufacturing, Working Paper (November). Clark, K. and T. Fujimoto, 1991, Product Development Perfor- mance: Strategy, Organization, and Management in the World Auto Industry (Harvard Business School Press, Boston). Clausing, D., 1993, Total Quality Development: A Step-by-Step Guide to World Class Concurrent Engineering (ASME Press, New York). Dosi, G. and L. Marengo, 1993, Some elements of an evolution- ary theory of organizational competencies, in: R.W. England (Editor), EvOlutionary .Concepts in Contemporary Economics (University of Michigan Press, Ann Arbor). Frischmuth, D.S. and T.J. Allen, 1969, A model for the descrip- tion and evaluation of technical problem solving, IEEE Trans- actions on Engineering Management EM-16(2), 58-64. Garcia, A.B. and J. Gocke, 1994, Virtual Prototyping: Concept to Production (Defense Systems Management College Press, Fort Belvoir). Haber, L., 1958, The Chemical Industry During the Nineteenth Century (Oxford University Press, Oxford). Hirsch, W., 1952, Manufacturing progress function, Review of Economics and Statistics 34, 143-155. lansiti, M. and K.B. Clark, 1994, Integration and dynamic capabil- ity: evidence from product development in automobiles and mainframe computers, Industrial and Corporate Change 3(3), 557-605. Kogut, B. and U. Zander, 1992, Knowledge of firm, combinative capabilities, and the replication of technology, Organization Science 3, 383-397. Leonard-Barton, D., 1992, The factory as a learning laboratory, Sloan Management Review 34(1), 23-38. Lieberman, M., 1984, The learning curve and pricing in the chemical processing industries, Rand Journal of Economics 15(2), 213-228. Marples, D., 1961, The decision of engineering design, IRE Transactions on Engineering Management 2, 55-71. McElvey, M., 1994, Evolutionary Innovation, Ph.D Thesis, Linkopings Universitet, Sweden. Nelson, R., 1982, The role of knowledge in R&D efficiency, Quarterly Journal of Economics (August), 453-470. Nelson, R. and S. Winter, 1977, In search of useful theory of innovation, Research Policy 5, 36-76. Newell, A. and H. Simon, 1972, Human Problem Solving (Pren- tice Hall, Englewood Cliffs, NS). Pavitt, K., 1984, Sectoral patterns of technical change: towards a taxonomy and a theory, Research Policy 13, 343-373. Pisano, G., 1994, Knowledge, integration, and the locus of learn- ing: an empirical analysis of process development, Strategic Management Journal 15(winter), 85-100. Pisano, G., 1996, The Development Factory: Unlocking the Poten- tial of Process Innovation (Harvard Business School Press, Boston). Polanyi, M., 1962, Personal Knowledge: Toward a Post-Critical Philosophy (Harper Torchbooks, New York). Rosenberg, N., 1982, Inside the Black Box: Technology and Economics (Cambridge University Press, New York). Simon, H.A., 1978, Rationality as process and as product of thought, American Economic Review 69(4), 1-16. Stobaugh, R. and P. Townsend, 1975, Price forecasting and strategic planning: the case of petrochemicals, Journal of Marketing Research 12, 19-29. Teece, D.J., 1976, The Multinational Corporation and the Re- source Cost of International Technology Transfer (Ballinger, Cambridge, MA). Ulrich, K. and S. Eppinger, 1995, Product Design and Develop- ment (McGraw-Hill, New York).
  23. 23. G.P. Pisano/ Research Policy 25 (1996) 1097-1119 1119 Utterback, J.M., 1994, Mastering the Dynamics of Innovations (Harvard Business School Press, Boston, MA). yon Hippel, E., 1994, The impact of 'sticky date' on innovation and problem solving, Management Science 40(4), 429-439. yon Hippel, E. and M. Tyre, 1995, How the 'learning by doing' is done: problem identification in novel process equipment, Re- search Policy 24(1), 1-12. Wheelwright, S. and K. Clark, 1992, Revolutionizing Product Development (Free Press, New York). Whitney, D., 1988, Manufacturing by design, Harvard Business Review 66, 83-91. Winter, S.G., 1987, Knowledge and competence as strategic as- sets, in: D.J. Teece (Editors), The Competitive Challenge (Ballinger Publishing Company, Cambridge, MA). Wright, T.P., 1936, Factors affecting the cost of airplanes, Journal of Aeronautical Science 3, 122-128.