The document proposes updated definitions for technology, manufacturing, and services readiness levels based on lean product development principles. It argues the current definitions promote a flawed "build-test-fix" approach and presents alternative "Lean TRL", "Lean MRL", and "SRL" definitions grounded in robust design, design for six sigma, and lean principles. The updated levels aim to characterize and validate performance earlier to reduce costly late iterations compared to the conventional approach.

1. 1 Copyright © 20xx by ASME
Proceedings of the ASME 2009 International Design Engineering Technical Conferences &
Design Theory and Design
IDETC/DTM 2009
August 30 – September 2, 2009, San Diego, California USA
DETC2009-86740
MULTIFUNCTIONAL ENTERPRISE READINESS: BEYOND THE POLICY OF BUILD-TEST-FIX CYCLIC REWORK
Victor Tang
Massachusetts Institute of Technology
Department of Mechanical Engineering
77 Massachusetts Avenue
Cambridge, Massachusetts, USA
phone 914-769-4040
victang@alum.mit.edu
Kevin N. Otto
Robust Systems and Strategy LLC
President
19 Edgewater Lane
Taunton, Massachusetts, USA 20780
phone 877-875-5087, fax 877-875-5087
otto@robuststrategy.com
ABSTRACT
NASA, the US Government and many companies attempt to manage the development and launch of new technology using Technology Readiness Levels, TRLs. Unfortunately, TRLs as generally defined are outdated and flawed, based on the extent of prototype or hardware use in the field. Urgency improving TRL levels drives early release of hardware before it is ready, and initiates cyclic rounds of debugging fixing failures in the field or laboratory. Such a build-test-fix approach to product development is now well documented to be inefficient and wasteful. We present updated definitions of technology readiness levels (TRLs) based on the lean and design-for-six- sigma product design methodology, a radical departure from the “build-test-fix” methodology of conventional TRLs. We argue that the iterative build-test-fix approach of cyclic rework is costly to product development, as well as, downstream manufacturing and services. We call our updated TRL the L- TRL, for Lean TRL. Consistent with our L-TRL, we also present updated definitions for Manufacturing Readiness Levels (MRLs) to address lean and six-sigma manufacturing principles. Hence we call them L-MRL. We address a void in the literature and unveil definitions for service readiness levels (SRLs).
1. INTRODUCTION
“Readiness” is a powerful general concept about an enterprise’s capability to successfully mobilize its resources execute specific functional processes; such as, technology evaluation in product development, manufacturing ability to meet demand with quality products, and services’ ability to serve customers , maintain quality, costs, and customer satisfaction. “Readiness Levels” are standard definitions of readiness for common understanding, providing a lingua franca for different disciplines to readily understand the readiness of different components, subsystems, products and entire systems. The purpose of readiness levels is sound – when a system is deemed readiness level 6, everyone can understand it is new and untried, but nonetheless ready for launch into the field. This communication is possible without having to describe underlying details or issues that may be foreign to others. Unfortunately, we find that the commonly used definitions of readiness promote poor practices, and run counter to well proven six sigma and lean practices. In this paper, we discuss common readiness level definitions, explore their flaws, and offer improved alternatives for different disciplines.
1.1 TECHNOLOGY READINESS
NASA introduced a document defining Technology Readiness Levels (TRL) as an assessment instrument to determine the extent to which the behavior and performance of a technology are sufficiently understood and characterized for use in a product system [1]. The goal is to help product development organizations adopt new technology with fewer unpleasant surprises, with more confidence, and less risk [2]. For example, of 50 major weapons systems, the Government Accountability Office (GAO) found that only 15% began development with demonstrably mature technology, and in those cases development costs increased only 1% versus 41% for those with technologies that were not ready [3]. The Department of Defense (DOD) now requires the use of TRLs for all its major acquisition programs [4, 5]. Many product development groups

2. 2 Copyright © 20xx by ASME
have integrated TRL into their product development processes for risky products with good results [6]. Experience with TRLs has resulted in improvements, refinements, and applications to new technical domains [2], 6-8].
However, important issues remain. For example, the TRLs do not fully address: the difficulties of subsystems integration, the interactions among subsystems in a complex product-system, the uncertainty of technical difficulties in the progression to maturation, and comparative analyses techniques for alternative technologies [9]. To fill this gap, Sauser proposes methods for System Readiness Levels (SRL) and Integration Readiness Levels (IRL) [10]. Majumbar [11] proposes ways for TRLs to address interoperability of different systems within a system-of-systems (SOS).
But, upon examining the definitions used in TRLs, we find that the TRL work is outdated. It predicated on an outmoded paradigm of build-test-fix. That is, the iterative nature of design has been historically assumed as a natural and acceptable feature of design methodology, e.g.
 “Teams make better decisions when they several iterations based on approximate information” [12] .
 “Product generation and evaluation are synergistic; they form an iterative loop” [13].
We now know that an ideal design process is rather one where all costly late iterations are eliminated [14, 15]. This is only possible by spending much more resources on early concepts, system engineering, critical requirement definition and characterization over a wide space of alternatives. Simply stated, characterize many alternatives, do not iterate only one alternative. Spending resources early far and away saves over spending larger resources later on build-test-fix. In practice, the means to do this are the methods of robust design, design-for- six-sigma and lean product development. Engineering results are proven more consistent, and potentially less costly, with these newer and proven engineering methods [16-19].
The reason premature iterations in cyclic design approaches are inherently risky and costly in time and dollar resources is the commonly understood heuristic on 80-20 rule of design decisions, 80% the decisions are made in first 20% of the resource expenditure in a traditional approach to design. In this paradigm, consider the cost of making a design change. This is well documented [20] as in Table 1. The message of Table 1 is clear; a better design process is one that reduces late phase iterations. Build-test-fix methods build in escalating future costs as the design freezes. The systems become more expensive to build and assemble, difficult and time consuming to debug determine root causes, more time consuming and expensive to implement any fix.
Table 1: The Cost of Making a Design Change Cost (Hamada1996), [21]. TRL $35 during the design phase 5 $177 before procurement 6 $368 before production 7 $17,000 before shipment 8 $690,000 on customer site 9
Given these observations, it is no surprise that studies reveal substantial benefits when using lean design for six-sigma methods [21]. MARKETWIRE reported “hard” (tied to financial statements) and “soft” benefits over standard practice are, respectively, for each project undertaken:
 $500K and 200K for companies with greater than 1 B in annual revenues,
 $200K and 100K mean for all companies, and $10K to $300K for the smallest companies.
A project is typically a 6-9 month design effort by a single engineer. The savings are real, instantiated by a paradigm shift in how to manage and execute product development.
Rather than working to build full system prototype hardware to learn by “seeing how it works”, lean DFSS is instead about first characterizing subsystems and components (not full systems), and through equations characterizing the system requirement targets as scalable equations over these subsystem and component changes. This approach greatly speeds the lengthy downstream design and manufacturing verification phases. That is, in the early phases when product requirements are defined, it is impossible to foresee exactly what the target should be for every design requirement, to provide a robust and reliable product. Traditional practices are to build a prototype, test it for gaps against requirements, and then fix it. Alpha, beta, and pre-production and prototypes are typically used. Often, these even codified into standard work, which institutionalizes this poor behavior. Lean DFSS is different. Instead, early in development all critical requirements are characterized as equations – design variables are changed in large sets of early subsystem and component prototypes and the requirements are measured to fit wide domain equations. The immediate impact is that there are much less problems during integration, since the characterization work of subsystems and components easily discovers resolves many issues early, with the need for root cause analysis on a full system alpha prototype that works inconsistently. Equally important, later in the development process during verification activities when the unknown- unknowns arise such as supplier problems, tooling or user environment problems, the previously developed equations can be used to very quickly fix these problems.
The benefits of lean six-sigma product development are also visible in the work of scholars [22-24] where design for
six-sigma, robust design, and lean principles are the core concepts. Dramatic results of lean PD are reported [22, 23]:

3. 3 Copyright © 20xx by ASME
 70% reduction in product development cycle time. 80% reduction in design hours.
 33% reduction in prototype development. 50% reduction inspections.
 25% time reduction between engineering and manufacturing.
 50% reduction in product cost. 90% conformance of design- to-cost targets.
 5-sigma design quality level.
We will present TRL definitions that are predicated on the robust design paradigm. We call this the Lean TRL, L-TRL.
1.2 MANUFACTURING READINESS
Manufacturing readiness gate reviews in industry are also not new, and have long been a standard industry practice [25-28]. However, codifying the manufacturing readiness review concept into a manufacturing readiness levels (MRL) specification is more recent trend. DOD and the defense community developed definitions for Manufacturing Readiness Levels (MRLs) that are consistent extensions of the conventional TRLs [29, 30]. Scholars have also taken an interest in MRLs [31, 32]. Unfortunately, the MRLs are again not only based on outdated policy of build-test-fix, but they also do not fully address the principles of Lean Manufacturing, which are widely researched and practiced [33]. Womack et al [33] report dramatic results of lean manufacturing implementation:
 50% reduction in human effort.
 50% less space.
 50% less investment in tools.
 50% less engineering hours, and time to develop products.
Motivated by these results, we also propose updated MRL definitions that reflect lean manufacturing principles. We will call this the Lean MRL or L-MRL.
1.3 SERVICES READINESS
OECD statistics reveal that services are a major contributor to national economic wealth. Similarly, the 2007 International Labor Organization reports that for the first time in history, employment in services (40%) exceeds agriculture 39.4%) and manufacturing (20.7%). Scholars note that economies are now undergoing a “servicization” transformation:. physical products and services are increasingly bundled as an integral offering to meet customer needs [34]. We consider that Services Readiness Levels (SRLs) definitions are a new and important subject.
Services readiness thinking is emerging. Heslop et al. [35] consider market readiness, commercial and management readiness all in conjunction with technology readiness. But their perspective is that of a senior business executive. Fine-grained texture and nuances of technology readiness are submerged in executive level abstractions. As such, the work is only indirectly useful to engineering or product development. Marketing scholars investigate technology readiness from psychometric measures of technology users [36- 38]. But their focus is on the demand side of technology, while useful to understand, it is only indirect applicability to engineering. Similarly, Lin and Hsieh [39] analyze the demand side implications of technology readiness. Significantly, there is a substantial asymmetry in the rigor of service’s conceptual foundations relative to technology and manufacturing. To address this deficit, IBM has articulated an overarching multi-disciplinary descriptive framework called Services, Science, Management, and Engineering (SSME) [40].
Unexpectedly, unlike technology and manufacturing, SSME is silent and the literature moot, as well, on a corpus of first-principles for services. Some scholars argue that without such first-principles, the service discipline will not attain the rigor that science, engineering, or operations management can bring to it [41, 42]. To address this gap, Tang and Zhou [44] have systematically derived and defined a set of first-principles for services, as well strong epistemic rules to test them. Given there are no service readiness levels (SRLs), we will call our SRL definitions simply as SRL.
1.4 STATE-OF-THE-ART AND NEW DIRECTIONS
Table 2 summarizes our multi-functional readiness discussion.
Table 2. Conceptual Foundations of readiness models
Technology readiness
Manufacturing readiness
Services readiness
Conventional
conceptual
foundations
build-test-fix
mass production and its variants:
people are interchangeable
customer satisfaction, consumer- product bias
Readiness models
TRL
MRL
none
New conceptual foundations
Robust design, Design for six-sigma, Lean product development.
Lean manufacturing, DMAIC six sigma
Services’ first- principles
new readiness models
L-TRL, i.e. Lean TRL
L-MRL, i.e. Lean MRL
SRL
The remainder of this paper is organized as follows. We begin with the definitions of conventional TRL as basis for discussion of the problems build-test-fix methodology of technology and product development. We then propose an updated TRL based on the new methods of robustness, six- sigma, and lean. We follow with a similar review of the MRL literature and, as with TRLs, point out that the predicate assumption of the MRLs on outdated model build-test-fix is not ideal as a base for MRLs. We turn our attention downstream to the services function with a literature review on service readiness. We note that the literature is dominated by the demand side of services, focusing on satisfaction and

4. 4 Copyright © 20xx by ASME
traditional concerns of services. The services development readiness perspective is barely visible in the literature. Following this review, we present our proposed definitions for updated TRLs, MRLs, and unveil a SRLs definitions; L-TRL, L-MRL, and SRL, respectively.
2. PRODUCT DEVELOPMENT AND TECHNOLOGY READINESS LEVELS
2.1 THE BUILD-TEST-FIX TECHNOLOGY READINESS LEVEL SYSTEM
We first examine the conventional TRL model, in Table 3. We concentrate on TRL 5 through 8 and present what the specifications require as supporting evidence. They are in effect exit criteria for a specific level [30]. L-TRL 1 is again the base line of no statistical proof the idea will work. L-TRL2 and L- TRL3 goals are largely predicated on robustness, six-sigma, and lean PD principles – not only should nominal response be characterized, but also noise factors that cause variability. L- TRL 4 is also now much more demanding. In addition to requiring that the elemental pieces put together will work, our L-TRL requires a deeper understanding of how the product- system will behave, to permit very rapid adjustment in downstream activities. The behavior must be represented not just by causal trees common to requirements management, but rather by complete transfer functions equations. The importance of this augmented set tasks cannot be understated.
Close examination at the supporting information required in TRL5 through TRL8 reveals that the specifications are grounded on repeated iterations of build-test-fix cycles. The wording is almost identical at each level. This process incents development teams toward building prematurely in order to proceed with ad-hoc testing to discover problems and then fix them. In section 1.1 and 1.3 of this paper, we presented evidence that lean PD and robust is superior to the policy of build-test-fix iterations. The classification is a good idea, the definitions are not.
2.2. IMPROVED LEAN TRL DEFINITIONS
We propose an updated TRL based on a policy grounded the robust six-sigma and lean PD principles. The definitions of our L-TRL are shown in Table 4. Instead of build-test-fix cycles, our strategy is to “characterize-validate-control” the performance and its variability to demonstrate robustness under progressively more demanding environments, from the laboratory, to relevant, representative, and customer operational under progressively more demanding environments, from the laboratory, to relevant, representative, and customer operational environments. The process proceeds with successively more mature embodiments in which the technology will operate, from subsystem, to prototype systems, increasingly valid systems.
Table 3. Traditional Build-test-fix TRL definitions
TRL 1
Basic principles observed and reported.
TRL 2
Technology concept and/or application formulated
TRL 3
Analytical and experimental critical function and/or characteristic proof of concept
TRL 4
Component and/or breadboard validation in laboratory environment.
TRL 5
Component and/or breadboard validation in relevant environment. Analyses and explanation of differences from predictions. Identify problems encountered and refinements made to match expected goals.
TRL 6
System/subsystem model or prototype demonstration in a relevant environment. Analyses and explanation of differences in the testing and operational environment in the differences results from predictions. Identify problems encountered and refinements made to match expected goals. Discuss actions to move next level of readiness
TRL 7
System prototype demonstration in an operational environment. Analyses and explanation of the differences in results from predictions. Identify problems encountered and refinements made to match expected goals. Discuss actions to move next level of readiness
TRL 8
Actual system completed and qualified through test demonstration. Results of testing final configuration under expected range of environmental conditions. Analyses and results that system will meet operational requirements. Identify problems encountered and refinements made to match expected goals. Discuss actions to remove problems and move finalizing design.
TRL 9
Actual system proven through successful mission operations.
L-TRL5 requires robustness work of the components. It also requires complete characterization of the noise conditions using the noise factors and predictions of their impact on the transfer function. L-TRL6 departs from the build-test-fix paradigm by intent and policy. Scalable transfer function equations must be developed and validated relating all input design variables noise variables with component-subsystem-system hierarchical output responnse variables that represent Performance is adjusted in downstream activities simply by using a subset of the available design variables (tuning variables) [44], not by iterative redesign. Unlike the conventional TRL7 where testing continues to uncover more problems fix or initiate more redesign, L-TRL7 is ready for commercialization. At L- TRL8 the technology is ready for manufacturing release, a non- event in our model, whereas traditional TRL8, the build-test- fix iterations continue. By L-TRL9 the product is used by the customer in their environment.
4. LEAN MANUFACTURING MATURITY READINESS (L-MRL)
Successful release to a manufacturing facility in order to begin production of new products is a critical and costly decision. Premature release of a facility before all problems are

5. 5 Copyright © 20xx by ASME
Table 4 Lean TRL Definitions
L-TRL 1
Basic principles observed and reported. Equations are ved describing the technology physics.
L-TRL 2
Technology concept and/or application formulated. Noise factors identified. Control Measurement response identified.
L-TRL 3
Technology performance behavior characterized. Range of noise factors identified. control factors identified. Measurement response identified. Measurement system GRR baselined. Proof of concept completed.
L-TRL 4
Technology Nominal Performance validated. Integration of basic technological components to establish they work together and produce the range of performance targets necessary. Integration uses “ad hoc” hardware in the laboratory. Transfer function equation predicts a validated nominal response. Measurement system GRR complete and capable.
L-TRL 5
Technology Performance Variability validated. Integration of basic technological components with reasonably realistic supporting elements to test technology in a simulated environment. Robustness work on the technology components is complete. The sum of squares response variation impact each noise factor varying is predicted in a validated transfer function equation.
L-TRL 6
Supersystem/system/subsystem interactions in relevant environment are demonstrated. Test representative prototype system in a stress test laboratory or simulated environment. Develop and validate scalable transfer function equations for the entire product as a system with the new technology. Equations include prediction of sum-of-squares performance variation and degradation for the entire product with applied off- nominal variation of the noise factors.
L-TRL 7
Product System Demonstrated Robust in representative environment. Technology prototype transferred to a product commercialization group, and they scaled it to fit within their real product application as an operational system. Demonstration of an actual full- product prototype in the field using new technology. Transfer function equations for the particular product system instantiation are completely verified. A limited set of remaining control factors are available to adjust the technology within product against unknown-unknowns. Technology is as robust as any other re-used module in the product.
L-TRL 8
Product Ready for Commercialization and Release to Manufacturing Full Production. Technology has been proven robust across the noise variations of extreme field conditions using hardware built with the production equipment purposefully set up and operated at their upper and lower control limits. Transfer to manufacturing is a non-event to the development staff if L-MRL processes are in place.
L-TRL 9
Experienced Customer Use. Product in use by the customer’s operational environment. This is the end of the last validation aspects of true system development. The performance of the product and technology perform to customer satisfaction in spite of uncontrollable perturbations in the system environment in which the product is embedded or other external perturbation. Transfer to the customer is a non-event to the engineering staff if L-TRL and L-MRL processes are in place.
worked out can lead to very expensive rework and poor quality products in the hands of customer. Scholars define manufacturing readiness, MR: “MR is designated as the ability to harness the manufacturing, production, quality assurance, and industrial functions to achieve operational capability that satisfies the product needs in quality and quantity needed at the best value as measured by product” [32]..
Similar to TRLs, manufacturing readiness levels have been defined to provide a means readily understand the uncertainty and readiness of new manufacturing production technology. Similar to TRLs, however, we consider conventional MRLs outdated because they are again grounded on the build-test-fix cyclic process of rework. MRLs need to be updated on more recent work and best practices. Six sigma, lean manufacturing and the best practices of Toyota Production System (TPS) are the foundations that we will use to base our work for updated definitions MRLs [33], [45, 46].. The core ideas of six sigma, lean manufacturing and TPS are to systematically identify and eliminate waste, reduce variability through elimination of defects, and to improve continuously. The practice demands a strategy of relentless waste elimination, defect reduction and customer pull wherein all processes are guided by customer demand and high quality.
We propose a Lean Six-Sigma Manufacturing Readiness Level Definition (L-MRL) in Table 5 (next page).
5. A SERVICES READINESS LEVELS MODEL (SRL)
Similar to technology product-development and manufacturing, new service offers also need taxonomy of readiness, for the same reasons of allowing communication the readiness. Our SRL takes Tang and Zhou’s [43] first- principles as a base. The interface between manufacturing and services as an important subject for research and firms for competitive advantage was first identified by Quinn [47]. Womack called for a lean services action plan [48]. As a response to these needs, we propose the SRL in Table 6. Our intent is for researchers and practitioners to refine, improve, build on it.
It is also our first-hand experiences in services for technology intensive products and systems that motivate us to propose SRL definitions. One of the co-authors held executive positions in engineering and services with IBM. He recalls his experience with the IBM AS/400 midrange server. In IBM, this product stands out as a technology and market success [49, 50].
The system was subjected to technology validation, system performance validation, and testing in customers’ operational environment that was unprecedented in IBM or by a competitor. Years in advance of product launch, 1800 systems

6. 6 Copyright © 20xx by ASME
Table 5. Lean MRL Definitions
L-MRL 1
Manufacturing pre-concept research. Research: new manufacturing concepts, technology, processes, materials, and potential investments. Scanning the manufacturing opportunity space. Manufacturing present at TRL processes. Value stream mapped.
L-MRL 2
Manufacturing concepts generated and down- selection done. Candidate concepts down-selected. implications on manufacturing feasibility and invention begin. Requirements on manufacturing technologies, materials analyzed. Cause-effect maps for all process steps done.
L-MRL 3
Manufacturing proof-of-concept characterized. Range of noise factors identified. Range of key process parameters (KPP) identified. Measured key quality characteristics (KQC) response identified. Baseline Measurement system GRR done. All conducted with experimental hardware in limited and not-integrated but highly controlled environments. Cost modeling begins. Proof-of- concept completed.
L-MRL 4
Manufacturing proof-of-concept Nominal Performance validated. Key value stream loops demonstrated as predicted by transfer function equations in lab environment. Nominal production capability demonstrated on KQCs, and key process parameter changes validate transfer function predictions. Takt times calculated. Investments case and cost models studies initiated. Supply chain studies and analyses initiated. Key constraints risks documented. Process FMEAs identify risks.
L-MR L 5
Critical process variability and Takt time. Manufacturing technology and cell work initiated. Process KQC sum-of-squares variability and the driving KPP are characterized. Investments and cost models completed. Initial sources of waste identified, analyzed qualitatively. Key value stream loops nominal performance validated.
L-MRL 6
Prototype manufacturing system performance variability in relevant environment. Majority of value stream defined and performance characterized with entire production line prototypes. Technologies’ producibility demonstrated. KPPs including costs, variability under noise characterized. Materials and tools proven. Personnel training requirements done. Supply chain infrastructure implementation initiated. Economic impact of waste known and programs initiated. Process FMEA risks reduced. Control plans available.
L-MRL 7
Pilot Line robustness demonstrated, ramp-up initiated. Hand-off from engineering is a non-event. Manufacturability and producibility in pilot line demonstrated. Yield performance and variability characterized and validated against the pilot production. In pilot, supply chain, materials, technology, tools, supermarkets, and personnel perform with no major surprises. Risk management procedures work as planned. Unit costs learning curve on track.
Yield variability is L-MRL 8
Ramp-up capability demonstrated, full-rate production initiated. All subsystems and systems are stable and meeting > 4-sigma quality levels on all
key parameters. Waste elimination/mitigation actions of previous L- MRLs confirmed, extend to this L-MRL and focus on overproduction wastes. Control plans complete.
L-MRL 9
At full-rate production. Manufacturing processes operated and controlled at greater than 4-sigma levels on all key process variables. Quality, costs and learning curves on track. Continuous quality improvement and lean waste elimination actions on track. Control plans validated.
were shipped worldwide for validation and use, in customers’ real operational environment, 70 millions lines of customer application code were tested, and over 200,000 programs procedures were validated [51]. During first customer installations, the AS/400 was at L-TRL9 and L-MRL9. In the year 2000, the AS/400 won Malcolm Baldrige US National Quality Award [50].
He also recalls another defining services-experiences, the 1996 Atlanta Olympic Summer Games and his first-hand role in the 1998 Nagano Olympic Winter Games. The Atlanta Olympics Games are a blot in IBM’s record on IT technology [52, 53]. In its eagerness to showcase technology, IBM used many products and systems that were only L-TRL6 during the Atlanta games. The entire Olympic IT system was a system of multivendor systems (SOS). The SOS was at L-TRL4. The SOS and business processes applications were never subjected to the rigor of end-end (value stream) operations in an Olympic competition environment. As a result, during the Games, services organization had to perform heroic acts to keep the systems running. And they had not been well trained.
However, for the 1998 Nagano Olympic Games, F.. Carrad, Director General of the IOC, declared, “Technology did win Gold in Nagano”. [54] What was different between the Atlanta Games and the Nagano Games? One, only products, which had a track record supported by demanding customer testimonials, were deployed for the Winter Games. All products had to be at L-TRL9 and L-MRL9. Moreover, the services organizations had to be fully trained and services infrastructure in place for all these products, i.e. at SRL9. Two, the SOS underwent comprehensive validation of user operations, systems interoperability, system performance, and business processes performance were all systematically conducted in end-end (entire value stream) customer operational environments. Since, there is only one Olympic Games, customer operational environment was provided by World Cup, Olympic qualifying competitions around the world. The scope of this effort was extraordinary. The validation included 75 M lines of software code, 60,000 test cases, 5000 PC’s, over 3000 additional server s and networking equipment, about 85,000 pages in the IBM internet Nagano Olympics homepage. Although the Variability criteria of SRL was not as Table 6.

7. 7 Copyright © 20xx by ASME
Service Maturity Levels (SRL) definitions
SRL 1
Services pre-concept research and awareness of R-TRL and L-MRL.
SRL 2
Services concept identified with implications and refinements. Based on R-TRL2 and L-MRL2, a services’ a strategic direction is formulated. Many candidate service concepts generated. Concepts analyzed against key assumptions of R-TRL and L-MRL. “as is” value stream of mapping done.
SRL 3
Services concept characterized via “to-be” value stream strategy. Down-selection complete with strategic consistency with R-TRL and L-MRL. Strategy for physical infrastructure and potential reuse of elemental processes documented. Customer, market, competitive, and investments case studies analyzed.
SRL 4
Nominal specifications for key-value service processes complete. Key-value services specified and demonstrated with L-MRL4. Changes to key service inputs demonstrate expected changes to service outcomes. Process interactions with manufacturing and services’ physical infrastructure, materials and people demonstrated. Strategy for business practices, fair and competitive terms and conditions complete.
SRL 5
End-end services processes specified. Material, information, skills (labor) flows and dependencies are also specified. Requirements and dependencies on contractual terms and conditions finalized. Services KPPs are characterized and their nominal performance targets set. End-to-end variations in the KPPs demonstrate expected changes to outcomes.
SRL 6
Prototype service process nominal performance validated relevant environment and in the R-TRL6 and L-MRL6 environment using services’ physical infrastructure. Waste drivers and constraints identified.
SRL 7
Key services processes variability validated in representative environment and infrastructure in R-TRL7 and L-MRL7. Waste and constraints elimination/reduction programs initiated. KPPs measured and analyzed, they variations measured against noise conditions analyzed. Field personnel training begins for services ramp-up.
SRL 8
Services infrastructure complete and poised for large scale field deployment. Waste drivers from previous SRL level fixed and completed. Supply chain dependencies confirmed and scale-up initiated. Critical mass of field personnel trained and ready for customers. Services physical infrastructure is robust.
SRL 9
Field deployment to entire customer base. Service delivery responsive to customer demand. KPPs from the field monitored and improvement programs initiated. Kaizen programs for services initiated. Star-burst* work initiated and on-going.
* Star-burst” refers to a best practice in services. Quinn [55] first observed that extending radially from an existing core-competency (as in a star-burst), a service provider can create a variety of other services centered on that core-competency. The idea is that once a service offering is proven to be effective, it disaggregated into elemental offerings. Its is an effective strategy to expand a firm’s services portfolio.
complete, the SRL were not at a “gentleman’s SRL9”. These and other similarly forceful experiences have strengthened our conviction that SRLs are overdue. We show in Table 5 our proposed definitions.
6. CLOSING REMARKS
Readiness levels are a useful and effective means to communicate to management and partners without disciplinary expertise the maturity and risk of a new system they are unfamiliar with. This can be new product technology, production systems, or new services. Traditional readiness level definitions, however, need to be updated modern practices that give much better indications of risk and readiness.
Technology readiness levels in the early phases need to be based not on the level of use in field, but rather on how ready they are to be released the field. This means level of mathematical and statistical characterization the technology is needed, and an accounting of the breadth characterization over the domain of application and production variances. An L-TRL 6 technology is one that has not been implemented in any product yet, but due to the level of characterization, one can still be fully confident it very easily be made to work as well a system commonly used in the field (L-TRL 8) through simple adjustments that are already characterized and understood. There are no surprises. The same holds true whether for manufacturing readiness levels and new production equipment or a new line.
Service readiness levels remain a new concept to the field. We propose a readiness level taxonomy similar to technology and manufacturing readiness levels that we have found useful. We hope to spark improvements in deployment of new services, as well spark an interest in fundamentals of effectively providing services.
ACKNOWLEDGEMENT
Our colleague Joern Hoppmann gave us valuable insight and comments in the preparation of this paper. It is a pleasure to acknowledge his assistance.
REFERENCES
1. Mankins J.C. 1995. “Technology Readiness Levels: A White Paper”. http://ipao.larc.nasa.gov/Toolkit/TRL.pdf April.6. Accessed: 8 February 2009.
2. DuBos G.F., Salch J.H. and Braun R. 2008. “Technology Readiness Level, Schedule Risk, and Slippage in Spacecraft Design”. Journal of Spacecraft and Rockets, 45(4) July- August, pp. 836-842.

8. 8 Copyright © 20xx by ASME
3. DuPont, D.G. 2008. “Proactive Prototypes”. Scientific American, pp. 24-25.
4. Graettinger C.P., S. Garcia, J. Siviy, R.J. Schenk, and P.J. Van Syckle. 2002. “Using the Technology Readiness Levels Scale to Support Technology Management in the DODs ATD/STO Environments”. Army CEOCOM, September.
5. DOD 5000. 2002. “Mandatory Procedures for Major Defense Acquisition Programs (MDAPS) and Major Automated Information Systems (MAIS) Acquisition Programs”, April 5.
6. Minning C.P., P.I. Moynihan and J.F. Stocky. 2003. “Technology Readiness Levels for the New Millennium Program”, AeroSpace Conference 2003, Proceeding 2003 IEEE 1, pp. 417-426
7. Moorhouse D.J. 2001. “Detailed Definitions and Guidance for Application of Technology Readiness Levels”. Journal Aircraft 39(1), pp. 190-192.
8. Mackey R.,R. Some, A. Aljabri. 2008. “Readiness Levels for Spacecraft Information Technologies”. AeroSpace Conference 2003, Proceeding 2003 IEEE 1, pp. 391-398.
9. Sauser, B., J. Ramirez-Marquez, D. Henry, and D. DiMarzio. (2008). “A System Maturity Index for the Systems Engineering Life Cycle”. International Journal of Industrial and Systems Engineering, 3(6), pp. 673-691
10. Sauser, B.J. 2008. “Technology Integration Maturity Metrics: Development of an Integration Readiness Level”. .Submitted to the Int. J. of Technology Management.
11. Majumdar, W.S. 2007. “System of Systems Technology Readiness Assessment”. Masters Thesis. Naval Post Graduate School. Monterey, CA.
12. Ulrich K.T., and S. D. Eppinger. 2000. “Product Design and Development”. McGraw Hill. New York, pp. 202.
13. Ullman, D. 1995. “The Mechanical Design Process”. MacGraw Hill, New York, pp. 95.
14. Ward, A. 2007. Lean Product and Process Development, Lean Enterprises Institute, Cambridge MA
15. Kato, J. 2005. “Development of a Process for Continuous
16. Creation of Lean Value in Product Development Organizations”, MS Thesis, Massachusetts Institute of Technology, Cambridge, MA..
17. Creveling, C., Slutsky, J. and D. Antis. 2002. “Design for Six Sigma in Technology and Product Development”, Prentice Hall PTR, Upper Saddle River, New Jersey.
18. Taguchi, G., S. Chowdhury, Taguchi. 2000. “Robust Engineering”. McGraw Hill, N.Y.
19. Taguchi, G. 1993. “Taguchi on Robust Technology Development: Bringing Quality Engineering Upstream”. ASME Press, New York
20. Otto, K. N. and Wood. 2001. “Product Design”. Prentice Hall. Upper Saddle River, New Jersey.
21. Hamada, H. 1996. “Quality what it means”. European Community Quarterly Review. Third Quarter.
22. MARKET WIRE. 2005. “$300K Typical Gain from Design for Six Sigma Projects”. SEATTLE, WA. July 18.
23. Oppenheim, B.W. 2004. “Lean Product Development Flow”. Systems Engineering, 7(4), pp. 352-376.
24. Fiore, C. 2005. “Accelerated Product Development: Combining Lean and Six-Sigma for Peak Performance”. Productivity Press, N.Y.
25. Anand, G. and R. Kodali. 2008. “Development of a Conceptual Framework for Lean Product Development Process”.. Int. J. Product Development 6(2), pp. 190-224.
26. Difranza, M.J. 1986. “R&D Transition to High Volume Production Excellence”. 40th Annual Quality Congress, May. Anaheim, CA, 40(0), May; pp. 62-65.
27. Benzin, R.W. 1991. “Evaluating Production Readiness: Improving Returns on the Research and development Investment”. IEEE Proceedings Annual Reliability and Maintainability Symposium, pp. 4-9.
28. Tung, T., L.V. DeArmond, R.F. Herald, P.E. Herning, M.H. Kalisher, D.A. Olson, R.F. Risser, A. Stevens, and S.J. Tighe. 1992. “State of the art of Hg-melt LPE HgCdTe at Santa Barbara Research Center”, Proc. SPIE, 1735, 109 (1992), pp. 109-134. DOI:10.1117/12.138616
29. Ho, B. 1997. “Manufacturing readiness model for fab startup”. Semiconductor Manufacturing Conference Proceedings, IEEE International Symposium, 6-8 Oct; A25- A28.
30. Acquisition Community Connection (a). 2009. “Manufacturing Readiness Assessments”. Available: https://acc.dau.mil/CommunityBrowser.aspx?id=18231&lang=en-US. Accessed on 12 February 2009.
31. Acquisition Community Connection (b). 2009. “Manufacturing Readiness Assessments”. Available: https://acc.dau.mil/CommunityBrowser.aspx?id=18545&lang-en-US. Accessed on 12 February 2009.
32. Pretorius M.W. and D. de Wet. 2000. “A model for the assessment of new technology for the manufacturing enterprise”. Technovation 20(1); pp. 3-10.
33. Genaidy, A., W. Karwowski. 2008. “A roadmap for a methodology to assess, improve and sustain intra-and inter- enterprise system performance with respect to technology- product life cycle in small and medium manufacturers”. Human Factors in Manufacturing 18(1), pp. 70-84.
34. Womack J.P., Jones D. T. and Roos 1991. “The Machine that Changed the World: The Story of Lean Production”. Harper Perennial, N.Y.
35. Müller P. and Blessing L. “Development of Product- Service-Systems–Comparison of Product and Service Development Process Models”. 16th International Conference in Engineering Design (ICED) 2007, pp. 245- 246.
36. Heslop, A.L., E McGregor, and M. Griffith. 2001. “Development of a Technology Readiness Assessment Measure: The cloverleaf Model of Technology Transfer”. Journal of Technology Transfer 26(4), pp. 369-384.
37. Parasuraman, A. 2000. “Technology Readiness Index (TRI): A Multiple-Item Scale to Measure Readiness to Embrace

9. 9 Copyright © 20xx by ASME
New Technologies”. Journal of service Research. May 2(4), pp.307-320.
38. Parasuraman, A. and C.I. Colby. 2001. “Techno Ready Marketing: How and Why Your Customers Adopt Technology”. Free Press, N.Y.
39. Tsikritsis, N. 2004. “A Technology Readiness-Based Taxonomy of Customers: A Replication and Extension”. Journal of Service Research, Aug 7(1), pp. 42-52.
40. Lin C. and P. Hsieh. 2006. “The role of technology readiness in customers’ perception and adoption of self- service technologies”. Int. Journal of Service Industry Management 17(5), pp. 497-517.
41. Spohrer J. and Riecken D. 2006. “Services Science”. Communications of the ACM 49(7), Jul., pp. 31-34.
42. Paulson, L.D. 2006. “Services Science: A New Field for Today’s Economy”. IEEE Computer, August, pp. 18-21
43. Chesbrough H. 2004. “A failing grade for the innovation academy”. Financial Times, 24 September.
44. Tang, V. and R. Zhou. 2009. “First-Principles for Services and Product-Services-Systems: An R&D Agenda”, Submitted to the International Conference on Engineering Design, ICED’09, 24-27 August. Stanford University, Stanford, CA.
45. Otto K. and E. Antonsson. 1993. “Tuning Parameters in Engineering Design”. Journal of Mechanical Design, 115(1), pp. 14-19.
46. Breyfogle, F. 2003. “Implementing Six Sigma”, John Wiley & Sons; 2nd edition, New York.
47. Ohno, T. 1995. “Toyota Production System: Beyond Large- scale Production”. Productivity Press.
48. Quinn, J.B. 1990. “Technology in Services: Rethinking Strategic Focus”. Sloan Management Review; Winter 31(2), pp. 79-87.
49. Womack, J.P. 2004. “An Action Plan for Lean Services”. Lean Service Summit. Amsterdam, June.
50. Bauer, R.A., E. Collar, and V. Tang. 1992. “The Silverlake Project: Transformation at IBM”. Oxford University Press, New York.
51. “Malcolm Baldrige National Quality Award 1990 Winner. IBM Rochester - AS/400 Division”. Available at: http://www.quality.nist.gov/IBM_90.htm. Accessed 26 Feb. 2009.
52. Pine, B.J. “Design, test, and validation of the Application System/400 through early user involvement”. 1989. The IBM Systems Journal 28(3), pp. 376-385.
53. Djurdjevic R. 1996. “Showcase of American IT supremacy Turns into a 5-Ring Circus”. Available: http://www.truthinmedia.org/truthinmedia/Columns/Atlanta96.html. Accessed 26 Feb. 2009.
54. Schwartz, J. 1996. “Olympics Stung by Technology’s False Starts”. July 22. New York Times. Available: http://www.nytimes.com/specials/olympics/0722/oly- failure.html. Accessed 26 Feb. 2009.
55. 1998 Nagano Olympic Winter Games: A Model of e- business Success. Available:
http://www-7.ibm.com/services/nz/success/nagano2.html,
Accessed 26 Feb. 2009
56. Quinn, J.B. 1992. The Intelligent Enterprise. The Free Press, New York.. pp. 148.