MRL and SBIR Success! Readiness Levels - Technology - Manufacturing - Business - Customer TRL MRL BR CR Manufacturing Readiness Level Joe Graben, Director USM Business & Innovation Assistance Center May 2009 www.usm.edu/biac/ IT’S NOT JUST THE SBIR TECHNOLOGY !
Just because you have developed an SBIR technology through a phase II contract and there is a “customer” with a “need” doesn’t necessarily mean you can sell that technology as a part of system, sub-system, or sub-assembly and that the customer will readily insert your product into their operational use.
There are many variables that must be addressed before you can succeed – you need to assure all your “ducks are in a row”.
The ultimate “success” in the SBIR program is for the small business to be “commercializing” their SBIR technology by producing a product that is being used in either the military and/or civilian marketplaces.
The ultimate “success” in the SBIR program is for the small business to be “commercializing” their SBIR technology by producing a product that is being used in either the military or civilian marketplaces.
Basic SBIR Philosophy TRL MRL BR CR All four “readiness” levels are critical to successfully reaching transition to operational use.
Introduction to Manufacturing Readiness and Its Role in Successful SBIR Commercialization
What is the small business “business” plan?
Will the small business do the soup to nuts?
Will the small business produce the end product?
Will the small business license the technology (patents)?
Will the small business sell the technology?
Just how does the small business plan on getting their SBIR Technology designed and manufactured into a usable end product the Government/Commercial markets can use?
A General Accounting Office (GAO) assessment of 54 major weapon programs found that the majority of programs were costing more and taking longer to develop than planned. Why did this happen? The programs were going ahead with less knowledge at critical junctures than suggested by best practices. These critical junctures known as “Knowledge Points” include:
Lets look at these knowledge points .
For more information see the GAO Report: GAO-05-301 at: http://www.gao.gov/new.items/d06391.pdf
Knowledge Points and associated indicators are defined as follows and all basically look at the maturity levels at a critical juncture:
Technology is mature. This means that technologies need to meet essential product requirements and have been demonstrated to work in their intended environment. This requires a close matching of customer requirements and resources. A gap between industry best practices and actual technology maturity indicates risk.
Product design is stable. This means that the design is stable at the system-level critical design review (midway through development). Best practices should have 90 percent of the drawings at the system-level completed. A gap between industry best practices and actual design stability indicates risk.
Production processes are mature. This means that all key manufacturing processes are in statistical control (repeatable, sustainable and capable) at the start of production. A gap between industry best practices and actual production maturity indicates risk.
What were the specific problems the GAO found with the Knowledge Points on the 54 programs they reviewed?
Desired level of knowledge Attainment of Product Knowledge Desired level of knowledge Production, design & technology maturity Design & technology maturity Technology maturity Source: GAO Report GAO 05-301 Development Start DoD Design Review Production Decision Gap Indicates Risk Gap Indicates Risk Gap Indicates Risk
Eighty-five percent of the programs began development not having demonstrated all of their technologies as mature. There was a major gap between what they should have known at that point and what they knew. Going forward required a “leap of faith” that somehow a miracle would occur to solve these gaps.
More often than not, programs sought to mature technologies well into system development when they should have been focusing on maturing the system design and preparing for production. These programs moved forward before the technologies were mature, but the miracle failed to appear and it caused problems.
Program acquisition cost:
Rose an average of 21 percent for those programs that preceded with immature technologies.
Rose an average of only one percent for programs with mature technologies!
Only 42 percent of programs held design reviews after achieving design stability. The majority moved forward with unstable designs. There was a major gap between what they should have known at that point and what they knew. Going forward required a “leap of faith” that somehow a miracle will occur to solve these gaps. These programs moved forward before the design was mature, the miracle did not occur and it caused problems.
The mature programs experienced:
A 6 percent increase in development costs and a schedule increase of 11 months
Immature programs , those that did not achieve design stability by CDR experienced:
A 46 percent increase in cost and a schedule slip of 29 months
It should be noted that design stability cannot be attained if key technologies are not mature.
Successful programs use statistical process control (SPC) as a best practice to bring manufacturing processes under control. Therefore, these processes are stable, capable and repeatable. Of the 54 programs reviewed, only 19 programs were in production or approaching a production decision within the next year. Of the 19, only two programs collected or even planned to collect statistical process control data. There was a major gap between what they should have known at that point and what they knew. Going forward required a “leap of faith” that somehow a miracle will occur to solve these gaps. These programs moved forward to production before the manufacturing processes were mature, the miracle failed to appear and it caused problems.
Unfortunately the GAO only looked at this one manufacturing element (SPC) to judge maturity. The development of Manufacturing Readiness Levels includes the definition of nine different threads or maturity areas for evaluation.
Title 10, Subtitle A, Part IV, Chapter 148 – National Defense Technology and Industrial Base, Defense Reinvestment, and Defense Conversion.
“ The Secretary of Defense shall establish a Manufacturing Technology Program to further the national security objectives of section 2501(a) of this title through the development and application of advanced manufacturing technologies and processes that will reduce the acquisition and supportability costs of defense weapon systems and reduce manufacturing and repair cycle times across the life cycles of such systems.
Manufacturing Readiness Levels are also addressed in the Defense Acquisition Guidebook (DAG): “ Engineering and Manufacturing Readiness Levels are a means of communicating the degree to which a technology is producible, reliable, and affordable. Their use is consistent with efforts to include the consideration of engineering, manufacturing, and sustainment issues early in a program.
More information can be found in the Manager's Guide to Technology Transition in an Evolutionary Acquisition Environment . Application of EMRLs should be tightly integrated with the technical reviews detailed in Section 4.3 .
You can access the DAG at http://akss.dau.mil/dag/DoD5000.asp?view=document
Why is manufacturing readiness really important? Because sometimes we go to war, and for that we need systems, equipment and supplies. The quality, availability and performance of those systems, equipment and supplies is directly tied to the manufacturing capability.
What if we are asked to surge? Do we have the capability to ramp up in time to meet deployment requirements?
What if we are asked to meet a new challenge with improved performance? Can we develop and deploy the solution in a timely and cost effective manner?
And sometimes things just happen, for example,
The canopies you have been manufacturing for the last 10-years suddenly show up with problems because somewhere you lost the expertise.
The Original Equipment Manufacturer (OEM) just notified you that they were no longer going to produce a critical part and now you need alternative sources.
The only factory in the world that produces your critical part just had a fire and there goes your supply chain.
Manufacturing Readiness is tied directly to producibility or to the design. Producibility can be defined as “ the measure of the relative ease of manufacturing .” That is, “is it easy to make?”
Producibility is a design accomplishment resulting from a coordinated effort by design engineering and all the functional engineering specialties to create a functional design that optimizes the ease and economy of fabrication, assembly, inspection, test, and acceptance without sacrificing function, performance or quality.
One of the basic producibility principles is to focus on the simplicity of design. Simple designs are actually more elegant and take more effort to achieve than complex designs and include dictates such as:
Use economical materials.
Standardize materials and components.
Minimize parts count.
Eliminate or minimize special tooling and testing.
Lets look at an example of complex design made into a simpler design.
The item on the right is a Bailout Bottle Holder used in the F/A-18 Hornet fighter aircraft. The original design was more complex than it needed to be.
This design was simplified using a technique called Design for Manufacturing and Assembly (DFMA). This technique asks three questions:
During operation, does this part move relative to the part it is connected to ?
Does this part need to be made from a different material than the pare it is connected to?
Does this part need to be removed?
If you can answer no to each of these three questions, then that part is a candidate for re-design. You start by comparing Part No.1 with Part No. 2, then do the same for 1 to 3, 1 to 4, etc., until all combinations have been assessed.
What do you think the effect is of the simpler design on manufacturing and assembly of the design to the right?
MRL – Why is it Important? F/A-18 Bailout Bottle Holder
Manufacturing Readiness Levels DoD and the services and agencies have been conducting Manufacturing Assessments for many years, and many of the services and agencies have checklists and other tools to help you to conduct these assessments. These assessments can include the following:
Production Readiness Review (PRR)
Manufacturing Management Production Capability Review (MMPCR)
Quality Assurance Surveys
Best Manufacturing Practices Center of Excellence Technical Risk Identification and Mitigation System (TRIMS)
Production relevant environment – An environment that contains key elements of production realism not normally found in the laboratory environment (e.g. uses production personnel, materials or equipment or tooling, or process steps, or work instructions, etc.). May occur in a laboratory or model shop if key elements of production realism are added.
Production representative environment – An environment (probably on the shop floor) that contains most of the key elements (tooling, equipment, temperature, cleanliness, lighting, personnel skill levels, materials, work instructions, etc) that will be present in the shop floor production areas where low rate production will eventually take place.
Pilot line environment – A shop floor production area that incorporates all of the key elements (equipment, personnel skill levels, materials, components, work instructions, tooling, etc.) required to produce production configuration items, subsystems or systems that meet design requirements in low rate production. To the maximum extent practical, the pilot line should be representative of processes to be used in rate production.
Full Rate Production demonstrated and lean production practices in place Low Rate Production demonstrated. Capability in place to begin Full Rate Production Pilot line capability demonstrated. Ready to begin low rate production Capability to produce systems, subsystems or components in a production representative environment Capability to produce a prototype system or subsystem in a production relevant environment Capability to produce prototype components in a production relevant environment Capability to produce the technology in a laboratory environment Manufacturing concepts identified MRL 10 MRL 9 MRL 8 MRL 7 MRL 6 MRL 5 MRL 4 MRL 3 Pre-Concept Refinement Concept Refinement Technology Development System Development & Demonstration Production & Deployment Manufacturing Readiness Levels
MRL 3 MRL 3 – Manufacturing Concepts Identified Identification of current manufacturing concepts or producibility needs based on laboratory studies. Materials characterized for manufacturability. Assumed that all corresponding TRL requirements are met for each MRL. Pre Concept Refinement Phase
Key Exit Criteria
Potential manufacturing sources identified (Commercial/Government and Domestic/Foreign)
Evaluate materials and processes for manufacturability and producibility
Budget request to achieve MRL 4 submitted
Materials assessed for manufacturability
Identification of manufacturing concepts/producibility needs
Industrial Base capability identified for key technologies
Identify Industrial Base capabilities and gaps/risks for Key Technologies
Assess Design for Producibility
Assessment of manufacturing processes for production and O&S
Key Performance Parameters (KPPs) identified
Budget request to achieve MRL 5 submitted
Prior use determined
Risk Reduction Plans in place for new materials
Assess current state-of-the-art of proposed processes
Yield and rate assessments complete
Quality strategy identified
MRL 4 MRL 4 – Produce the Technology in a Laboratory Environment Required investments, such as manufacturing technology development identified. Processes to ensure manufacturability, producibility and quality are in place and are sufficient to produce technology demonstrators. Manufacturing risks identified for prototype build. Manufacturing cost drivers emerging. Producibility assessments of design concepts have been completed. Key Performance Parameters (KPP) identified. Special needs identified for tooling, facilities, material handling and skills. Concept Refinement (CR) Phase leading to a Milestone A decision
Industrial Base assessed for potential manufacturing sources
Initiate assessment of key technology/component producibility
Validate design choices against industrial base capabilities
Identification of enabling/critical technologies and components
Initiate evaluation of design Key Characteristics
Value Stream Map of cost model for all CTEs
Budget request to achieve MRL 6 submitted
Materials produced in a prototype environment
Supply chain sources identified
Process capability requirements identified
Yields and rates established for production and issues identified
Industrial Base capabilities/gaps identified
Special Tooling/Special Test Equipment identified
MRL 5 MRL 5 – Produce Prototype Components in a Production Relevant Environment Manufacturing strategy refined and integrated with Risk Management Plan. Identification of enabling/critical technologies and components is complete. Prototype materials, tooling and test equipment, as well as personnel skills have been demonstrated on components in a production relevant environment, but many manufacturing processes and procedures are still in development. Manufacturing technology development efforts initiated or ongoing. Producibility assessments of key technologies and components ongoing. Component Design to Cost (DTC) goals set. Early Technology Development (TD) Phase
Industrial Capability Assessment for Milestone B completed
Producibility assessments completed
All enabling/critical technologies/components tested and validated
Budget request to achieve MRL 7 submitted
Supply chains in place
Yields and rates evaluated in a production relevant environment
Quality system in place
Workforce skills available
Integrated Master Plan/Integrated Master Schedule developed
Prototype tooling concepts demonstrated in a production relevant environment
MRL 6 MRL 6 – Produce a Prototype System or Subsystem in a Production Relevant Environment Initial manufacturing approach developed. Majority of manufacturing processes have been defined and characterized, but there are still significant engineering/design changes. Preliminary design of critical components completed. Producibility assessments of key technologies complete. Prototype materials, tooling and test equipment, as well as personnel skills have been demonstrated on subsystems/systems in a production relevant environment. Production cost drivers/goals analyzed. Producibility considerations shape system development plans. System level DTC goals set. Long lead and key supply chain elements identified. Industrial Capabilities Assessment (ICA) for MS B completed. Later Technology Development (TD) Phase leading to a Milestone B decision
Product requirements support detailed systems design
Budget request to achieve MRL 8 submitted
Long lead procurement identified/planned for LRIP
Effective supply chain management
Yields and rates estimated in production representative environment
Quality targets established
Workforce/training requirements identified
Facilities for LRIP and FRP identified
MRL 7 MRL 7 – Produce Systems, Subsystems or Components in a Production Representative Environment Detailed design is underway. Material specifications are approved. Materials available to meet planned pilot line build schedule. Manufacturing processes and procedures demonstrated in a production representative environment. Detailed producibility trade studies and risk assessments underway. Cost reduction efforts underway, incentives in place. Supply chain and supplier QA assessed. Long lead procurement plans in place. Production tooling and test equipment design & development initiated. System Development and Demonstration (SDD) Phase leading to Design Readiness Review (DRR)
Industrial Capabilities Assessment completed for Milestone C
Producibility improvements implemented
Detailed design complete
Budget request to achieve MRL 9 submitted
Materials proven and validated during SDD
Long lead procurement initiated for LRIP
Supply chain stable to support LRIP
Yields and rates verified for LRIP
Quality targets demonstrated
Workforce trained and certified
Manufacturing plan completed
Tooling and test equipment proven on pilot line
MRL 8 MRL 8 – Pilot Line Capability Demonstrated and Ready to Begin Low Rate Initial Production Detailed system design essentially complete and sufficiently stable to enter Low Rate Initial Production (LRIP). All materials are available to meet planned LRIP schedule. Manufacturing and quality processes and procedures proven in a pilot line environment, under control and ready for LRIP. Known producibility risks pose no significant risk for LRIP. Program has budget estimate to reach MRL 9, including investments for Full Rate Production (FRP). Supply chain established and stable. ICA for MS C completed. System Development and Demonstration (SDD) Phase leading to a Milestone C decision
Producibility risks mitigated and do not threaten FRP
Three sigma or other appropriate quality levels
LRIP cost goals met
Budget estimate for Lean implementation during FRP
Long lead procurement initiated for FRP
Supply chain stable to support FRP
Yield and rate targets achieved
Workforce requirements for FRP identified
Manufacturing plan updated and validated for FRP
Tooling and test equipment proven during LRIP
MRL 9 MRL 9 – Low Rate Initial Production Demonstrated and Ready to Begin Full Rate Production Major system design features are stable and proven in test and evaluation. Materials are available to meet planned full rate production schedules. Manufacturing processes and procedures are established and controlled to three-sigma or some other appropriate quality level to meet design key characteristic tolerances in a Low Rate Initial Production environment. Production risk monitoring ongoing. LRIP cost goals met, learning curve validated. Production and Deployment Phase leading to a Full Rate Production (FRP) decision
Industrial capability supports modifications, upgrades, surge, and other potential requirements
Ongoing producibility improvements
Product design stable
FRP cost goals met
Budget sufficient for production at required rates and schedule
Supply chain proven
Six sigma or other appropriate quality levels
Yield and rate targets achieved
Workforce skill sets maintained
Manufacturing risks mitigated
Proven tooling and testing equipment in place
MRL 10! MRL 10 – Full Rate Production Demonstrated and Lean Production Practices in Place This is the highest level of production readiness. Engineering/design changes are few and generally limited to quality and cost improvements. System, components or items are in rate production and meet all engineering, performance, quality and reliability requirements. All materials, manufacturing processes and procedures, inspection and test equipment are in production and controlled to six-sigma or some other appropriate quality level. Production actual costs meet FRP goals. Lean practices well established and continuous process improvements ongoing. Latter Stages of Production and Deployment Phase plus Operations and Support Phase
MRL Assist Tool Website A useful MRL assist tool developed under the direction of the Joint Defense Manufacturing Technology Panel is available on the web at: https://www.mrlassist.bmpcoe.org/ The Mississippi Manufacturing Extension Program (MEP.ms) may be another useful source of assistance in helping to address MRL issues: http://mep.ms/
TRLs should not be used interchangeably with MRLs. TRLs focus on a technologies maturity, while MRLs look at the maturity of the manufacturing system and processes that will deliver that design as a final product. A critical technology might have be very mature yet the manufacturing processes needed to produce it may be very immature. This will be especially true if manufacturing is not involved early in the design and development process. One of the classic roles of manufacturing is to take a look at the design and make it producible using tools like Design for Manufacturing and Assembly (DFMA). Some manufacturing considerations are:
Use economical materials. Design solutions seldom involve just one material. If you have a choice of materials that provide the same performance, then choose the least expensive material.
Use economical manufacturing techniques. If you have a choice on which machine or method to use to fabricate or assemble a product choose the least expensive approach.
Standardize materials and components. Often components or materials used in one product can be used in other products.
Design for process repeatability. Use quality control tools to make your processes more repeatable. If your factory floor processes are capable and in control then you stand a better chance of achieving your design goals.
Design for Product inspectability. Consider how you are going to inspect or verify that the product will meet its objectives. If you are inventing a new material you need to ask yourself how you are going to determine its acceptability.
Use acceptable materials and processes. You should be aware that some materials like methylene chloride is an ozone-depleting compound and is not an unacceptable material for use. So do not embed it in the product or use it in the manufacturing processes you employ.
Minimize your total part count. The number of parts drives a design’s efficiency, reliability, and maintainability. It is usually wise to study the design and work to reduce part count.
Minimize your skill levels required to manufacture. Henry Ford used simplicity to enable workers to focus on only one task. By doing this Ford was then able to connect the line to a chain and pull the vehicle through the factory giving the world its first moving assembly line.
Approach for Conducting MRAs INTRODUCE TRAIN ASSESS MANAGE INCORPORATE
Meet with PM to get buy-in and gather program info
Customize MRL approach for program
Train program IPT on manufacturing tools to support manufacturing lll maturity efforts
Determine current MRL
Develop plan, actions, and estimate costs to get to target MRL
Manufacturing risk/maturity is not the only cost/schedule/performance driver, but we need to manage manufacturing readiness integral to the acquisition process – increase successful technology transition
The intent for SBIRs that address manufacturing is the same as for other programs – to make the product less risky for the customer
Products made by mature manufacturing processes generally:
Are less prone to quality problems
Make the product / process perform the same, and perform better as a whole
Are more reliable in service
Have less difficult time delivering on schedule
Likewise we want to address not just manufacturing technology readiness and the manufacturing capability, but also the business case for the SBIR
Establish the Business Case
Technology is ready for transition – and customers exist for product / process
Long term agreements are potentially available for delivering product
Capital investments--when does the company invest? Government?
It only takes one duck to be “out of line” to keep your SBIR company from successfully inserting your technology into operational use!
Ultimate Success! This presentation based in part and a modification of original “Readiness Level” materials provided by the U.S. Air Force SBIR/STTR Program: http://www.sbirsttrmall.com/Library/Default.aspx
To learn more about how the MS-FAST Program can help your company compete in federal R&D programs contact:
Joe Graben, MBA
Director – USM/BIAC
Phone: (228) 688-2280
"This U.S. Small Business Administration (SBA) Cooperative Agreement is partially funded by the SBA. SBA's funding is not an endorsement of any products, opinions, or services. All SBA funded programs are extended to the public on a nondiscriminatory basis."