“How do you know so much about our program?” was a question the quality manager asked after reading the assessment report. The assessment took one day with eight interviews. The reliability that results is going to happen whether or not the team designing the product or production line deliberately use reliability engineering tools or not. The elements of a product or system will respond to the environment and either work or fail.

1. fmsreliability.com http://www.fmsreliability.com/education/assess-reliability-program/
Fred
How to Assess Your Reliability Program
“How do you know so much about our program?” was a
question the quality manager asked after reading the
assessment report. The assessment took one day with
eight interviews.
The reliability that results is going to happen whether or
not the team designing the product or production line
deliberately use reliability engineering tools or not. The
elements of a product or system will respond to the
environment and either work or fail.
While working at Hewlett-Packard I had the opportunity
to conduct reliability program assessment of about 50
product divisions. One hypothesis related the number of
reliability tasks the team actively used would correlate to
their warranty expenses.
That worked to a point.
The teams that did not understand basic tools and did no
overt or organized reliability engineering had high
warranty expenses (as percent of revenue). The teams
that did a large number of tasks (FMEA, HALT, ALT,
Predictions, etc.) did have lower warranty expenses.
The surprise was the teams that had the lowest warranty
expenses also did very few reliability activities. The
difference was the best performing teams understood
the range of available reliability engineering activities
and only used the tools that would provide value for a
given circumstance.
Less mature organization would attempt to do as many
reliability related activities including a long list of product
tests of which many provided little actual value. It was
the application of the right tool at the right time that made the difference.
Maturity and Activity
Hiring a reliability engineer or running a lot of life tests does not improve your product’s reliability performance. It is
not the organization or activities that you call a reliability program, rather your reliability performance relates to the
culture concerning reliability.
Reliability occurs at the point of decision. Therefore, during interviews the intent is to understand how decisions are
currently accomplished. To what extent does reliability considerations influence decisions and what tools or methods
are used to form decisions.

2. For example, if we ask, “To what extent do you do HALT?” (HALT being Highly Accelerated Life Testing). In two
circumstance the answer may be, “We rarely use HALT.”
In one case, it may be the engineer doesn’t know what HALT is and isn’t sure if the testing they accomplish is similar
to HALT or not. Or, they don’t do HALT as they are unfamiliar with that type of testing.
In another case, the engineer knows about HALT, how and why it is used. Then says they have rarely used HALT
because they have not had appropriate situations to do HALT. They understand that it is a useful tool for specific
applications and recently have not needed to conduct HALT.
Some respond that they do HALT. Again there are two common responses. In one case the team does HALT all the
time because it is required, independent of whether it may be useful or not. In the other case, they do HALT as it is
the right tool for the current situation.
One team didn’t know what HALT was and the other fully understood and choose to not do HALT. The difference is
the understanding and application, or maturity.
Assessment Process
To understand how an organization’s reliability maturity, use the following assessment process.
1. Select survey topics. This is a list of activities and tools common to reliability practices in your industry. It may
include items rarely used. It should include the breadth of topics related to reliability in your field. See the DFR
Methods Survey for one possible list of topics.
Some topics are broad, such as on ownership/responsibility of product reliability, or reactive or proactive approach of
management. Some topics are very specific, such as specific tools like FMEA or HALT.
2. Establish the interview format. One to one, small group, via phone, invited survey with follow up conversations,
or some other method. I have found the one-to-one discussions the most useful as it permits immediate follow up and
exploration of the rationale or motivation behind specific behaviors or responses.
3. Conduct the interviews (collect information). Arrange to interview or survey a cross section of people in the
organization. Select individuals with experience with the organization and products are typically designed and
manufactured. My recommended list of position titles comprises:
Design & development engineers (electrical, mechanical, software)
Design& development managers (electrical, mechanical, software)
Reliability or Quality engineers and/or managers
Procurement engineering (someone that works with suppliers)
Manufacturing engineering and/or managers (design for manufacturing, sustaining and/or production
engineering)
Select about 8 individuals for interviews, more or less depending on the specific situation, size, complexity, etc of the
program.
In general the interview question is based on the phrase, ’to what extent’. For example, you might ask, “To what
extent do you use HALT?” Depending on the response you may explore the motivations or rationale behind the
decision both to conduct HALT, and how the HALT results are used within the organization.
4. Document the business environment – volume, cost, brand position, revenue, cost of unreliability as percent of
net revenue, etc. Document any regulatory or customer imposed restrictions or requirements. Summarize to convey

3. the atmosphere around the reliability program.
5. Document the collected information. A summary back to participant asking for additional input or corrections
helps with the acceptance of the assessment results, plus may help avoid a mistake in your understanding.
6. Analyze the data. This is not done during the interviews – let them talk. Review the notes and information
provided and map to the maturity matrix. Look for consistent approaches to making reliability related decisions. Look
for patterns of behavior and underlaying motivations or causes.
7. Report on assessment findings. Document and explain what you heard and how it related to the overall
organization’s maturity. The report my include the interview summary, strengths, weaknesses, and recommendations
for improvement.
Summary
The assessment process should provide a view of the overall organization’s approach to making decisions and to
what extent and how their reliability program influences those decisions. With that basic understanding you can
identify strengths to build upon, weaknesses that need attention, and recommendations to improve the maturity of the
reliability program.