This document discusses the differences between individual reliability (IndRel) and population reliability (PopRel) for aging systems. IndRel provides the reliability of a single system at a given age, while PopRel provides the probability that a randomly selected system from a population will work at a given time, taking into account the age distribution of systems in the population. The document outlines methods to estimate both IndRel and PopRel, including using Weibull and probit models on failure data. Examples are provided to demonstrate estimating IndRel and PopRel for projects using different statistical models and failure data.

1. Comparing
Individual
Reliability
to
Popula6on
Reliabil6y
for
Aging
Systems
Dr.
Chris6ne
M.
Anderson-­‐Cook
©2013
ASQ
&
Presenta6on
Anderson-­‐Cook
hHp://reliabilitycalendar.org/
webinars/

2. ASQ
Reliability
Division
English
Webinar
Series
One
of
the
monthly
webinars
on
topics
of
interest
to
reliability
engineers.
To
view
recorded
webinar
(available
to
ASQ
Reliability
Division
members
only)
visit
asq.org/reliability
To
sign
up
for
the
free
and
available
to
anyone
live
webinars
visit
reliabilitycalendar.org
and
select
English
Webinars
to
find
links
to
register
for
upcoming
events
hGp://reliabilitycalendar.org/
webinars/

3. Comparing Individual Reliability to
Population Reliability for Aging Systemsp y g g y
Dr. Christine M. Anderson-Cook, LANL (candcook@lanl.gov)
Dr Lu Lu University of South FloridaDr. Lu Lu, University of South Florida
July 2013
https://sites google com/site/poprellu/home
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
https://sites.google.com/site/poprellu/home

4. | Los Alamos National Laboratory |
Outline
1. Individual and Population Reliability
a. Definition
b. When to use which
c. Overview of methods for population reliability
2. Age Only examples (QE paper, 2011)
a. Weibull (observations: time to failure)
b. Probit (obs: Pass/Fail at specific age)
3. Age + Usage example (QREI, 2011)
( / f &a. Probit (obs: Pass/Fail at specific age &
usage)
4 Conclusions
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 2
4. Conclusions

5. | Los Alamos National Laboratory |
Focus on ReliabilityFocus on Reliability
 Definition of reliability:Definition of reliability:
“the probability that a system will continue to
perform its intended functions until a specified
point in time under encountered use conditions.”
Define boundaries of system
(peripherals, human interface)
Often exposure to
environmental conditions
Multiuse systems may have different
thresholds for working (most severe,
typical)
environmental conditions
may impact reliability
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 3
yp )

6. | Los Alamos National Laboratory |
Individual vs Population Reliability
ility
ability
ystemReliabi
pulationReli
Age (months) Time into future (months)
Sy
Po
 Individual System Summary (IndRel): For a given
system with specified age, what is its reliability?
 Population Aggregate Summary (PopRel): For a Population Aggregate Summary (PopRel): For a
population of systems (each with possibly different
ages), what is the probability that a randomly chosen
system will work at the current or some future time?
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 4
system will work at the current or some future time?

7. | Los Alamos National Laboratory |
Two Different SummariesTwo Different Summaries
 Relevance
– IndRel: for managing individual units, perhaps to
remo e them from the pop lation if the become tooremove them from the population if they become too
unreliable, or to send them in for scheduled
maintenance.
– PopRel: for managing the population and require aPopRel: for managing the population and require a
given performance level across the population at this or
some future points in time.
 Information needed
– Summary of results from testing various systems (both)
– An appropriate statistical model for the reliability given
age (both)
lplus
– the age demographics of the population of interest at
the current time (PopRel)
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 5

8. | Los Alamos National Laboratory |
Calculation of PopRel
IndRelIndRel
Age
Demographics
ReliabilitySystem
For an individual system we
Age (months) Age (months)
For an individual system, we
can predict its reliability now
and into the future given its
current age
onReliability
Time into future (months)
Populatio
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 6
Time into future (months)

9. | Los Alamos National Laboratory |
Calculation of PopRel (cont’d)
IndRelIndRel
Age
Demographics
ReliabilitySystem
Age (months) Age (months)
abilitySystemRelia
For each system in the population, we can determine its predicted
li bilit b d it t
Time into future (months) Time into future (months) Time into future (months)
S
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 7
reliability based on its current age

10. | Los Alamos National Laboratory |
Calculation of PopRel (cont’d)Calculation of PopRel (cont d)
iabilitySystemReli
Now we use the estimates of all the individual predicted reliabilities to determine
Time into future (months) Time into future (months) Time into future (months)
PopRel
the overall reliability of the population
ReliabilityPopulation
Time into future (months)
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 8
Note: this could be calculated for any sub-population
Time into future (months)

11. | Los Alamos National Laboratory |
Reliability summary questionsReliability summary questions
IndRel: For a given system with specified, what is its reliability?
PopRel: For a population of systems (each with possibly different ages),p p p y ( p y g )
what is the probability that a randomly chosen system will work at a
given point in time? Or what fraction of the parts will work at a given
point in time?
Questions: Which summary is more of interest,
1. If you own a single item? IndRely g
2. If you are own a collection of items used by your department?
3. If you work on maintaining the systems?
4 If id i h i t t l t h t
IndRel
PopRel
IndRel or PopRel
4. If you are considering purchasing new systems to supplement what
is currently available? PopRel
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 9

12. | Los Alamos National Laboratory |
Example 1:
LCD j t l i W ib ll Di t ib tiLCD projector lamps using Weibull Distribution
LCD Projection LCD Projection LCD Projection
M d l H M d l H M d l H
Observed failure time of 31
Model Hours Model Hours Model Hours
1 182 1 974 2 380
1 230 1 1755 2 418
1 244 2 50 2 584
lamps (3 different models)
182 h
230 h 1 387 2 81 2 1205
1 464 2 131 2 1407
1 473 2 158 2 1752
1 600 2 174 3 34
230 h
244 h
…
1895 h
1 627 2 300 3 39
1 660 2 332 3 274
1 798 2 345 3 1895
1 954
1895 h
1 954
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 10

13. | Los Alamos National Laboratory |
Step 1: Estimate Individual
Reliabilityy
 Weibull model:
1
( | ) exp( )f t t t 
   

 Bayesian analysis – specify priors
( | , ) exp( )f t t t 
    
~ (2.5, 2350)
~ (1 1)
G am m a
G am m a


 Estimate the posterior (WinBUGS)
~ (1,1)G am m a
( , | ) ( | , ) ( , )f y f y f     
NMCMC λ, β estimates are generated by WinBUGS to
i t th t i di t ib ti
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 11
approximate the posterior distribution

14. | Los Alamos National Laboratory |
Individual Reliability EstimatesIndividual Reliability Estimates
ility
ility
Model 1 Model 2
ystemReliab
ystemReliab
Age (hours)
Sy
Age (hours)
Sy
  eliability
Model 3
 
At age=t
NMCMC
estimates
SystemRe
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 12
Age (hours)

15. | Los Alamos National Laboratory |
Step 2: Estimate Population Reliability
 For a population of 51 Model 1 units
Frequency
A (h )Age (hours)
 
At each time,t,
estimate reliability for
eliability
 
NMCMC
estimates
estimate reliability for
each unit, then combine
to get PopRel estimate
1
( ) ( )ip t p t 
PopulationRe
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 13
( ) ( )r i
i U
p t p t
N 

P
Time into future (hours)

16. | Los Alamos National Laboratory |
Population Reliability ResultsPopulation Reliability Results
ncy
PopRel
nReliability
Frequen
Population
Age (hours)
Time into future (hours)
Reliability
IndRel
Could we have predicted this poor
population reliability? SystemR
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 14
Age (hours)

17. | Los Alamos National Laboratory |
Answering Questions – IndRel or PopRel
1. For a unit that is 100 hours old, what is the
probability that it will work?
2. What is the probability that a random unit will work
?now?
3 What is the probability of the unit I currently have3. What is the probability of the unit I currently have
working when I turn it on?
4. My team has 5 units which we use regularly. What
is the probability that a random unit from there will
k?
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 15
work?

18. | Los Alamos National Laboratory |
PopRel for More Complex PopulationPopRel for More Complex Population
Overall
abilityPopulationRelia
Time into future (hours)
Model 1 Model 2 Model 3
Reliability
nReliability
nReliability
Population
Population
Population
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 16
Time into future (hours) Time into future (hours) Time into future (hours)

19. | Los Alamos National Laboratory |
Frequentist Options for EstimationFrequentist Options for Estimation
 Estimate λ,β (and their covariance matrix) using
maximum likelihood
 IndRel: From this confidence intervals for reliability
are possible at all ages of the system
 PopRel:
a. Generate M draws from the bivariate normal
di t ib ti th d t bt i Mdistribution, use these draws to obtain M
estimates of λ,β and use to build an empirical
C.I. for PopRelp
b. Sample with replacement from the original data,
use this to obtain M estimates of λ,β ….
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 17

20. | Los Alamos National Laboratory |
Example 2: Missiles using Probit ModelExample 2: Missiles using Probit Model
227 missiles tested (destructive testing)
Current Population
Model: ~ ( )i iY Bernoulli p 1 Pass
Y


Age = 40 months
Age = 90 months
0 1( )i
i
age
p
  
  
 
0
iY
Fail
 

Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 18
s
 
 

21. | Los Alamos National Laboratory |
Step 1: Estimating IndRel
 Bayesian analysis to estimate
parameters (WinBUGS)
0 1( )i
i
age
p
s
  
  
 
0 1 s 
NMCMC
estimatesestimates
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 19

22. | Los Alamos National Laboratory |
Step 2: Estimate PopRel
 Current Population At each time,t,
estimate reliability for
each unit, then combine
to get PopRel estimate
0 1 s 
to get PopRel estimate
1
( ) ( )r i
i U
p t p t
N 
 
NMCMC
estimates
abilityPopulationRelia
PopRel
IndRel P
Time into future (months)
PopRel
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 20

23. | Los Alamos National Laboratory |
Modeling Reliability as a Function of Age
d Oth I f ti (U E )and Other Information (Usage or Exposure)
 For Example 2, reliability was estimated as a
function of the age of the systemfunction of the age of the system
– If a system is x months old today, then in a month
it will be (x+1) months old
 But what if reliability is a function of age and usage?
(eg. car reliability typically modeled with age and
mileage)mileage)
– If a car is 24 months old and has gone 30000
miles, what will these values be in 1 month?
– Historical usage pattern can be helpful for
prediction, but will introduce some additional
variability
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 21
variability.

24. | Los Alamos National Laboratory |
Example3:
Missiles Modeled with Probit for Ageg
and Usage
Model: ~ ( )Y Bernoulli pModel: ~ ( )i iY Bernoulli p
1
0
i
Pass
Y
Fail

 

0 1 2( ) ( )i i
i
age usage
p
    
  
 
0 Fail
ip
s
 
 
Estimation of IndRel – unchanged
C ld B i i i f d l
Here,
#t f- Could use Bayesian estimation for model parameters
- Could use maximum likelihood to get estimates
usage = #transfers
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 22

25. | Los Alamos National Laboratory |
Estimating PopRel – need to predict
f tfuture usage
 By looking at the rate at
hi h i
ge
which usage increases,
we can predict what
future usage values are
Usag
g
likely, assuming the
same pattern of usage
P ibl t
Age
Usage range
predicted from
historical data
 Possible sources to
describe the pattern
– Test data
ge
Current
values
Test data
– Current population
– User specified
Usag
Added variability
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 23
distribution
Age

26. | Los Alamos National Laboratory |
Obtaining the Usage Rate Distribution
 Historical (test data, current population, or both)
Index Age Usage Usage Rate Use as population to draw
from by sampling with
1 a1 u1
2 a2 u2
u1/a1
u2/a2
from by sampling with
replacement
Create a distribution
 User specified distribution
… … C eate a d st but o
which adequately
represents usage rate
center and spread
 User specified distribution
– Allows flexibility to specify change in anticipated usage
Create a distribution which adequately
represents usage rate center and spread
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 24

27. | Los Alamos National Laboratory |
Estimating PopRel R t f tiEstimating PopRel
Individual system reliability PopRel at
Repeat for many times
0 1 2, , ,s   Usage rate
r(1)
Individual system reliability
estimates at each time, t
(1) (1) (1) (1)
0 1 2, , ,s  
PopRel at
each time, t
r(2)
…
0 1 2  
(2) (2) (2) (2)
0 1 2, , ,s  
… ( ) ( ) ( ) ( ) ( ) ( )
( ) 0 1 2( ) ( * )
( )
j j j j j j
j i i iage usage rate age     
 
NMCMC
…… ( ) 0 1 2
( )
( ) ( )
( )j i i i
i j
age usage ate age
p t
s
   
  
 
1
( ) ( )t tMCMC
estimates
Obtain NMCMC values
from characterizing distribution
( ) ( )r i
i U
p t p t
N 
 
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 25
from characterizing distribution

28. | Los Alamos National Laboratory |
PopRel for Missile population
Used historical
usage information
from both testedfrom both tested
samples and current
population
onReliabilityPopulatio
PopRel (assuming continued
same pattern of usage for
overall population)
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 26
Time into future (months)
p p )

29. | Los Alamos National Laboratory |
Conclusions
 Understanding which summary is appropriate to answer which
question is key to good decision-making
– IndRel answers “For a given system with specified age (and
) h i i li bili ?usage), what is its reliability?
– PopRel answers “For a population of systems, what is the
probability that a randomly chosen system will work at the
current or some future time?current or some future time?
 What is needed?
– Summary of results from testing various systems [both]
A statistical model for the reliability given age and usage [both]– A statistical model for the reliability given age and usage [both]
plus
– The age (and usage) demographics of the population at the current
time [PopRel]
 Predicting the age of systems into the future is straightforward, but
additional assumptions about future usage of units in the population are
needed to obtain a sensible PopRel estimate
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 27

30. | Los Alamos National Laboratory |
ReferencesReferences
1. Lu, L., Anderson-Cook, C.M. (2011) “Prediction of, , , ( )
Reliability of an Arbitrary System from a Finite
Population” Quality Engineering 23 71-83.
2. Lu, L., Anderson-Cook, C.M. (2011) “Using Age and
Usage for Prediction of Reliability of an ArbitraryUsage for Prediction of Reliability of an Arbitrary
System from a Finite Population” Quality and
Reliability Engineering International 27 179-190.
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
UNCLASSIFIED
July 2013 | UNCLASSIFIED | 28