RAMS 2013 Establishing Product Reliability Goals


Published on

Paper given at RAMS 2013 on best practices and consideration when establishing reliability goals for the development team.

The way we communicate goals directly impacts the achievement of the goal. Creating and succinctly stating a reliability goal necessary for the leadership it provides an organization when designing a new product.
The reliability goal statement includes four elements: function, environment, probability of success, and duration. The function definition provides one means to define failure conditions. The environment includes elements such as weather and elements related to use frequency.
The probability of success and duration should always be stated in a couplet. Setting more than one probability and duration couplet further defines reliability over the expected lifespan of the product. And setting multiple couplets enables various focuses of the goal that concern different periods of time or constituents.
Setting and stating a clear and complete reliability goal delineates the boundary between a reliable enough product and one that is not. The goal enables the design team to balance the myriad of other design considerations along with product reliability in a meaningful manner.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

RAMS 2013 Establishing Product Reliability Goals

  1. 1. Establishing Product Reliability GoalsFred Schenkelberg, Ops A La Carte, LLCKey Words: reliability, specification, goal, process SUMMARY & CONCLUSIONS 2 A RELIABILITY GOAL The way we communicate goals directly impacts the The common understanding of „reliable‟ involvesachievement of the goal. Creating and succinctly stating a concepts like trust and consistency. Saying a product isreliability goal necessary for the leadership it provides an reliable generally means it works as expected over time. Whenorganization when designing a new product. I say my car is reliable, I may mean it starts every morning, on The reliability goal statement includes four elements: demand.function, environment, probability of success, and duration. These are not useful definitions even if the sentiment isThe function definition provides one means to define failure correct for a product design engineering team. One way toconditions. The environment includes elements such as define reliability that is more useful for engineers isweather and elements related to use frequency. “The probability that an item will perform a required The probability of success and duration should always be function without failure under stated conditions for a statedstated in a couplet. Setting more than one probability and period of time.” [1]duration couplet further defines reliability over the expected In this definition, we have four elements that needlifespan of the product. And setting multiple couplets enables refinement to fit a particular „item‟. Each of the four elementsvarious focuses of the goal that concern different periods of may or may not be well defined for the item and they may ortime or constituents. may not be stated all in one place. Setting and stating a clear and complete reliability goal For the purpose of this paper, we will address the fourdelineates the boundary between a reliable enough product elements as: Function, Environment, Probability of success,and one that is not. The goal enables the design team to and Duration.balance the myriad of other design considerations along with Creating a succinct reliability goal statement provides aproduct reliability in a meaningful manner. means to clearly communicate the desired reliability or 1 INTRODUCTION performance over time of the product. Rather than an implied, vague or scattered set of requirements or goals, create a single Nearly every product has a clear set of functional or statement with supporting information.performance objectives, possibly a cost goal and often a time The reliability goal should be easy to understand andto market deadline. Product reliability should be part of this promote the desired behavior and outcomes. As with any goalinitial set of criteria for product design and it should be very the normal guidelines apply. The goal should be specific,clearly stated. All too often goals such as „as good as or better quantifiable and be met within a specific period of time. [2]than the last product‟, or „5 year life‟, or „2-year warranty‟, is The goal setting process is a business process and shouldall the design team receives as a reliability goal. reflect the interests and capabilities of the business, Stating a complete reliability goal includes the references technology and customer. The intent is to provide a realisticto product function(s) and expected operating environment objective that can be achieved with a product that is profitableand use profiles. It also includes stating the probability of for the business and meets or exceeds the customer‟ssurvival over a time period. This probability and time couplet expectations. Profitable here may mean the product meetsmay have multiple statements in order to consider early life business objectives, which may or may not include profit,failure period, a warranty period, and a design life for the although profit is a common business objective.product. Each period has specific and often different risks,considerations and business impact. 3 FUNCTION AND ENVIRONMENT This paper explores the practical means to establish The first two elements of a reliability goal to explore areproduct reliability goals. The use of field performance of past the function and environment. The first assists us in defining aproducts, market positioning, cost of failures, technical failure, the second the expected set of external stresses orcapabilities, and engineering judgment all come to play when conditions. Both are necessary for a complete reliability goalestablishing the reliability goal. Setting a meaningful goal, that statement.is measureable during product development, is often one ofthe first steps necessary to create a reliable product.
  2. 2. 3.1 Function Tests.[3] Another with a focus on the telecommunications industry equipment is the set of documents provided by the The function or functions of an item are essentially what European Telecommunications Standards Institute (ETSI).[4]the item is to do. For example, a passenger plane transports Rather than use rated limits or absolute maximum storagepassengers from one location to another, or, a pair of temperature expectations which would only apply to a verysunglasses provides UV radiation eye protection from the sun. few situations and units, I recommend you use the 90 th The product may be very complex or relatively simple. percentile values for daily average temperature and dailyThe product may include multiple functions or only one. Yet, average temperature range. The National Climatic Data Centereven with the sunglasses, ultraviolet protection is probably not [5] has available worldwide weather station daily datathe only function. Style, color, comfort, and other aspects of readings. Besides temperature there is dew point (related tothe glasses may be as or more important than UV protection. relative humidity) and rainfall, windspeed and similar For a reliability goal, select the primary or salient function information for each day at each of the thousands of weatherof the product and reference the remaining functions in other reporting stations from around the world. The database accessdocuments. While I was at Hewlett-Packard Company, I permits one to select a region or country set of stations.noticed that many product development teams developed early Having years of daily or hourly weather data permits one toin the design process a detailed Product Requirements use the actual distribution of the values or a specific percentileDocument (PRD), which included sufficient definition of all of the data, say the 90th percentile, which improves thefunctions to deliver to the customer, often including color, accuracy of any consideration related to the weather.weight, location of the logo, pages printed per minute in Depending on the products industry or use expectations,various printing modes, etc. The PRD was often 50 pages or the use of standards or tabulated data may prove useful. If not,more. then gathering readings from actual or similar products in use When setting a reliability goal, use the primary function would provide the necessary information to establish ato communicate the purpose of the product. Keep it simple and complete set of environmental conditions.clear. For products with a detailed PRD or similar document, The other element of environment is the use profile. Howprovide a reference to the full set of expected functions. something is used as well as how often it is used may directly3.2 Environment affectits reliability. A long haul international plane may have one or two flights per day, whereas, a local commuter plane The product has to work for the customer to be of use. It may experience a dozen flights per day. The difference inalso has to function where and when the customer is expecting stresses included the number of take offs and landings, maybethe product to work. For example, a large passenger aircraft is altitude, and many other factors that detail the expectedexpected to operate around the world, and function properly frequency and nature of use.during taxi, take off, level flight and landing. Another example The use profile may also influence the types of failureis sunglasses should function when exposed to full sun. mechanisms and related stresses. A bicycle used by a The weather (temperature, humidity, contaminants, etc.) professional during a major race, will likely have stressesaround the product constitutes part of the environment. The imparted by the rider that are significantly higher than by ause profile is another element. The weather is a short way of recreational rider. The maintenance practices, hours of use perstating the list of stresses that exist external to the product and day, and other conditions may be quite different also.may impact the reliability of the product. If a product is It may be necessary to create different profiles andexpected to operate outdoors in all weather, then include associated environmental conditions. The aircraft examplecondensation, rain, and hail as expected conditions. may include details related to long haul and commuter and The weather should include all stresses that the product is cargo uses for the same aircraft. Each type of use would haveexposed to during storage, transport, installation and use. And, slightly different expected frequency of use, loadingall stresses that related to an expected failure mechanism conditions, weather conditions, and possibly maintenanceshould have sufficient information to permit characterization practices.of the products performance when exposed to those stresses. The same considerations apply for the design of a bicycle.For example, the sunglasses material should not deteriorate or A bicycle that is rarely maintained or used may be more pronechange color when exposed to sunlight. Knowing how much to corrosion related failure. The professional bike may have asunlight, expected spectrum and intensity, would assist in the higher probability for metal fatigue related failure.design, evaluation of alternatives, and validation & Understanding and detailing the full set of environmentalverification testing. (weather and use) conditions provides the design team a clear The environmental set of conditions are not just weather, set of expectations. The weather and use permit the evaluationit may include radiated emissions, power line anomalies, of materials and designs to function within the customers‟insect or animal exposure, dust, or something as possibly world.damaging as the interior of a purse or briefcase. One freely available standard that lists a wide range of 4 PROBABILITY AND DURATIONenvironmental conditions is the MIL-STD-810G The probability and duration for a reliability goalEnvironmental Engineering Considerations and Laboratory complete the goal statement. I recommend always pairing and
  3. 3. stating the two elements together to avoid any confusion. In practice, this line does not have to be straight. We mayStating 98% probability of success over 2 years is easier to want very few failures during the first month, a relatively lowunderstanding than stating only one of the following: chance of failure over the warranty period, and a modest Two year life failure rate out to the expected useful life of the product. The 2% failure rate The first implies duration and leaves open the expectednumber of units that would survive the two years withoutfailure. The second example, may apply to one year, onemonth, one hour or some other duration. Neither is complete.4.1 Probability There are a few ways to state the probability of success. Ilike probability of success or reliability as its positive(proportion not failing); it‟s easy to multiply with expectednumber of shipping units to estimate the number of units stilloperating over the related duration; and it is difficult tomisunderstand. Another way to state the probability term is to use theprobability of failure, which is the complement of theprobability of success, F(t) = 1 - R(t), where F(t) is the Figure 1 CDF lot of example to assist goal settingcumulative distribution function or probability of failure; and,R(t) is the reliability function or the probability of success. Other ways to state the probability include annualizedfailure rate (AFR) or mean time between failure (MTBF). Idon‟t recommend these as they assume the failure rate isconstant (which it often is not) and one must use theexponential distribution when the failure rate is constant. TheR(t) or F(t) do not assume a particular distribution. Setting this number is a balance between the desire tohave no failures whatsoever, the cost, technology andcomplexity of the product, and business profit or relatedobjectives. The „positioning‟ of the product as a high qualityand reliability product may or may not command a higherprice. This may depend on the marketing, competition, brandperception and consumer acceptance. Some makes chargemore for their cars in part due to the brand perception ofreliability. Even if Yugo built a very reliable car it would have Figure 2 CDF plot with multiple slopesto overcome the brand perception and earn market acceptance. As a consumer I do not want my specific unit to fail and next section discusses these and other duration periods.as a reliability engineer I understand that there is a finite risk Figure 2 shows three different slopes on the CDF plot thatthat my unit will fail. If it fails quickly, I may return the reflect the balance between all the elements related toproduct and buy a competitor‟s product as a replacement. If it probability of success.fails after many years of service, I am likely to buy the same In this example, figure 2, the reliability at 30 days isbrand or model. This implies a changing acceptance of failure 99.8%, and at one year is 98%, and at five years is greater thanover time. 90%. To set an appropriate reliability goal, it may be necessary Establishing the probability values is to some extent anto establish multiple couplets of probability and duration for exercise in trial and error. Understanding the customerthe product. The various probabilities of success define the expectations may provide one view, which has to balance withline between the decreasing reliability over time, and the cost of the solution and available technology. One optionthecustomers‟ acceptance of a failure. One way to represent to bracket the probability values is to review past productsthis goal is to draw a line on a cumulative distribution (CDF) field reliability performance, or estimate the reliability ofplot (assuming a particular distribution, in this case Weibull), similar products in the field. Another method is to conductsee figure 1. reliability analysis and testing on competitor products. The At any point in time, the plot in figure 1 provides the intent is to find an acceptable level of product reliability forprobability of failure (one minus the probability of success). the customer and the producer.For example, at 30 days the reliability is 99.7%, and at oneyear it is 97%, and at five years is less than 90%.
  4. 4. 4.2 Duration Useful life is the period of time the product is expected to function or operate for the customer. It is often well beyond The discussion in section 4.1 alluded to a few types of the warranty period. While there isn‟t an obligation to repairdurations: early life, warranty period, and useful life. In or replace the product if it fails, there may be an impact toaddition, durations such as economical life, mission time, and repeat sales, to sales of other products offered by themany others may apply. In this paper we will explore firs three company, or to the purchase of consumables associated withmentioned. The discussion for the establishment of each may the product. The last example is illustrated well by inkjetapply directly to many other types of durations. printers, which have a long expected useful life, such that the Early life failures (ELF), also called infant mortality, is consumer will continue to purchase inkjet cartridges. The firstdefined by the producer to reflect the time period over which two examples are related to brand image and the purchaseproduct failures are included in a different group for analysis decision based on perception of product reliability.and tracking. The reasons for doing this include, detection of The useful life may have technology limitations, maysupply chain or assembly issues, detection of shipping or have industry or market expectations (possibly also contractinstallation issues, and to provide focus on the failures that obligations), or it may have practical business limitations.occurred very early in the expected life of the unit. Another Consider the washing machines built in the 1950‟s. They werereason is customers with ELF may reduce the brand‟s cast iron, very simple in design and very durable. They alsoreliability image, may complain more to potential customers, washed clothes just fine. Some are probably still in service, asand may have their reliability expectation eroded. is the one in my grandparents‟ home. Today‟s machines are The ELF period is variable and is often in the range from lighter (less transport costs), feature rich (marketing?), andinstallation or first use, up to 3 months of use by the customer. tend to last about 10 to 12 years. One reason for this reductionDepending on the product and market the actual period of time in useful life is the business desire to have customers buymay or may not start with first use, as it may be difficult to another washing machine every 10 to 15 years. “Let‟s notactually determine. Instead starting the clock with shipment make it too reliable,” might be heard in discussions of suchdate may serve as surrogate. product lines. The intention is to identify those failures that occur very The useful life duration is the reliability goal elementsearly. The focus on these helps to isolate and resolve issues that address how long the product should last. The useful lifequickly. Also, the ELFs tend to have root causes unrelated to is a major concern for the selection of materials andwear out or end of useful life type failure mechanisms. Setting components, and determining if and how to implement repairs.the duration is local policy and enough duration to include atleast a month of use by the customer. Warranty period is the length of time the manufacturer is 5 EXAMPLEfinancially responsible to repair or replace a faulty unit. It is Consider a bicycle as a product under development. Itbeyond the scope of this paper to discuss the types of needs to have a clear reliability goal established. Even withoutwarranties, so we will only focus on the broad definition. mentioning reliability, some questions may arise from the As warranty is an expense to the company it is of design team: What type of bicycle? Intended use? Size andfinancial interest. Warranty Week [6] reports publicly traded weight options? Custom accessories or fittings for customercompanies spend roughly 2 to 4% of net revenue on warranty selected accessories? Color? And many more.claims. Therefore, one way to establish a warranty period is to These natural questions begin to develop the function anddetermine the products‟ expected failure rate and associated environmental elements of a reliability goal. Let‟s say the newcost of warranty over various time periods. Then set the product is to be a road bike suitable for the amateur triathlete.warranty such that the warranty expense is reasonable. The full list of specifications for the model may evolve into a In reality the above financial concern is balanced by complete list of specifications called a product requirementsmarketing, customer expectations and in some cases local document. Succinctly let‟s say the function is a triathlon roadlaws. If the current market expects a 1-year warranty, offering bicycle.a shorter warranty period may signal an inferior product. The environment may need further refinement. Is the bikeWhereas, offering a longer warranty may indicate a better to be available only in certain markets? What is the price pointproduct or one worth „taking a chance on‟ as it is „under and expected level of riders? Will it be maintained by awarranty‟. There are no hard and fast rules for setting a professional staff or by the customer only? How many mileswarranty period and often is already established for the market and over what terrain is expected?the product enters. The answers to these environmental questions may Setting the warranty period as one of the durations initially take the form of just an outline. And, later evolve topermits the entire organization to understand the financial include temperature distributions, miles per day distributions,impact of product reliability on the business bottom line. One maintenance frequency expectations, torque loadingway to discuss warranty within the company is to calculate the distributions, and more. To summarize for the goal, we canwarranty cost per unit shipped. This has the advantage of say the bike is expected to be used by amateur triathletes inplacing warranty in the same units as most of the products bill Olympic to Ironman distance races, and including theirof materials list of component costs. training rides.
  5. 5. Consider the probability of success and duration as 6.1 Marketcouplets. The distributors and resellers are concerned with Part of marketing a product includes setting expectationsELF within 1 month of purchase. Discussions with resellers for customers concerning product reliability. This may beand with customers and review of customer satisfaction done with clear reliability claims or comparisons to competitorsurveys for the product line and industry suggest keeping the products.ELF to less than 1% over the first month is not damaging to Often the marketing team is the voice of the nextthe brand‟s reputation nor significantly impact brand future customer. Especially for new products or for new markets, thesales. Therefore, the first couplet is stated as 99% probability marketing team‟s input is essential for a well-positionedof success over first month of ownership. product within that marketplace. The industry‟s warranty period is commonly one year forparts and workmanship (doesnt cover normal use or 6.2 Reliable Enoughaccidents). A brief discussion with senior management reveals Using more expensive materials and technologies maythere is not a desire to differentiate on reliability with this provide a more robust and reliable product. The washingproduct so the one-year warranty is fine. A review of similar machine example illustrates the possibility of creating amodels suggests that a 1.75% failure rate over a year is the product that lasts an undesirably long time.company‟s average for the past few models of triathlon bikes. It may be possible to create a cell phone that would lastAlso, a discussion with the project manager suggests the for 20 years without failure. Given the expectation of newweight of the bike is the primary design goal, and lowering features enticing the customer to buy a new phone every twowarranty cost is a major concern. Therefore, the second years negates the necessity or desire for a 20-year phone.couplet may be stated as 98.25% probability of success over Create the goal that is in line with customer and marketthe one-year warranty period. expectations, balanced with cost and actual market drivers for The third of the couplets addresses the useful life of the purchase. If reliability is not the most important factor forbicycle. A serious amateur may replace their bike every year, purchase, it should not be the primary concern or constraintwhile others may use the same equipment for many years. A for the design team.(hypothetical) survey conducted at regional and nationaltriathlon events found most bikes are 3 years old or newer, 6.3 Safetywhile a few are as much as 10 years old. Let‟s say for this Product reliability and safety are related. One of thepaper that 90% of triathletes are riding bikes less than 5 years considerations in setting a reliability goal concerning safety isold. Therefore, a reasonable useful life probability/duration considering how the product fails. If it has a reasonablecouplet is set as 95% probability of success over 5 years. possibility of unavoidably failing in an unsafe manner (i.e. In summary, we can state a reliability goal with all four ignition) then one option is to reduce the chance of failure.elements included and three probability of success (reliability) The increase in reliability to avoid un-safe failure modesand duration couplets. may be feasible, yet it may preclude features, cost or other Design and build a triathlon road bicycle intended for use objectives. Of course it is preferred to only permit productin training and races for triathlons ranging from Olympic to failures into a safe mode. Another possibility is to add fail-safeIronman distances, with 99% reliability over the first month, features to the product to mitigate the harm possibility causedwith 98.25% reliability over the one-year, and, with 95% by un-safe failure modes.reliability over the 5 years. Details for product specificationsand functions are in the PRD and environmental and use 6.4 Customer Expectationsdetails are in the Environment document. Product failures, degraded performance, fracture, bugs, 6 CONSIDERATIONS and related are part of the risk we encounter with every product we employ. Failures happen. A goal statement is a balance between the desired and the The expectation is not for a totally failure free experiencepossible. A well-crafted goal will provide sufficient with every product, yet each person does have an internalinformation and guidance to enable decision-making during gauge for what is acceptable. Understanding you customerthe development process that leads to attainment of the goal. It expectations related to reliability is part customer satisfaction,must be measureable, which is sometimes difficult for part brand loyalty, part market share and part the nature ofreliability. A well-crafted goal also is clear, concise and easily your customers (i.e. early adopters may have more toleranceunderstood.The balance for a reliability goal is between „never for failure).A consideration for the reliability goal is thefails‟ and „fails too often or early‟. perception of customer acceptance of the products reliability. Each consideration or factor that impacts the setting of thegoal should be part of the discussion. The lack of any 6.5 Technology and Materialsreliability goal permits everyone in the organization to Consider, for example, alow density poly(ethylene) meltsprioritize product reliability according to their local objectives. at a relatively low temperature (98-115°C) compared to NylonSetting a goal that is tied to the organizations strategy, 66 (255-265°C), thus enabling the correct choice of materialincludes all stakeholders and is clearly stated enables the based on temperature when compared to use temperature. Ofentire organization to strive toward the same objective.
  6. 6. course, polymer selection as with all materials may consider Considerations and Laboratory Tests, US Department ofmany different factors. For reliability, the consideration is the Defense, 31 October 2008.amount of margin the materials and technologies have beyond 4. ETSI En 300 119 series Environmental Engineering (EE):the expected use stresses. Environmental conditions and environmental tests for Sometimes there are relatively obvious and low cost telecommunications equipment; Part 1-0: Classificationsimprovements, sometimes there are no choices. Adjust the of environmental conditions; Introduction – andreliability goal to accommodate any reliability limitations associated documents, freely available at www.etsi.org,related to technology or a material expected within the product visited July 29th, 2012.helps to communicate the priority reliability has in this area. 5. National Climatic Data Center, U.S. Department of Commerce, as of July 6, 2012, http://www7.ncdc.noaa.gov/CDO/cdoselect.cmd?dataseta6.6 Value bby=GSOD&countryabby=&georegionabby= The final judge of a product value is the customer or end 6. Eric Arnum, Warranty Week newsletter,user. If the product provides sufficient value (benefits www.warrantyweek.com, visited on July 30th, 2012.outweigh costs) then the product is successful. This judgment 7. Quote attributed to Charles Coonradt at Chatsworthis always in context of other possible solutions. Also, product Consulting Group blog, visited on July 31 st, 2012,reliability may provide a major element of the value, or it may http://chatsworthconsulting.com/2011/06/20/want-more-not be a consideration. The way customers perceive value is a of-it-measure-it/major influence on product reliability goal setting. If we BIOGRAPHIESunderstand the value placed on product reliability by thosepurchasing the product we have a better opportunity to meet or Fred Schenkelbergexceed their expectations. 15466 Los Gatos Blvd #109-371 Another value to consider is the impact a reliability goal Los Gatos, CA, 95032, USAand it‟s measurement has on the design process. “You have to e-mail: fms@opsalacarte.commeasure what you want more of.” Charles Coonradt [7] Thesimple idea is to clearly communicate what you want from the Fred Schenkelberg is a reliability engineering andorganization. If reliability is not included or not clear, then it is management consultant with Ops A La Carte, LLC, with areaspossibly considered not important. Setting and achieving a of focus including reliability engineering management traininggoal also enables the entire organization to plan appropriately and accelerated life testing. Previously, he co-founded andfor warranty accruals, manufacturing capacity, spare stocking, built the HP corporate reliability program, includingetc. Guessing the reliability objectives can cause expensive consulting on a broad range of HP products. He is a lecturermistakes in these areas. with the University of Maryland teaching a graduate level course on reliability engineering management. He earned a REFERENCES Master of Science degree in statistics at Stanford University in 1996. He earned hisbachelor‟s degrees in Physics at the1. Patrick D. T. O‟Connor and Andre Kleyner, Practical United State Military Academy in 1983. Fredis the immediate Reliability Engineering, 5th Edition, Chichester, Wiley, Past-Chair of the American Society of Quality Reliability 2012, p 1. Division, active with IEEE and IEC reliability standards2. Joseph M. Juran and A. Blanton Godfrey, Juran‟s Quality development teams. Fred is also the founder of the No MTBF Handbook, 5th Edition, New York, McGraw-Hill, 1999, p movement and website nomtbf.com. He is a Senior Member of 13.9. ASQ and IEEE. He is an ASQ Certified Quality and3. MIL-STD-810G, Environmental Engineering Reliability Engineer.