NG BB 47 Basic Design of Experiments

3,279 views

Published on

Published in: Education, Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,279
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
605
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

NG BB 47 Basic Design of Experiments

  1. 1. UNCLASSIFIED / FOUO UNCLASSIFIED / FOUO National Guard Black Belt Training Module 47 Basic Design of Experiments (DOE) UNCLASSIFIED / FOUO UNCLASSIFIED / FOUO
  2. 2. UNCLASSIFIED / FOUOCPI Roadmap – Improve 8-STEP PROCESS 6. See 1.Validate 2. Identify 3. Set 4. Determine 5. Develop 7. Confirm 8. Standardize Counter- the Performance Improvement Root Counter- Results Successful Measures Problem Gaps Targets Cause Measures & Process Processes Through Define Measure Analyze Improve Control TOOLS ACTIVITIES •Brainstorming • Develop Potential Solutions •Replenishment Pull/Kanban • Develop Evaluation Criteria •Stocking Strategy • Select Best Solutions •Process Flow Improvement • Develop Future State Process Map(s) •Process Balancing • Develop Pilot Plan •Standard Work • Pilot Solution •Quick Change Over • Develop Full Scale Action/ •Design of Experiments (DOE) Implementation Plan •Solution Selection Matrix • Complete Improve Gate •‘To-Be’ Process Mapping •Poka-Yoke •6S Visual Mgt •RIE Note: Activities and tools vary by project. Lists provided here are not necessarily all-inclusive. UNCLASSIFIED / FOUO 2
  3. 3. UNCLASSIFIED / FOUO Learning Objectives  Learn benefits of DOE methodology  Discuss differences between DOE and trial and error (one-factor-at-a-time) approaches to experimentation  Learn basic DOE terminology  Distinguish between the concepts of full and fractional factorial designs  Use Minitab to run and analyze a DOE  Use results of DOE to drive statistically significant improvements Basic Design of Experiments UNCLASSIFIED / FOUO 3
  4. 4. UNCLASSIFIED / FOUO Helicopter Simulation Phase One UNCLASSIFIED / FOUO 4
  5. 5. UNCLASSIFIED / FOUO Exercise: Helicopter Simulation  Customers at CHI (Cellulose Helicopters Inc.) have been complaining about the limited flight time of CHI helicopters  Management wants to increase flight time to improve customer satisfaction  You are put in charge of this improvement project UNCLASSIFIED / FOUO 5
  6. 6. UNCLASSIFIED / FOUO Exercise: Constraints  Project Mission: Find the combination of factors that maximize flight time  Project Constraints:  Budget for testing = $1.5 M  Cost to build one prototype = $100,000  Cost per flight test = $10,000  Prototype once tested can not be altered  See allowable flight test factors and parameters on the next page UNCLASSIFIED / FOUO 6
  7. 7. UNCLASSIFIED / FOUO Exercise: Test Factors and Parameters  Paper Type Regular Card stock  Paper Clip No Yes  Taped Body No 3 in of tape  Taped Wing Joint No Yes  Body Width 1.42 in 2.00 in  Body Length 3.00 in 4.75 in  Wing Length 3.00 in 4.75 in UNCLASSIFIED / FOUO 7
  8. 8. UNCLASSIFIED / FOUO Exercise: Roles & Responsibilities  Lead Engineer – Leads the team and makes final decision on which prototypes to build and test  Test Engineer – Leads the team in conducting the test and has final say on how test are conducted  Assembly Engineer – Leads the team in building prototypes and has final say on building issues  Finance Engineer – Leads the team in tracking expenses and keeping the team on budget  Recorder – Leads the team in recording data from the trials UNCLASSIFIED / FOUO 8
  9. 9. UNCLASSIFIED / FOUO Exercise: Phase One Deliverables  Prepare a Phase One Report showing:  Recommendation for optimal design  Predicted flight time at optimal setting  How much money was spent  Description of experimental strategy used  Description of analysis techniques used  Recommendations for future tests UNCLASSIFIED / FOUO 9
  10. 10. UNCLASSIFIED / FOUO Introduction to DOE UNCLASSIFIED / FOUO 10
  11. 11. UNCLASSIFIED / FOUO How Do We Learn? 1. Significant Event 2. Somebody Sees It 3. Research How can we learn more efficiently? Basic Design of Experiments UNCLASSIFIED / FOUO 11
  12. 12. UNCLASSIFIED / FOUO How Do We Learn? (Cont.)  Products and processes are continually providing data that could lead to their improvement - so what has been missing? There are several possibilities:  We are not collecting and analyzing the data provided  We are not proactive in data collection  We are unable to translate the data into information  A significant event has not occurred “In order to learn, two things must occur simultaneously: something must happen (informative event) and someone must see it happen (perceptive observer).” – George Box Basic Design of Experiments UNCLASSIFIED / FOUO 12
  13. 13. UNCLASSIFIED / FOUO How Do We Improve?  By creating significant events and observing them, we can obtain knowledge faster  That is basically what occurs in a designed experiment  Let‟s look at an example of these two things occurring (significant event and perceptive observer) simultaneously Basic Design of Experiments UNCLASSIFIED / FOUO 13
  14. 14. UNCLASSIFIED / FOUO Champagne Example  Wine – The fermented juice of fresh grapes used as a beverage. Wine has been in existence since the beginning of recorded history  Champagne – A clear, sparkling liquid made by way of the second fermentation of wine. First discovered by a French monk in the late 1600s Basic Design of Experiments UNCLASSIFIED / FOUO 14
  15. 15. UNCLASSIFIED / FOUO Need Improved Observation  Need to make sure that naturally occurring informative events are brought to the attention of the perceptive observer!  Improved observation increases the probability of observing naturally occurring informative events so appropriate action can be taken. SPC tools and techniques improve observation, but we must wait for an event to happen in order to observe it Basic Design of Experiments UNCLASSIFIED / FOUO 15
  16. 16. UNCLASSIFIED / FOUO Passive Observation Is Not Enough  We need to induce occurrence of informative events.  An experiment is set-up so that an informative event will occur! Designed Experimentation – The manipulation of controllable factors (independent variables) at different levels to see their effect on some response (dependent variable)  By manipulating inputs to see how the output changes, we can understand and model Y (a dependent variable) as a function of X (an independent variable). Basic Design of Experiments UNCLASSIFIED / FOUO 16
  17. 17. UNCLASSIFIED / FOUO What Is Experimental Design? Inputs (Factors) Outputs (Responses) People Responses related to Material producing a product Equipment Experimental Process: A controlled blending Responses related to Policies of inputs which completing a task Procedures generates corresponding measurable outputs. Responses related to Methods performing a service EnvironmentFrom Understanding Industrial Designed Experiments, Schmidt & Launsby Basic Design of Experiments UNCLASSIFIED / FOUO 17
  18. 18. UNCLASSIFIED / FOUO Example of a Recruiting Process Inputs Outputs (Factors) (Responses) Job Description Marketing Candidate Pool Economic Environment Process: Recruiting Type of Job Location of Job Hire Quickly Job Application Process Hire Best Candidate EEO Requirements Hire at Competitive Pay DOE was originally used for manufacturing quality applications - it has now expanded to many other areas where performance characteristics are of interest Basic Design of Experiments UNCLASSIFIED / FOUO 18
  19. 19. UNCLASSIFIED / FOUO Methods of Experimentation  Experimentation has been used for a long time. Some experiments have been good, some not so good  Our early experiments can be grouped into the following general categories: 1. Trial and Error 2. One-Factor-at-a-Time (OFAT) 3. Full Factorial 4. Fractional Factorial 5. Others Basic Design of Experiments UNCLASSIFIED / FOUO 19
  20. 20. UNCLASSIFIED / FOUO Trial and Error - Increase Gas Mileage  Problem: Gas mileage for car is 20 mpg. Would like to get > 30 mpg.  Factors:  Change brand of gas  Change octane rating  Drive slower  Tune-up car  Wash and wax car  New tires  Change tire pressure  Remove hood ornament and external radio antenna Basic Design of Experiments UNCLASSIFIED / FOUO 20
  21. 21. UNCLASSIFIED / FOUO One-Factor-at-a-Time (OFAT)- Gas Mileage  Problem: Gas mileage for vehicle is 20 mpg. Would like to get > 30 mpg MPG Results Speed Octane Tire Pressure Miles per Gallon 55 85 30 25 65 85 30 23 55 91 30 27 55 85 35 27  How many more runs would you need to figure out the best configuration of variables?  How can you explain the above results?  If there were more variables, how long would it take to get a good solution?  What if there‟s a specific combination of two or more variables that leads to the best mileage (the optimum)? Basic Design of Experiments UNCLASSIFIED / FOUO 21
  22. 22. UNCLASSIFIED / FOUO Results Miles per Gallon as a Function of Speed and Tire Pressure 75 17 70  How would we find 65 Speed (mph) 26 this optimum with 60 OFAT testing? 55 18 23 26 26 20 50 26  How would we 45 know that we‟d 18 found it? 40 35 35 30 Optimum MPG 26 28 30 32 34 36 38 Tire Pressure (lbs.) Basic Design of Experiments UNCLASSIFIED / FOUO 22
  23. 23. UNCLASSIFIED / FOUO One-Factor-at-a-Time (OFAT)  While OFAT is simple, it is inefficient in determining optimal results:  Unnecessary experiments may be run  Time to find causal factors is significant  Don‟t know the effects of changing one factor while other factors are also changing (no model)  Inability to detect or learn about how factors work together to drive the response Is there a better way? Basic Design of Experiments UNCLASSIFIED / FOUO 23
  24. 24. UNCLASSIFIED / FOUO Cake Example - Interactions  An Interaction occurs when the effect of one factor, X1, on the response, Y, depends on the setting (level) of another factor, X2: Y = f(x)  For example, when baking a cake, the temperature that you set the oven at is dependent on the time that the cake will be in the oven. Basic Design of Experiments UNCLASSIFIED / FOUO 24
  25. 25. UNCLASSIFIED / FOUO Cake Example - Interactions  Where would you set Time to get a good cake?  How would you experiment on this process to learn about this interaction? Temp = 100 degrees Temp = 500 degrees Time = 45 minutes Time = 20 minutes Time Temp = 100 degrees Temp = 500 degrees Time = 20 minutes Time = 45 minutes 100 Temp 500 Basic Design of Experiments UNCLASSIFIED / FOUO 25
  26. 26. UNCLASSIFIED / FOUO Cake Example - Interactions  Duncan Hines used designed experiments in the 50‟s on their cake mixes.  Their goal was a robust design for the most consistent product. Basic Design of Experiments UNCLASSIFIED / FOUO 26
  27. 27. UNCLASSIFIED / FOUO Why Use DOE? “Often we have used a trial and error approach to testing, or just changed one variable at a time. Why is a statistically designed experiment better?”  The structured methodology provides a directed approach to avoid time wasted with “hunt and peck” - don‟t need 30 years of experience to design the tests  The designed experiment gives a mathematical model relating the variables and responses - no more experiments where you can‟t draw conclusions  The model is easily optimized, so you know when you‟re done  The statistical significance of the results is known, so there is much greater confidence in the results  Can determine how multiple input variables interact to affect results Basic Design of Experiments UNCLASSIFIED / FOUO 27
  28. 28. UNCLASSIFIED / FOUO Full Factorial DOE  Full Factorial examines every possible combination of factors at the levels tested. The full factorial design is an experimental strategy that allows us to answer most questions completely.  Full factorial enables us to:  Determine the Main Effects that the factors being manipulated have on the response variable(s)  Determine the effects of factor interactions on the response variables  Estimate levels at which to set factors for best results Basic Design of Experiments UNCLASSIFIED / FOUO 28
  29. 29. UNCLASSIFIED / FOUO Full Factorial Minimum number of tests for a full factorial experiment: Xk X = # of levels, k = # of factors Factors Level 2 3 4 2 4 8 16 # of Tests 3 9 27 81 Adding another level significantly increases the number of tests!  Full Factorial Advantages  Information about all effects  Information about all interactions What can we do when  Quantify Y=f(x) resources are limited?  Limitations  Amount of resources needed  Amount of time needed Basic Design of Experiments UNCLASSIFIED / FOUO 29
  30. 30. UNCLASSIFIED / FOUO Full Factorial Notation  2 level designs are the most common because they provide a lot of information, but require the fewest tests.  The general notation for a full factorial design of 2 levels is: 2k = # Runs  2 is the number of levels for each factor (Range = High and Low)  k is the number of factors to be investigated  This is the minimum number of test runs required for a full factorial Basic Design of Experiments UNCLASSIFIED / FOUO 30
  31. 31. UNCLASSIFIED / FOUO Full Factorial Experiment Problem: Gas Mileage is 20 mpg Speed Octane Tire Pressure Miles per Gallon 55 85 30 25 65 85 30 23 55 91 30 27 65 91 30 23 55 85 35 27 65 85 35 24 55 91 35 32 65 91 35 25 OFAT Runs Do we think 32 is best?  What conclusion do you make now?  How many runs? MPG = f(Speed, Octane, Tire Pressure)  How many runs at each level? Basic Design of Experiments UNCLASSIFIED / FOUO 31
  32. 32. UNCLASSIFIED / FOUO Fractional Factorial  Looks at only a fraction of all the possible combinations contained in a full factorial.  If many factors are being investigated, information can be obtained with smaller investment.  Resources necessary to complete a fractional factorial are manageable.  Limitations - give up some interactions  Benefits  Economy  Speed  Fewer runs Basic Design of Experiments UNCLASSIFIED / FOUO 32
  33. 33. UNCLASSIFIED / FOUO Fractional Factorial Notation  The general notation to designate a fractional factorial design is: k p 2R = # Runs  2 is the number of levels for each factor  k is the number of factors to be investigated  2-p is the size of the fraction (p = 1  1/2 fraction, p = 2  1/4 fraction, etc.)  2k-p is the number of runs  R is the resolution Basic Design of Experiments UNCLASSIFIED / FOUO 33
  34. 34. UNCLASSIFIED / FOUO Fractional Factorial Notation – Resolution When we go to a fractional factorial design, we are not able to estimate all of the interactions The amount that we are able to estimate is indicated by the resolution of an experiment The higher the resolution, the more interactions we can measure  Example: The designation below means fifteen factors will be investigated in 16 runs. This design is a resolution III: 1511 2 III Note: A deeper discussion of design resolution is beyond the scope of the lesson. The content, above, is intended to only provide a brief explanation of the design resolution term. Basic Design of Experiments UNCLASSIFIED / FOUO 34
  35. 35. UNCLASSIFIED / FOUO Gas Mileage Example  Problem: Gas mileage for vehicle is 20 mpg Speed Octane Tire Pressure Mileage (A) (B) (C) (Y) 55 85 35 27 65 85 30 23 55 91 30 27 65 91 35 25  Compare with previous full factorial:  How many runs?  How much information? Basic Design of Experiments UNCLASSIFIED / FOUO 35
  36. 36. UNCLASSIFIED / FOUO DOE Will Help Us Identify Factors  Factors which shift the average A1 A2  Longer line = greater effect Main Effects Plot (data means) for Mileage 55 65 85 91 30 35 27.8 26.8 In the gas mileage Mileage 25.8 example, Speed, Octane, and Tire 24.8 Pressure all look to have an effect on 23.8 average mileage. Speed Octane Tire Pressure Basic Design of Experiments UNCLASSIFIED / FOUO 36
  37. 37. UNCLASSIFIED / FOUO DOE Will Help Us Identify Factors  Factors which affect variation B2 B1  Flat line = no effect Main Effects Plot for Standard Deviation 55 65 85 91 30 35 3.0 2.5 Only Tire Pressure Standard Dev is affecting standard 2.0 1.5 deviation 1.0 Speed Octane Tire Pressure Basic Design of Experiments UNCLASSIFIED / FOUO 37
  38. 38. UNCLASSIFIED / FOUO DOE Will Help Us Identify Factors  Factors which shift the average and C2 affect variation C1 Main Effects Plot (data means) for Mileage Main Effects Plot for Standard Dev 55 65 85 91 30 35 55 65 85 91 30 35 27.8 3.0 26.8 2.5 Standard Dev Mileage 25.8 2.0 24.8 1.5 23.8 1.0 Speed Octane Tire Pressure Speed Octane Tire Pressure Only Tire Pressure affects both the average mileage and also the variability Basic Design of Experiments UNCLASSIFIED / FOUO 38
  39. 39. UNCLASSIFIED / FOUO DOE Will Help Us Identify Factors  Factors which have no effect D1 = D2 Main Effects Plot (data means) for Mileage Main Effects Plot for Standard Dev -1 1 -1 1 -1 1 -1 1 28 3.0 27 2.5 Standard Dev Mileage 26 2.0 25 1.5 24 1.0 Driver Radio Driver Radio An expanded study investigated the effect of driver and radio on mileage. These factors show no effect. This is also valuable information, because these factors can be set at their most economical (least cost) or most convenient levels. Basic Design of Experiments UNCLASSIFIED / FOUO 39
  40. 40. UNCLASSIFIED / FOUO Benefits of DOE  Determine input settings which optimize results and minimize costs  Quick screening for significant effects  Obtain a mathematical model relating inputs and results  Reduction in the number of tests required  Verification of the statistical significance of results  Identification of low-impact areas allows for increased flexibility/tolerances  Standardized methodology provides a directed approach Basic Design of Experiments UNCLASSIFIED / FOUO 40
  41. 41. UNCLASSIFIED / FOUO When Could I Use Design of Experiments?  Identification of critical factors to improve performance  Identification of unimportant factors to reduce costs  Reduction in cycle time  Reduction of scrap/rework  Scientific method for setting tolerances  Whenever you see repetitive testing Basic Design of Experiments UNCLASSIFIED / FOUO 41
  42. 42. UNCLASSIFIED / FOUO DOE Review  What does DOE offer us that trial and error experimentation and OFAT do not?  What are the differences between full and fractional factorial DOE‟s?  What is the minimum number of runs required for a 2-level, 3-factor full factorial experiment? Basic Design of Experiments UNCLASSIFIED / FOUO 42
  43. 43. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  A contract airline is interested in reducing overall late take-off time in order to improve Soldier satisfaction  Previous Black Belt work has identified 4 key process input variables (KPIVs) that affect late time:  Dollars spent on training  Number of jets  Number of employees  % Overbooked Basic Design of Experiments UNCLASSIFIED / FOUO 43
  44. 44. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Using the Airline DOE Data.mpj file and Minitab, the instructor will walk the class through the following activities:  A DOE to identify which factors affect “minutes late” in terms of both the mean and standard deviation  Use the DOE results to determine new process settings  Hypothesis test to prove statistical significance of change. Basic Design of Experiments UNCLASSIFIED / FOUO 44
  45. 45. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Current settings for these factors are as follows:  Dollars spent on training 200  Number of jets 52  Number of employees 850  % Overbooked 15  The target is zero minutes late, with a specification of +/- 10 minutes. Basic Design of Experiments UNCLASSIFIED / FOUO 45
  46. 46. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Our goal in this experiment is to reduce late take- off times - we will measure late time in minutes  Here are the factors and their levels that we are going to investigate: Factors Levels  Dollars spent on training 100 300  Number of Jets 50 55  Number of Employees 800 900  % Overbooked 0 25 Basic Design of Experiments UNCLASSIFIED / FOUO 46
  47. 47. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  First, we need to set up the test matrix  Select Stat>DOE>Factorial>Create Factorial Design Basic Design of Experiments UNCLASSIFIED / FOUO 47
  48. 48. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  We left the default at: 2-level factorial design. That means we will test each factor at 2 different levels  We also selected 4 factors, since there are 4 variables that we want to test in this experiment. Select Display Available Designs to display possible experiments we can run… Basic Design of Experiments UNCLASSIFIED / FOUO 48
  49. 49. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  For 4 factors, we can either do an 8 run half-fraction or a 16 run full factorial. We will go with the 16 run full factorial experiment.  Click OK to return to the previous dialog box. Basic Design of Experiments UNCLASSIFIED / FOUO 49
  50. 50. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Click the Designs button and highlight the 16-run Full Factorial design.  Leave the other settings at their defaults, click on OK. Basic Design of Experiments UNCLASSIFIED / FOUO 50
  51. 51. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Next, click the Factors button  Enter the names of the Factors – Change -1 and 1 to actual levels per chart below Factors Levels Dollars spent on training 100 300 Number of Jets 50 55 Number of Employees 800 900 Basic Design of Experiments UNCLASSIFIED / FOUO 51
  52. 52. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Click the Options button and uncheck Randomize runs.  We do want to randomize our tests when we actually run an experiment. However, for this in-class demo, it will be easiest if everyone‟s screen is the same. Basic Design of Experiments UNCLASSIFIED / FOUO 52
  53. 53. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Here are the tests that we need to run. Example, row 1 indicates that we first need to collect data at the low level for all four factors. (Tip: First check that you have the same test matrix. If you don‟t, it‟s likely that you did not uncheck “Randomize Runs.”) Basic Design of Experiments UNCLASSIFIED / FOUO 53
  54. 54. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Next, we collect the data.  To allow us to measure variation, we need to run 3 repetitions at each set of settings.  Copy the data from the DOE data worksheet and paste into the design as shown below. Min Late 1 = C9 Min Late 2 = C10 Min Late 3 = C11 Basic Design of Experiments UNCLASSIFIED / FOUO 54
  55. 55. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Run #1, with settings of $100 spent on training, 50 jets, 800 employees, and 0% overbooked, was 46.35 minutes late on the first repetition, 61.92 minutes late on the second repetition, and 75.18 minutes late on the third repetition. $100.00 50 Jets 800 Employees 0% Overbooked Note: This row of coded variables were all at their Low (or –1) settings Basic Design of Experiments UNCLASSIFIED / FOUO 55
  56. 56. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Since we ran our DOE with 3 repetitions, and we want to analyze the variation in our DOE results, we need to prepare the worksheet by having Minitab calculate means and standard deviations.  First, we need to name some blank columns. Name a blank column StdDev Min Late, a second blank column Count Min Late, and a third blank column Mean Min Late. Basic Design of Experiments UNCLASSIFIED / FOUO 56
  57. 57. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Now, we will have Minitab calculate the means and standard deviations.  Select Stat>DOE>Factorial>Pre-Process Responses for Analyze Variability Basic Design of Experiments UNCLASSIFIED / FOUO 57
  58. 58. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Click on Compute for repeat responses across rows, then click in the cell under Repeat responses across rows of: and select Min late 1, Min late 2, and Min late 3 from the columns pane.  Click in the box Store standard deviations in and select the column you named StdDev Min Late  Click in the box Store number of repeats in and select the column you named Count Min Late  Click in the box Store Means (optional) in and select the column you named Mean Min Late  Click OK Basic Design of Experiments UNCLASSIFIED / FOUO 58
  59. 59. UNCLASSIFIED / FOUO Minitab: Airline DOE Example Minitab calculates means and standard deviations for each combination of factors. (Remember: there were 24, or 16, combinations.) Minitab also determines the counts. (Remember: there were 3 data points at each combination, since we ran 3 repetitions at each setting of the DOE.) Looking at this data Practically, there appears to be some significance to the factors, but nothing definitive…yet. Basic Design of Experiments UNCLASSIFIED / FOUO 59
  60. 60. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Before we view the statistics, we always start with the graphs.  Select Stat>DOE>Factorial>Factorial Plots. Basic Design of Experiments UNCLASSIFIED / FOUO 60
  61. 61. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Select Main Effects Plot.  Choose Setup and click on OK to go to next dialog box Basic Design of Experiments UNCLASSIFIED / FOUO 61
  62. 62. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Select StdDev Min Late and Mean Min Late for Responses, and move all four factors from Available to the Selected box to have them included in the analysis. Click on OK. > Basic Design of Experiments UNCLASSIFIED / FOUO 62
  63. 63. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  The Main Effects Plot shows that the number of Employees is the only driver for Mean Min Late Main Effects Plot for Mean Min Late Data Means Training Dollars Jets 60 45 30 15 0 Mean -1 1 -1 1 Employees %Overbooked 60 45 30 15 0 -1 1 -1 1 Looking at this data Graphically, it appears that Employees might be a significant factor influencing the Mean of Time Late Basic Design of Experiments UNCLASSIFIED / FOUO 63
  64. 64. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  The Main Effects Plot shows that the number of Jets AND Employees are driving the StdDev Min Late Main Effects Plot for StdDev Min Late Data Means Training Dollars Jets 8 6 4 2 Mean -1 1 -1 1 Employees % Overbooked 8 6 4 2 -1 1 -1 1 Looking at this data Graphically, it appears that Jets AND Employees might be a significant factor influencing the StdDev of Time Late Basic Design of Experiments UNCLASSIFIED / FOUO 64
  65. 65. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Now we will run the analysis.  Select Stat>DOE>Factorial>Analyze Factorial Design Basic Design of Experiments UNCLASSIFIED / FOUO 65
  66. 66. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Select Mean Min Late and StdDev Min Late as the Response.  Choose Terms to get to next dialog box Basic Design of Experiments UNCLASSIFIED / FOUO 66
  67. 67. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Include Terms in the model up through second order (2). This will include the main effects and two-way interactions. Click on OK to go back to previous dialog box. Basic Design of Experiments UNCLASSIFIED / FOUO 67
  68. 68. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Click on Graphs and select the Pareto Chart. Click OK in both dialog boxes. Basic Design of Experiments UNCLASSIFIED / FOUO 68
  69. 69. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  This chart confirms what we saw earlier in the Main Effects Plot – the number of Employees has a significant impact on Mean Min Late. We also see that the interaction term BD* is significant. This is the „Critical F-statistic‟ used to determine significance. Pareto Chart of the Standardized Effects (response is Mean Min Late, Alpha = 0.05) 2.57 F actor N ame C A Training D ollars B Jets BD C E mploy ees D % O v erbooked D AD BC Term B * - BD is the interaction A between the factors Jets AB and %Overbooked. Pareto Chart for AC Mean Min Late CD 0 10 20 30 40 50 60 70 80 90 Standardized Effect Basic Design of Experiments UNCLASSIFIED / FOUO 69
  70. 70. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  This chart confirms only part of what we saw earlier in the Main Effects Plot – the number of Jets has a significant impact on StdDev Min Late but Employees does not. Pareto Chart of the Standardized Effects (response is StdDev Min Late, Alpha = 0.05) 2.571 F actor N ame B A Training Dollars B Jets AD C E mploy ees D % O v erbooked C AC BC Term AB BD Pareto Chart for CD StdDev Min Late D A 0 1 2 3 4 Standardized Effect Basic Design of Experiments UNCLASSIFIED / FOUO 70
  71. 71. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  In the Session window, we see that Employees and the interaction Jets*%Overbooked are the only statistically significant factors for Mean Min Late. All other main effects and 2-way interactions have a p-value > 0.05. Factorial Fit: Mean Min Late versus Training Dollars, Jets, ... Estimated Effects and Coefficients for Mean Min Late (coded units) This data shows Term Effect Coef SE Coef T P Constant 30.42 0.3191 95.34 0.000 Analytically that Training Dollars Jets -0.71 -0.77 -0.35 -0.39 0.3191 0.3191 -1.11 -1.21 0.319 0.279 Employees and Employees -53.89 -26.94 0.3191 -84.44 0.000 the Jets*% %Overbooked -1.28 -0.64 0.3191 -2.01 0.101 Training Dollars*Jets 0.68 0.34 0.3191 1.06 0.337 Overbooked Training Dollars*Employees Training Dollars*%Overbooked 0.12 1.13 0.06 0.56 0.3191 0.3191 0.18 1.77 0.861 0.138 interaction are Jets*Employees -0.81 -0.41 0.3191 -1.27 0.259 statistically Jets*%Overbooked 2.14 1.07 0.3191 3.36 0.020 Employees*%Overbooked -0.10 -0.05 0.3191 -0.16 0.881 significant. S = 1.27632 PRESS = 83.4049 R-Sq = 99.93% R-Sq(pred) = 99.28% R-Sq(adj) = 99.79% Basic Design of Experiments UNCLASSIFIED / FOUO 71
  72. 72. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  In the Session window, we see that Jets is the only statistically significant factor for Stdev Min Late. The negative sign for Effect indicates that standard deviation decreases as Jets increases. All other main effects and 2-way interactions have a p-value > 0.05. Factorial Fit: StdDev Min Late versus Training Dollars, Jets, ... Estimated Effects and Coefficients for StdDev Min Late (coded units) Term Effect Coef SE Coef T P Constant 4.575 0.7241 6.32 0.001 Training Dollars -0.105 -0.052 0.7241 -0.07 0.945 Jets -5.930 -2.965 0.7241 -4.09 0.009 Employees -2.477 -1.239 0.7241 -1.71 0.148 % Overbooked 0.146 0.073 0.7241 0.10 0.923 Training Dollars*Jets -0.987 -0.493 0.7241 -0.68 0.526 Training Dollars*Employees 2.032 1.016 0.7241 1.40 0.219 Training Dollars*% Overbooked 2.786 1.393 0.7241 1.92 0.112 Jets*Employees 1.655 0.828 0.7241 1.14 0.305 Jets*% Overbooked -0.935 -0.468 0.7241 -0.65 0.547 Employees*% Overbooked 0.932 0.466 0.7241 0.64 0.548 S = 2.89641 PRESS = 429.527 R-Sq = 84.84% R-Sq(pred) = 0.00% R-Sq(adj) = 54.52% Basic Design of Experiments UNCLASSIFIED / FOUO 72
  73. 73. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Summarizing what we have found in the initial analysis:  Employees and the interaction between Jets and %Overbooked had a significant impact on Mean Min Late.  As seen from the Effects, increasing Jets or %Overbooked decreases Mean Min Late.  In addition, when the product of Jets*%Overbooked is positive, Mean Min Late will increase. If the product is negative, Mean Min Late will decrease. Term Effect Coef SE Coef T P Employees -53.89 -26.94 0.3191 -84.44 0.000 Jets -0.77 -0.39 0.3191 -1.21 0.279 %Overbooked -1.28 -0.64 0.3191 -2.01 0.101 Jets*%Overbooked 2.14 1.07 0.3191 3.36 0.020  Jets had a significant impact on Stdev Min Late. As seen from its Effect, increasing Jets decreases Stdev. Term Effect Coef SE Coef T P Jets -5.930 -2.965 0.7241 -4.09 0.009 Basic Design of Experiments UNCLASSIFIED / FOUO 73
  74. 74. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  The next step in the DOE analysis is to eliminate the insignificant terms. This is called “reducing the model.”  Every study is different; in this particular case, let‟s take the following approach:  Reduce the StdDev model to identify the needed setting for „Jets‟ since it:  is the only significant factor influencing StdDev  plays only a small role in driving the Mean (Coef = -0.39)  will determine where to set the factor % Overbooked in the interaction term. Basic Design of Experiments UNCLASSIFIED / FOUO 74
  75. 75. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Select Stat>DOE>Factorial>Analyze Factorial Design Basic Design of Experiments UNCLASSIFIED / FOUO 75
  76. 76. UNCLASSIFIED / FOUO Minitab: Airline DOE Example 1. Click on Terms 2. Remove all Selected Terms: except B:Jets 3. Select OK and OK 2 1 3 Basic Design of Experiments UNCLASSIFIED / FOUO 76
  77. 77. UNCLASSIFIED / FOUO Minitab: Airline DOE Example The mathematical model is taken from the „Coef‟ column of the Session Window: StdDev Min Late* = 4.575 – (2.965 x Jets) Factorial Fit: StdDev Min Late versus Jets Estimated Effects and Coefficients for StdDev Min Late (coded units) Term Effect Coef SE Coef T P Constant 4.575 0.7792 5.87 0.000 Jets -5.930 -2.965 0.7792 -3.81 0.002 S = 3.11695 PRESS = 177.653 R-Sq = 50.84% R-Sq(pred) = 35.79% R-Sq(adj) = 47.33% Conclusion: To reduce StdDev Min Late, we should set the factor „Jets‟ to the +1 level (55). Basic Design of Experiments UNCLASSIFIED / FOUO 77
  78. 78. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Now, let‟s reduce the model for the response, Mean.  Select Stat>DOE>Factorial>Analyze Factorial Design Basic Design of Experiments UNCLASSIFIED / FOUO 78
  79. 79. UNCLASSIFIED / FOUO Minitab: Airline DOE Example 1. Click on Terms 2. Remove all Selected Terms: except B:Jets, C: Employees, D: % Overbooked and the interaction BD. 3. Select OK and OK 2 1 3 Basic Design of Experiments UNCLASSIFIED / FOUO 79
  80. 80. UNCLASSIFIED / FOUO Minitab: Airline DOE Example The mathematical model is taken from the „Coef‟ column of the Session Window: Mean Min Late = 30.42 – (0.39 x Jets) – (26.94 x Employees) –(0.64 x %Overbooked) + (1.07 x Jets x %Overbooked) Factorial Fit: Mean Min Late versus Jets, Employees, %Overbooked Estimated Effects and Coefficients for Mean Min Late (coded units) Term Effect Coef SE Coef T P Constant 30.42 0.3354 90.71 0.000 Jets -0.77 -0.39 0.3354 -1.15 0.273 Employees -53.89 -26.94 0.3354 -80.34 0.000 %Overbooked -1.28 -0.64 0.3354 -1.91 0.082 Jets*%Overbooked 2.14 1.07 0.3354 3.19 0.009 S = 1.34148 PRESS = 41.8811 R-Sq = 99.83% R-Sq(pred) = 99.64% R-Sq(adj) = 99.77% Basic Design of Experiments UNCLASSIFIED / FOUO 80
  81. 81. UNCLASSIFIED / FOUO Minitab: Airline DOE Example  Of the four factors that were investigated, two (Employees and Jets), plus the Jets*%Overbooked interaction, were significant.  Jets – to reduce variation, we need to increase Jets to 55.  Employees - to reduce the average late time from 30 minutes, we need to increase Employees from 850 to 900.  Training Budget - had no effect, and can be reduced to $100k as a budget savings.  % Overbooked - had marginal effect on time late and on variation – should be reduced to 0% to improve customer satisfaction.  What would you do if this were your organization? Basic Design of Experiments UNCLASSIFIED / FOUO 81
  82. 82. UNCLASSIFIED / FOUO Is the Change Significant? 1. Conduct a hypothesis test. 2. Open the Capability Data worksheet within the Airline DOE Data.mpj file. 3. Since we have the baseline sample and the improved sample, select Stat>Basic Statistics>2-Sample t… Basic Design of Experiments UNCLASSIFIED / FOUO 82
  83. 83. UNCLASSIFIED / FOUO Is the Change Significant? (continued) 4. Select „Samples in different columns‟ 5. Select „Baseline Data‟ for First: and „New Data‟ for Second: 6. Select „Graphs…‟ 4 5 6 Basic Design of Experiments UNCLASSIFIED / FOUO 83
  84. 84. UNCLASSIFIED / FOUO Is the Change Significant? (continued) 7. Select Boxplots of data 8. Click on OK 9. Interpret boxplot. Does there appear to be a graphical difference? 7 50 Boxplot of Baseline Data, New Data 40 30 Data 20 8 9 10 0 Baseline Data New Data Basic Design of Experiments UNCLASSIFIED / FOUO 84
  85. 85. UNCLASSIFIED / FOUO Is the Change Significant? (continued) 10. Review Minitabs Session Window output. 11. Can we state, with 95% confidence, that there is a statistical difference between our Baseline Data and the New data? (i.e. Does the Confidence Interval contain „0‟ or is the P-value less than 0.05?) 10 11 Basic Design of Experiments UNCLASSIFIED / FOUO 85
  86. 86. UNCLASSIFIED / FOUO Helicopter Simulation Phase Two UNCLASSIFIED / FOUO 86
  87. 87. UNCLASSIFIED / FOUO Exercise: Helicopter Simulation  Customers at CHI (Cellulose Helicopters Inc.) have been complaining about the limited flight time of CHI helicopters  Management wants to increase flight time to improve customer satisfaction  You are put in charge of this improvement project UNCLASSIFIED / FOUO 87
  88. 88. UNCLASSIFIED / FOUO Exercise: Constraints  Project Mission: Find the combination of factors that maximize flight time  Project Constraints:  Budget for testing = $1.5 M  Cost to build one prototype = $100,000  Cost per flight test = $10,000  Prototype once tested can not be altered  See allowable flight test factors and parameters on the next page UNCLASSIFIED / FOUO 88
  89. 89. UNCLASSIFIED / FOUO Exercise: Test Factors and Parameters  Paper Type Regular Card stock  Paper Clip No Yes  Taped Body No 3 in of tape  Taped Wing Joint No Yes  Body Width 1.42 in 2.00 in  Body Length 3.00 in 4.75 in  Wing Length 3.00 in 4.75 in UNCLASSIFIED / FOUO 89
  90. 90. UNCLASSIFIED / FOUO Exercise: Roles & Responsibilities  Lead Engineer – Leads the team and makes final decision on which prototypes to build and test  Test Engineer – Leads the team in conducting the test and has final say on how test are conducted  Assembly Engineer – Leads the team in building prototypes and has final say on building issues  Finance Engineer – Leads the team in tracking expenses and keeping the team on budget  Recorder – Leads the team in recording data from the trials UNCLASSIFIED / FOUO 90
  91. 91. UNCLASSIFIED / FOUO Exercise: Phase Two Deliverables  Prepare a Phase Two Report showing:  Recommendation for optimal design  Predicted flight time at optimal setting  How much money was spent  Description of experimental strategy used  Description of analysis techniques used  Recommendations for future tests  Comparison of Phase One and Phase Two approaches Which Team Has The Best Design? UNCLASSIFIED / FOUO 91
  92. 92. UNCLASSIFIED / FOUO Takeaways  Types of experiments – Trial and Error, OFAT, DOE  Introductory DOE terminology  Benefits of full factorial vs. fractional designs  How to use Minitab to design, run, and analyze a DOE  Use DOE results to drive statistical improvements Basic Design of Experiments UNCLASSIFIED / FOUO 92
  93. 93. UNCLASSIFIED / FOUO What other comments or questions do you have? UNCLASSIFIED / FOUO 93
  94. 94. UNCLASSIFIED / FOUO References  Schmidt & Launsby, Understanding Industrial Designed Experiments Basic Design of Experiments UNCLASSIFIED / FOUO 94

×