Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. DX8-02A-Gen1Factor-P1.docx Rev. 12/8/09General One-Factor Tutorial(Part 1 – The Basics)Introduction In this tutorial you will build a general one-factor design using Design-Expert® software. This type of design is very useful for simple comparisons of categorical treatments, such as: Who will be the best supplier, Which type of raw material should be selected, What happens when you change procedures for processing paperwork. If you wish to experiment on a continuous factor, such as time, which can be adjusted to any numerical level, consider using response surface methods (RSM) instead. This is covered in a separate tutorial. The data for this example come from the Stat-Ease bowling league. Three bowlers (Pat, Mark, and Shari) are competing for the last team position. They each bowl six games in random order – ideal for proper experimentation protocol. Results are: Game Pat Mark Shari 1 160 165 166 2 150 180 158 3 140 170 145 4 167 185 161 5 157 195 151 6 148 175 156 Mean 153.7 178.3 156.2 Bowling scores Being a good experimenter, the team captain knows better than to simply pick the bowler with the highest mean score. The captain needs to know if the average scores are significantly different, given the variability in individual games. Maybe it’s a fluke that Mark’s score is highest. This one-factor case study provides a good introduction to the power of simple comparative design of experiments (DOE). It exercises many handy features found in Design-Expert software. We won’t explain all features displayed in this current exercise because most will be covered in later tutorials. Many other features and outputs are detailed only in the help system, which you can access by clicking Help in the main menu, or in most places via a right click, or by pressing the F1 key (context sensitive).Design-Expert 8 User’s Guide General One-Factor Tutorial – Part 1 1
  2. 2. Design the Experiment We will assume that you are familiar with your computer’s graphical user interface and your mouse. Start the program by double clicking the Design-Expert icon. You will then see the main menu and icon bar. Click on File in the main menu. Unavailable items are dimmed. (If you prefer using your keyboard, press the Alt key and underlined letter simultaneously, in this case Alt F.) File menu Select the New Design item with your mouse. (The blank-sheet icon  on the left of the toolbar is a quicker path to this screen. To try this, press Cancel to re- activate the tool bar.) Using either path, you now see four yellow tabs on the left of your screen. The Factorial tab comes up by default. Select General Factorial for this design because the factor is categorical. (If your factor is numerical, such as temperature, then you would use the One Factor option under the Response Surface tab.) Note the helpful description: “Design for 1 to 12 factors where each factor may have a different number of levels.” General Factorial design2 General One-Factor Tutorial – Part 1 Design-Expert 8 User’s Guide
  3. 3. DX8-02A-Gen1Factor-P1.docx Rev. 12/8/09Enter the Design Parameters Leave the number of factors at its default level of 1 but click the entry format Vertical (easier than Horizontal for multiple levels). Enter Bowler as the name of the factor. Tab down to the Units field and enter Person. Next tab to Type. For details on the options for factor type, click the light bulb icon ( ) in the toolbar to access our context-sensitive screen tips. Screen tips on factor Type Leaving Type at its default of Nominal, tab down to the Levels field and enter 3. Now tab to L(1) (level one) and enter Pat. Type Mark, and Shari for the other two levels (L2 and L3). General Factorial design-builder dialog box – completed Press Continue to specify the remaining design options. In the Replicates field, which becomes active by default, type 6 (each bowler rolls six games). Tab to the “Assign one block per replicate” field but leave it unchecked. Design-Expert now recalculates the number of runs for this experiment: 18. Design options entered Press Continue. Let’s do the easy things first. Leave the number of Responses at the default of 1. Now click on the Name box and enter Score. Tab to the Units field and enter Pins.Design-Expert 8 User’s Guide General One-Factor Tutorial – Part 1 3
  4. 4. Response name dialog box – completed At this stage you can skip the remainder of the fields and continue on. However, it is good to gain an assessment of the power of your planned experiment. In this case, as shown in the fields below, enter the value 10 because the bowling captain does not care if averages differ by fewer than 10 pins. Then enter the value 5 for standard deviation (derived from league records as the variability of a typical bowler). Design-Expert then compute a signal-to-noise ratio of 2 (10 divided by 5). Optional power calculator – necessary inputs entered Press Continue to view the happy outcome – power that exceeds 80 percent probability of seeing the desired difference. Results of power calculation Click on Continue for Design-Expert to create the design and take you to the design layout window. Before moving on, take a look at the unique branching interface provided by Design-Expert for the design and analysis of experiments and resulting optimization.4 General One-Factor Tutorial – Part 1 Design-Expert 8 User’s Guide
  5. 5. DX8-02A-Gen1Factor-P1.docx Rev. 12/8/09 Design-Expert software’s easy-to-use branching interface You will explore some branches in this series of tutorials and others if you progress to more advanced features, such as response surface methods for process optimization.Save the Design When you complete the design setup, save it to a file by selecting File, Save As. Type in the name of your choice (for this tutorial, we suggest Bowling) for your data file, which is saved as a *.dxp type. Save As dialog box Click on Save. Now you’re protected in case of a system crash.Design-Expert 8 User’s Guide General One-Factor Tutorial – Part 1 5
  6. 6. Create a Data Entry Form In the floating Design Tool click Run Sheet (or go to the View menu and select Run Sheet) to produce a recipe sheet for your experiment with your runs in randomized order. A printout provides space to write down the responses. (Note: this view of the data does not allow response entry. To type results into the program you must switch back to the home base – the Design Layout view.) Run Sheet view (your run order may differ) It’s not necessary for this tutorial, but if you have a printer connected, you can select File, Print, and OK (or click the printer icon) to make a hard copy. (You can do the same from the basic design layout if you like that format better.)Enter the Response Data When performing your own experiments, you will need to go out and collect the data. Simulate this by clicking File, Exit. Click on Yes if you are prompted to Save. Now re-start Design-Expert and use File, Open Design or click the open file icon on the toolbar)) to open your data file (Bowling.dxp). You should now see your data tabulated in the randomized layout. For this example, you must enter your data in the proper order to match the correct bowlers. To do this, right-click the Std column header and choose Sort by Standard Order. Sort runs by standard (std) order Now enter the responses from the table on page one, or use the following screen. Except for run order, your design layout window must look like that shown below.6 General One-Factor Tutorial – Part 1 Design-Expert 8 User’s Guide
  7. 7. DX8-02A-Gen1Factor-P1.docx Rev. 12/8/09 Design Layout in standard order with response data entered When you conduct your own experiment, be sure to do the runs and enter the response(s) in randomized order. Standard order should only be used as a convenience for entering pre-existing design data. If you are a real stickler for precision, replace (type over) your run numbers with the ones shown above, thus preserving the actual bowlers’ game sequence. Bowling six games is taxing but manageable for any serious bowler. However, short and random breaks while bowling six games protects against time-related effects such as learning curve (getting better as you go) and/or fatigue (tiring over time). Save your data by selecting File, Save from the menu (or via the save icon  on the toolbar). Now you’re backed up in case you mess up your data. This backup is good because now we’ll demonstrate many beneficial procedures Design-Expert features in its design layout. Start by pressing the screen tips button (or select Tips, Screen Tips). These tips, found throughout Design-Expert, clarify the screen currently displayed. Be sure to play the video clips that are offered for most of the detailed features – these movies make it far clearer than static, written tutorials can about how to use our advanced tools.Design-Expert 8 User’s Guide General One-Factor Tutorial – Part 1 7
  8. 8. Tips on features for Design Layout screen Try some features highlighted in the Design Layout tips. For example, right-click the Select button. This allows you to control what Design-Expert displays. Select button for choosing what you wish to display in the design layout In the comments column above we added a notation that after run 8, the bowling alley proprietor re-oiled the lane – for what that was worth. Seeing Pat’s scores, the effect evidently was negligible). Try this if you like. If comments exceed allotted space, double click the right border for automatic column re-sizing. That’s handy! Press the red X (close button) of any help screen to close it. Now, to better grasp the bowling results, order them from low-to-high as shown below by right-clicking the Response column header and selecting Sort by This Response.8 General One-Factor Tutorial – Part 1 Design-Expert 8 User’s Guide
  9. 9. DX8-02A-Gen1Factor-P1.docx Rev. 12/8/09 Sorting a response column (also works in the factor column) You’ll find sorting a very useful feature. It works on factors as well as responses. In this example, you quickly see that Mark bowled almost all the highest games.Analyze the Results Now we’ll begin data analysis. Under the Analysis branch of the program (on the left side of your screen), click the Score node. Transform options appear in the main window of Design-Expert on a progressive tool bar. You’ll click these buttons from left to right and perform the complete analysis. It’s a very easy process. The Transform option gives you the opportunity to select a transformation for the response. This may improve the analysis’ statistical properties. Transformation button – the starting point for the statistical analysis If you need some background on transformations, first try Tips. For complete details, go to the Help command on the main menu. Click the Search tab and enterDesign-Expert 8 User’s Guide General One-Factor Tutorial – Part 1 9
  10. 10. “transformations.” As shown at the bottom of the Transformscreen above, the program provides data-sensitive advice, so press ahead with the default of None by clicking the Effects button.Examine the Analysis By necessity, the tutorial now turns a bit statistical. If this becomes intimidating, we recommend you attend a basic class on regression, or better yet, a DOE workshop such as Stat-Ease’s computer-intensive Experiment Design Made Easy. Design-Expert may default to the mean (average) response as its model for prediction. If so, you will see “A-Bowler” established as an error (“e”) term as shown below. But if it’s already set up to be modeled (indicated by it being identified as “M”), then skip the next few instructions. Effect button result – default mean model Right-click the Bowler term and place this in your prediction model by clicking Model. Effects button results Focus on the F-value column and associated probability (“Prob>F”). In this case, there’s a very small probability, near 0.06% (p = 0.0006), that the differences in bowling averages (the term “A-Bowler” that you designated “M” for model) are due to chance variation (the term “Pure Error” labeled “e” for error, generated by the within-bowler multiple games). In other words, it appears at this stage that the difference between bowlers is significant. To get more statistical details, press the ANOVA (Analysis of Variance) button. Notice to the far right side of your screen that Design-Expert verifies that the results are significant.10 General One-Factor Tutorial – Part 1 Design-Expert 8 User’s Guide
  11. 11. DX8-02A-Gen1Factor-P1.docx Rev. 12/8/09 ANOVA results (annotated), with context-sensitive Help enabled via right-click menu Now select View, Annotated ANOVA from the menu atop the screen and uncheck () this option. Note that the blue textual hints and explanations disappear so you can make a clean printout for statistically savvy clients. Re-select View, Annotated ANOVA to ‘toggle’ back all the helpful hints. Before moving on, try the first hint shown in blue: “Use your mouse to right click on individual cells for definitions.” For example, perform this tip on the p-value of 0.0006 as shown above (select Help at the bottom of the pop-up menu). There’s a wealth of information to be brought up from within the program with a few simple keystrokes: Take advantage! Now click the ‘floating’ (moveable) R-squared Bookmark button (or press the scroll-down arrow at the bottom right screen) to see various summary statistics. Summary statistics These summary annotations reveal what you need to know, but don’t be shy about clicking on a value and getting online Help via a right-click (or try the F1 key). In most cases you will access helpful advice about the particular statistic. Now click the Coefficients Bookmark button to view the output illustrated below.Design-Expert 8 User’s Guide General One-Factor Tutorial – Part 1 11
  12. 12. Coefficient estimates Here you see statistical details such as coefficient estimates for each model terms and their confidence intervals (“CI”). The intercept in this simple one-factor comparative experiment is simply the overall mean score of the three bowlers. You may wonder why only two terms, A1 and A2, are provided for a predictive model on three bowlers. It turns out that the last model term, A3, is superfluous because it can be inferred once you know the mean plus the averages of the other two bowlers. Now let’s move on to the next section within this screen: “Treatment Means.” Treatment means Here are the averages for each of the three bowlers. As you can see below, these are compared via pair-wise t-tests in the following part of the ANOVA report. Treatment means You can conclude from the treatment comparisons that: Pat differs significantly (24.68 pins worse!) when compared with Mark (1 vs 2) The 2.5 pins mean difference between Pat and Shari (1 vs 3) is not significant (nor is it considered important by the bowling team’s captain –12 General One-Factor Tutorial – Part 1 Design-Expert 8 User’s Guide
  13. 13. DX8-02A-Gen1Factor-P1.docx Rev. 12/8/09 recall in the design specification for power that a 10-pin difference was the minimum of interest) Mark differs significantly (22.17 pins better!) when compared with Shari (2 vs 3). Before moving ahead, press Top on the floating Bookmark. This is a very handy way of moving through long reports, so it’s worth getting in the habit of using it.Analyze Residuals Click the Diagnostics button to bring up the normal plot of residuals. Ideally this will be a straight line, indicating no outlying abnormalities. If you have a pencil handy (or anything straight), hold it up to the graph. Does it loosely cover up all the points? The answer is “Yes” in this example – it passes the “pencil test” for normality. You can reposition the thin red line by dragging it (place the mouse pointer on the line, hold down the left button, and move the mouse) or its “pivot point” (the round circle in the middle). However, we don’t recommend you bother doing this – the program generally places the line in the ideal location automatically. If you need to re-set the line, simply double-click your left mouse button over the graph. Notice that the points are coded by color to the level of response they represent – going from cool blue for lowest values to hot red for the highest. In this example, the red point is Mark’s outstanding 195 game. Pat and Shari think Mark’s 195 game should be thrown out because it’s too high. Is this fair? Click this point so it will be highlighted on this and all the other residual graphs available via the Diagnostics Tool (the ‘floating’ palette on your screen). Normal probability plot of studentized residuals (195 game highlighted)Design-Expert 8 User’s Guide General One-Factor Tutorial – Part 1 13
  14. 14. Notice on the Diagnostics Tool that “Studentized” is checked on by default. This converts raw residuals, reported in original units (‘pins’ of bowling in this example), to dimensionless numbers based on standard deviations, which come out in plus or minus scale. More details on studentization reside in Help. Raw residuals can be displayed by un-checking this default studentized mode on the Diagnostics Tool. Check it out! However, when runs have greater leverage (another statistical term to look up in Help), only the Studentized form of residuals produces valid diagnostic graphs. In this example, if Pat and Shari succeed in getting Mark’s high game thrown out (don’t worry – they won’t!), then each of Mark’s remaining five games will exhibit a leverage of 0.2 (1/5) versus 0.167 (1/6) for each of the others’ six games. Due to potential imbalances of this sort, we advise that you always leave the Studentized feature checked (as done by default). On the Diagnostics Tool, select ei vs. Pred. to generate a plot of residuals for each individual game (the label “ei” referring to the error in fitting each individual result) versus what is predicted by the response model. [Sidebar: Supposedly, “residuals” were originally termed “error” by statisticians, but the management people got upset at so many mistakes being made!] Let’s make it easier to see which residual goes with which bowler by pressing the down-list arrow for the Color by option in the Diagnostics Tool and selecting A:Bowler. Residuals versus predicted values, colored by bowler The size of the studentized residual should be independent of its predicted value. In other words, the vertical spread of the studentized residuals should be approximately the same for each bowler. In this case the plot looks OK. Don’t be alarmed that Mark’s games stand out as a whole. The spread from bottom-to-top is not out of line with his competitors, despite their protestations about the highest score (still highlighted).14 General One-Factor Tutorial – Part 1 Design-Expert 8 User’s Guide
  15. 15. DX8-02A-Gen1Factor-P1.docx Rev. 12/8/09 Bring up the next graph on the Diagnostics Tool list – ei vs Run (residuals versus run number). (Note: your graph may differ due to randomization.) Residuals versus run chart (Note: your graph may differ due to randomization) Here you might see trends due to changing alley conditions (the lane re-oiling, for example), bowler fatigue, or other time-related lurking variables. (The pattern on your graph may differ from what we show here due to the randomized run order, but that will not be relevant to this discussion.) In this example things look relatively normal. However, even if you see a pronounced upward, downward, or shift change, it will probably not bias the outcome because the runs are completely randomized. To ensure against your experiment being sabotaged by uncontrolled variables, always randomize! At the top of the Diagnostics Tool, click the Influence button. The default is Ext. Student ei (externally studentized residuals), which show if any points stand out. Externally studentized residuals graph (your graph may differ due to random runs)Design-Expert 8 User’s Guide General One-Factor Tutorial – Part 1 15
  16. 16. On this graph, also known as “outlier t,” we are looking for points outside the plus and minus control limits. Again, for details on the meaning of external studentization, refer to Help, but briefly, this graph differs from the run chart in the previous figure by excluding each run, such as the one in question by Mark, prior to calculating the residual on a scale of standard deviation. In this case, all points fall within the limits (calculated at the 95 percent confidence level). In other words, Mark’s high game does not exhibit anything more than common-cause variability, so it should not be disqualified. Before moving on, at the bottom of the Diagnostics Tool, try this surprising feature: Pop-Out View. Then on the new Diagnostics Tool, go back to the Diagnostics button, click it, and re-select ei vs Run (residuals versus run number). Now you can see the two views, internal versus external, side by side. Pop-out view of residual versus run number Close the pop-out view by clicking the X in the upper right corner of the graph. Because there’s no indication of abnormality, it’s OK to move on to the model graph. This will reveal the outcome of a change in the bowling team membership. It will make a good final report to the Stat-Ease team on who they should invite as a new member. Unfortunately only one person can be chosen. View the Means and Data Plot Select the Model Graphs button from the progressive tool bar to display a plot containing all the response data and the average value at each level of the treatment (factor). This plot gives an excellent overview of the data and the effect of the factor levels on the mean and spread of the response. Note how conveniently Design-Expert scaled the Y axis from 140 to 200 pins in increments of 10.16 General One-Factor Tutorial – Part 1 Design-Expert 8 User’s Guide
  17. 17. DX8-02A-Gen1Factor-P1.docx Rev. 12/8/09 One-factor effects graph with Mark’s predicted score (mean) highlighted The squares in this effects plot represent predicted responses for each factor level (bowler). Click the square representing Mark’s top score. Notice that Design- Expert displays the prediction for this treatment level (reverting to DOE jargon) on the legend at the left of the graph. Vertical ‘I-beam-shaped’ bars represent the 95% least significant difference (LSD) intervals for each treatment. Mark’s LSD bars don’t overlap horizontally with Pat’s or Shari’s, so with at least 95% confidence, Mark’s mean is significantly higher than the means of the other two bowlers. Oh, by the way, maybe you noticed that the numerical value for the height of the LSD bar appeared when you clicked Mark’s square. You can also click on any round point to see the actual scores. Check it out! Pat and Shari’s LSD bars overlap horizontally, so we can’t say which of them bowls better. It seems they must spend a year in a minor bowling league and see if a year’s worth of games reveals a significant difference in ability. Meanwhile, Mark will be trying to live up to the high average he exhibited in the tryouts and thus justify being chosen for the Stat-Ease bowling team. That’s it for now. Save your results by going to File, Save. You can now Exit Design-Expert if you like, or keep it open and go on to the next tutorial – part two for general one-factor design and analysis. It delves into advanced features via further adventures in bowling.Design-Expert 8 User’s Guide General One-Factor Tutorial – Part 1 17