Your SlideShare is downloading. ×
0
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Exploring Best Practises in Design of Experiments
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Exploring Best Practises in Design of Experiments

1,301

Published on

Published in: Software, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,301
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
269
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Copyright © 2014, SAS Institute Inc. All rights reserved. Exploring Best Practises in Design of Experiments A Holistic Approach to DOE, Increasing Robustness, Efficiency and Effectiveness
  • 2. Copyright © 2014, SAS Institute Inc. All rights reserved. Contents  Background to DOE  Why Use DOE?  Tips for Effective DOE with Classical Designs  Definitive Screening  Case Studies 1-3  Role of Statistical Modelling and DOE in Learning  Holistic DOE  Case Study 4
  • 3. Copyright © 2014, SAS Institute Inc. All rights reserved. BACKGROUND TO DESIGN OF EXPERIMENTS (DOE)
  • 4. Copyright © 2014, SAS Institute Inc. All rights reserved. FATHER OF DOE RONALD A. FISHER Rothamstead Experimental Station, England – Early 1920’s
  • 5. Copyright © 2014, SAS Institute Inc. All rights reserved. FISHER’S FOUR DESIGN PRINCIPLES 1. Factorial Concept - rather than one-factor-at-a-time 2. Randomization - to avoid bias from lurking variables 3. Blocking - to reduce noise from nuisance variables 4. Replication - to quantify noise within an experiment
  • 6. Copyright © 2014, SAS Institute Inc. All rights reserved. AGRICULTURAL IMPACT US corn yields Cornell University, http://usda.mannlib.cornell.edu/MannUsda
  • 7. Copyright © 2014, SAS Institute Inc. All rights reserved. WHY USE DOE?
  • 8. Copyright © 2014, SAS Institute Inc. All rights reserved. Typical Process The properties of products and processes are often affected by many factors: In order to build new or improve products and processes, we must understand the relationship between the factors (inputs) and the responses (outputs). Typical Process Machine Operator Temperature Pressure Humidity Yield Cost … Inputs Factors Outputs Responses
  • 9. Copyright © 2014, SAS Institute Inc. All rights reserved. Traditional One-Factor-at-a-Time  A common approach is one-factor-at-a-time experimentation.  Consider experimenting one-factor-at-a-time to determine the values of temperature and time that optimise yield.
  • 10. Copyright © 2014, SAS Institute Inc. All rights reserved. Traditional One-Factor-at-a-Time
  • 11. Copyright © 2014, SAS Institute Inc. All rights reserved. Traditional One-Factor-at-a-Time
  • 12. Copyright © 2014, SAS Institute Inc. All rights reserved. Traditional One-Factor-at-a-Time
  • 13. Copyright © 2014, SAS Institute Inc. All rights reserved. Traditional One-Factor-at-a-Time
  • 14. Copyright © 2014, SAS Institute Inc. All rights reserved. Traditional One-Factor-at-a-Time  One-factor-at-a-time experimentation frequently leads to sub-optimal solutions.  Assumes the effect of one factor is the same at each level of the other factors, i.e. factors do not interact.  In practice, factors frequently interact.
  • 15. Copyright © 2014, SAS Institute Inc. All rights reserved. Interaction between factors
  • 16. Copyright © 2014, SAS Institute Inc. All rights reserved. Experimental Design  Most efficient way of investigating relationships.  Runs (factor combinations) chosen to maximize the information  Ideally balanced for ease of analysis and interpretation
  • 17. Copyright © 2014, SAS Institute Inc. All rights reserved. ITERATIVE AND SEQUENTIAL NATURE OF CLASSICAL DOE
  • 18. Copyright © 2014, SAS Institute Inc. All rights reserved. TIPS FOR EFFECTIVE DOE WITH CLASSICAL DESIGNS
  • 19. Copyright © 2014, SAS Institute Inc. All rights reserved. Stages of Experimental Design  Designing an experiment involves much more than just selecting the sequence of experimental runs:  Historically, improper planning is the most common cause of failed experiments. Plan Design Conduct Analyse Confirm
  • 20. Copyright © 2014, SAS Institute Inc. All rights reserved. Some Planning Steps  Review what we know • Have peer discussions  Determine new questions to answer  Identify factors and ranges to investigate  Define responses • Easy and precise to measure
  • 21. Copyright © 2014, SAS Institute Inc. All rights reserved. Common Experimental Objectives Identify Important Factors Screening Design Classical Fractional Factorial Optimise Process RSM Design Classical Central Composite Optimise Ingredients Mixtures Classical Simplex & Extreme Vertices
  • 22. Copyright © 2014, SAS Institute Inc. All rights reserved. Sequential Experimentation Reduces Total Cost Common Experimental Objectives Identify Important Factors Screening Design Classical Fractional Factorial Optimise Process RSM Design Classical Central Composite Optimise Ingredients Mixtures Classical Simplex & Extreme Vertices
  • 23. Copyright © 2014, SAS Institute Inc. All rights reserved. Sequential Experimentation Common Experimental Objectives Identify Important Factors Screening Design Classical Fractional Factorial Optimise Process RSM Design Classical Central Composite Optimise Ingredients Mixtures Classical Simplex & Extreme Vertices Definitive Screening Design Simplifies Experimental Workflow
  • 24. Copyright © 2014, SAS Institute Inc. All rights reserved. Sequential Experimentation Common Experimental Objectives Identify Important Factors Screening Design Classical Fractional Factorial Optimise Process RSM Design Classical Central Composite Optimise Ingredients Mixtures Classical Simplex & Extreme Vertices Definitive Screening Design Optimal Design Manages Experimental Constraints
  • 25. Copyright © 2014, SAS Institute Inc. All rights reserved. Determining the Appropriate Factors  Determining the factors to be included in your experiment is a critical part of planning. • Exploring too many factors may be costly and time consuming. • Exploring too few may limit the success of your experiment.  Prior knowledge and analysis of existing data are useful aids to identifying and prioritising factors for study. Other methods may include: • Brainstorming • Ishikawa
  • 26. Copyright © 2014, SAS Institute Inc. All rights reserved. Selection of Factor Range is Critical With Two Level Designs …
  • 27. Copyright © 2014, SAS Institute Inc. All rights reserved. Selection of Factor Range is Critical With Two Level Designs … By experimenting at the two settings in yellow, X would be declared unimportant
  • 28. Copyright © 2014, SAS Institute Inc. All rights reserved. Selection of Factor Range is Critical With Two Level Designs … By using half and often times much less than than half the factor range X is declared important
  • 29. Copyright © 2014, SAS Institute Inc. All rights reserved. Selection of Factor Range is Critical With Two Level Designs … By using half and often times much less than than half the factor range X is declared important Often leads to narrow factor ranges to force linear relationships but consequence is high risk of determining sub-optimal solution
  • 30. Copyright © 2014, SAS Institute Inc. All rights reserved. Determining the Appropriate Responses  Selection of your responses will also be critical to the success of your experiment. Whenever possible: • Choose variables that correlate to internal or external customer requirements • Find responses that are easy to measure • Make sure your measurement systems are precise, accurate, and stable  Analysis of current data, prior knowledge, measurement systems analysis are useful aids.
  • 31. Copyright © 2014, SAS Institute Inc. All rights reserved. DEFINITIVE SCREENING
  • 32. Copyright © 2014, SAS Institute Inc. All rights reserved. Fractional Factorials: Complex workflow from many factors to optimum settings Tempting to miss out middle step which can result in selection of wrong factors and decisions
  • 33. Copyright © 2014, SAS Institute Inc. All rights reserved. Definitive Screening Design  Identifies active main effects, uncorrelated with other effects.  May identify significant quadratic effects, uncorrelated with main effects and at worst weakly correlated with other quadratic effects.  If few factors turn out to be important, can identify significant two-way interactions uncorrelated with main effects and weakly correlated with other higher order effects.  One stage experiment if three or fewer factors important: • progress straight to full quadratic model • optimise process with no further experimentation • otherwise augment DSD for optimization goals
  • 34. Copyright © 2014, SAS Institute Inc. All rights reserved. New Class of Screening Design  Three-level screening design • 2m + 1 runs based on m fold-over pairs and an overall center point, where m is number of factors • the values of the ±1 entries in the odd- numbered runs are determined using optimal design.
  • 35. Copyright © 2014, SAS Institute Inc. All rights reserved. Use of Three Level Designs Advantageous  Scientists and engineers are uncomfortable using two-level designs • Restricting factor ranges may result in sub-optimal solutions • Scientific/engineering judgment suggests relationships nonlinear over wide ranges  Investigators frequently have an opinion regarding the “best” levels of each factor for optimizing a response • Experimental region centered at these levels. • Two-level design might screen out an important factor when experimental region centred at “best” • Adding centre points allows test for curvature • However ambiguity over factors causing curvature • DSD avoids ambiguity by making it possible to uniquely identify the source(s) of curvature.
  • 36. Copyright © 2014, SAS Institute Inc. All rights reserved. CASE STUDIES
  • 37. Copyright © 2014, SAS Institute Inc. All rights reserved. Case Study 1: Optimising a Chemical Process Why Consider Definitive Screening Designs?
  • 38. Copyright © 2014, SAS Institute Inc. All rights reserved. Background  Five factors  One response yield  Goal optimise yield  Keep total cost of experimentation to minimum  Contrast traditional approach of main effect screening design plus augmentation to RSM with DSD
  • 39. Copyright © 2014, SAS Institute Inc. All rights reserved.  Traditional screening approach correlates main effects with two factor interaction effects  Cost constraint and inexperience with such designs can lead to missed DOE steps  Investigator missed step of augmenting main effect design to separate correlated interaction effects from assumed important main effects  Resulted in wrong set of factors selected for RSM design which results in wrong solution Background
  • 40. Copyright © 2014, SAS Institute Inc. All rights reserved. Traditional Approach with Missed Step
  • 41. Copyright © 2014, SAS Institute Inc. All rights reserved. Resolution III Design Perfectly Correlates Main Effects With Interaction Effects
  • 42. Copyright © 2014, SAS Institute Inc. All rights reserved. Model Interpretation  Fitted Model Y = b0 + b1*X1 + b2*X2 + b3*X3 + Error  Correct Interpretation of Fitted Model Y = b0 + b1*(X1+X2X3) + b2*(X2+X1X3) + b3*(X3+X1X2) + Error
  • 43. Copyright © 2014, SAS Institute Inc. All rights reserved. Missed Step Augments Initial Design to Separate Main Effects From Interactions
  • 44. Copyright © 2014, SAS Institute Inc. All rights reserved. Model Interpretation of Augmented Design  Correct Interpretation of Model Fitted to Augmented design Y = b0 + b1*X1 + b2*X2 + b3*X3 + b12*X1X2 + b13*X1X3 + b23*X2X3 + Error  Allows clear separation of main and interaction effects  This step was missed in case study prior to modelling curvature
  • 45. Copyright © 2014, SAS Institute Inc. All rights reserved.  DSD results in correct identification of important factors due to non correlated main and two factor interaction effects  Because just three factors are important DSD results in one step design: • In addition to correctly identifying correct factors • DSD requires no augmentation to identify optimal settings of important factors Background
  • 46. Copyright © 2014, SAS Institute Inc. All rights reserved. CASE STUDY 1
  • 47. Copyright © 2014, SAS Institute Inc. All rights reserved. Conclusions  Fractional factorial designs can lead to selection of wrong factor set  Complex workflow for avoiding this risk which may be misunderstood or not applied by users new to DOE  May lead to conclusion that DOE does not work for us!  DSD simplifies DOE process and removes risk of selecting wrong factor set  Provides one step DOE when three or fewer important factors • Sufficient to identify correct factor set and determine best settings of selected factors
  • 48. Copyright © 2014, SAS Institute Inc. All rights reserved. Case Study 2: Optimising Reaction Conditions for Chemical Methods Augmenting Definitive Screening Designs
  • 49. Copyright © 2014, SAS Institute Inc. All rights reserved. Background  How effective are DSD when more than three factors are important?  Use example from literature • Response-Surface Co-optimization of Reaction Conditions in Clinical Chemical Methods, Gopal S. Rautela, Ronald 0. Snee,’ and Warren K. Miller, CLINICAL CHEMISTRY, Vol. 25, No. 11, 1979 • CCF RSM in six factors • Five factors are important • Use model from this experiment to contrast CCF with augmented DSD
  • 50. Copyright © 2014, SAS Institute Inc. All rights reserved.  Aspartate Aminotransferase Assay: http://www.chem.qmul.ac.uk/iubmb/enzyme/EC2/6/1/1.html  Six factors (reagent conditions): tris(hydroxymethyl)aminomethane, pH, L-aspartic acid, pyridoxal-5’-phosphate, 2-oxoglutarate, and malate dehydrogenase  Response: aspartate aminotransferase activity measured for human serum with above normal activity at 30C  Goal: select reagent conditions that maximise aspartate aminotransferase activity  Example selected to stress DSD when >3 factors are important, in this case 5 factors are important. Background
  • 51. Copyright © 2014, SAS Institute Inc. All rights reserved. CASE STUDY 2
  • 52. Copyright © 2014, SAS Institute Inc. All rights reserved. Conclusions  When >3 factors are important, augmenting DSD works  When >3 factors are important, an augmented DSD approach is more efficient than classical Response Surface Designs
  • 53. Copyright © 2014, SAS Institute Inc. All rights reserved. CASE STUDY 3
  • 54. Copyright © 2014, SAS Institute Inc. All rights reserved. Case Study 3: Optimising Yield What About Constrained Factor Spaces?
  • 55. Copyright © 2014, SAS Institute Inc. All rights reserved. Background  From chapter 5 of Goos & Jones  Chemical reaction  Goal: maximise yield  2 factors: Temperature and Time
  • 56. Copyright © 2014, SAS Institute Inc. All rights reserved. Background  Expert knowledge tells us • Certain conditions will give poor results (hence, constraints) • Behaviour very non-linear  We will show • Design where prior knowledge is ignored. • Fitting the design to the problem
  • 57. Copyright © 2014, SAS Institute Inc. All rights reserved. Example of Process Constraint
  • 58. Copyright © 2014, SAS Institute Inc. All rights reserved. Shrink Experimental Range to Factorial
  • 59. Copyright © 2014, SAS Institute Inc. All rights reserved. Shrink Experimental Range to Factorial
  • 60. Copyright © 2014, SAS Institute Inc. All rights reserved. Shrink Experimental Range to Factorial
  • 61. Copyright © 2014, SAS Institute Inc. All rights reserved. Optimal Design: Use Actual Factor Range
  • 62. Copyright © 2014, SAS Institute Inc. All rights reserved. … optimal designs allow investigation of complete factor space properly adjusted for constraints Typical Process Machine Operator Temperature Pressure Humidity Yield Cost … Inputs Factors Outputs Responses Optimal Design: Fit to Model Model Y = f(X) The process is not seen as a black box anymore…
  • 63. Copyright © 2014, SAS Institute Inc. All rights reserved. CASE STUDY 3
  • 64. Copyright © 2014, SAS Institute Inc. All rights reserved. Conclusions  Custom Design permits studying any: • combination of factors with or without constraints, • number of factor levels, • blocking structure.  Build your design to suit the problem instead of fitting the problem into a design
  • 65. Copyright © 2014, SAS Institute Inc. All rights reserved. Case Study 4: Designing Products People Want to Buy Holistic DOE
  • 66. Copyright © 2014, SAS Institute Inc. All rights reserved. ROLE OF STATISTICAL MODELLING AND DOE IN LEARNING
  • 67. Copyright © 2014, SAS Institute Inc. All rights reserved. Data Sources  DOE and/or observational (historical)  Potential problems with observational data: • X’s are correlated – identification of “best” model difficult • Outliers (potential or real) - bias model estimation • Missing data cells – result in loss of whole data rows with traditional least squares based analysis • Range over which X’s varied may be limited – restricting model usefulness • May not have measured all relevant X’s  In some situations these can also be issues with DOE datasets
  • 68. Copyright © 2014, SAS Institute Inc. All rights reserved. WHAT IS HOLISTIC DOE?
  • 69. Copyright © 2014, SAS Institute Inc. All rights reserved. Holistic DOE Approach: Integrating Statistical Modelling and DOE  Learning is incremental and effective statistical modelling of observational data aids design of next experiment.  Analysis approach needs to manage real (messy) data simply • Correlated X’s, outliers, missing cells • Quickly deliver “best” current model to revise with new DOE data • Aid better analysis of new experimental data when unexpected occurs • Build models based on individual datasets and aggregated data  Good statistical modelling integrated with DOE helps reduce total learning time, effort and cost  It would be a shame to not use pre-existing data that comes for free
  • 70. Copyright © 2014, SAS Institute Inc. All rights reserved. Holistic DOE Example
  • 71. Copyright © 2014, SAS Institute Inc. All rights reserved. Background  PC retailer is observing appreciably retail price variation in its laptop computer line.  Goals: • Investigate factors associated with retail price variation. • Perform further experimentation in key factors to optimise and standardise pricing across stores.
  • 72. Copyright © 2014, SAS Institute Inc. All rights reserved. CASE STUDY 4
  • 73. Copyright © 2014, SAS Institute Inc. All rights reserved. Conclusions  Analysis of prior data helps identify factors and ranges to use in next DOE.  Analysis of prior data helps reduce risk and increase efficiency and effectiveness of future experiments.  DOE is not just for science and engineering.
  • 74. Copyright © 2014, SAS Institute Inc. All rights reserved. Holistic DOE: Integrated Statistical Modelling and DOE  Supports wide range of user skills  Exploratory analysis and statistical modelling of historical messy data simplifies and shortens whole DOE process.  Next generation DOE enables more staff to apply DOE with reduced learning and implementation effort  Interact with model predictions to build consensus  Integrated simulation capabilities enables rapid progression from models to decisions  Drag and drop charts help monitor processes and identify potential causes of issues  Manage risk better by correctly identifying signal from noise

×