Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Experimental design part 1

180 views

Published on

Part 1 of a 5 part lecture series on experimental design.
This section deals with hypothesis generation, correlative vs manipulative experiments and choosing an appropriate model system.

Text version of this content available at www.lantsandlaminins.com

Published in: Science
  • Be the first to comment

  • Be the first to like this

Experimental design part 1

  1. 1. Experimental Design
  2. 2. Welcome to the experimental design lecture series The series is broken up into 5 parts Each part has examples and questions associated with them to help you test your knowledge 1. Experimental Preliminaries and hypotheses 2. Measurements 3. Controls 4. Independence and assigning groups 5. Sample size determination Kevin LeeBurt Krn Conro
  3. 3. Experimental Preliminaries and Hypotheses Experimental design: part 1
  4. 4. Formulate a clear primary research question Big Question Sub-question 1 Sub-question 2 Sub-question 3 Likely you won’t be able to directly answer this in one experiment Usually you will design experiments to answer a single sub question Experiment 1 Experiment 2 It’s better to answer 1 question well than 3 questions badly Using multiple experiments that answer the same question will increase your confidence in the findings
  5. 5. Most experimental design happens at the individual experiment level Big Question Sub-question 2Each experiment needs to be as robust as possible for it to contribute meaningful data Experiment 1 There are three types of experiment 1. Those that yield interesting data, no matter the outcome 2. Those that yield interesting data, but only if the experiment turns out one way 3. Those that yield ambiguous, uninterpretable results regardless of outcome Type 3 experiments are a waste of time, money and could be ethically unforgiveable
  6. 6. Core Concept: Hypothesis testing We call the prediction a “hypothesis” Experiments don’t “prove” something It’s (your predicted) answer to the experimental question Rather they are used to test a prediction At the end of an experiment your hypothesis is either supported or refuted
  7. 7. Core Concept: Hypothesis testing Having a clear, well defined hypothesis is a key first step in experimental design Hypotheses have to TESTABLE If you don’t know, and aren’t able to explicitly state, what you are trying to discover, it will be very difficult to design you experiment well Hypotheses have to SPECIFIC Key points:
  8. 8. How do you come up with a hypothesis? First you need a key set of observations the first experiments of an experimental series might be designed to generate these observations Descriptive Studies These could be published work or experimental data We call these sort of hypothesis generating studies; descriptive studies Descriptive studies still need to be carefully designed if they are to be useful
  9. 9. Descriptive Studies Often ‘omics or bioinformatics type studies are viewed “hypothesis generating” These studies might be presented as testing a hypothesis that populations are different with whatever manipulation is being investigated Gonna catch me a hypothesis However, they may have a more general aim of characterising differences
  10. 10. Hypotheses have to TESTABLE Hypotheses have to SPECIFIC OK, let’s do it!Before you start, remember: Example time!
  11. 11. Examples Normal Cancer Protein A Protein B Protein A and Protein B are genetically related to each other. Previous experiments have shown that in normal tissue protein A is highly expressed (brown) whereas protein A shows low expression. In cancer the pattern is reversed.
  12. 12. Examples Normal Cancer Protein A Protein B These sorts of observations could generate lots of questions Big Question Sub-question 1 Sub-question 2 Sub-question 3 The big question might be related to the clinical use But you might also be interested in how this switch in expression is controlled or what it means in terms of cancer cell behaviour
  13. 13. Examples Big Question Sub-question 1 Sub-question 2 Sub-question 3 Let’s have a look at some different types of questions Normal Cancer Protein A Protein B
  14. 14. Examples Q1 How often does this change happen? In which type of cancers? Does the change in expression of protein A and/or B indicate disease severity? Could measuring these proteins have diagnostic or prognostic value? Q2 Q3 What causes the switch in expression to happen? Is it at the genetic level or protein level? What is driving the switch? What effect does the increase in protein B or decrease in protein A have upon cancer cell invasion and metastasis? Each of these questions are still too big for a single experiment so let’s simplify each to the first part
  15. 15. Examples Q1 How frequently does the change in expression of protein B happen? Q2 Q3 Do the changes in expression happen at the mRNA level? Does the increase in protein B influence cancer cell invasion? Focus on the most important questions Normal Cancer Protein A Protein B
  16. 16. Examples Q1 OK, we’ve got a questions, let’s turn it them into hypotheses Give it a go…. What would be appropriate here? How frequently does the change in expression of protein B happen? Changes in expression of protein B occur in more than 50% of cancers Good start but you need your hypothesis to be specific and testable. What type of cancer? What direction will the change be?
  17. 17. Examples Q1 How frequently does the change in expression of protein B happen? Protein B expression is increased in more than 50% of squamous cell carcinoma cancers Better
  18. 18. Examples Q1 How frequently do the changes in expression of protein A and protein B happen? Although we are studying protein B, it might be possible to study protein A at the same time. Let’s change the experimental question. Can you come up with a hypothesis that captures a potential answer to this question?
  19. 19. Examples Q1 How frequently does the change in expression of protein A and protein B happen? Protein B expression is increased in more than 50% of squamous cell carcinoma cancers Protein A expression is decreased in more than 50% of squamous cell carcinoma cancers Protein A is decreased and Protein B expression is increased in more than 50% of squamous cell carcinoma cancers An experiment can test multiple hypotheses BUT make sure you design it so that it can test all of them effectively
  20. 20. Examples Q1 How frequently does the change in expression of protein A and protein B happen? Protein A is decreased and Protein B expression is increased in more than 50% of squamous cell carcinoma cancers Rank your hypotheses/questions based on how important they are “primary hypothesis, secondary hypotheses etc…” Later in this series, we will discuss sample sizes: you will use your primary hypothesis for those calculations
  21. 21. Examples Q1 How frequently does the change in expression of protein A and protein B happen? Why 50%? Do be honest, it is was just an arbitrary number! In situations like this, pick a number that either comes from your pilot data or is biologically meaningful What would be a useful number? For example here, would a certain frequency of change mean the information is useful as a diagnostic or prognostic biomarker
  22. 22. Examples Q2 Q3 Do the changes in A and B protein expression also occur at the mRNA level? Does the increase in protein B influence cancer cell invasion? What about the other questions? Try these now
  23. 23. Examples Q2 Q3 Do the changes in A and B protein expression also occur at the mRNA level? Does the increase in protein B influence cancer cell invasion? How about something like: The mRNA for protein A is decreased and mRNA for protein B is increased in RNAs extracted from squamous cell carcinoma tissue compared to RNAs isolated normal skin Squamous cell carcinoma cells induced to overexpress protein B display increased invasion compared with control treated cells.
  24. 24. OK, almost done with hypotheses. But one final point: Be careful with your wording! Note also that you may need to revisit your hypothesis as you proceed through the rest of the design stages When you say “due to” or “influences” then your experiment needs to establish causality
  25. 25. Correlative or Manipulative? My data shows a positive correlation! Great! What with what? The number of experiments I do and how confused I become!
  26. 26. Correlative: Looking for associations between one observation and one or more other observations For example: “Smokers have lower lung capacity than non-smokers” In manipulative studies you deliberately change something to determine if it has an effect For example: “Lung capacity increases after stopping smoking” Let’s look at the two main types of experiment Correlative or Manipulative?
  27. 27. Correlative or Manipulative? Correlative studies are used where manipulation is impossible Whereas manipulative studies are used to gain mechanistic insight Correlative studies are often the source of observations that allow hypotheses to be generated Manipulative studies will allow you to control for confounding variables Or allow you to determine directionality, establish causality or control for “reverse causation” How do you choose? Let’s have a look at some of these points
  28. 28. Key Questions Is manipulation possible? Ethics, sample availability or timeframe may mean you can’t actually manipulate the system For example; a study investigating the impact of childhood diet on lifespan in humans would take 80 years to completeOr where you predict a manipulation could cause long- term, unnecessary harm it would be ethically wrong to do the experiment
  29. 29. Key Questions Next you should ask: Can you actually produce biologically realistic manipulations? Manipulations can introduce things that would never happen or in incomplete/irrelevant contexts If the data you obtain can’t be interpreted in the real world then is it worth asking?
  30. 30. Key Questions Can you actually infer causation? Just because two things happen together it doesn’t automatically mean they are causally linked! What about correlative studies? What do you need to consider there? http://www.tylervigen.com/spurious-correlations
  31. 31. Key Questions Also, you need to think about 3rd variables Also known as confounding variables. These are other things that could influence your interpretation We’ll talk about controls later, but if you can’t find a way to eliminate 3rd variables then you may not be able to interpret your data. Extra variables Diet Predators Terrain Weather Genetics For example, if you were comparing experimental animals to wild animals it might not be possible to segregate one extra variable from the others Does X cause Y Or is it Z that causes Y
  32. 32. Key Questions Does X cause Y or is it really that Y that causes X? You also need to ask; can you control for reverse causation? Lower BMI More exercise I exercise a lot because I want to stay thin Causation Lower BMI More exercise I don’t exercise because I am embarrassed about being overweight Reverse Causation
  33. 33. Correlative or Manipulative? Q1 Q2 Q3 Protein B expression is increased in more than 50% of squamous cell carcinoma cancers The mRNA for protein A is decreased and mRNA for protein B is decreased In RNA extracted from squamous cell carcinoma tissue compared to RNA isolated normal skin Squamous cell carcinoma cells induced to overexpress protein B display increased invasion compared with control treated cells. Your turn; What approaches would you use for these hypotheses?
  34. 34. Correlative or Manipulative? Q1 Protein B expression is increased in more than 50% of squamous cell carcinoma cancers Correlative: Measure protein B expression in lots of squamous cell carcinoma samples from human patients Or, induce squamous cell carcinoma in an animal model and measure protein B expression Note that even though you are manipulating (inducing cancer) the output measured is still a correlation
  35. 35. Correlative or Manipulative? Q2 The mRNA for protein A is decreased and the mRNA for protein B is increased In RNA extracted from squamous cell carcinoma tissue compared to RNA isolated normal skin Correlative Again, acquire samples and analyse mRNA levels If you wanted to do a manipulative study you would need a different hypothesis For example: increasing expression of the mRNA for protein A causes a decrease in the mRNA for protein B
  36. 36. Correlative or Manipulative? Q3 Squamous cell carcinoma cells induced to overexpress protein B display increased invasion compared with control treated cells. Can you rephrase this hypothesis to make it suitable for a correlative study? Manipulative Here you would modify the system: introducing overexpression of protein B and measuring the effect Correlative Squamous cell carcinomas with increased protein B expression are more likely to have metastatic spread compared with those with normal protein B expression
  37. 37. Choosing a Model System
  38. 38. Points to consider in your model system choice Ease of manipulation Direct vs indirect effects Cost and time Ethics and approvals Biological relevance Complexity Simplicity and tractability Biological accuracy
  39. 39. The decision about what system to use should be dependent on the question you want to answer, not the other way round! Simplicity and tractability Biological accuracy Adding biological relevance by using more complex systems may seem good but won’t help if your data become impossible to interpret You should also be considering research ethics in your study design Is the added benefit you get from using animals/humans in your studies actually worth the cost to the subject
  40. 40. Squamous cell carcinoma cells induced to overexpress protein B display increased invasion compared with control treated cells. Example Let’s look at one of our examples from before… In the literature there are three widely used assays to assess cell invasion The end decision depends on the question, and what else is already known Chick Amniotic Membrane invasion assay $$ 4-6 weeks Variable Invasion into more biologically relevant substrate Confounders Migration through tissue Colony growth and proliferation 3D cell culture invasion assay $ 2-3 weeks Variable Ability to invade into a substrate Confounders Mouse graft model $$$$ 3-6 months Variable Invasion into more biologically relevant substrates Confounders Migration through tissue Colony growth and proliferation Interaction with immune cells
  41. 41. Part 1 Recap. Identify the important, interesting question you want to answer Form a clear discrete hypothesis that provides a plausible answer to your main question Decide the best approach; manipulation or correlation? Choose a model system that will allow you to address the key question
  42. 42. Advice from Students SamJess Danielle Adam

×