Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Experimental design part 3 controls

132 views

Published on

Part 3 of 5 part experimental design lecture series. This presentation deals with controls and the different roles they play in your design. Interpretation, calibration, biological controls, experimental controls, blinding and multiple observers.

Text form of this content available via www.lantsandlaminins.com

Published in: Science
  • Be the first to comment

  • Be the first to like this

Experimental design part 3 controls

  1. 1. Controls Experimental design: part 3
  2. 2. What controls do you need? Your experiment is only as good as your controls You should “control” for anything that could influence your interpretation of your data “Controls” are additional components of your experiment that help you interpret your data in the way that you want to
  3. 3. Think about what the harshest critic of your work would think and try and design ways so that your experiment stands up to that criticism
  4. 4. Controls have different roles in your experiment. We’ll talk about each of them in turn. Experimental Biological Interpretation Calibration
  5. 5. Experimental Experimental controls help you with troubleshooting problems with your experiment They are extra samples that you process that you already know “work” in this experiment. Including a sample like this at each stage will allow you to work out where problems have come up. Examples please
  6. 6. Experimental The mRNA for protein A is decreased and the mRNA for protein B is decreased In RNA extracted from squamous cell carcinoma tissue compared to RNA isolated normal skin In this example experiment we want to study mRNA abundance To decide which experimental controls are needed, I need to think what could go wrong Basically, every stage of the experiment RNA isolation Cancer tissue Reverse transcription Quantitative PCR
  7. 7. Experimental The mRNA for protein A is decreased and the mRNA for protein B is decreased In RNA extracted from squamous cell carcinoma tissue compared to RNA isolated normal skin Reverse transcription Quantitative PCR RNA isolation Cancer tissue Now think, what sample can I use that will definitely work at each step Tissue that I have used successfully before RNA that I have already used and know is OK cDNA that I have used previously with these primers It won’t always be possible to have everything. Especially with a new experiment. Or primers that you have used before on the new cDNA But the more you can include, the better!
  8. 8. Experimental The mRNA for protein A is decreased and the mRNA for protein B is decreased In RNA extracted from squamous cell carcinoma tissue compared to RNA isolated normal skin RNA isolation Reverse transcription Quantitative PCR Cancer tissue In addition you will want negative controls Samples that will give a negative result at every stage of the experiment No tissue Water only as template in RT reaction Water instead of cDNA as template for qPCR
  9. 9. Biological Whereas experimental controls are to help you troubleshoot problems Biological controls are used to show that your data are really what you think they are Let’s move on
  10. 10. Biological You can’t prove a negative without a positive! You can’t prove a positive without a negative! This is big! Remember this throughout your planning
  11. 11. Biological Let’s talk about positive controls First consider what are you measuring and what a positive result would look like This could be the presence of something new Or an increase or decrease in something
  12. 12. Biological Your positive control should give you a similar response The control could be a treatment that has been published before A cell line / tissue or whatever known to express your protein or mRNA of interest
  13. 13. Biological Positive controls increase your confidence that if there is a difference you would be able to see it
  14. 14. Biological OK, let’s move on to negative controls I can feel an example coming! No surprises here, negative controls are treatments that you are confident won’t produce the response you are looking at
  15. 15. Biological In this experiment we are using an antibody to detect our protein of interest But how do we know that the signal we have imaged is real? What else could it be? Well, no antibody is perfect. So you will always get some signal that is non-specific Using controls can help us distinguish between what is real versus what is non-specific
  16. 16. Biological sample The process of preparing a sample involves two key steps First you probe the sample with the primary antibody Then you detect the primary antibody by probing with a secondary antibody with coloured tag We need to think about what could go “wrong” and design ways to identify them Let’s look at ways we can check that the signal we get is ”real”
  17. 17. Biological sample First the simplest one. If the secondary antibody bound to the sample without the primary being present, we would get a false positive signal So, we control for that by having a sample probed with just the secondary antibody alone.Secondary only A secondary only control doesn’t help us identify if our antibody has bound specifically to our protein of interest or to other proteins So we add a second negative control: a sample that doesn’t express our protein of interest but is otherwise as similar as possible as our real sample Sample that doesn’t express our protein
  18. 18. Biological sample Let’s add a positive experimental control too: a sample that definitely does express our protein Secondary only Sample that doesn’t express our protein Sample that definitely does express your protein (experimental control) And a biological control: a treatment that does cause the effect your are expecting to see Treatment that causes the change you are looking for
  19. 19. Biological sample It’s not unusual to have more controls than samples! Secondary only Sample that doesn’t express our protein Sample that definitely does express your protein (experimental control) Some of these can serve multiple roles, so you may not need everything. But the more robust your controls are, the more confident you will be in your data Treatment that causes the change you are looking for That’s a lot
  20. 20. Interpretation Biological and experimental controls help you identify what is real, but controls can also help you interpret your data, rule out confounding variables As you are thinking about the design, try to consider every other possible reason you might be obtain the effect you are looking for and try to think of a way to rule out those alternative mechanisms
  21. 21. Interpretation Let’s say you are studying the effect of altitude training on VO2 max. You have two groups of people; group 1 training at sea level, group 2 training up a mountain. Other than the location of their training, what else could influence the results? Why am I in the altitude group!
  22. 22. Smoking Previous altitude training Starting fitness levels Interpretation Age Weight BMI Gender You could control for some or all of these during the group selection process Depending on the sample size, you might also be able to determine the effect the confounder has on the outcome (as a secondary metric) Your pilot study might also help you to determine which of your potential confounders are likely to have the biggest effect, this will help you identify which ones are most important to control for
  23. 23. B + Reading 2 A + Reading 1 OK, let’s look at an experiment comparing a new drug to the current best drug In your pilot experiments you found that the response varied a lot Paired analyses And you think the variation is due to differences in confounding variables Example time!
  24. 24. A + Treatment 1 Measurement One way to control for these variables is to pool the study population and do a paired analysis In this experiment, the same people will get both treatments Baseline Reading 1 Baseline Reading 2 B + Treatment 2 Measurement Paired analyses And you can compare the same individual’s response to the different drugs
  25. 25. A + Treatment 1 Measurement B + By reducing variation in your data, they can make your stats stronger Baseline Reading 1 Baseline Reading 2 Treatment 2 Measurement Paired analyses You can often use more powerful tests When you have a limited sample size, and can’t control for confounders by grouping, a paired design can really help!
  26. 26. A + Treatment 1 Measurement B + Baseline Reading 1 Baseline Reading 2 Treatment 2 Measurement Hopefully by this point you can spot the potential problem here? And a pretty clear potential solution Paired analyses
  27. 27. Break the population in 2 and reverse the treatment order between groups A+ B+ Baseline Reading 1 Baseline Reading 2 B+ A+ Baseline Reading 1 Baseline Reading 2 When you do your data analysis the first thing you do is determine if treatment order matters Yes, this way you can control for order effects
  28. 28. Calibration One more quick point on controls Often you will use known samples to calibrate your system concentration absorbance This can involve using measurements to set up a standard curve However, in other situations you might use relative quantification…. But relative to what? In these cases the calibration might be really obvious Feels like you are setting up another example!
  29. 29. Note that this approach isn’t very common for this type of application The mRNA for protein A is decreased and mRNA for protein B is decreased In RNA extracted from squamous cell carcinoma tissue compared to RNA isolated normal skin Let’s use a study question from before We could calibrate this experiment in two ways mRNA amount Cycle number We could set up a standard curve using mRNA that we have synthesised Our readout would then be mg of protein B mRNA per mg of total RNA or per mg of tissue
  30. 30. The mRNA for protein A is decreased and mRNA for protein B is decreased In RNA extracted from squamous cell carcinoma tissue compared to RNA isolated normal skin A second option would be to compare protein B mRNA abundance to one or more reference mRNAs Tumour Sample Control Sample Protein B mRNA 150 140 Reference mRNA 400 500 150 400 140 500 Relative abundance But how do you choose what to use as your reference
  31. 31. The mRNA for protein A is decreased and mRNA for protein B is decreased In RNA extracted from squamous cell carcinoma tissue compared to RNA isolated normal skin Good question! Sometimes your understanding of the system will help you identify something that will stay constant within your study But, you should still test that assumption! Again, identifying appropriate reference points is something that your pilot experiments should be designed to do As with other aspects of your data you should be able to justify the decisions you have made along the way
  32. 32. Time for you to do something! Squamous cell carcinoma cells induced to overexpress protein B display increased invasion compared with control treated cells. Squamous cell carcinoma cells will be induced to express protein B or not treated then seeded onto a skin substitute After 48 hours the distance migrated into each substrate will be measured Remember this experiment? Can you identify what controls you might need and what they would be? Experimental? Biological? Interpretation? Calibration? Skin substitute Normal cells + protein B
  33. 33. Experimental controls -ve: Use a non-invasive cell line, should see no invasion Treat with an invasion inhibiting drug +ve: Use a well-established highly invasive cell line Treat with an invasion promoting drug Biological controls -ve: Introduce a protein that is known to not affect invasion Treat cells with all the reagents needed to induce expression of the protein (i.e. everything but the protein) +ve: Use the same expression system to introduce a protein already established as increasing or decreasing invasion Interpretation controls A lot of overlap here! That’s OK! Include cell proliferation inhibiting drugs so that measurements are solely due to invasion Use multiple different cell donors, to control for donor specific responses Did you think of any more?
  34. 34. Controlling for human bias Let’s talk about a way we can control for human bias; “blinding”
  35. 35. Controlling for human bias You’ve probably heard about clinical studies being described as ”double blind”. This means that person in the trial doesn’t know if they have received the placebo (control) treatment or the actual treatment In addition, the clinicians do not know what the participants have received.
  36. 36. Controlling for human bias By not knowing, both parties won’t introduce sub-conscious bias. In lab studies, it can be harder to introduce blinding but if it is possible then you should. The most obvious times would be when the data will be subjectively analysed, such as scoring of images
  37. 37. Controlling for human bias Another option, when you have subjective data, is for multiple observer to be used and some consensus scoring system developed
  38. 38. Part 3 Recap. Every experiment needs a positive and negative control that can tell you whether it worked or not You can’t prove a positive without a negative. You can’t prove a negative without a positive. Try to control for every variable or confounder that could affect your interpretation of your data Think about what the harshest critic of your work would think and try and design ways so that your experiment stands up to that criticism
  39. 39. Jess Neil Conro John

×