Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Dealing with negative results

221 views

Published on

Invited lecture in the course "Research Methods and Proposal Writing", Dublin Institute of Technology, kindly hosted by Dr. Luca Longo

Published in: Science
  • Be the first to comment

Dealing with negative results

  1. 1. Dealing with negative results Research methods and proposal writing Dr M Benini Università degli Studi dell’Insubria kindly hosted by Dr L Longo marco.benini@uninsubria.it March 1st, 2016
  2. 2. Science is positive The vast majority of scientific results are stated as answers to questions of the kind why how The first question asks for an explanation of some phenomenon; the second one requires to provide a justified method to achieve some result of interest ( 2 of 19 )
  3. 3. Science is positive Consider the negated questions: why not how not The first question requires to provide a way to falsify an explanation; the second question asks to show a faulty way to achieve a result, and to put in evidence its failure As a matter of fact, we have a huge number of negative answers in our experience, and we need them! ( 3 of 19 )
  4. 4. Science is positive Consider the negated questions: why not how not The first question requires to provide a way to falsify an explanation; the second question asks to show a faulty way to achieve a result, and to put in evidence its failure As a matter of fact, we have a huge number of negative answers in our experience, and we need them! ( 3 of 19 )
  5. 5. Science is positive When we do research, we collect many negative answers: when an experiment fails, we have to understand why it did not succeed when a (mathematical) proof fails to be found, we have to ask ourselves what prevents it, eventually searching for a counterexample when a procedure fails to achieve the desired result, we have to understand what went wrong, and why ( 4 of 19 )
  6. 6. Science is positive When we do research, we collect many negative answers: when an experiment fails, we have to understand why it did not succeed when a (mathematical) proof fails to be found, we have to ask ourselves what prevents it, eventually searching for a counterexample when a procedure fails to achieve the desired result, we have to understand what went wrong, and why ( 4 of 19 )
  7. 7. Science is positive When we do research, we collect many negative answers: when an experiment fails, we have to understand why it did not succeed when a (mathematical) proof fails to be found, we have to ask ourselves what prevents it, eventually searching for a counterexample when a procedure fails to achieve the desired result, we have to understand what went wrong, and why ( 4 of 19 )
  8. 8. Science is positive When we do research, we collect many negative answers: when an experiment fails, we have to understand why it did not succeed when a (mathematical) proof fails to be found, we have to ask ourselves what prevents it, eventually searching for a counterexample when a procedure fails to achieve the desired result, we have to understand what went wrong, and why ( 4 of 19 )
  9. 9. Science is positive When we reach the final stage of our research efforts, publishing the results, we focus on the positive achievements, And we discard the negative experience we have accumulated An experienced researcher, instead, should be more effective, trying to use the negative results and to maximise the outcomes of his efforts ( 5 of 19 )
  10. 10. Science is positive When we reach the final stage of our research efforts, publishing the results, we focus on the positive achievements, And we discard the negative experience we have accumulated An experienced researcher, instead, should be more effective, trying to use the negative results and to maximise the outcomes of his efforts ( 5 of 19 )
  11. 11. Negative content, positively presented Although some results we have found are negative, they can often be positively presented when a procedure fails, precisely delimit the domain of applicability, excluding the cases we know are bad: the limit becomes a necessary condition, and the failure, a counterexample to a direct generalisation when a mathematical proof fails, find a counterexample, and use it to add a condition that prevents its applicability: the additional hypothesis becomes necessary because of the counterexample ( 6 of 19 )
  12. 12. Negative content, positively presented Although some results we have found are negative, they can often be positively presented when a procedure fails, precisely delimit the domain of applicability, excluding the cases we know are bad: the limit becomes a necessary condition, and the failure, a counterexample to a direct generalisation when a mathematical proof fails, find a counterexample, and use it to add a condition that prevents its applicability: the additional hypothesis becomes necessary because of the counterexample ( 6 of 19 )
  13. 13. Negative content, positively presented Although some results we have found are negative, they can often be positively presented when a procedure fails, precisely delimit the domain of applicability, excluding the cases we know are bad: the limit becomes a necessary condition, and the failure, a counterexample to a direct generalisation when a mathematical proof fails, find a counterexample, and use it to add a condition that prevents its applicability: the additional hypothesis becomes necessary because of the counterexample ( 6 of 19 )
  14. 14. Negative content, positively presented When stating a result, use a positive language Theorem [Gödel] Peano Arithmetic is not complete. Theorem [Gödel] There is a sentence G in Peano Arithmetic such that both G and ¬G are not provable. Theorem [Gödel] There is a sentence G in Peano Arithmetic such that G is true on natural numbers, but G is not provable. ( 7 of 19 )
  15. 15. Negative content, positively presented When stating a result, use a positive language Theorem [Gödel] Peano Arithmetic is not complete. Theorem [Gödel] There is a sentence G in Peano Arithmetic such that both G and ¬G are not provable. Theorem [Gödel] There is a sentence G in Peano Arithmetic such that G is true on natural numbers, but G is not provable. ( 7 of 19 )
  16. 16. Negative content, positively presented When stating a result, use a positive language Theorem [Gödel] Peano Arithmetic is not complete. Theorem [Gödel] There is a sentence G in Peano Arithmetic such that both G and ¬G are not provable. Theorem [Gödel] There is a sentence G in Peano Arithmetic such that G is true on natural numbers, but G is not provable. ( 7 of 19 )
  17. 17. Negative content, positively presented When stating a result, use a positive language Theorem [Gödel] Peano Arithmetic is not complete. Theorem [Gödel] There is a sentence G in Peano Arithmetic such that both G and ¬G are not provable. Theorem [Gödel] There is a sentence G in Peano Arithmetic such that G is true on natural numbers, but G is not provable. ( 7 of 19 )
  18. 18. Negative content, positively presented When writing a statement, provide a positive content Theorem An arbitrary list cannot be sorted in linear time. Theorem An arbitrary list can be sorted in O(nlogn) steps. Theorem The heap sort algorithm sorts any list in at most O(nlogn) steps. ( 8 of 19 )
  19. 19. Negative content, positively presented When writing a statement, provide a positive content Theorem An arbitrary list cannot be sorted in linear time. Theorem An arbitrary list can be sorted in O(nlogn) steps. Theorem The heap sort algorithm sorts any list in at most O(nlogn) steps. ( 8 of 19 )
  20. 20. Negative content, positively presented When writing a statement, provide a positive content Theorem An arbitrary list cannot be sorted in linear time. Theorem An arbitrary list can be sorted in O(nlogn) steps. Theorem The heap sort algorithm sorts any list in at most O(nlogn) steps. ( 8 of 19 )
  21. 21. Negative content, positively presented When writing a statement, provide a positive content Theorem An arbitrary list cannot be sorted in linear time. Theorem An arbitrary list can be sorted in O(nlogn) steps. Theorem The heap sort algorithm sorts any list in at most O(nlogn) steps. ( 8 of 19 )
  22. 22. Negative content, positively presented When stating a counterexample, be positive The real-valued function f (x) = |x| does not admit a derivative in x = 0. The real-valued function f (x) = |x| admits f (x) = −1 as derivative on the interval (−∞,0), and f (x) = 1 on the interval (0,+∞). Thus, f (0) is not defined. ( 9 of 19 )
  23. 23. Negative content, positively presented When stating a counterexample, be positive The real-valued function f (x) = |x| does not admit a derivative in x = 0. The real-valued function f (x) = |x| admits f (x) = −1 as derivative on the interval (−∞,0), and f (x) = 1 on the interval (0,+∞). Thus, f (0) is not defined. ( 9 of 19 )
  24. 24. Negative content, positively presented When stating a counterexample, be positive The real-valued function f (x) = |x| does not admit a derivative in x = 0. The real-valued function f (x) = |x| admits f (x) = −1 as derivative on the interval (−∞,0), and f (x) = 1 on the interval (0,+∞). Thus, f (0) is not defined. ( 9 of 19 )
  25. 25. Negative content, positively presented When stating a limit to a procedure, be positive Using a 12,800 ISO film speed in a digital camera prevents getting a noise-free photograph when shooting in a dark environment. Using a 12,800 ISO film speed in a digital camera provides a noisy photograph when shooting in a dark environment, which can be useful to calculate the correct exposure at a lower sensibility, and get a noise-free image. ( 10 of 19 )
  26. 26. Negative content, positively presented When stating a limit to a procedure, be positive Using a 12,800 ISO film speed in a digital camera prevents getting a noise-free photograph when shooting in a dark environment. Using a 12,800 ISO film speed in a digital camera provides a noisy photograph when shooting in a dark environment, which can be useful to calculate the correct exposure at a lower sensibility, and get a noise-free image. ( 10 of 19 )
  27. 27. Negative content, positively presented When stating a limit to a procedure, be positive Using a 12,800 ISO film speed in a digital camera prevents getting a noise-free photograph when shooting in a dark environment. Using a 12,800 ISO film speed in a digital camera provides a noisy photograph when shooting in a dark environment, which can be useful to calculate the correct exposure at a lower sensibility, and get a noise-free image. ( 10 of 19 )
  28. 28. Negative content, positively presented When stating a limit to a procedure, be positive Using a 12,800 ISO film speed in a digital camera prevents getting a noise-free photograph when shooting in a dark environment. Using a 12,800 ISO film speed in a digital camera provides a noisy photograph when shooting in a dark environment, which can be useful to calculate the correct exposure at a lower sensibility, and get a noise-free image. ( 10 of 19 )
  29. 29. Impossibility results Some results are inherently negative. If they are also significant, it is worth stating them in a negative way Theorem [Turing] Deciding whether a program terminates on all inputs, is not a computable problem. Theorem [Turing] There is no computer program that, taking as input a computer program, decides whether it terminates on all inputs. ( 11 of 19 )
  30. 30. Impossibility results Some results are inherently negative. If they are also significant, it is worth stating them in a negative way Theorem [Turing] Deciding whether a program terminates on all inputs, is not a computable problem. Theorem [Turing] There is no computer program that, taking as input a computer program, decides whether it terminates on all inputs. ( 11 of 19 )
  31. 31. Impossibility results Some results are inherently negative. If they are also significant, it is worth stating them in a negative way Theorem [Turing] Deciding whether a program terminates on all inputs, is not a computable problem. Theorem [Turing] There is no computer program that, taking as input a computer program, decides whether it terminates on all inputs. ( 11 of 19 )
  32. 32. Dead ends When we do research, experiments fail, proofs do not come through, and procedures give crazy results We use these failures to better understand our problems, and eventually to fix what did not work properly Sometimes, we get in a situation where there seem to be no way out, in which we are unable to find a way to fix the incorrect behaviours we encounter. We call these situation dead ends ( 12 of 19 )
  33. 33. Dead ends When we do research, experiments fail, proofs do not come through, and procedures give crazy results We use these failures to better understand our problems, and eventually to fix what did not work properly Sometimes, we get in a situation where there seem to be no way out, in which we are unable to find a way to fix the incorrect behaviours we encounter. We call these situation dead ends ( 12 of 19 )
  34. 34. Dead ends When you get into a dead end, you suffer from a lack of alternatives. There are many ways to try to get out Write down what you got as clearly as possible. The effect to organise the knowledge collected so far often shows alternatives Show your attempts to a colleague or to your supervisor. A fresh look often provides a new point of view Leave the problem apart for a while. When we do not think consciously to a problem, our mind follows different paths, and we leave room for new intuitions ( 13 of 19 )
  35. 35. Dead ends When you get into a dead end, you suffer from a lack of alternatives. There are many ways to try to get out Write down what you got as clearly as possible. The effect to organise the knowledge collected so far often shows alternatives Show your attempts to a colleague or to your supervisor. A fresh look often provides a new point of view Leave the problem apart for a while. When we do not think consciously to a problem, our mind follows different paths, and we leave room for new intuitions ( 13 of 19 )
  36. 36. Dead ends When you get into a dead end, you suffer from a lack of alternatives. There are many ways to try to get out Write down what you got as clearly as possible. The effect to organise the knowledge collected so far often shows alternatives Show your attempts to a colleague or to your supervisor. A fresh look often provides a new point of view Leave the problem apart for a while. When we do not think consciously to a problem, our mind follows different paths, and we leave room for new intuitions ( 13 of 19 )
  37. 37. Dead ends When you get into a dead end, you suffer from a lack of alternatives. There are many ways to try to get out Write down what you got as clearly as possible. The effect to organise the knowledge collected so far often shows alternatives Show your attempts to a colleague or to your supervisor. A fresh look often provides a new point of view Leave the problem apart for a while. When we do not think consciously to a problem, our mind follows different paths, and we leave room for new intuitions ( 13 of 19 )
  38. 38. Dead ends It is tempting to get some result out of a long period of hard work. But, when the result is a dead end, do not publish it! A dead end leads to nowhere, by definition. You have few chances to motivate your partial achievements You will spend time writing, submitting the paper, and receiving comments. You know in advance that the results are not complete, and that the direction leads nowhere: honestly, any reviewer will recognise these weaknesses, lowering your chances. Are you sure you are not wasting more time? ( 14 of 19 )
  39. 39. Dead ends It is tempting to get some result out of a long period of hard work. But, when the result is a dead end, do not publish it! A dead end leads to nowhere, by definition. You have few chances to motivate your partial achievements You will spend time writing, submitting the paper, and receiving comments. You know in advance that the results are not complete, and that the direction leads nowhere: honestly, any reviewer will recognise these weaknesses, lowering your chances. Are you sure you are not wasting more time? ( 14 of 19 )
  40. 40. Dead ends It is tempting to get some result out of a long period of hard work. But, when the result is a dead end, do not publish it! A dead end leads to nowhere, by definition. You have few chances to motivate your partial achievements You will spend time writing, submitting the paper, and receiving comments. You know in advance that the results are not complete, and that the direction leads nowhere: honestly, any reviewer will recognise these weaknesses, lowering your chances. Are you sure you are not wasting more time? ( 14 of 19 )
  41. 41. Publishing or discarding? A basic rule in Economy says to buy when the market is low, and to sell when the market is high Consider pursuing a scientific result as an investment. Publishing is the act which provides you with a revenue. It makes no sense to submit an article when you know in advance that it will not get published. You spend time and effort with no revenue ( 15 of 19 )
  42. 42. Publishing or discarding? A positive result has bigger chances to get published than a negative one. But negative results have some chances, too Try to positively present your negative outcomes Strong impossibility results are landmarks. Prepare yourself to defend them, but be sure they will be important in the long term Weak impossibility results are interesting only for specialists. You should carefully evaluate if your audience is wide enough to invest time and effort to try to publish such a negative achievement ( 16 of 19 )
  43. 43. Publishing or discarding? A positive result has bigger chances to get published than a negative one. But negative results have some chances, too Try to positively present your negative outcomes Strong impossibility results are landmarks. Prepare yourself to defend them, but be sure they will be important in the long term Weak impossibility results are interesting only for specialists. You should carefully evaluate if your audience is wide enough to invest time and effort to try to publish such a negative achievement ( 16 of 19 )
  44. 44. Publishing or discarding? A positive result has bigger chances to get published than a negative one. But negative results have some chances, too Try to positively present your negative outcomes Strong impossibility results are landmarks. Prepare yourself to defend them, but be sure they will be important in the long term Weak impossibility results are interesting only for specialists. You should carefully evaluate if your audience is wide enough to invest time and effort to try to publish such a negative achievement ( 16 of 19 )
  45. 45. Publishing or discarding? A positive result has bigger chances to get published than a negative one. But negative results have some chances, too Try to positively present your negative outcomes Strong impossibility results are landmarks. Prepare yourself to defend them, but be sure they will be important in the long term Weak impossibility results are interesting only for specialists. You should carefully evaluate if your audience is wide enough to invest time and effort to try to publish such a negative achievement ( 16 of 19 )
  46. 46. Publishing or discarding? Dead ends are the result of our efforts. What matters in Science are our findings, not the effort you made You should carefully judge what parts of your efforts are significant, and what are irrelevant to other people, with no regards for the time you invested Do not try to publish what you think is irrelevant! If you think so, how could you possibly convince a reader? You have to believe in the value of your results before trying to explain them ( 17 of 19 )
  47. 47. Experience Get into the habit of writing down, as clearly as possible, everything you find during your research you can reuse this pieces when writing the final paper you can use them to help a new collaborator to enter in your team you have an evidence of your work you can use them to illustrate the point you reached to, e.g., a colleague or to your supervisor you keep track of what you have understood. In the future, you may want to reuse these partial results elsewhere, and you do not want to rework all the details from scratch ( 18 of 19 )
  48. 48. Experience Get into the habit of writing down, as clearly as possible, everything you find during your research you can reuse this pieces when writing the final paper you can use them to help a new collaborator to enter in your team you have an evidence of your work you can use them to illustrate the point you reached to, e.g., a colleague or to your supervisor you keep track of what you have understood. In the future, you may want to reuse these partial results elsewhere, and you do not want to rework all the details from scratch ( 18 of 19 )
  49. 49. Experience Get into the habit of writing down, as clearly as possible, everything you find during your research you can reuse this pieces when writing the final paper you can use them to help a new collaborator to enter in your team you have an evidence of your work you can use them to illustrate the point you reached to, e.g., a colleague or to your supervisor you keep track of what you have understood. In the future, you may want to reuse these partial results elsewhere, and you do not want to rework all the details from scratch ( 18 of 19 )
  50. 50. Experience Get into the habit of writing down, as clearly as possible, everything you find during your research you can reuse this pieces when writing the final paper you can use them to help a new collaborator to enter in your team you have an evidence of your work you can use them to illustrate the point you reached to, e.g., a colleague or to your supervisor you keep track of what you have understood. In the future, you may want to reuse these partial results elsewhere, and you do not want to rework all the details from scratch ( 18 of 19 )
  51. 51. Experience Get into the habit of writing down, as clearly as possible, everything you find during your research you can reuse this pieces when writing the final paper you can use them to help a new collaborator to enter in your team you have an evidence of your work you can use them to illustrate the point you reached to, e.g., a colleague or to your supervisor you keep track of what you have understood. In the future, you may want to reuse these partial results elsewhere, and you do not want to rework all the details from scratch ( 18 of 19 )
  52. 52. Experience Get into the habit of writing down, as clearly as possible, everything you find during your research you can reuse this pieces when writing the final paper you can use them to help a new collaborator to enter in your team you have an evidence of your work you can use them to illustrate the point you reached to, e.g., a colleague or to your supervisor you keep track of what you have understood. In the future, you may want to reuse these partial results elsewhere, and you do not want to rework all the details from scratch ( 18 of 19 )
  53. 53. Experience In the long term, negative results will form the fundamental part of your experience you will know what directions and tools have few hopes to lead to a result you have a track of open problems, not-yet-attempted alternatives, variants to explore negative results, well documented, can speed up your achievements when you hear about a new result that overcomes a difficulty which blocked you in the past All these aspects of experience qualify you as an expert ( 19 of 19 )
  54. 54. Experience In the long term, negative results will form the fundamental part of your experience you will know what directions and tools have few hopes to lead to a result you have a track of open problems, not-yet-attempted alternatives, variants to explore negative results, well documented, can speed up your achievements when you hear about a new result that overcomes a difficulty which blocked you in the past All these aspects of experience qualify you as an expert ( 19 of 19 )
  55. 55. Experience In the long term, negative results will form the fundamental part of your experience you will know what directions and tools have few hopes to lead to a result you have a track of open problems, not-yet-attempted alternatives, variants to explore negative results, well documented, can speed up your achievements when you hear about a new result that overcomes a difficulty which blocked you in the past All these aspects of experience qualify you as an expert ( 19 of 19 )
  56. 56. Experience In the long term, negative results will form the fundamental part of your experience you will know what directions and tools have few hopes to lead to a result you have a track of open problems, not-yet-attempted alternatives, variants to explore negative results, well documented, can speed up your achievements when you hear about a new result that overcomes a difficulty which blocked you in the past All these aspects of experience qualify you as an expert ( 19 of 19 )
  57. 57. Experience In the long term, negative results will form the fundamental part of your experience you will know what directions and tools have few hopes to lead to a result you have a track of open problems, not-yet-attempted alternatives, variants to explore negative results, well documented, can speed up your achievements when you hear about a new result that overcomes a difficulty which blocked you in the past All these aspects of experience qualify you as an expert ( 19 of 19 )

×