Author(s): Rajesh Mangrulkar, MD, 2009License: Unless otherwise noted, this material is made available under the terms oft...
Citation Key                          for more information see: http://open.umich.edu/wiki/CitationPolicyUse + Share + Ada...
Patients and Populations    Medical Decision-Making: Diagnostic             Reasoning I and II              Rajesh S. Mang...
Ask                Acquire                    Apply                            Thread 3: Diagnostic Reasoning             ...
Initial Diagnostic Reasoning           The Odyssey Reloaded      The Mechanic               The Clinician•  Failure to ent...
The Odyssey: Conclusion                                 50 Prime, flickrInitial Possibilities                             ...
Learning Objectives•  Acquire a basic understanding of diagnostic   question formulation•  Be able to define and calculate...
Recall case: Diagnostic Reasoning•  The case: A 56 year old man without heart   disease presents with sudden onset of   sh...
Diagnostic Reasoning: Your Intake•  Q: What other symptoms were you   feeling at the time?•  A: He has had no chest pain, ...
Diagnostic Reasoning: First StepsThe differential diagnosisYour tasks:•  Assign likelihoods to each possibility   –  E.g. ...
Your list
My listMy differential diagnosis  –  Pulmonary embolism  –  Congestive heart failure  –  Emphysema exacerbation  –  Asthma...
Probabilities  (1) PE                  P(PE) = 40%  (2) CHF                 P(CHF) = 30%  (3) Emphysema           P(emphys...
Prior Probabilities•  Based on many factors:  –  Clinician experience  –  Patient demographics  –  Characteristics of the ...
More information•  Family history: father and brother have had DVTs in   the past•  Physical Exam:   –  His blood oxygen s...
Diagnostic Reasoning: Testing•  If a Test existed that could rule in PE   as the diagnosis with 100% certainty:   then P(P...
Diagnostic Testing•  Facilitates the modification of probabilities.•  Can include any/all of the following:  –  Further hi...
PICO: The Anatomy of a Diagnostic      Foreground Question D P   •  Patient: define the clinical condition or disease     ...
Practice PICO    Case: A 56 year old man without heart diseaseP     presents with sudden onset of shortness of      breath...
Studies of Diagnostic Tests1. A well-defined group of people being  evaluated for a condition undergo:      - an experimen...
Diagnostic Testing: Statistical             Significance•  Statistical significance: strength of the   association between...
Diagnostic Testing: Clinical             Significance•  Clinical significance: how likely is the   diagnostic test going t...
What are the results - Diagnosis    Diagnostic performance is an association     between test result and diagnosis of a   ...
Which test characteristics?•  There are prevalence-dependent and   prevalence-independent measures in   diagnostic tests.•...
Test Characteristics: SeNsitivitySensitivity:•  The probability that the test will be positive   when the disease is prese...
Test Characteristics: SPecificitySpecificity:•  The probability that the test will be   negative when the disease is absen...
Test Characteristics: Predictive Values•  Positive predictive value: the probability   that a patient has a disease, given...
Diagnostic Test Characteristics•  Sens = A/(A+C)          Dx+   Dx-•  Spec = D/(B+D)    T+     A    B•  PPV = A/(A+B)     ...
To reflect upon... Why are sensitivity and specificity       prevalence-INdependent  characteristics, while positive andne...
Let s try it out                                          V/Q scanCase: To determine the diagnostic  accuracy of V/Q scans...
Let s try it outCase: To determine the  diagnostic accuracy of V/Q                PE+   PE-  scans for detecting pulmonary...
Let s try it out        PE+    PE-     •  Sens = 75/(75+75)                               = 50%VQ hi   75      25     •  S...
Modification of Probability     Pretest                           Test result   Probability            Test        changes...
Test Characteristics and Prevalence•  Sens = A/(A+C)                Dx+   Dx-•  Spec = D/(B+D)          T+    A     B•  PP...
Prevalence        PE+   PE-                    •    Sens = 50%VQ hi   75     25   •    Spec = 83%                    •    ...
Populations and PatientsPopulation view          Patient view•  Prevalence reflects   •  Same concept   the number of     ...
Modification of Probability     Pretest                           Test result   Probability            Test        changes...
An Important Question and                AssumptionQuestion: Are certain test characteristics fixed?Answer: Generally, yes...
Modification of Probability     Pretest                                Test result   Probability            Test          ...
Importance of Pre-Test Probability•  Hi-prob V/Q:   Sens = 50%, Spec = 83%                          Post-TP               ...
Importance of Pre-Test Probability•  Hi-prob V/Q:   Sens = 50%, Spec = 83%                         Post-TP                ...
Importance of Pre-Test Probability•  Hi-prob V/Q:   Sens = 50%, Spec = 83%                         Post-TP                ...
What did we just do?        100                                                    75% = P(PE|T+)                         ...
Modification of Probability     Pretest                                Test result   Probability            Test          ...
Fundamental AssumptionsSensitivity and specificity are constants,  regardless of the prevalence of the  disease in the stu...
Now, what do we do?                              Choices:                              • Treat as if patient has PE       ...
What if we change our pretest            probability?•  In essence, we are simultaneously   changing the prevalence:  –  O...
Importance of Pre-Test Probability    •  Hi-prob V/Q:        Sens = 50%, Spec = 83%                                       ...
Decision time                 Choices:                 • Treat as if patient has PE                 • Decide to get anothe...
Let s change it again…•  Again, we are changing the prevalence:  –  Young woman, no risk factors, some dyspnea,     no fam...
Importance of Pre-Test Probability  •  Hi-prob V/Q:   Sens = 50%, Spec = 83%                                              ...
What did we just do?                              Observation         As prevalence (pre-test probability) decreases,     ...
Fundamentally...Question: If you get a high probability V/Q scan  for the diagnosis of pulmonary embolism, is it  more lik...
Another example•  Question: Why don t we screen the general   population for HIV with an ELISA in the USA?•  Sensitivity: ...
Importance of Pre-Test Probability  •  HIV ELISA:   Sens = 99%, Spec = 95%                                            D+  ...
Pearls•  Studies of tests give you test characteristics,   not predictive values.•  The better the test, the greater the d...
Diagnostic Reasoning            The Odyssey Returns      The Mechanic               The Clinician•  Failure to entertain  ...
Ask      Acquire                    Apply                      AppraiseR. Mangrulkar
Additional Source Information                           for more information see: http://open.umich.edu/wiki/CitationPolic...
Upcoming SlideShare
Loading in …5
×

08.18.08: Diagnostic Reasoning I and II

1,009 views

Published on

Slideshow is from the University of Michigan Medical School's M1 Patients and Populations: Medical Decision-Making Sequence

View additional course materials on Open.Michigan:
openmi.ch/med-M1PatientsPopulations

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,009
On SlideShare
0
From Embeds
0
Number of Embeds
46
Actions
Shares
0
Downloads
9
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

08.18.08: Diagnostic Reasoning I and II

  1. 1. Author(s): Rajesh Mangrulkar, MD, 2009License: Unless otherwise noted, this material is made available under the terms ofthe Creative Commons Attribution–Non-commercial–Share Alike 3.0 License:http://creativecommons.org/licenses/by-nc-sa/3.0/We have reviewed this material in accordance with U.S. Copyright Law and have tried to maximize your ability to use, share, andadapt it. The citation key on the following slide provides information about how you may share and adapt this material.Copyright holders of content included in this material should contact open.michigan@umich.edu with any questions, corrections, orclarification regarding the use of content.For more information about how to cite these materials visit http://open.umich.edu/education/about/terms-of-use.Any medical information in this material is intended to inform and educate and is not a tool for self-diagnosis or a replacement formedical evaluation, advice, diagnosis or treatment by a healthcare professional. Please speak to your physician if you have questionsabout your medical condition.Viewer discretion is advised: Some medical content is graphic and may not be suitable for all viewers.
  2. 2. Citation Key for more information see: http://open.umich.edu/wiki/CitationPolicyUse + Share + Adapt { Content the copyright holder, author, or law permits you to use, share and adapt. } Public Domain – Government: Works that are produced by the U.S. Government. (USC 17 § 105) Public Domain – Expired: Works that are no longer protected due to an expired copyright term. Public Domain – Self Dedicated: Works that a copyright holder has dedicated to the public domain. Creative Commons – Zero Waiver Creative Commons – Attribution License Creative Commons – Attribution Share Alike License Creative Commons – Attribution Noncommercial License Creative Commons – Attribution Noncommercial Share Alike License GNU – Free Documentation LicenseMake Your Own Assessment { Content Open.Michigan believes can be used, shared, and adapted because it is ineligible for copyright. } Public Domain – Ineligible: Works that are ineligible for copyright protection in the U.S. (USC 17 § 102(b)) *laws in your jurisdiction may differ { Content Open.Michigan has used under a Fair Use determination. } Fair Use: Use of works that is determined to be Fair consistent with the U.S. Copyright Act. (USC 17 § 107) *laws in your jurisdiction may differ Our determination DOES NOT mean that all uses of this 3rd-party content are Fair Uses and we DO NOT guarantee that your use of the content is Fair. To use this content you should do your own independent analysis to determine whether or not your use will be Fair.
  3. 3. Patients and Populations Medical Decision-Making: Diagnostic Reasoning I and II Rajesh S. Mangrulkar, M.D.Fall 2008
  4. 4. Ask Acquire Apply Thread 3: Diagnostic Reasoning AppraiseR. Mangrulkar
  5. 5. Initial Diagnostic Reasoning The Odyssey Reloaded The Mechanic The Clinician•  Failure to entertain •  Entertain all important all possibilities possibilities•  Failure to pay •  Elicit and pay attention attention to all to description of all symptoms symptoms•  Failure to inform •  Inform and involve customer patients•  Failure to perform •  Perform effective diagnostic tests diagnostic tests
  6. 6. The Odyssey: Conclusion 50 Prime, flickrInitial Possibilities The Answer#1: Trunk latch defect (recall pending)#2: Ajar sensing defect on #2: Ajar sensing defect on side door side door#3: Side door not closing properly
  7. 7. Learning Objectives•  Acquire a basic understanding of diagnostic question formulation•  Be able to define and calculate sensitivity, specificity, and predictive values for diagnostic tests•  Understand how risk factors drive prior probabilities, and how this concept relates to prevalence•  Be able to modify probabilities from test results through 2x2 table calculations or Bayesian reasoning
  8. 8. Recall case: Diagnostic Reasoning•  The case: A 56 year old man without heart disease presents with sudden onset of shortness of breath.•  Description of the problem: Yesterday, after flying in from California the day before, the patient awoke at 3AM with sudden shortness of breath. His breathing is not worsened while lying down.
  9. 9. Diagnostic Reasoning: Your Intake•  Q: What other symptoms were you feeling at the time?•  A: He has had no chest pain, no leg pain, no swelling. He just returned yesterday from a long plane ride. He has no history of this problem before. He takes an aspirin every day. He smokes a pack of cigarettes a day.
  10. 10. Diagnostic Reasoning: First StepsThe differential diagnosisYour tasks:•  Assign likelihoods to each possibility –  E.g. P(X) = probability that X is the cause of the patient s symptoms•  Place the possibilities in descending order of likelihood
  11. 11. Your list
  12. 12. My listMy differential diagnosis –  Pulmonary embolism –  Congestive heart failure –  Emphysema exacerbation –  Asthma exacerbation
  13. 13. Probabilities (1) PE P(PE) = 40% (2) CHF P(CHF) = 30% (3) Emphysema P(emphysema) = 20% (4) Asthma P(asthma) = 10%•  What is the probability that the shortness of breath is due to either PE or CHF? 70%* *provided that both do not happen simultaneously. If there is a 10% chance that both may happen, then this number is 60%
  14. 14. Prior Probabilities•  Based on many factors: –  Clinician experience –  Patient demographics –  Characteristics of the patient presentations (history and physical exam) –  Previous testing –  Basic science knowledge•  Quite variable but can be standardized –  Clinical Prediction Rules –  http://www.mdcalc.com/wells-criteria-for-pulmonary-embolism-pe
  15. 15. More information•  Family history: father and brother have had DVTs in the past•  Physical Exam: –  His blood oxygen saturation is 89% on room air (low) –  His respiratory rate is 16, but his pulse rate is 105 beats per minute –  Examination of his lungs reveals some crackles and wheezes, but no pleural rub or evidence of consolidation. –  Swollen right leg, with firm vein below the knee•  CXR: normal•  EKG: sinus tachycardia http://www.mdcalc.com/wells-criteria-for-pulmonary-embolism-pe
  16. 16. Diagnostic Reasoning: Testing•  If a Test existed that could rule in PE as the diagnosis with 100% certainty: then P(PE | Test+) = 100%•  Two questions: –  What is this test called? Gold Standard –  Does P(CHF | Test+) = 0%? No
  17. 17. Diagnostic Testing•  Facilitates the modification of probabilities.•  Can include any/all of the following: –  Further history taking –  Physical Examination maneuver –  Simple testing (laboratory analysis, radiographs) –  Complex technology (stress testing, $$$ angiography, CT/MRI, nuclear scans)
  18. 18. PICO: The Anatomy of a Diagnostic Foreground Question D P •  Patient: define the clinical condition or disease clearly. I T •  Intervention: define the diagnostic test clearly •  Comparison group: define the accepted gold G C standard diagnostic test to compare the results against. O •  are the properties of the test itself (e.g.,interest A Outcomes of interest: the outcomes of accuracy and others we ll discuss).
  19. 19. Practice PICO Case: A 56 year old man without heart diseaseP presents with sudden onset of shortness of breath. Considering PE. Diagnostic Test to consider:I Ventilation / Perfusion ScanningC Gold standard: Pulmonary Source Undetermined angiographyO Need: Diagnostic accuracy Source Undetermined
  20. 20. Studies of Diagnostic Tests1. A well-defined group of people being evaluated for a condition undergo: - an experimental test, and - the gold standard test.2. Comparison is made between the accuracy of the new test and that of the gold standard.
  21. 21. Diagnostic Testing: Statistical Significance•  Statistical significance: strength of the association between… –  Diagnostic study results (for the diagnosis of a particular disease) –  Gold standard results (for the diagnosis of the same disease, in the same population)•  Strength = degree of accuracy
  22. 22. Diagnostic Testing: Clinical Significance•  Clinical significance: how likely is the diagnostic test going to affect patient care? –  Magnitude of the association between test results and the accepted gold standard –  Other literature (including those of the gold standard) –  Cost of the test, reproducibility of test –  Disease characteristics (will the test result affect management of the disease?)
  23. 23. What are the results - Diagnosis Diagnostic performance is an association between test result and diagnosis of a condition (as assessed by the gold standard) Disease + Disease - BONUS Test + A BWhat type of variable is TP FP FN TN disease state? Test - C D
  24. 24. Which test characteristics?•  There are prevalence-dependent and prevalence-independent measures in diagnostic tests.•  Prevalence-independent: sensitivity and specificity.•  Prevalence-dependent: positive and negative predictive values.
  25. 25. Test Characteristics: SeNsitivitySensitivity:•  The probability that the test will be positive when the disease is present.•  Of all the people WITH the disease, the percentage that will test positive.•  A seNsitive test is one that will detect most of the patients who have the disease (low false-Negative rate).
  26. 26. Test Characteristics: SPecificitySpecificity:•  The probability that the test will be negative when the disease is absent.•  Of all the people WITHOUT the disease, the percentage that will test negative.•  A sPecific test is one that will rarely be positive in patients who don t have the disease (low false-Positive rate).
  27. 27. Test Characteristics: Predictive Values•  Positive predictive value: the probability that a patient has a disease, given a positive result on a test. P (Disease + | Test +)•  Negative predictive value: the probability that a patient does not have a disease, given a negative result on a test. P (Disease - | Test -)
  28. 28. Diagnostic Test Characteristics•  Sens = A/(A+C) Dx+ Dx-•  Spec = D/(B+D) T+ A B•  PPV = A/(A+B) T- C D•  NPV = D/(C+D) A+C B+D
  29. 29. To reflect upon... Why are sensitivity and specificity prevalence-INdependent characteristics, while positive andnegative predictive values prevalence- DEpendent?
  30. 30. Let s try it out V/Q scanCase: To determine the diagnostic accuracy of V/Q scans for detecting pulmonary embolism, a study was conducted where 300 patients underwent both a V/Q and pulmonary angiogram. 150 Source Undetermined patients were found to have a PE Pulmonary Angiogram by PA gram. Of those, 75 patients had a high probability VQ scan. Of the 150 patients without a PE, 125 had a non-high probability VQ scan. Source Undetermined
  31. 31. Let s try it outCase: To determine the diagnostic accuracy of V/Q PE+ PE- scans for detecting pulmonary embolism, a study was conducted where 300 patients VQ hi 75 25 underwent both a V/Q and pulmonary angiogram. 150 patients were found to have a VQ PE by PA gram. Of those, 75 patients had a high probability other 75 125 VQ scan. Of the 150 patients without a PE, 125 did not have a high probability VQ scan (VQ other). 150 150
  32. 32. Let s try it out PE+ PE- •  Sens = 75/(75+75) = 50%VQ hi 75 25 •  Spec = 125/(125+25) = 83% VQ •  PPV = 75/(75+25) 75 125 = 75%other •  NPV = 125/(125+75) = 63% 150 150
  33. 33. Modification of Probability Pretest Test result Probability Test changes the P (Disease) Result probability of disease P (Disease|Test Result)R. Mangrulkar
  34. 34. Test Characteristics and Prevalence•  Sens = A/(A+C) Dx+ Dx-•  Spec = D/(B+D) T+ A B•  PPV = A/(A+B) T- C D•  NPV = D/(C+D) Disease A+C B+D Prevalence
  35. 35. Prevalence PE+ PE- •  Sens = 50%VQ hi 75 25 •  Spec = 83% •  PPV = 75% VQ •  NPV = 63%other 75 125 •  Prevalence = 50% ??? 150 150
  36. 36. Populations and PatientsPopulation view Patient view•  Prevalence reflects •  Same concept the number of implies how likely an people with the individual patient disease at a given has the disease moment •  P (Disease)
  37. 37. Modification of Probability Pretest Test result Probability Test changes the P (Disease) Result probability of disease P (Disease|Test Result) Disease PrevalenceR. Mangrulkar
  38. 38. An Important Question and AssumptionQuestion: Are certain test characteristics fixed?Answer: Generally, yes.Sensitivity and specificity are constants, regardless of the prevalence of the disease in the studied population (prevalence-INdependent)**Exceptions and caveats to this assumption are real, but are beyond the scope of this course
  39. 39. Modification of Probability Pretest Test result Probability Test changes the P (Disease) Result probability of disease P (Disease|Test Result) Disease sensitivity Prevalence specificityR. Mangrulkar
  40. 40. Importance of Pre-Test Probability•  Hi-prob V/Q: Sens = 50%, Spec = 83% Post-TP PV D+ D- Pre-TP/Prev PPV NPV T+ 75 25 50% 75% 63% T- 75 125 How do our predictive values relate to our probability after the test result is obtained (our post-test probabilities)?
  41. 41. Importance of Pre-Test Probability•  Hi-prob V/Q: Sens = 50%, Spec = 83% Post-TP PV D+ D- Pre-TP/Prev PPV NPV T+ 75 25 50% 75% 63% T- 75 125•  If our Pre-test Probability was 50%, and we obtain a hi-prob V/Q scan on this patient, what is our Post-test probability? 75%
  42. 42. Importance of Pre-Test Probability•  Hi-prob V/Q: Sens = 50%, Spec = 83% Post-TP PV D+ D- Pre-TP/Prev PPV NPV T+ 75 25 50% 75% 63% T- 75 125•  If our Pre-test Probability was 50%, and we obtain a V/Q-other scan on this patient, what is our Post-test probability? 37% (tricky: 1-63%)
  43. 43. What did we just do? 100 75% = P(PE|T+) i VQ h 50% VQ oth er 37% = P(PE|T-) 0 P (PE) P (PE | Test)R. Mangrulkar
  44. 44. Modification of Probability Pretest Test result Probability Test changes the P (Disease) Result probability of disease P (Disease|Test Result) Disease sensitivity Prevalence specificity Predictive Values (Positive and Negative)R. Mangrulkar
  45. 45. Fundamental AssumptionsSensitivity and specificity are constants, regardless of the prevalence of the disease in the studied population (prevalence-INdependent)*Positive and Negative Predictive Values are dependent on the prevalence of the disease in the studied population (prevalence-DEpendent)*with exceptions
  46. 46. Now, what do we do? Choices: • Treat as if patient has PE 75% = P(PE|T+) • Decide to get another test • Decide that patient does not have a PE Choices: 37% = P(PE|T-) • Treat as if patient has PE • Decide to get another test • Decide that patient does not have a PEWhat factors do you consider when making the next decision?
  47. 47. What if we change our pretest probability?•  In essence, we are simultaneously changing the prevalence: –  Original pre-TP = P(PE) = 50% HIGH RISK –  New pre-TP = P(PE) = 25% MED RISK•  Assuming that sensitivity and specificity are fixed…then we must recalculate our predictive values to determine our new post-test probabilities.
  48. 48. Importance of Pre-Test Probability •  Hi-prob V/Q: Sens = 50%, Spec = 83% D+ D- Post-TP T+ 75 25 Pre-TP/Prev PPV NPV T- 75 125hi risk 50% 75% 63% D+ D-med risk 25% 50% 83% 38/(38+38) 187/(187+37) T+ 38 38Our Pre-test Probability was 25%, we obtain a V/Q-other scan T- 37 187 on this patient, our Post-test probability is now…17%
  49. 49. Decision time Choices: • Treat as if patient has PE • Decide to get another test • Decide that patient does not have a PE50% = P(PE|T+) Choices: • Treat as if patient has PE • Decide to get another test17% = P(PE|T-) • Decide that patient does not have a PE
  50. 50. Let s change it again…•  Again, we are changing the prevalence: –  Young woman, no risk factors, some dyspnea, no family history, normal exam (except hr of 105) –  Lets consult our clinical prediction rule: –  http://www.mdcalc.com/wells-criteria-for-pulmonary-embolism-pe –  New pre-TP = P(PE) = 5%: LOW RISK
  51. 51. Importance of Pre-Test Probability •  Hi-prob V/Q: Sens = 50%, Spec = 83% D+ D- Pred Val T+ 75 25 Pre-TP/Prev PPV NPV T- 75 125hi risk 50% 75% 63% D+ D-lo risk 5% 15% 97% 8/(8+47) 238/(238+7) T+ 8 47 T- 7 238
  52. 52. What did we just do? Observation As prevalence (pre-test probability) decreases, positive tests are more likely to be false-positives 75% = P(PE|T+) i VQ h 50% VQ oth er 37% = P(PE|T-) VQ hi 15% = P(PE|T+) 5% 0 VQ other 3% = P(PE|T-) P (PE) P (PE | Test)R. Mangrulkar
  53. 53. Fundamentally...Question: If you get a high probability V/Q scan for the diagnosis of pulmonary embolism, is it more likely to represent a false positive test if the patient presented with…(a) many clinical features of PE (shortness of breath, chest pain, long plane ride), or(b) no clinical features of PE (no shortness of breath, no chest pain, no leg swelling, no long plane ride)?
  54. 54. Another example•  Question: Why don t we screen the general population for HIV with an ELISA in the USA?•  Sensitivity: 99%, Specificity: 95%•  A positive ELISA for HIV would most likely represent true HIV infection in which of these 2 patients? –  28 year old woman with history of iv drug abuse, husband has HIV. –  68 year old woman, monogamous her entire life, no blood transfusions, no ivda.
  55. 55. Importance of Pre-Test Probability •  HIV ELISA: Sens = 99%, Spec = 95% D+ D- Post-TP T+ 248 37 Pre-TP PPV NPV T- 2 713hi risk 25% 87% 99.7% D+ D-lo risk 1% 17% 99.9% T+ 99 495 T- 1 9405
  56. 56. Pearls•  Studies of tests give you test characteristics, not predictive values.•  The better the test, the greater the difference between pre-test and post-test probability.•  If your pre-test probability is very low (<10%) or very high (>90%), it is rare that a single test can help you, unless it is a nearly perfect test.
  57. 57. Diagnostic Reasoning The Odyssey Returns The Mechanic The Clinician•  Failure to entertain •  Entertain all important all possibilities possibilities•  Failure to pay •  Elicit and pay attention attention to all to description of all symptoms symptoms•  Failure to inform •  Inform and involve customer patients•  Failure to perform •  Perform effective diagnostic tests diagnostic tests
  58. 58. Ask Acquire Apply AppraiseR. Mangrulkar
  59. 59. Additional Source Information for more information see: http://open.umich.edu/wiki/CitationPolicySlide 4: Rajesh MangrulkarSlide 6: 50 Prime, flickr, http://www.flickr.com/photos/pernett/1544045987/, CC: BY http://creativecommons.org/licenses/by/2.0/deed.enSlide 19: Sources UndeterminedSlide 30: Sources UndeterminedSlide 33: Rajesh MangrulkarSlide 38: Rajesh MangrulkarSlide 39: Rajesh MangrulkarSlide 43: Rajesh MangrulkarSlide 44: Rajesh MangrulkarSlide 52: Rajesh MangrulkarSlide 58: Rajesh Mangrulkar

×