Junk Science On Trial


Published on

Quality of the Evidence and Reliability of the Conviction

Published in: News & Politics, Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • See Boris Geller et al., A Chronological Review of Fingerprint Forgery, 44 J. Forensic Sci. 963 (1999) (discussing history of fingerprint forgeries); Mark Hansen, Troopers' Wrongdoing Taints Cases, A.B.A. J., Mar. 1994, at 22 (discussing numerous cases of evidence fabrication, "usually [involving] fingerprints," by New York State Police troopers). ; . In Actual Innocence, Barry *1320 Scheck, Peter Neufeld, and Jim Dwyer reported the findings of a Cardozo Law School Innocence Project study of 62 DNA exonerations secured in the United States; one of the more astounding conclusions was that a third of these cases involved "tainted or fraudulent science. ;
  • U.S. Dep't of Justice, Office of the Inspector Gen., The FBI Laboratory: An Investigation into Laboratory Practices and Alleged Misconduct in Explosives-Related Cases 217-43 (1997); see also John F. Kelly & Phillip K. Wearne, Tainting Evidence: Inside the Scandals at the FBI Crime Lab (1998); David Johnston, F.B.I. Lab Practices Faulted in Oklahoma Bomb Inquiry, N.Y. Times, Jan. 31, 1997, at A1.
  • f/n 14 philosopher C.A.J. Coady
  • 153 One suspects that control problems are perceived to be more worrisome where the law operates through an adversary system
  • He had earned numerous degrees, including a Ph.D. in mechanical engineering from Princeton, and he had taught mechanical engineering at the University of Kentucky since 1973. He had testified as an expert witness on multi-piece rims in over one hundred cases in forty-nine states, and he had studied this type of rim for twenty-six years.
  • Junk Science On Trial

    1. 1. Junk Science Quality of the Evidence and Reliability of the Conviction
    2. 2. Science is Everywhere <ul><li>Justice Breyer noted the pervasive use of expert witnesses in modern litigation: </li></ul><ul><li>&quot;Scientific issues permeate the law. Criminal courts consider the scientific validity of . . . DNA sampling or voiceprints, or expert predictions of defendants' 'future dangerousness,' which can lead courts or juries to authorize or withhold the punishment of death.&quot; </li></ul>
    3. 3. So – what is the problem with that? <ul><li>Science is becoming more complex </li></ul><ul><li>Witnesses remain as fallible as ever </li></ul><ul><li>Counsel lack the tools to evaluate and challenge </li></ul><ul><li>Courts lack the skills to decide </li></ul>
    4. 4. The ‘Junk Science’ Debate <ul><li>Some legal scholars, editorialists, and others argued that juries must be protected from the scourge of junk science. </li></ul><ul><li>Like-minded judges began to scrutinize scientific evidence closely before admitting it at trial and to exclude evidence that did not meet their strict standards. </li></ul><ul><li>Other judges were content to rely on the adversarial system to reveal any flaws in tendered scientific evidence. These judges favored admitting any scientific evidence that seemed relevant, and therefore rarely excluded scientific evidence before trial. </li></ul>
    5. 5. <ul><li>(1) the junk science problem is real; </li></ul><ul><li>(2) juries are not competent to decide complex scientific issues; </li></ul><ul><li>(3) tightening the rules for the admissibility of scientific evidence is a second best solution to the problem of junk science in tort; </li></ul><ul><li>(4) tightening the rules for the admissibility of scientific evidence will not solve the junk science problem in criminal cases -- broader reforms (within science) are needed. </li></ul>
    6. 6. The Complexity of Science: DNA Analysis <ul><li>The advent of DNA evidence, dramatically changed the legal landscape. </li></ul><ul><li>The initial DNA skirmishes over laboratory protocols quickly morphed into fights over statistical interpretation and population genetics. </li></ul><ul><li>The initial technique, Restriction Fragment Length Polymorphism (RFLP), was soon supplanted by Polymerase Chain Reaction (PCR) involving the DQ-alpha loci, which was followed by Polymarkers (PM) and D1S80. </li></ul><ul><li>These, in turn, were replaced by Short Tandem Repeats (STR), the current procedure. </li></ul><ul><li>In addition to nuclear DNA, mitochondrial DNA analysis and animal and plant DNA evidence have been introduced in criminal cases, raising further evidentiary questions. </li></ul><ul><li>Finally, the use of DNA databases for &quot;cold hits&quot; has presented its own fundamental issue: whether DNA evidence alone is sufficient to uphold a conviction. </li></ul>
    7. 7. Social Science <ul><li>Beginning in the 1980s, the use of social science research has had a significant impact on criminal litigation. </li></ul><ul><li>Battered Woman Syndrome, </li></ul><ul><li>Rape Trauma Syndrome, </li></ul><ul><li>Child Sexual Abuse Accommodation Syndrome (CSAAS), </li></ul><ul><li>Child Interviewing Techniques, </li></ul><ul><li>Munchausen syndrome by Proxy </li></ul>
    8. 8. Social Science <ul><li>Various types of profile evidence: of battering parents, drug couriers, sex offenders and child molesters </li></ul><ul><li>Neonaticide syndrome-postpartum psychosis. </li></ul><ul><li>The use of hypnotically refreshed testimony also produced vigorous debate. </li></ul><ul><li>The related and controversial phenomenon of repressed memory syndrome. </li></ul><ul><li>the deficiencies of eyewitness identifications </li></ul><ul><li>the existence of false confessions. </li></ul>
    9. 9. The Fallibility of Witnesses: Scientific Fraud <ul><li>Forged fingerprints </li></ul><ul><li>faked autopsies </li></ul><ul><li>false laboratory reports </li></ul><ul><li>perjured testimony (including the falsification of credentials) have all been reported </li></ul>
    10. 10. Scientific Fraud & Incompetence <ul><li>The Department of Justice's 1997 report on the FBI laboratory, issued by the Inspector General, graphically described negligence, misconduct, and other shortcomings of the premier crime laboratory in the country. </li></ul><ul><li>The investigation found scientifically flawed testimony, inaccurate testimony, testimony beyond the competence of examiners, improperly prepared laboratory reports, insufficient documentation of test results, inadequate record management and retention, and failures of management &quot;to resolve serious and credible allegations of incompetence.&quot; </li></ul>
    11. 11. Richard H. Underwood <ul><li>&quot;(one) can concede the important, even essential, role of the expert witness . . . </li></ul><ul><li>(and yet worry about) whether the vastly increased role of experts in the law poses a threat to the proper exercise of the court's arbitral role.&quot; </li></ul>
    12. 12. Evaluating Scientific Evidence <ul><li>One suspects that control problems are perceived to be more worrisome where the law operates through an adversary system </li></ul><ul><li>relationship between the law and expert testimony in English law as a &quot; shotgun marriage&quot; in which the men of law have attempted to keep the expert in a &quot;subordinate role.&quot; </li></ul>
    13. 13. Evaluating Scientific Evidence <ul><li>Role of Counsel </li></ul><ul><li>Role of the Courts </li></ul><ul><li>Role of the Experts </li></ul>
    14. 14. Counsel <ul><li>Plaintiffs' lawyers in civil cases wants a loose standard for the admission of scientific evidence-the plaintiffs' lawyers would prefer that the question be one of evidentiary weight or sufficiency </li></ul><ul><li>This is a large and increasingly well organized group that knows how to get its points across. </li></ul><ul><li>Conservative politicians refer to this group as &quot;The Trial Lawyers.&quot; </li></ul>
    15. 15. Counsel <ul><li>Defense lawyers in civil cases want a strict admissibility standard </li></ul><ul><li>This group is less organized than the plaintiff bar </li></ul><ul><li>Civil defense lawyers support and are supported by corporations and insurance companies. </li></ul>
    16. 16. Counsel <ul><li>Prosecutors (and other government lawyers-lawyers in enforcement agencies, who use scientific evidence) want a liberal standard of admissibility, or at least a standard that will apply in the same way to both the prosecution and the defense </li></ul>
    17. 17. Counsel <ul><li>Criminal defense lawyers want a strict, standard to be applied to the prosecution, but want a more liberal standard or no standard at all to be applied to the defense. </li></ul>
    18. 18. Judges & Juries <ul><li>IN HIS BOOK LABORATORY OF JUSTICE, David Faigman takes the Supreme Court to task for persistently failing to inquire into the merits of the scientific evidence figuring in the cases before it. </li></ul><ul><li>&quot;For 200-plus years the Court has demonstrated remarkable insensitivity to empirical questions,&quot; he contends. &quot;Its factual jurisprudence is slapdash, sloppy, and, too often, supercilious.&quot; </li></ul>
    19. 19. The Courts: Daubert (Mohan) <ul><li>&quot;By a preponderance of the evidence, the proponent of the expert evidenc e has to demonstrate to the judge that it is good evidence, perhaps in spite of what other experts think about it.“ </li></ul><ul><li>The judge must decide at least three questions: </li></ul><ul><li>(1) whether the witness is indeed expert in the field; </li></ul><ul><li>(2) whether the field is a genuine area of science; </li></ul><ul><li>(3) whether, given a positive answer to (1) and (2), his particular depositions are credible. . . . </li></ul><ul><li>All three of these questions pose difficulty for a legal tribunal since they seem to be questions that only an expert can answer. </li></ul><ul><li>Hence the specter of a vicious logical regress arises. </li></ul>
    20. 20. Applying the Rule: Novel, social, psychological science <ul><li>Goodyear Tire & Rubber Co. v. Thompson, a products liability case against arising from the plaintiff's injuries that he suffered when he was changing a multi-piece tire rim manufactured by Goodyear. </li></ul><ul><li>Dr. O.J. Hahn would have testified that the wheel multi-piece tire rim was defectively designed, and a safer technology was available. </li></ul>
    21. 21. ‘ Novel’ Science <ul><li>Dr. Hahn was very well qualified. </li></ul><ul><li>His opinion was based on the application of his knowledge of engineering principles, and his study of wheel bolting systems </li></ul><ul><li>These systems were successfully used on B-52s, and Dr. Hahn sensibly concluded that a similar bolting system would work on wheel assemblies on trucks and trailers. </li></ul>
    22. 22. ‘ Novel’ Science <ul><li>because Dr. Hahn had never: </li></ul><ul><li>(1) used a B-52 system on a truck, </li></ul><ul><li>(2) submitted his views to any tire manufacturer, </li></ul><ul><li>(3) submitted his suggested design for peer review, nor </li></ul><ul><li>(4) published his &quot;B-52 bolting theory&quot; in any article, </li></ul><ul><li>his testimony was not admissible </li></ul>
    23. 23. ‘ Novel’ Science <ul><li>Was this a proper exercise of (Daubert) gatekeeping power? </li></ul><ul><li>The dissent questioned whether it was not more sensible to admit testimony based on accepted scientific principles of engineering, and let the jury consider the points made by defendant as matters going to the weight and not the admissibility of the evidence </li></ul>
    24. 24. ‘ Social’Science <ul><li>In these difficult and thorny cases, the court &quot;used science to disguise a policy choice.&quot; </li></ul><ul><li>Justice Blackmun delved into the science in Roe v Wade drawing on his experience as counsel for the Mayo Clinic when he spent a summer enmeshed in the study of fetal viability. </li></ul><ul><li>much of the expertise in play in Supreme Court cases concerns social science, an area of frequently unsettled knowledge. </li></ul><ul><li>Unsettled science today may become established science tomorrow, or it may fall by the wayside. </li></ul><ul><li>In either case, the claimant science of the moment offers a hazardous foundation on which to erect principles of constitutional law. </li></ul>
    25. 25. Over value the Expert <ul><li>opinion (may) have a weight and authority that it may not deserve, </li></ul><ul><li>not just because he may not be a particularly good specimen of &quot;homo scientificus&quot; </li></ul><ul><li>but also because what he testifies to may be much more contestable than the deferential lay person is inclined to believe. </li></ul>
    26. 26. Accuracy Problems <ul><li>&quot;(f)orensic experts often present their findings with great confidence, but infallibility is unfortunately not a characteristic of forensic laboratories&quot; </li></ul><ul><li>&quot;(t)he problem with DNA evidence is no longer one of validity, but one of proficiency&quot;). </li></ul>
    27. 27. Over Value the Science <ul><li>some procedures have been subjected to little or no testing, </li></ul><ul><li>&quot;most of forensic science operates outside of (any) peer review systems.&quot; </li></ul><ul><li>Publication is limited. </li></ul><ul><li>Many techniques are not used &quot;outside,&quot; and </li></ul><ul><li>may not be generally accepted, at least by the larger scientific community. </li></ul>
    28. 28. Error Rates <ul><li>little is . . . known about the true error rates for almost all forensic science techniques </li></ul><ul><li>. . (and that) (t)he few disclosed error rates . . . are shockingly high.&quot; </li></ul>
    29. 29. Error Rates <ul><li>Are we to consider the degree of error inherent in a test or technique even when it is properly conducted (theoretical accuracy)? </li></ul><ul><li>Are we talking about performance rates on a particular lab's or a particular individual's proficiency tests? </li></ul><ul><li>Should an expert be required to combine the probability of a match due to lab error with the random-match probabilities (RMP)? (DNA) </li></ul><ul><li>Should the jury be told anything about &quot;error rates&quot;? </li></ul><ul><li>Should we demand some evidence of actual human error in the particular case before we inject &quot;possibilities&quot; into the trial? </li></ul><ul><li>Can the risks be minimized by providing opportunities for reciprocal discovery and testing? </li></ul>
    30. 30. Error Rates <ul><li>Investigation of the West Virginia State Police Crime Laboratory, Serology Division: </li></ul><ul><li>(1) no written documentation of testing methodology; </li></ul><ul><li>(2) no written quality assurance program; </li></ul><ul><li>(3) no written internal or external auditing procedures </li></ul><ul><li>(4) no routine proficiency testing of laboratory technicians; </li></ul><ul><li>(5) no technical review of work product; </li></ul><ul><li>(6) no written documentation of instrument maintenance and calibration; </li></ul><ul><li>(7) no written testing procedures manual; </li></ul><ul><li>(8) failure to follow generally accepted scientific testing standards with respect to certain tests; </li></ul><ul><li>(9) inadequate record-keeping; </li></ul><ul><li>10) failure to conduct collateral testing </li></ul>
    31. 31. Error Rates <ul><li>Reports of investigations of other crime laboratories are also available on several forensic web pages on the internet </li></ul><ul><li>De Paul University Center for Law and Science, at http://www. law.depaul.edu (last visited Nov. 6, 2000); </li></ul><ul><li>Scientific Testimony: An Online Journal, (last visited Nov. 6, 2000) at http://www.scientific.org; </li></ul><ul><li>Kim Kruglick, Forensic Resource and Criminal Law Search Site, at http:// www.kruglaw.com/lawsearch.htm (last visited Nov. 6, 2000); </li></ul><ul><li>The Evidence Site, at http://www.law.umich.edu/thayer/ (last visited Nov. 6, 2000). </li></ul>
    32. 32. Honesty Problems <ul><li>&quot;(t)he adversary system is probably the best tool we have for detecting . . (overconfidence, self-deception, and dishonesty) . . . </li></ul><ul><li>ironically, it is itself responsible for one common defect, namely, the expert's temptation to identify overmuch with the cause of his 'side.&quot; </li></ul>
    33. 33. Expert Witnesses <ul><li>expert witnesses can be &quot;co-opted&quot; by the prosecution-they may be little more than hired guns of the state. </li></ul><ul><li>Even honest witnesses who are trying to be objective may not be asked the right questions and may have their testimony filtered or contorted by the advocate. </li></ul><ul><li>&quot;Untoward results follow when expert evidence in complex cases is presented in adversarial fashion: Expert witnesses are manipulated for partisan purposes, some relevant scientific findings are never introduced, and unwarranted conclusions are not distinguished from valid research “ . . . </li></ul>
    34. 34. Interest in the Outcome <ul><li>Good detectives follow the evidence and do not manipulate the evidence to fit their pre-conceived notions or theories. </li></ul><ul><li>We all know, however, that there are bad detectives as well as over-zealous prosecutors. </li></ul>
    35. 35. Interest in the Outcome <ul><li>Forensic scientist D.H. Garrison, Jr. puts it this way: </li></ul><ul><li>&quot;Bad science is what forensic science becomes when an attorney or prosecutor, who often display(s) all the ethics of a full-grown hamster, get(s) a forensic scientist to play ball, to get with (the advocate's) program and see (the advocate's) big picture.&quot; </li></ul>
    36. 36. Interest in the Outcome <ul><li>Sometimes prosecutors will &quot;shop around&quot; until they find an expert who will tell them what they want to hear. </li></ul><ul><li>This happened in the infamous Rolando Cruz case. ] </li></ul><ul><li>The first lab guy says it's not the boot. . . . We don't like that answer, so there's no paper (report). We go to a second guy who used to do our lab. He says yes. So we write a report on Mr. Yes. Then Louise Robbins arrives. This is the boot, she says. That'll be $10,000. So now we have evidence </li></ul>
    37. 37. A troubling tale <ul><li>Lawyer and criminology Professor William Thompson tells a troubling tale about DNA. </li></ul>
    38. 38. A troubling tale <ul><li>(T)he laboratory report indicated that the DNA test had produced powerful evidence against both suspects </li></ul><ul><li>a five locus match between each suspect and the DNA found in semen extracted from </li></ul><ul><li>the victim. The report gave no indication that the evidence against suspect 1 was weaker than that against suspect 2. . . . </li></ul><ul><li>Examination of the underlying autorads confirmed a clear, unambiguous match with suspect 2, but indicated the evidence against suspect 1 was ambiguous and equivocal. </li></ul>
    39. 39. A troubling tale <ul><li>. . . initial suspicion . . . examiner bias. </li></ul><ul><li>prosecution position - the autorads had been scored objectively by a computer-assisted imaging device. </li></ul><ul><li>The prosecution claimed a scanner was used to create a digital image of each autorad . . . making the process entirely objective. </li></ul>
    40. 40. A troubling tale <ul><li>. . . a court order to re-score the autorads with the computer imaging device watched by counsel and an independent expert </li></ul><ul><li>In order to detect bands in the male vaginal extract lane that corresponded to those of suspect 1, the analyst had to increase the sensitivity of the computer to the point that it detected many additional &quot;bands&quot; that matched neither suspect. </li></ul><ul><li>The analyst then performed a &quot;manual override&quot; of the computer's scoring, instructing the computer to &quot;delete&quot; (i.e., ignore) all of the bands that matched neither suspect. </li></ul><ul><li>The analyst’s explanation? </li></ul><ul><li>He could tell by looking that the undeleted bands, were &quot;true&quot; bands while the others were not. </li></ul><ul><li>In fact, a number of the deleted bands he deleted had higher optical densities than the bands scored as matching suspect # 1 </li></ul>
    41. 41. The trap of “Consistent With” <ul><li>&quot;'(c)onsistent with' is one of those catch-all phrases that means (little more than) something could be possible. It is used a lot . . . especially on the witness stand when evidence can be interpreted in more than one way.&quot; </li></ul>
    42. 42. Sacco & Vanzetti <ul><li>Q: Have you an opinion as to whether bullet no. 3 was fired from the Colt automatic which is in evidence (Sacco's pistol)? </li></ul><ul><li>A: I have. </li></ul><ul><li>Q: And what is your opinion? </li></ul><ul><li>A: My opinion is that it is consistent with being fired by that pistol </li></ul>
    43. 43. Judgement: <ul><li>The presiding judge accepted this as if it were a statement that </li></ul><ul><li>&quot;it was . . . (Sacco's) pistol that fired the bullet that caused the death of Berardelli.&quot; </li></ul>
    44. 44. Reality Check <ul><li>Later Captain Proctor would give an affidavit that he was never able to find any convincing evidence to support another expert's testimony that the bullet was marked with scratches to prove it went through Sacco's pistol. </li></ul><ul><li>He also said he told the district attorney he would answer &quot;no&quot; if he were asked if he had found any such &quot;affirmative&quot; evidence, but that he had not been asked that question at the trial. </li></ul>
    45. 45. Would an expert lie? <ul><li>Mouse: Who are you? </li></ul><ul><li>Fox: I'm Robin Hood. </li></ul><ul><li>Mouse: You don't look like Robin Hood, but if you say you are then it must be true, because Robin Hood wouldn't tell a lie </li></ul>
    46. 46. Would an expert lie? <ul><li>Perhaps the most spectacular cases of lying involved: </li></ul><ul><li>Fred Zain, the former Chief of Serology at the West Virginia State Police Crime Laboratory, </li></ul><ul><li>Dr. Ralph Erdmann, a pathologist for forty-two Texas counties, </li></ul><ul><li>and the late Dr. Louise Robbins, an anthropologist for hire who is remembered for her bogus shoeprint and &quot;Cinderella&quot; testimony </li></ul>
    47. 47. Daniel J. Brodsky <ul><ul><ul><li>Barristers Chambers </li></ul></ul></ul><ul><ul><ul><li>11 Prince Arthur Avenue </li></ul></ul></ul><ul><ul><ul><li>Toronto, Ontario M5R 1B2 </li></ul></ul></ul><ul><ul><ul><li>Telephone: (416) 964-2618 </li></ul></ul></ul><ul><ul><ul><li>[email_address] </li></ul></ul></ul>