Presentation made as part of APA (Div 33) invited symposium on Atkins MR/ID cases and the death penalty. Reviewed 5 major issues re: FE "adjust or not" issue and 4 possible positions for experts that vary as a function of two dimensions (adjust or not and how; attention to "bright line" real-world legal context.

1. The Flynn Effect in Atkins MR/ID cases To adjust or not to adjust?—that is the question Kevin S. McGrew, Ph.D. Educational Psychologist Director Institute for Applied Psychometrics Presentation as part of invited symposium: Perspectives on Intellectual Disability and the Death Penalty—Toward More Effective Contributions of Psychologists in Atkins Cases. Aug, 4, 2011. American Psychological Association Annual Convention, Washington, DC..

3. Woodcock-Munoz Foundation (WMF Research Director)

4. Measurement Learning Consultants (MLC Associate Director)

5. WJ III author team (co-author with financial interest) or Riverside Publishing

7. (n = 143 references/articles)

8. For more information

11. Is the Flynn Effect a Scientifically Accepted Fact? Is it Real?

12. Should a Flynn Effect “Adjustment” be made in Atkins Cases?

13. Commentary—Bright lines and shades of gray

14. Recommended readings(Primary focus of presentation today)

15. Flynn Effect Definition “a gradual rise of IQ level that has been observed since the time when records of IQ first were kept. Although the average IQ remains 100 due to periodic renorming of IQ tests, raw scores have been rising. These increases have been roughly 9 points per generation (i.e., 30 years). ”

17. Sometimes a different example or perspective helps others understand the concept of norm obsolescence (aka, FE)

19. (b) subtract the same constant value from the patients current BMI and then evaluate that “adjusted” BMI against the BMI normative data from 1999-2000. Flegal, K. M. et al. JAMA 2010;303:235-241. Copyright restrictions may apply.

21. Is the Flynn Effect a Scientifically Accepted Fact? Is it Real?

22. Should a Flynn Effect “Adjustment” be made in Atkins Cases?

23. Commentary—Bright lines and shades of gray

25. 09 (8.0%) not supportive ("no"),

26. 04 (3.5%) mixed or neutral

27. 14 (12.4%) not relevantOf relevant papers (n = 99): 86.9 % yes 09.1 % no 04.0 % mixed

28. (2010) “The consensus of most (but not all) intelligence scholars is that the Flynn effect (FE), at the level of the global IQ score, is real…”

29. (2010) “The Flynn effect (FE) is well known” and the “rate of increase in the United States has apparently remained a fairly constant 3 points per decade since the 1930s.” “The Flynn Effect (FE) is real” and “The FE has been shown to be near 3 points per decade on average across a large number of studies, countries, and tests.” (2010)

30. (2010) “We agree that mean IQ scores shift over time, that psychologists should use the most current appropriate IQ tests, and that test developers should update norms. We also agree with McGrew (2010) that ‘the consensus of most (but not all) intelligence scholars is that the Flynn Effect (FE), at the level of the global IQ score, is real’ “

31. (1999) “ Even with a healthy dose of skepticism, the effect rises above purely methodological interpretation, and appears to have substantive import.”

33. Is the Flynn Effect a Scientifically Accepted Fact? Is it Real?

34. Should a Flynn Effect “Adjustment” be made in Atkins Cases?

35. Commentary—Bright lines and shades of gray

37. Greenspan (2006, 2007)

38. Moore (2006)..in response to Greenspan (2006)

39. Point/Counterpoint of 5 major FE (norm obsolescence) issues in professional(not legal) literature

40. 5 major FE (norm obsolescence) professional issues # 1. Is norm obsolescence (aka, Flynn Effect) an established scientific fact or has it only become a significant issue post-Atkins primarily for agenda-driven legal outcome purposes ? #2. What do the test manuals (publishers) say about the adjustment of an individual IQ score for norm obsolescence ? #3. Can a group statistical phenomena (norm obsolescence) be applied to scores of an individual? #4. Is the adjustment of an individual IQ score for norm obsolescence consistent with established professional standards of practice? #5. Is there sufficient scientific evidence upon which to base a norm obsolescence adjustment rule-of-thumb?

41. # 1 Is norm obsolescence (aka, Flynn Effect) a real established scientific fact or has it only become a significant issue post-Atkins primarily for agenda-driven legal outcome purposes?

42. Point: FE adjustment was given birth by Atkins more for advocacy/legal reasons #1 (2010) “For the 18 years between Flynn (1984) and the Atkins v. Virginia (2002) decision, the discourse about score shifts over time was not about the death penalty (DP). The Atkins decision changed the conversation. Scientific methods and the data did not change appreciably, although there was more of the latter. The way practitioners reported scores did not change noticeably.What changed was the opportunity to impact the outcome of DP sentencing proceedings.” “The fact that so many practice-focused publications on the score shifts over time have almost exclusively addressed DP cases strongly suggests that this is an outcome-focused issue. There is no voice calling for the reduction of obtained IQ scores in the high end or mid-range”

43. Counterpoint: FE adjustment recommendations pre-dates Atkins and is not MR/ID-specific #1 (1985) “Flynn proposed an IQ score of 70 on a “reference” IQ test (i.e., WAIS-R) be set in as the absolute criterion for mental retardation. Whenever a new test was published, it would be given together with the established reference test (e.g., WAIS-R), and the average mean IQ difference between the new test and the reference test would be used to “derive a new score equivalent to the old cutting line.” Flynn has been the canary in the coal mine since 1985 – proposed a form of an “adjustment” in 1985

44. Counterpoint: FE adjustment recommendations pre-dates Atkins and is not MR/ID-specific #1 (2000) “It is certain that over the past 50 years, literally millions of Americans evaded the label of mentally retarded designed for them by the test manuals. Whether this was good or bad depends on what one thinks of the label. Some will say millions avoided stigma. Others will say that millions missed out on needed assistance and classroom teachers were left unaided to cope with pupils for whom aid was needed.” Flynn sounds the canary alarm again in 2000---pre-Atkins.

45. Counterpoint: FE adjustment recommendations pre-dates Atkins and is not MR/ID-specific #1 Truscott and colleagues sounded the FE alarm for LD IQ-Achievement identification in two articles—one prior to Atkins and neither that mention Atkins DP cases (2001) (2003)

46. Counterpoint: FE adjustment recommendations pre-dates Atkins and is not MR/ID-specific #1 “Recency of norms” or FE had been mentioned in many IQ interpretation books prior to Atkins as a reason for score differences in general—not specific to MR/ID Just three select examples (1986) (1998) (1994)

47. Yes—but…. #1 (2010) “Most (79%) contemporary applied textbooks cite Flynn’s research and mention the FE by name or as a concept. In contrast to the claim in Walker v. True (2005), none recommend adjusting (italics in original)scores or recalculating norm means as generally accepted practice.

48. # 2 What do the test manuals (publishers) say about the adjustment of individual IQ scores for norm obsolescence ?

49. Point: IQ test manuals don’t mention it, which is an indicator that it is not customary best practice #2 (2008) “Six IQ tests met the inclusion criteria…We examined each test manual for citations of Flynn’s publications, references to the FE, and any specific recommendation for dealing with the increase in scores over time...The Stanford–Binet Intelligence Scales and the Kaufman Adolescent and Adult Intelligence Test manuals cite Flynn (1987) but make no specific recommendation for dealing with this statistical observation beyond the general admonition to follow the scoring rules strictly. The Reynolds Intellectual Assessment Scales, the Multidimensional Aptitude Battery, and the Woodcock–Johnson test do not reference the FE, either conceptually or by name.”

50. Point IQ test manuals don’t mention it, which is an indicator that it is not customary best practice #2 (2010) Examined test manuals from 6 major IQ batteries with adult norms “…it is well established that psychologists should follow the test manual instructions or risk violating the standardization. None of the test manuals … directs psychologists to engage in post-hoc score revisions”

51. Counterpoint: IQ test manuals don’t mention it due to context and vested interests. #2 (2011) “…context and vested interests likely make a difference. Moreover, test publishers have a vested interest in ignoring the Flynn Effect in test manuals because of the tacit admission attendant to discussing this phenomenon that tests have a limited shelf life and need to be updated frequently (Kaufman, 2010; Weiss, 2007, 2010). One exception is the following content from the WAIS-III Manual (Wechsler, 1997).

52. Counterpoint: Decreasing IQ test revision interval is implied recognition by test publishers #2 Notice the obvious trend towards shorter re-norming/revisions

53. # 3 Can a group statistical phenomena (norm obsolescence) be applied to the score of an individual?

54. Point: The FE is a group finding and should not be applied to the adjustment of an individual scores #3 (2008) “Of particular importance to the evaluating psychologist is whether the observed changes in group mean scores over time apply reliably to a specific individual. The question here is whether the FE’s broad construct applies to a specific evaluee’s IQ test scores, particularly when the individual’s obtained score is offered as evidence in support of a theory to prove a legal fact. Specifically, is it the generally accepted practice in the field of psychological testing to adjust a particular person’s earned IQ scores or to recalculate norm means on the basis of the FE?”

55. Counterpoint: The application of group data to individual IQ test performance constitutes virtually the entire field of psychometrics #3 (2010) “This is a curious point of contention, at best. The interpretation of any IQ score involves utilizing information from the standardization group (which almost never contained the individual being assessed) to interpret the performance of a specific individual. Indeed, this application of group data to the individual constitutes virtually the entirety of the field of psychometrics, as well as being the scientific foundation for the practice of medicine and mental health sciences. Gresham & Reschly (2011) make similar point – “a specious argument”. So do Reynolds, Niland, Wright & Rosenn (2010) – “This is really a straw man argument.”

56. Counterpoint: The application of group data to individual IQ test performance constitutes virtually the entire field of psychometrics #3 (2011) This is, frankly, a specious argument simply because any individual's IQ is entirely dependent upon group mean scores of the standardization sample. If the group mean has shifted upward, then the score that meets the intellectual disability standard has likewise increased by the same amount (Flynn, 1985). If this standardization sample is obsolete, then any individual score calculated in reference to the obsolete norms will be inflated by a factor of 0.3 points per year, or 3 points per decade from when the test was standardized.

57. Counterpoint: The application of group data to individual IQ test performance constitutes virtually the entire field of psychometrics #3 (2010) First, nearly all effects in psychology are based on aggregated data and groups and subsequent probability estimations from groups to individuals. Any prediction formula, and these are used often by most all psychologists involved in forensic cases, in employment decisions, prediction of achievement levels, diagnosis of specific learning disabilities, college admissions, and so on to name a few, is based on groups and then the formulae are applied to individual cases. However, to argue the FE should not be applied to individuals belies the fact that all IQs, obtained or otherwise, are to a significant extent based on a group effect and derived from aggregated data.

58. Yes—but…. #3 (2010) “An all-inclusive declaration about ‘every individual’ does not, however, adequately acknowledge the probabilistic nature of group data and potential inconsistency when applied to individuals”

59. # 4 Is the adjustment of an individual IQ score for norm obsolescence consistent with established professional standards of practice?

61. Survey of Diplomates in School Psychology

63. Counterpoint: Standards of practice are set by professional association consensus reports written by experts #4 (2010) “Taken to its logical conclusion, tying the standard of practice (or even “best practice”) to prevailing convention may impose a veritable straightjacket of circularity on the ability of professional psychology to remain scientifically abreast. To illustrate the circularity problem of anchoring “standards of practice” to prevailing convention: 1. Prevailing convention defines standards of practice. 2. Practice outside of prevailing convention is pejoratively inconsistent with the standard. 3. Scientific advancements cannot be legitimately incorporated into professional practice until they become the prevailing convention. 4. The standard of practice does not allow the adoption of scientific advancements until they are the prevailing convention.”

64. Counterpoint: Standards of practice are set by professional association consensus reports written by experts #4 (2011) Hagan et al. (2008) concluded that adjusting IQ scores and recalculating scores based on the Flynn Effect do not represent custom or standard of practice in professional psychology based on a survey with a participation rate among those surveyed. This so-called standard of practice, however, was based on a survey in which over one third of the sample responding was fundamentally unfamiliar with the concept at issue—namely, the Flynn Effect.

65. Counterpoint: Standards of practice are set by consensus reports written by experts #4 (2010) However, standards of practice are set by consensus reports written by experts. The most prominent guidelines for the assessment of ID represent the 11 editions of the manual for diagnosis by the American Association of Intellectual and Developmental Disabilities (Schalock et al., 2010), not cited by Hagan et al. (2008). Since 2002, this manual has explicitly recommended correcting IQ scores for norms obsolescence, with other researchers agreeing (e.g., Gresham, 2009; Kanaya & Ceci, 2007; Widaman, 2007).

66. Counterpoint: Standards of practice are set by consensus reports written by experts #4 “As discussed in the Users Guide (Schalock et al., 2007) that accompanies the 10th edition of this Manual, best practices require recognition of a potential Flynn Effect when older editions of an intelligence test (with corresponding older norms) are used in the assessment or interpretation of an IQ score.” “The main recommendation resulting from this work [regarding the Flynn Effect] is that all intellectual assessment must use a reliable and appropriate individually administered intelligence test. In cases of tests with multiple versions, the most recent version with the most current norms should be used at all times. In cases where a test with aging norms is used, a correction for the age of the norms is warranted.”

67. Yes—but…. #4 (2010) A national survey of American Board of Professional Psychology school psychologists and training directors of American Psychological Association–accredited clinical, counseling, and school psychology doctoral programs showed that most report or teach the practice of reporting obtained scores and—consistent with the dictates of test manuals—do not train future psychologists to alter IQ scores due to the FE (Hagan, Drogin, & Guilmette, 2008). Comment: Note shift away from citing Hagan et al. (2008) as establishing “standards”

68. Yes—but…. #4 (2010) “Our disagreement lies in how psychologists should convey IQ scores in light of the observation that mean scores drift over time….Our earlier contention that ‘adjusting the numerical scores”’ is inappropriate (Hagan et al., 2008, p. 623) remains unchanged (Hagan, Drogin, & Guilmette, 2010), and we appreciate this opportunity to state plainly that since research scientists are unable to agree on how much the FE might have affected an examinee's obtained score…” Comment: Again note shift away from interpretation of Hagan et al. (2008) as evidence of “standard.” Their last comment leads directly into probably the most salient scientific issue at this time - #5

69. # 5 Is there sufficient scientific evidence upon which to base a norm obsolescence adjustment rule-of-thumb?

70. Point: Extant norm obsolescence (FE) research does not provide a solid scientific basis for prescribing an agreed upon adjustment rule-of-thumb #5 (2010) “The drift in mean scores has not been shown to be reliable in degree or direction. For almost a quarter century, studies have repeatedly demonstrated a wide range of score shifts. The data for at least eight IQ tests across 13 countries showed per-year shifts from 1.250 to 0.133 (Flynn, 1987….Recent data continue to substantiate a wide range of per-year shifts from 0.917 to 0.117 (Flynn, 2009a), which is triple the proposed adjustment, even when the field is limited to U.S. scores on the Wechsler and Stanford-Binet tests.

71. Counterpoint: Scholars who have reviewed the literature have generally reached a professional consensus re: the FE 3/decade rule-of-thumb #5 (2010) “The Flynn effect (FE) is well known” and the “rate of increase in the United States has apparently remained a fairly constant 3 points per decade since the 1930s.” “The Flynn Effect (FE) is real” and “The FE has been shown to be near 3 points per decade on average across a large number of studies, countries, and tests.” (2010)

72. Counterpoint: Approximately true is preferable to the certainly false #5 (2010) Admittedly, debate and varied perspectives continue on precisely what score correction should be made to the WAIS-III in light of norms that were contemporaneous at the time of any particular administration. This variation in correction makes a strange argument, however, for making no correction at all to WAIS-III scores, or other tests in the Wechsler series for that matter (see Flynn, 2009). In agreement with Flynn, we would argue that the approximately true is preferable to the certainly false.

73. Counterpoint: Approximately true is preferable to the certainly false #5 (2008) (2008). In L. M. Glidden (Ed.), International Review of Mental Retardation (Vol. 35, pp. 121- 149). Boston: Elsevier “Some might complain about the lack of precision in our formula, but this formula is clearly better than having no adjustment of any kind. Proceeding without an adjustment formula during a time of enhanced normative performance ensures that persons, perhaps many persons, who deserve the label of MR will fail to receive it.”

74. Counterpoint: Quantitative research methods suggest that the 3/decade rule-of-thumb is supported by accepted scientific methods #5 (2010) A meta-analytic weighted mean of the 14 studies in Flynn (2009) is 2.80 (2.50, 3.09), close to Flynn’s (2009) unweighted average (2.99). More psychometric research would be helpful, but this level of precision supports the Flynn adjustment of 3 points per decade. (Fletcher, Stuebing & Hughes, 2010)

75. Counterpoint: Quantitative research methods suggest that the 3/decade rule-of-thumb is supported by accepted scientific methods #5 A meta-analytic weighted mean of the 14 studies in Flynn (2009) is 2.80 (2.50, 3.09), close to Flynn’s (2009) unweighted average (2.99). More psychometric research would be helpful, but this level of precision supports the Flynn adjustment of 3 points per decade. (Fletcher, Stuebing & Hughes, 2010)

76. Counterpoint: Quantitative research methods suggest that the 3/decade rule-of-thumb is supported by accepted scientific methods Box-whisker plot #5 Some exploratory data analyses of Fletcher et al 14 tabled studies (McGrew, 7-27-11)

77. Counterpoint: Quantitative research methods suggest that the 3/decade rule-of-thumb is supported by accepted scientific methods #5 Freq. polygon plot Kernel curve density plot Some exploratory data analysis of Fletcher et al 14 tabled studies (McGrew, 7-27-11)

79. Is the Flynn Effect a Scientifically Accepted Fact? Is it Real?

80. Should a Flynn Effect “Adjustment” be made in Atkins Cases?

81. Commentary—Bright lines and shades of gray

83. Bright lines and shades of gray: My (K. McGrew) analysis and commentary 5 (4) possible professional positions for psychological experts in Atkins cases “… my clinical intuition tells me that the moral spectacles though which each of us views the death penalty issue has likely swayed our interpretation of ambiguous data in ways which we may not even be aware…. …the ethical position of an expert witness providing testimony is not to argue either for or against FE adjustments but to inform the court about the extant research on the topic.” (Weiss, 2010) The dispassionate scientist who informs the court “Just the facts….ma’am--Dragnet”

84. Bright lines and shades of gray: My (K. McGrew) analysis and commentary 5 (4) possible professional positions for psychological experts in Atkins cases “Our disagreement lies in how psychologists should convey IQ scores” The proper exercise of clinical judgment should not be idiosyncratic. The slippery slope of clinical judgment goes both ways. (K. McGrew) “…it is characterized by its being systematic, formal (i.e., explicit and reasoned), and transparent” (Shalock & Luckasson, 2005) The practitioner- scientist (the astute clinician) “Forensic psychologists are explicitly encouraged to address likely ranges of IQ score variability and to discuss in relevant detail the strengths and weaknesses of the specific studies—however much at odds these may be—that attempt to define and quantify mean score shifts.” (Hagan et al., 2010)

85. Bright lines and shades of gray: My (K. McGrew) analysis and commentary 5 (4) possible professional positions for psychological experts in Atkins cases The practitioner- scientist (the astute clinician) Practical issue —the reporting of ranges may not be accepted in all courts. For example, TN Supreme Court (Coleman v TN, 2010) abandoned its strict bright line cutoff of 70 and issued a new standard allowing the TN courts to take testimony on things like the SEM and the FE. Specifically, the court stated : “However, consistent with the plain language to Tenn. Code Ann. § 39-13-203(a)(1), as interpreted in Howell v. State, an expert’s opinion regarding a criminal defendant’s I.Q. cannot be expressed within a range (i.e., that the defendant’s I.Q. falls somewhere between 65 to 75) but must be expressed specifically (i.e., that the defendant’s I.Q. is 75 or is “seventy (70) or below” or is above 70)

86. Bright lines and shades of gray 5 (4) possible professional positions for psychological experts in Atkins cases The scientist- practitioner (Cunningham & Tasse, 2010) Report the obtained scores from the historical testing. Describe the Flynn effect and associated studies demonstrating the progressive inflation in the group mean and the effect of this on observed IQ scores, including in the zone of ambiguity (IQ 71–80). Report the corrected IQ scores calculated from the interval between the year the test was normed and the year the test was administered, multiplied by the associated annual inflation rate from the best synthesis of available normative data. The comparative norm group at the time the test was administered is specified as this is the most meaningful interpretation of a norm referenced performance, i.e., what did the obtained score mean in relation to the contemporaneous norm group at the time that it was obtained?

87. Bright lines and shades of gray 5 (4) possible professional positions for psychological experts in Atkins cases The scientist- practitioner We assert that this procedure constitutes a scientifically informed, ethically sound, and clinically transparent practice at capital sentencing (see APA, 2002a, 2.04 Bases for Scientific and Professional Judgments, 3.04 Avoiding Harm, 9.02 Use of Assessments; Committee on Ethical Guidelines for Forensic Psychologists, 1991: VI. Methods and Procedures, Section A). In place of convention, prevailing practice, and authority, we assert that best science illuminates best practice and is fundamental to ethical conduct and professional standards. We find that a sufficient body of science supports interpreting obtained IQ scores in capital mental retardation hearings in reference to best estimates of norms that were contemporaneous to date of test administration, rather than historical standardization means” (Cunningham & Tasse, 2010)

88. Bright lines and shades of gray 5 (4) possible professional positions for psychological experts in Atkins cases The scientist- practitioner (2009)

89. Bright lines and shades of gray 5 (4) possible professional positions for psychological experts in Atkins cases The standard bearing ethicist “it is imperative that the FE, if it is real, be taken into account in capital cases….. If the FE is real, the failure to apply the Flynn correction as we have described it is tantamount to malpractice.” (Reynolds, et al., 2010) The fact that most program directors would never teach students to interpret scores in light of the Flynn Effect is to ignore scientific reality and potentially could be in violation of the Standards for Educational and Psychological Testing. “We maintain that failure to account for the Flynn Effect in death penalty cases can produce adverse social consequences for individual.” (unintended consequences standard) (Gresham & Reschly, 2011) ---- [continued next slide] If the norms are unjust….you MUST adjust !

90. Bright lines and shades of gray 5 (4) possible professional positions for psychological experts in Atkins cases The standard bearing ethicist “To this we would add that Principle 9.08 (Obsolete Tests and Outdated Test Results) of the ‘‘Ethical Principles of Psychologists and Code of Conduct'' (American Psychological Association, 2002) states in part: ‘‘(B) Psychologists do not base such decisions or recommendations on tests and measures that are obsolete and not useful for the current purpose [italics added].'' Failure to account for the Flynn Effect in test score interpretation in Atkins or any other cases is a violation of this ethical principle. In addition, failure to ensure the accurate interpretation of test scores in Atkins cases may possibly be a violation of the ethical Principle A: Beneficence and Nonmaleficence of the APA Code of Ethics. The principle states, in part, ‘‘Psychologists strive to benefit those with whom they work and take care to do no harm [italics added].'' In their professional actions, psychologists seek to safeguard the welfare and rights of those with whom they interact professionally and other affected persons (Gresham & Reschly, 2011) If the norms are unjust….you MUST adjust !

91. Possible psych. expert Atkins positions vary by two fundamental continua (K. McGrew) 5 (4) possible professional positions for psychological experts in Atkins cases “Adjustment” continuum Clinical/qualitativeEmpirical/quantitative The scientist- Practitioner The standard bearing ethicist Just say “no” to the FE or use “junk science” The dispassionate scientist who informs the court The practitioner- scientist (the astute clinician) Consider the legal “bright line”

92. Cunningham, M. D., & Tasse, M. J. (2010). Looking to Science Rather Than Convention in Adjusting IQ Scores When Death Is at Issue. Professional Psychology Research and Practice, 41(5), 413-419. Fletcher, J. M., Stuebing, K. K., & Hughes, L. C. (2010). IQ Scores Should Be Corrected for the Flynn Effect in High-Stakes Decisions. Journal of Psychoeducational Assessment, 28(5), 469-473. Flynn, J. R. (2006a). Tethering the elephant: Capital cases, IQ, and the Flynn effect. Psychology, Public Policy, and Law, 12, 170-189. Flynn, J. R. (2007b). Capital offenders and the death sentence: A scandal that must be addressed. Psychology in Mental Retardation and Developmental Disabilities, 32(3), 3-7 Flynn, J. R., & Widaman, K. F. (2008). The Flynn effect and the shadow of the past: Mental retardation and the indefensible and indispensible role of IQ. In L. M. Glidden (Ed.), International Review of Mental Retardation (Vol. 35, pp. 121-149). Boston: Elsevier. Greenspan, S. (2006). Issues in the use of the "Flynn effect" to adjust IQ scores when diagnosing MR. Psychology in Mental Retardation and Developmental Disabilities, 31(3), 3-7. Greenspan, S. (2007). Flynn-adjustment is a matter of basic fairness: Response to Roger B. Moore, Jr. Psychology in Mental Retardation and Developmental Disabilities, 32(3), 7-8. Greshman, F. & Reschly, D. J. (2011). Standard of practice and Flynn effect testimony in death penalty cases. Intellectual and Developmental Disabilities, 49 (3), 131-140. Hagan, L. D., Drogin, E. Y., & Guilmette, T. J. (2010a). IQ Scores Should Not Be Adjusted for the Flynn Effect in Capital Punishment Cases. Journal of Psychoeducational Assessment, 28(5), 474-476.

93. Hagan, L. D., Drogin, E. Y., & Guilmette, T. J. (2010b). Science Rather Than Advocacy When Reporting IQ Scores. Professional Psychology Research and Practice, 41(5), 420-423. McGrew, K. S. (2010). The Flynn Effect and Its Critics: Rusty Linchpins and ''Lookin' for g and Gf in Some of the Wrong Places''. Journal of Psychoeducational Assessment, 28(5), 448-468. McVaugh, G. S. & Cunningham, M. D. (2009). Atkins v. Virginia: Implications and recommendations for forensic practice. The Journal of Psychiatry and Law, 37, 131-187. Moore, R. B. (2006). Letter to the Editor: Modification of individual’s IQ scores is not acceptable professional practice. Psychology in Mental Retardation and Developmental Disabilities, 32(3), 11-12. Reynolds, C. R., Niland, J., Wright, J. E., & Rosenn, M. (2010). Failure to Apply the Flynn Correction in Death Penalty Litigation: Standard Practice of Today Maybe, but Certainly Malpractice of Tomorrow. Journal of Psychoeducational Assessment, 28(5), 477-481. Rodgers, J. L. (1999). A critique of the Flynn Effect: Massive IQ gains, methodological artifacts, or both? Intelligence, 26(4), 337-356. Scullin, M. H. (2006). Large state-level fluctuations in mental retardation classifications related to introduction of renormed intelligence test. American Journal of Mental Retardation, 111, 322-335. Weiss, L. G. (2010). Considerations on the Flynn Effect. Journal of Psychoeducational Assessment, 28(5), 482-493. Young, B., Boccacini, M. T., Conroy, M. A., & Lawson, K. (2007). Four practical and conceptual issues that evaluators should address in capital case mental retardation evaluations. Professional Psychology: Research and Practice, 38, 169-178