INTRODUCTION TO PSYCHOLOGICAL RESEARCH - 1 Journal article: E. Schreiber, & K. Schreiber (1995).Using relaxation techniques and positive self-esteem to improve academic achievement of college students. Psychological Reports, 76 (3), 929-930 Westerberg, VM Date: 19 March 2009
WESTERBERG, VM Page 2 of 8 INTRODUCTION TO PSYCHOLOGICAL RESEARCH - 1Question 11a.- Draw a construct flowchart that communicates the author´s theory about themuscle relaxation and academic performance. To the two constructs in the flowchartadd a credible moderating variable of your choice. Academic Muscle Relaxation Performance Room Temperature1b.- On the basis of your moderated flowchart draw a credible graph of the constructrelationships. Academic Performance Warm Cold Minimal Moderate Maximal Muscle relaxation
WESTERBERG, VM Page 3 of 81c.- Describe what your graph depicts (short paragraph)The above graph shows the increase in academic performance following a course onmuscle relaxation techniques. However, the students whose room was cold during therelaxation procedure showed a lower academic performance than those whose room waswarm during the procedure, indicating that, in the present study, room temperature hada moderating effect in the muscle relaxation – academic performance relationship.Question 2The authors were looking at the effects of muscle relaxation on various dependentvariables. In this context, critically evaluate the internal validity of the study.The internal validity evaluates how well the operational definition measures the effectand if something is being missed or measured inadvertently.The criteria indicated in the table below will be used to discuss the construct validity ofthe study regarding academic performance, self-esteem and anxiety. All points will beconsidered in the critical evaluation but only the relevant ones will be mentionedspecifically. Standardised scale (preferable) Strengths Adequate variable level number ( ≥2, otherwise it cannot vary) Correct magnitude levels (10, 20, 30, 32, 33, 38, 40, 50) Internal Sensible variable relationship (linear, curvilinear) validityevaluation Poor construct validity (inadequate cause-effect relationship) Poor extraneous variable control (noise) Weaknesses Sample bias Drop outs (not notified, not managed) Measures invalid / unreliable (sometimes, twice a week, often) Statistical analysis inadequate (lacking data, wrong calculations)
WESTERBERG, VM Page 4 of 8With regard to academic performance, both the control and the relaxation group wereevaluated comparing their mid- and final term grades. The scores obtained by therelaxation group were significantly superior to those of the control group in both terms.Academic scores for educational institutions are standardised and approved by thenational and local government educational administrations, that means that they shouldbe generally considered a valid means to measure academic performance as they wouldmeet all the strengths and lack (or control) all the weaknesses mentioned in the tableabove. Still, in the present study, no note has been made about differences inperformance according to age or gender. Additionally, the control group is larger than therelaxation group, and the male:female ratio is not well balanced in either group, muchmore so in the relaxation group. If these conditions were intentionally obviated in thearticle because the data obtained were irrelevant for the study of academic performance,then, it could be concluded that the internal validity of this construct is good. On theother hand, if there was in fact a difference and it was not mentioned, either intentionallyor mistakenly, then validity could not be confidentially rated.As for anxiety, a valid, standardised and accepted scale was used to measure it, namely,the Cattell & Scheier Anxiety Scale and the result found was that no significant differencewas noted between the groups. However, in this case, a note is made about thedifferences in anxiety level scores between men and women as, in fact, there is a strikingdifference in variability between genders. Looking at the differences in SD of both groups,anxiety levels are much more spread out among women. This, together with the widerrange in scores, indicates that anxiety levels are very heterogeneous amongst women and
WESTERBERG, VM Page 5 of 8that the resulting mean anxiety level score from the addition of male and female scoresmay be misleading. Separate evaluation of men and women should be considered tocontrol gender as a modifying variable.Construct validity evaluation in terms of self-esteem differs significantly from that of theprevious two variables. In the first place, the scale used in this study to measure self-esteem is Cattell & Scheier´s Anxiety Scale, which is designed to measure anxiety, not self-esteem. Additionally, self-esteem was measured on the basis of response to 2 questionseach with only 2 levels of variability and whose validity cannot been proven. Ifresearchers had used a standardised (reliable) test, like Rosenberg´s Self-Esteem Scale,they would have obtained valid results for this variable. Instead, due to a deficientprocedure used, the internal validity of the evaluation of self-esteem should be rated aspoor and be regarded as unreliable.Summarizing, the internal validity of 2 out of the 3 constructs is good, and given thatinternal validity is a pre-requisite to evaluate external validity, it has to be assumed thatthe overall internal validity is fair and therefore external validity can be assessed.Question 3Describe, and comment on, the external validity of the study, systematically covering allrelevant issues.
WESTERBERG, VM Page 6 of 8External validity refers to how well the research results can be generalised acrossparticipants (target and general population), across different operational definitions andacross settings (experimental and mundane realism and ecological validity).Participants were 52 college students aged 19 to 40. The sample is small and very possiblynot representative of the overall college (target) population with regard to male : femaleratio. The results could possibly be generalised to other country´s college students but itcould hardly do so to the general population, where the age range is wider (students-other adults). It could be assumed that gender ratio in colleges reflects that of the generalpopulation, but there are significant differences in gender distribution depending on thestudies, and one should expect the proportion of women studying midwifery to be higherthan that of men. Conversely, technical and engineering studies are more commonlychosen by men.With regard to the evaluation of external validity across different operational definitions,no graphs have been provided showing how levels were labelled and how many levels pervariable were considered. The C- score average prior to relaxation increased, but did itreach B, B+, A or A+? No reference is made to the magnitudes or levels used to measureanxiety. Self-esteem was rated as negative or positive, which is rather vague, and nomiddle values were given. Lower than usual, as usual, and higher than usual would havebeen a better choice.
WESTERBERG, VM Page 7 of 8To assess generalisation across settings, it should be noted that the experiment is notdated, and therefore it is hard to extrapolate settings in terms of time. If the test wasdone in the 70s maybe the percentage of males enrolled in tertiary education was higherthan that of women and that would explain, in part, gender distribution in the samples.As for experimental realism (engagement of participants in the study), although there isno direct mention to it, drop outs did not occur as, according to the data in the tablesgiven, all the students in both groups were considered in the various measurements ofthe different variables. This could by itself be an indicator of an elevated involvement ofparticipants in the experiment, the outcome of which would be the possibility to gethigher grades which is a good incentive to any responsible student.The reasons why this experiment may have a good ecological validity, at least in terms ofincreasing performance / skills, will now be discussed. Although the relaxation sessionsvery possibly took place in a classroom and in any case in a controlled environment,without distractions of note, it should be possible to reproduce them in the real world, inan uncontrolled environment. Therefore, if a student (or any adult of the generalpopulation) wishes to carry on / start up the relaxation techniques in the future, s/hecould represent the experimental conditions in an adequate real-life environment andcarry on. This concept could be the object of a follow-up experiment if the trainingsessions were recorded and sold as DVDs and the general population who used it tested
WESTERBERG, VM Page 8 of 8on how much their professional or academic performance improved after a 10-weekperiod with twice a week sessions.Real life conditions (a classroom, an office, a room at home) are adequately mimicked(mundane realism) in this study and individuals in the target and general populationsshould expect results to be equivalent in terms of skill improvement.This experiment carried out with the above mentioned modifications, namely the controlof gender as a modifying variable in anxiety evaluation, the use of an adequate scale tomeasure self-esteem, samples being more homogeneous in numbers and gender, theindication of the test date, and better labelling of scales among the most relevant ones,would make that this study turned into one with a much better overall validity, reliabilityand with a high potential of generalisation.