The Research Process: Data, Measurement, and Variables
Research Process 1The Research Process: Data, Measurement, and Variables Edgardo Donovan RES 600 – Dr. Yufeng Tu Module 1 – Case Analysis Monday, October 20, 2008
Research Process 2 The Research Process: Data, Measurement, and Variables In a 1992 study by Melissa Chase entitled “The Role of Sports as a Social StatusDeterminant in Children”, the variables are poorly defined, sampling techniques are not applied,the research model mimics a 1976 project that fails to take into account recent video gameprogress, and preconceived unfounded notions are incorrectly cited as evidence. She providesextensive analysis of a large amount of survey data collected to better assess the determinants ofsocial popularity among school age children. A direct comparison with an earlier study fromBuchanam, Blankenbaker, and Cotton (1976) attempts to track societal changes on how childrenhave come to perceive popularity over time. The study unsuccessfully theorizes that in 1992more so than 1976 girls care more about their appearance and that boys continue to view sportsas their primary vehicle towards achieving social success. Developing a research model to prove any thesis is very difficult especially when it dealswith postulating a series of nominal variables with enough qualifiers to enable to act as reliableindicators of a particular phenomenon. In much social research and in program evaluation, weconsider the treatment or program to be made up of one or more variables. An educationalprogram can have varying amounts of time on task, classroom settings, student-teacher ratios,and so even the program can be considered a variable (Trochim). Chase attempts to apply a set of ordinal responses in terms of preference ranking from 1stto 4th for a variety of nominal variables. This is difficult enough to do in surveys that involveadults and is much more problematic when dealing with children. The variables selected caneasily apply both to people, groups, or even concepts making them difficult to use in a survey
Research Process 3involving children. How do children perceive appearance, having a lot of money, or being goodat sports? Is it related to fads? Are these imaginary concepts for them or are they rooted in realitylike owning the latest Barbie or BMX bicycle for example? If any amount of imagination isinvolved with a child grasping the aforementioned concepts how can research based on thesevariables be valid? Given the magnitude of this work one must ask whether there is a law of decreasingreturns in terms of validity and reliability when you present a survey of any length forcompletion to a group of elementary school children. Perhaps the survey was partitioned andsampled out randomly thereby avoiding this problem. However, we have no way of knowingthis. Limiting oneself to a very distinct demographic when attempting to analyze a quasi-universal phenomenon cannot be deemed as an effective example of sampling and is prone toresult in huge inconsistencies. I was surprised to see that this study was actually carried outfocusing on a restricted local geographical area in Michigan rather than a sample of the nationalelementary school population. I believe that if Melissa Chase would have conducted a traditionalsampling of the US elementary school population her results would have portrayed a much morediverse and nuanced audience. Over-dependence on a similar 1976 study contributes to the results of Chase’s workbeing less valid and ultimately unreliable. Chase theorizes that boys today continue to depend onthe same unhealthy fixation with sports as they did in 1976. However, the 1976 study did notinclude time spent on and importance allocated to playing video games. Unfortunately, thisimportant generational change in how schoolchildren and particularly boys spend their time was
Research Process 4left out of the more recent study. It would be interesting to see a study that incorporated videogames’ impact as a social determinant in children and whether that activity has been inverselyproportional to time spent playing sports. If so, it would be interesting to see if Chase would stillstand by her unsubstantiated belief that boys’ emphasis on sports is "unfortunate" since sports donot contribute to success beyond school (Chase) even when sports have been proven to be ahealthy way to get kids to exercise whereas video games are mostly a sedentary activity. Planning is one of the most critical phases of social science complex causal research.Constructing a research model is not easy. It forces you pick subject matter in which you will tryto prove or disprove a hypothesis. When complex causal research studies are concerned one mustdevise a research model that minimizes the structural limitations of previous contextual researchattempts rather than duplicate them as in Chase’s work. A research model must also be flexibleenough so that its structure conforms to the nature of the quantitative and qualitative data onewill gather and not vice-versa. Intuition and experience researching related subject matter mayhelp during the planning phase. Unfortunately, many research projects are not planned well andare limited by quantitative or qualitative metrics based on preconceived notions. Sometimes thisis realized when it is too late to start over increasing the pressure for the research team tocompensate for what their project lacks in substance by molding the research results to support athesis popular within the academic community. Initially, one can develop what may seem a very strong blueprint via the research modelby rationalizing a series of quantitative and/or qualitative metrics that will either support thehypothesis or not. However, as research is conducted it may not be unusual to come acrossnuances in the data possibly such as the video game phenomena or diverse demographics that
Research Process 5challenge some of the initial research model assumptions. Sometimes a researcher may think thatthey have isolated a set of quantitative data that will point towards or away from their hypothesisonly to find that the latter is correlated to a series of unexpected qualitative factors requiringfurther research. At this point the validity of the research project may be called into question.One may start over from scratch or they may feel pressure to work with what they have andcontextualize the information they have gathered to fit their pre-existing research model as muchas possible. A negative aspect of the Chase study was that in an attempt to bolster its thesis it madereferences to macro-social trends without offering any supporting evidence. In the study, boys’emphasis on sports is deemed "unfortunate" since sports do not contribute to success beyondschool whereas girls would like sports as much as boys if “properly socialized”. Furthermore,appearance alone has few long-term benefits that would prepare children for future careers andmeaningful contributions to society (Chase). The preceding arguments have been debated byeducators since the dawn of civilization and are far from settled. Since no attempt was made toreference studies ascertaining the preceding potentially controversial points of view one mustwonder if an unacceptable amount of intellectual bias runs unchecked throughout the rest of theresearch. In a 1992 study by Melissa Chase entitled “The Role of Sports as a Social StatusDeterminant in Children”, the variables are poorly defined, sampling techniques are not applied,the research model mimics a 1976 project that fails to take into account recent video gameprogress, and preconceived unfounded notions are incorrectly cited as evidence. She providesextensive analysis of a large amount of survey data collected to better assess the determinants of
Research Process 6social popularity among school age children. A direct comparison with an earlier study fromBuchanam, Blankenbaker, and Cotton (1976) attempts to track societal changes on how childrenhave come to perceive popularity over time. The study unsuccessfully theorizes that in 1992more so than 1976 girls care more about their appearance and that boys continue to view sportsas their primary vehicle towards achieving social success.
Research Process 7 BibliographyChase, Melissa. (1992). The role of sports as a social determinant for children. ResearchQuarterly for Exercise and Sport, 63, 418-424.Martens, B. (2004). Hitchhikers guide to hypotheses. Retrieved on 10 August 2008 fromhttp://www.theorywatch.com.Stead, Richard. (2008). The nature and methods of research. Leeds MetropolitanUniversity.Trochim, William. (2006). Variables. Social Research Methods Knowledge Base.