Download to read offline
Uploaded byMike Blamires
A
This document discusses quantitative and qualitative methods in educational research. It provides examples of quantitative approaches like standardized tests, surveys, and experimental methods. Qualitative approaches include action research, interviews, and documentary evidence. Both approaches are useful but have limitations, so a mixed methods approach is best. Important variables to consider include socioeconomic status, social disadvantage, ethnicity, gender, and school policies. Reliability and validity are also discussed.
More Related Content
A
- 1. Quantitative analysis within methodologicalapproaches in education Mike Blamires mb1@canterbury.ac.uk Faculty of Education
- 2. Relationships of Respectand Challenge Inquiry Habit of MindUsing Relevant Data Evidence-informed Conversations Evidence-Informed Conversations about Data Earl and Timperley (2000)
- 3. Which QTS stddoes this apply to? “Hook-ups shift electrical energy from the survival centres in the hindbrain to the reasoning centres in the midbrain and neocortex, thus activating hemispheric integration … the tongue pressing into the roof of the mouth stimulates the limbic system for emotional processing in concert with more refined reasoning in the frontal lobes”
- 4. Learning is facilitatedwhen teachers encourage risk-taking, tolerate mistakes and provide feedback that allows learners to monitor progress. That is ‘learning oriented’ rather than ‘performance oriented’ cultures (Dweck, 1989) Risk Challenge & Feedback
- 5. Teacher Inquiry andKnowledge Building Cycle Timperley, H. (2009)
- 6. Formative assessment Many teachers’previous training and approaches to teaching practice did not require them to interpret and use these kinds of data, because assessment information was about labelling and categorising students, and not for guiding and directing teaching How have we contributed to existing student outcomes? What do we already know that we can use to promote improved outcomes for students? What do we need to learn to do to promote these outcomes? What sources of evidence or knowledge can we utilise?
- 7. Two interrelated orientationsto research In simple terms we can think of two approaches to investigations in educational research: qualitative and quantitative. In the former we use words to describe the outcomes and in the latter we use numbers. Berry, J. & Hohmann (2005)
- 8. Quantitative approaches • Standardisedtests • Criterion references test • Surveys • Measurements, data collection and analysis • Experimental and quasi experimental methods
- 9. Shall I slicethe pizza into four or eight pieces? Make it four ! I could never eat eight ! Shall I slice the pizza into four or eight pieces? Make it four ! I could never eat eight ! How useful is quantitative data alone ?
- 10. (www.docs.google.com – Forms withinspreadsheets) (www.docs.google.com – Forms within spreadsheets)
- 11. Qualitative approaches • Actionresearch (sometimes) • Biography • Interviews • Ethnography • Use of documentary evidence (Hermeneutics)
- 12. How useful isqualitative evidence? • It (Factory Act, 1844) provides nothing more than that the children shall on certain days of the week, and for a certain number of hours (three in each day) be enclosed within the four walls of a place called a school and that the employer of the child shall receive weekly a certificate to that effect signed by a person designated by the subscriber as a school master or schoolmistress. ‘ On one occasion, on visiting a place called a school, from which certificates of attendance had issued I was so struck by the ignorance of the master that I said to him; “Pray sir, can you read?” His reply was “Aye summat!”: as a justification of his right to grant certificates, he added “At any rate I am before my scholars”. • Leonard Horner in Reports of the Inspectors of Factories 30th April 1837 p17 cited in Marx, K. Das Kapital (1873)
- 13. Why Mixed Methods? Allquantitative data is based upon qualitative judgments; and all qualitative data can be described and manipulated numerically.
- 14. Peter Tymms: EvidenceHierarchy
- 15. New Evidence Centresfor Education See a short article on these centres on the TTRB http://www.ttrb3.org.uk/?p=8885 • This follows on from the Goldacre report on BUILDING EVIDENCE INTO EDUCATION http://media.education.gov.uk/assets/files/pdf/b/ben%20golda
- 16. How good arethe numbers? What practice in your school do you think has the strongest evidence base and why? What practice in your school do you think has the weakest evidence base and why? Which area of practice are you unsure of in terms of its evidence? Where could you look for this?
- 17. Variables A variable isany entity that can take on different values. the independent variable is what you (or nature) manipulates -- a treatment or program or cause. The dependent variable is what is affected by the independent variable -- your effects or outcomes. For example, if you are studying the effects of a new educational program on student achievement, the program is the independent variable and your measures of achievement are the dependent ones.
- 18. What are themost important variables to consider in education? Independent Variable Dependent Variable Possible Measurement Socio Economic Status Academic Achievement ? Social Disadvantage Degree of exclusion and difficulty encountered in education Numbers of children registered as having SEN or a statement of SEN Ethnicity ? Explained and unexplained absences recorded according to ethnicity Gender Engagement with Physics % take up of GCSE and A Level Physics School ethos and policy on Behaviour ? Number of short term and permanent exclusions
- 19.
- 20. Units of analysis •Individuals • Classes • Schools • Artefacts (programmes, books, photos, newspapers) • Geographical units (town, census tract, state) • Social interactions (dyadic relations, divorces, arrests)
- 21. Levels of themeasurement • Stevens, 1946 measure of central tendency Geometric Mean Arithmetic Mean Median Mode
- 22. Gives examples ofdifferent levels of data Level of Data Example Ratio: has a absolute zero Eg Height, Age and Weight Interval: distance between measures Where means mean something eg Date & time Ordinal: attribute can be ordered Eg social class, grade and education level, levels of agreement Nominal: attribute can be named Eg Gender, Colours, Country,
- 23. Standard Deviation • isa widely used measurement of variability or diversity used in statistics and probability theory. It shows how much variation or "dispersion" there is from the average (mean, or expected value). A low standard deviation indicates that the data points tend to be very close to the mean, whereas high standard deviation indicates that the data are spread out over a large range of values. • Read more: http://www.answers.com/topic/standard- deviation#ixzz1IqwFC900
- 24. Same mean differentSD Example of two sample populations with the same mean and different standard deviations. Red population has mean 100 and SD 10; blue population has mean 100 and SD 50. Read more: http://www.answers.com/topic/standard-deviation#ixzz1Iqzf9FSm
- 25. Two Research Fallacies •Theecological fallacy occurs when you make conclusions about individuals based only on analyses of group data •An exception fallacy is sort of the reverse of the ecological fallacy. It occurs when you reach a group conclusion on the basis of exceptional cases.
- 26. Validity Validity to refersto the quality of various conclusions you might reach based on a research project. Conclusion Validity: In this study, is there a relationship between the two variables? Internal Validity: Assuming that there is a relationship in this study, is the relationship a causal one? Construct Validity: Assuming that there is a causal relationship in this study, can we claim that the program reflected well our construct of the program and that our measure reflected well our idea of the construct of the measure? External Validity: Assuming that there is a causal relationship in this study between the constructs of the cause and the effect, can we generalize this effect to other persons, places or times?
- 27. Reliability has to dowith the quality of measurement. In its everyday sense, reliability is the "consistency" or "repeatability" of your measures. •Inter-Rater or Inter-Observer Reliability Used to assess the degree to which different raters/observers give consistent estimates of the same phenomenon. •Test-Retest Reliability Used to assess the consistency of a measure from one time to another. •Parallel-Forms Reliability Used to assess the consistency of the results of two tests constructed in the same way from the same content domain. •Internal Consistency Reliability Used to assess the consistency of results across items within a test.
- 28. Challenge A A) Youhave been asked to develop a means of measuring progression in the development of pupil creativity that can be used across subject areas. How would you measure it and how would you demonstrate that learners were developing creativity?
- 29. Challenge B You area consultant to the Belizean government who wish to improve the quality of provision for pupils with disabilities in mainstream schools who are supported by specialist advisory teachers. What would you do and how would you prove it was effective?
- 30. Questions & Answers •Invite questions from the audience
- 31. references • In L.Earl and H. Timperley (ed.) Professional Learning Conversations: Challenges in Using Evidence for Improvement. London, Springer Academic Publishers, 121-126, 2008. • Dweck, C. S. (1989). Motivation. In A. Lesgold and R. Glaser (Ed.), Foundations for a Psychology of Education (pp. 87-136). Hillsdale, NJ: Erlbaum.
- 32. What are themost important variables to consider in education? Independent Variable Dependent Variable Possible Measurement Socio Economic Status Academic Achievement Exam results ? University Course? Career destination Social Disadvantage Degree of exclusion and difficulty encountered in education Numbers of children registered as having SEN or a statement of SEN Ethnicity Degree of participation in education Explained and unexplained absences recorded according to ethnicity Gender Engagement with Physics % take up of GCSE and A Level Physics School ethos and policy on Behaviour The success of the behaviour policy – the quality of behaviour Number of short term and permanent exclusions
- 33. Resources Trochim, W.M.K (2006)Social Research Methods http://www.socialresearchmethods.net/kb/index.php Berr, J. & Hohmann , U. (2005)Quantitative Methods in Education Research Centre for Teaching Mathematics, University of Plymouth http://www.edu.plymouth.ac.uk/resined/Quantitative/qu anthme.htm#A.%20%20%20%20INTRODUCTION The TTRB has a large range of reviews on evidence in education: www.ttrb3.org.uk
Editor's Notes
- #2 This is an introduction to the use of evidence in Education that takes a look at its importance and some things to consider when you are evaluating evidence.
- #3 As an Educational professional you are constantly using evidence to make decisions about your teaching. Some of this may be based on memories of you past experiences of teaching but this should not be your only source of evidence when you justify or challenge what you or colleagues do?
- #4 We do not want to be gullible teachers who accept ideas and approached without criticality. This slide perhaps shows how bad it can get sometimes. I look at this picture and wonder and hope. Who is going to be the one who spots the design flaw in the emperor's new outfit?
- #5 Dweck in 1989 focused on defining the positive classroom climate or culture for learning and so we can use evidence about learning and teaching to learn about how to enhance our teaching.
- #6 We can see this a cycle and the knowledge that we use to deepen and refine our skills can be gained from many sources. From our past experiences, from the experiences of colleagues but also from written accounts, videos of practice and also from evidence gained by research in classrooms in this country or from throughout the world. The challenge is finding what is relevant.
- #7 ‘ Assessment for Learning’ or formative assessment are widely promoted for example in the Cambridge Primary Review and this approach can be central to harnessing evidence in education.
- #8 This is very simplistic distinction but it serves to show us what might be different ends of a continuum. Although some would argue that they are very distinct.
- #9 These are broadly the main quantitative approaches that you may be familiar with. Standardised tests will include tests that have been applied to large samples of a target population to produce results that are readily comparible to this population. Eg IQ tests Reading Tests Criterion tests do not have that rigour but will be a test for the performance of a skill or skill set. Experiments will be discussed later but as we teach in classes rather than laboratories, experiments tend to make compromises on what can be controlled so we call them quasi experiments.
- #10 Clearly you need to develop some understanding of the numbers being used. See the consider-ed for Graham Birrell’s comment on the use of number in education This almost by way of aside is a cartoon that appeared in one of the early newspaper reports on PISA For me it’s a wonderful example of the misconceptions that many young students hold. I think it would make a wonderful piece of stimulus for materials that tap into students mathematical beliefs.
- #11 On line survey tools such as forms in google docs allow the rapid creation of a survey and also provides graphical summaries of the results.
- #12 With qualitative approaches we have a range of approaches that require classification and interpretation and they sometimes overlap. One can reflect upon one’s experience, implement a change and record what happens, interview people about their experiences of teaching and learning, observe a particular group in detail even joining the group in some cases or you can analyse legislation, policy and guidance.
- #13 Qualitative data gathered from the inspection of factories where there were child workers in the North of England led to the introduction of requirements for qualified teacherstatus. Recent deregulation and emerging curriculum requirements may remind some of you of Dicken’s ‘Hard Times’ Mr Chokemchild
- #15 Durham university have done a good deal of work with quantitative data sets and they suggest a hierarchy of evidence sources which is widely recognised. Eg NICER
- #17 With the previous slide in mind you might want to consider the evidence base for the practice that you are involved in ? Are like the Brain Gym enthusiasts on the earlier slide?
- #18 We are now going to look at aspects of methodology.
- #19 This table has some gaps where you might want to stop the video and see if you can work out what to put in the box (answers are at the end)
- #21 When you undertake research there are range of sources of data to be considered. This is a list of some examples.
- #22 Levels of measurement show the ‘statistical strength’ of the data has been gathered. The higher up the steps, the more sophisticated statistics can be applied.
- #23 Here we have examples of the different levels of data that we may encounter.
- #24 The greek symbols here denote the mean and the standard deviation from the mean. This shows us how the values are dispersed close to the mean or spread out. We can then see how common or exceptional an individual result is.
- #25 This illustration shows how this dispersion can differ when the mean is the same and the SD differs
- #26 The ecological fallacy occurs when you make conclusions about individuals based only on analyses of group data An exception fallacy is sort of the reverse of the ecological fallacy. It occurs when you reach a group conclusion on the basis of exceptional cases.
- #27 Validity refers to the quality of the relationships between the constructs, the measures and other relevant situations.
- #28 In its everyday sense, reliability is the "consistency" or "repeatability" of your measures.
- #33 This table has some gaps where you might want to stop the video and see if you can work out what to put in the box (answers are at the end)