0
Methoden en Technieken Ning Ding September 19 2011
Overview <ul><li>Review of the last course </li></ul><ul><ul><li>Research Problem </li></ul></ul><ul><ul><li>Variables and...
Structure of the course <ul><li>Requirement: </li></ul><ul><ul><li>read the book </li></ul></ul><ul><ul><li>Online exercis...
Review of the last course <ul><li>Chapter 1 The Nature of Research </li></ul><ul><ul><li>Why do we need a research in educ...
Review of the last course <ul><li>Chapter 1 The Nature of Research </li></ul><ul><ul><li>Why do we need a research in educ...
Review of the last course <ul><li>Chapter 1 The Nature of Research </li></ul><ul><ul><li>Why do we need a research in educ...
Review of the last course <ul><li>Chapter 2 The Research Problem </li></ul><ul><ul><li>Characteristics of good research pr...
Review of the last course <ul><li>Chapter 2 The Research Problem </li></ul><ul><ul><li>Characteristics of good research pr...
Review of the last course <ul><li>Chapter 3 Variables and Hypotheses </li></ul><ul><ul><li>Quantitative vs Categorical Var...
Review of the last course <ul><li>Chapter 3 Variables and Hypotheses </li></ul><ul><ul><li>Quantitative vs Categorical Var...
Review of the last course <ul><li>Chapter 3 Variables and Hypotheses </li></ul><ul><ul><li>Quantitative vs Categorical Var...
Review of the last course <ul><li>Chapter 3 Variables and Hypotheses </li></ul><ul><ul><li>Advantages vs Disadvantages of ...
Review of the last course <ul><li>Chapter 3 Variables and Hypotheses </li></ul><ul><ul><li>Advantages vs Disadvantages of ...
Today’s course <ul><li>Chapter 7 Instrumentation </li></ul><ul><li>Chapter 8 Validity and Reliability </li></ul><ul><li>Ch...
Chapter 7 Instrumentation <ul><li>7.1 Data </li></ul><ul><li>7.2 Data Collection </li></ul><ul><ul><li>7.2.1 Where will th...
Chapter 7 Instrumentation 7.3.1 Validity:   is an important consideration in the choice of an instrument to be used in a r...
Chapter 7 Instrumentation <ul><li>7.3.4 Usability:   how easy the instrument will actually be to use. </li></ul><ul><ul><u...
Chapter 7 Instrumentation <ul><li>7.4 Who provides you the information? </li></ul><ul><li>Wie biedt de informatie? </li></...
Chapter 7 Instrumentation <ul><li>Who provides you the information? </li></ul><ul><li>Wie biedt de informatie? </li></ul><...
Chapter 7 Instrumentation <ul><li>Who provides you the information? </li></ul><ul><li>Wie biedt de informatie? </li></ul><...
Chapter 7 Instrumentation <ul><ul><li>Rating scales </li></ul></ul><ul><ul><li>interview list </li></ul></ul><ul><ul><li>o...
Chapter 7 Instrumentation <ul><ul><li>Rating scales </li></ul></ul><ul><ul><li>interview list </li></ul></ul><ul><ul><li>o...
Chapter 7 Instrumentation <ul><ul><li>Rating scales </li></ul></ul><ul><ul><li>interview list </li></ul></ul><ul><ul><li>o...
Chapter 7 Instrumentation <ul><ul><li>Rating scales </li></ul></ul><ul><ul><li>interview list </li></ul></ul><ul><ul><li>o...
Chapter 7 Instrumentation <ul><ul><li>Rating scales </li></ul></ul><ul><ul><li>interview list </li></ul></ul><ul><ul><li>o...
Chapter 7 Instrumentation <ul><li>7.7 Examples of Data-Collection Instruments </li></ul><ul><ul><li>7.7.1 The researcher h...
Chapter 7 Instrumentation <ul><li>7.7.3 Examples of Data-Collection Instruments </li></ul><ul><ul><li>The research himself...
Chapter 7 Instrumentation <ul><li>Examples of Data-Collection Instruments </li></ul><ul><ul><li>The subject himself </li><...
Chapter 7 Instrumentation <ul><li>Examples of Data-Collection Instruments </li></ul><ul><ul><li>The subject himself </li><...
Chapter 7 Instrumentation <ul><li>7.8 Item-format </li></ul><ul><ul><li>True-false </li></ul></ul><ul><ul><li>Matching </l...
Chapter 7 Instrumentation <ul><ul><li>7.9 Unobtrusive Measures </li></ul></ul><ul><ul><li>To eliminate the reactive effect...
Chapter 7 Instrumentation <ul><li>7.10 Types of Scores </li></ul><ul><ul><li>Raw  </li></ul></ul><ul><ul><li>Derived score...
Chapter 7 Instrumentation <ul><li>Norm-referenced vs Criterion-referenced instruments </li></ul><ul><ul><li>Norm: score co...
Chapter 7 Instrumentation <ul><li>7.12 Measurement Scales </li></ul><ul><ul><li>Nominal  </li></ul></ul><ul><ul><li>Ordina...
Chapter 7 Instrumentation <ul><li>Measurement Scales </li></ul><ul><ul><li>Nominal  </li></ul></ul><ul><ul><li>Ordinal  </...
Nominal:  Ordinal:  No logical order Ranked or ordered Chapter 7 Instrumentation
Chapter 7 Instrumentation <ul><li>Measurement Scales </li></ul><ul><ul><li>Nominal  </li></ul></ul><ul><ul><li>Ordinal  </...
Chapter 8 Validity and Reliability <ul><li>8.1 Basic concepts </li></ul><ul><ul><li>Validity: appropriate, meaningful, cor...
Chapter 8 Validity and Reliability <ul><li>8.2 Validity </li></ul><ul><li>Do the results of the assessment provide useful ...
Chapter 8 Validity and Reliability <ul><li>8.2 Validity </li></ul><ul><ul><li>8.2.1 Content-related evidence of validity <...
Chapter 8 Validity and Reliability <ul><li>8.2 Validity </li></ul><ul><ul><li>Content-related evidence of validity </li></...
Chapter 8 Validity and Reliability <ul><li>8.2 Validity </li></ul><ul><ul><li>Content-related evidence of validity </li></...
Chapter 8 Validity and Reliability <ul><li>8.3 Reliability </li></ul>Validity Instrument A Instrument B
Chapter 8 Validity and Reliability <ul><li>8.3 Reliability </li></ul>Instrument A Instrument A Week1 Week2 <ul><ul><li>Can...
Chapter 8 Validity and Reliability <ul><li>8.3 Reliability </li></ul><ul><ul><li>8.3.1 Errors of Measurement </li></ul></u...
Chapter 8 Validity and Reliability <ul><li>8.3 Reliability </li></ul><ul><ul><li>Can be reliable, but not valid </li></ul>...
Chapter 9 Internal Validity <ul><li>9.1 Internal Validity </li></ul><ul><ul><li>Observed differences on the dependent vari...
Chapter 9 Internal Validity <ul><li>Internal Validity </li></ul><ul><ul><li>Observed differences on the dependent variable...
Chapter 9 Internal Validity <ul><li>9.1 Internal Validity </li></ul><ul><ul><li>Observed differences on the dependent vari...
Chapter 9 Internal Validity <ul><li>Internal Validity </li></ul><ul><ul><li>Observed differences on the dependent variable...
Chapter 9 Internal Validity <ul><li>9.2 Threats to Internal Validity </li></ul><ul><ul><li>Subject Characteristics </li></...
Chapter 9 Internal Validity <ul><li>9.2 Threats to Internal Validity </li></ul><ul><ul><li>Testing: e.g. pretest </li></ul...
Chapter 9 Internal Validity <ul><li>9.2 Threats to Internal Validity </li></ul><ul><ul><li>Testing: e.g. pretest </li></ul...
Chapter 9 Internal Validity <ul><li>9.3 How to minimize these threats </li></ul>
Today’s course <ul><li>7 Instrumentation  </li></ul><ul><ul><li>Data </li></ul></ul><ul><ul><li>Data collection </li></ul>...
Upcoming SlideShare
Loading in...5
×

Sept19 college 2

629

Published on

SPO students at Rijksuniversiteit Groningen 2011-2012

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
629
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Unlike tests that are &amp;quot;criterion-referenced,&amp;quot; meaning that they compare the scores of each student to a given standard, norm-referenced tests compare the performance of the students to each other. 
  • Transcript of "Sept19 college 2"

    1. 1. Methoden en Technieken Ning Ding September 19 2011
    2. 2. Overview <ul><li>Review of the last course </li></ul><ul><ul><li>Research Problem </li></ul></ul><ul><ul><li>Variables and Hypotheses </li></ul></ul><ul><li>Today’s course </li></ul><ul><ul><li>Instrumentation </li></ul></ul><ul><ul><li>Validity and Reliability </li></ul></ul><ul><ul><li>Internal Validity </li></ul></ul>
    3. 3. Structure of the course <ul><li>Requirement: </li></ul><ul><ul><li>read the book </li></ul></ul><ul><ul><li>Online exercises </li></ul></ul><ul><li>130 </li></ul><ul><li>zmhzfx </li></ul>
    4. 4. Review of the last course <ul><li>Chapter 1 The Nature of Research </li></ul><ul><ul><li>Why do we need a research in education? </li></ul></ul><ul><ul><li>Waarom hebben we onderzoek naar het onderwijs nodig? </li></ul></ul><ul><ul><li>Beschrijven, voorspellen, uitleggen </li></ul></ul>Describe, Predict, Explain Mixed method = quantitative + qualitative
    5. 5. Review of the last course <ul><li>Chapter 1 The Nature of Research </li></ul><ul><ul><li>Why do we need a research in education? </li></ul></ul><ul><ul><li>Several types of research </li></ul></ul><ul><ul><li>Verschillende soorten onderzoek </li></ul></ul><ul><ul><li>Overview of the research process </li></ul></ul>Experimental research Correlational research Causal-comparative research Survey Ethnographic research Historical research Action research Compare; Single over time Relationship Cause, consequence Characteristics Everyday experience Past Active involvement
    6. 6. Review of the last course <ul><li>Chapter 1 The Nature of Research </li></ul><ul><ul><li>Why do we need a research in education? </li></ul></ul><ul><ul><li>Several types of research </li></ul></ul><ul><ul><li>Overview of the research process </li></ul></ul><ul><ul><li>Overzicht over het onderzoeksproces </li></ul></ul>
    7. 7. Review of the last course <ul><li>Chapter 2 The Research Problem </li></ul><ul><ul><li>Characteristics of good research problem </li></ul></ul><ul><ul><ul><li>Feasible </li></ul></ul></ul><ul><ul><ul><li>Clear </li></ul></ul></ul><ul><ul><ul><li>Significant </li></ul></ul></ul><ul><ul><ul><li>Ethical </li></ul></ul></ul>
    8. 8. Review of the last course <ul><li>Chapter 2 The Research Problem </li></ul><ul><ul><li>Characteristics of good research problem </li></ul></ul><ul><ul><ul><li>Feasible </li></ul></ul></ul><ul><ul><ul><li>Clear </li></ul></ul></ul><ul><ul><ul><li>Significant </li></ul></ul></ul><ul><ul><ul><li>Ethical </li></ul></ul></ul>Is computer-supported learning an effective learning method? Is computer-supported learning an effective learning method for secondary school students? Is computer-supported learning an effective learning method for secondeary school students in physics problem-solving? Is computer-ondersteund leren een effectieve leermethode voor middelbare school leerlingen voor het oplossen van natuurkundeproblemen in vergelijking met face-to-face leren?
    9. 9. Review of the last course <ul><li>Chapter 3 Variables and Hypotheses </li></ul><ul><ul><li>Quantitative vs Categorical Variables </li></ul></ul><ul><ul><ul><li>Vary in degree / do not vary in degree </li></ul></ul></ul><ul><ul><ul><li>Verschillen in gradatie/ verschillen niet in gradatie </li></ul></ul></ul><ul><ul><li>Independent vs Dependent Variables </li></ul></ul><ul><ul><ul><li>Study its effect / presumed to be affected </li></ul></ul></ul><ul><ul><ul><li>Het effect ervan bestuderen/ verondersteld beinvloed te worden </li></ul></ul></ul><ul><ul><li>Moderator Variables </li></ul></ul><ul><ul><li>Exaneous Variables </li></ul></ul>
    10. 10. Review of the last course <ul><li>Chapter 3 Variables and Hypotheses </li></ul><ul><ul><li>Quantitative vs Categorical Variables </li></ul></ul><ul><ul><ul><li>Vary in degree / do not vary in degree </li></ul></ul></ul><ul><ul><li>Independent vs Dependent Variables </li></ul></ul><ul><ul><ul><li>Study its effect / presumed to be affected </li></ul></ul></ul><ul><ul><li>Moderator Variables </li></ul></ul><ul><ul><li>Exaneous Variables </li></ul></ul>
    11. 11. Review of the last course <ul><li>Chapter 3 Variables and Hypotheses </li></ul><ul><ul><li>Quantitative vs Categorical Variables </li></ul></ul><ul><ul><ul><li>Vary in degree / do not vary in degree </li></ul></ul></ul><ul><ul><li>Independent vs Dependent Variables </li></ul></ul><ul><ul><ul><li>Study its effect / presumed to be affected </li></ul></ul></ul><ul><ul><li>Moderator Variables </li></ul></ul><ul><ul><li>Exaneous Variables </li></ul></ul>Are secondary school students’ physics performances influenced by their gender ? Worden de leerprestaties van de middelbare school leerlingen beinvloed door het geslacht ? As above, the relationship is moderated by SES Homework, teacher, etc.
    12. 12. Review of the last course <ul><li>Chapter 3 Variables and Hypotheses </li></ul><ul><ul><li>Advantages vs Disadvantages of stating hypotheses </li></ul></ul><ul><ul><li>Voordelen en Nadelen van het uitgaan van hypothesis </li></ul></ul><ul><ul><ul><li>Think deeply / bias </li></ul></ul></ul><ul><ul><ul><li>Prediction / unnecessary for some research types </li></ul></ul></ul><ul><ul><li>Significant Hypotheses </li></ul></ul><ul><ul><li>Directional vs Nondirectional Hypotheses </li></ul></ul><ul><ul><ul><li>Richtinggevende vs. Niet richtinggevende </li></ul></ul></ul>
    13. 13. Review of the last course <ul><li>Chapter 3 Variables and Hypotheses </li></ul><ul><ul><li>Advantages vs Disadvantages of stating hypotheses </li></ul></ul><ul><ul><ul><li>Think deeply / bias </li></ul></ul></ul><ul><ul><ul><li>Prediction / unnecessary for some research types </li></ul></ul></ul><ul><ul><li>Significant Hypotheses </li></ul></ul><ul><ul><li>Directional vs Nondirectional Hypotheses </li></ul></ul>In secondary school physics problem-solving, the learning performances of students learning via computers are different in comparison with that of students learning in face-to-face learning. Voor het oplossen van natuurkundenproblemen op de middelbare school, is de leerprestatie van de leerlingen die computers gebruiken beter dan die van leerlingen die leren met behulp van face-to-face leren.
    14. 14. Today’s course <ul><li>Chapter 7 Instrumentation </li></ul><ul><li>Chapter 8 Validity and Reliability </li></ul><ul><li>Chapter 9 Internal Validity </li></ul>
    15. 15. Chapter 7 Instrumentation <ul><li>7.1 Data </li></ul><ul><li>7.2 Data Collection </li></ul><ul><ul><li>7.2.1 Where will the data be collected? </li></ul></ul><ul><ul><li>Waar kunnen wij de data verzamelen </li></ul></ul><ul><ul><li>7.2.2 When will the data be collected? </li></ul></ul><ul><ul><li>Wanneer kunnen we de data verzamelen </li></ul></ul><ul><ul><li>7.2.3 How often are the data to be collected? </li></ul></ul><ul><ul><li>Hoe vaak worden de data verzameld </li></ul></ul><ul><ul><li>7.2.4 Who is to collect the data? </li></ul></ul><ul><ul><li>Wie verzamelt de data? </li></ul></ul><ul><li>7.3 Validity, Reliability, Objectivity, Usability </li></ul>
    16. 16. Chapter 7 Instrumentation 7.3.1 Validity: is an important consideration in the choice of an instrument to be used in a research investigation 7.3.2 Reliability: is another important consideration, since researchers want consistent results from instrumentation 7.3.3 Objectivity: refers to the absence of subjective judgments Belangrijk als je een instrument kiest voor een onderzoek Belangrijk als je een betrouwbare instrument wilt Er is geen subjectieve oordeel.
    17. 17. Chapter 7 Instrumentation <ul><li>7.3.4 Usability: how easy the instrument will actually be to use. </li></ul><ul><ul><ul><li>Is het gebruiksvriendelijk? </li></ul></ul></ul><ul><ul><ul><li>How long will it take to administer? </li></ul></ul></ul><ul><ul><ul><ul><li>Hoe lang duurt het uitvoeren? </li></ul></ul></ul></ul><ul><ul><ul><li>Are the directions clear? </li></ul></ul></ul><ul><ul><ul><ul><li>Zijn de aanwijzingen duidelijk? </li></ul></ul></ul></ul><ul><ul><ul><li>How easy is it to score? </li></ul></ul></ul><ul><ul><ul><ul><li>Is het gemakkelijk te scoren? </li></ul></ul></ul></ul><ul><ul><ul><li>Do equivalent forms exist? </li></ul></ul></ul><ul><ul><ul><li>Is er een gelijkwaardige vorm? </li></ul></ul></ul><ul><ul><ul><li>Have any problems been reported by others who used it? </li></ul></ul></ul><ul><ul><ul><li>Hebben anderen die het gebruikt hebben problemen gemeld? </li></ul></ul></ul>
    18. 18. Chapter 7 Instrumentation <ul><li>7.4 Who provides you the information? </li></ul><ul><li>Wie biedt de informatie? </li></ul><ul><ul><li>the researcher (observation) De onderzoeker (observatie) </li></ul></ul><ul><ul><li>the subject (test, questionnaire, log book) De subject </li></ul></ul><ul><ul><li>the informant (teacher, parents, interviewee) De informant </li></ul></ul><ul><li>Example </li></ul><ul><li>A researcher in an elementary school administers a weekly maths test that requires students to solve maths problems correctly. </li></ul>A researchers asks teachers to use a rating scale to rate each of their students on their phonic reading skills. Subject Informant
    19. 19. Chapter 7 Instrumentation <ul><li>Who provides you the information? </li></ul><ul><li>Wie biedt de informatie? </li></ul><ul><ul><li>the researcher (observation) De onderzoeker (observatie) </li></ul></ul><ul><ul><li>the subject (test, questionnaire, log book) De subject </li></ul></ul><ul><ul><li>the informant (teacher, parents, interviewee) De informant </li></ul></ul><ul><li>7.5 Where can you get the instrument? </li></ul><ul><ul><li>Waar kan ik het instrument krijgen? </li></ul></ul><ul><ul><li>b.v. Eric database </li></ul></ul>
    20. 20. Chapter 7 Instrumentation <ul><li>Who provides you the information? </li></ul><ul><li>Wie biedt de informatie? </li></ul><ul><ul><li>the researcher (observation) De onderzoeker (observatie) </li></ul></ul><ul><ul><li>the subject (test, questionnaire, log book) De subject </li></ul></ul><ul><ul><li>the informant (teacher, parents, interviewee) De informant </li></ul></ul><ul><li>Where can you get the instrument? </li></ul><ul><ul><li>Waar kan ik hetinstrument krijgen? </li></ul></ul><ul><ul><li>b.v. Eric database </li></ul></ul><ul><li>7.6 Voorbeeld van Data-Collection Instruments </li></ul><ul><ul><li>Rating scales, interview list, observation form, flow chart, behavior checklist, anecdotal information, time-and motion log </li></ul></ul>
    21. 21. Chapter 7 Instrumentation <ul><ul><li>Rating scales </li></ul></ul><ul><ul><li>interview list </li></ul></ul><ul><ul><li>observation form </li></ul></ul><ul><ul><li>flow chart </li></ul></ul><ul><ul><li>behavior checklist </li></ul></ul><ul><ul><li>anecdotal information </li></ul></ul><ul><ul><li>time-and motion log </li></ul></ul>Instructions: For each of the behaviors listed below, circle the appropriate number, using the following key: 5 = Excellent, 4 = Above Average, 3 = Average, 2 = Below Average, 1 = Poor. A. Explains course material clearly. 1 2 3 4 5 B. Establishes rapport with students. 1 2 3 4 5 C. Asks high-level questions. 1 2 3 4 5 D. Varies class activities. 1 2 3 4 5 Researcher
    22. 22. Chapter 7 Instrumentation <ul><ul><li>Rating scales </li></ul></ul><ul><ul><li>interview list </li></ul></ul><ul><ul><li>observation form </li></ul></ul><ul><ul><li>flow chart </li></ul></ul><ul><ul><li>behavior checklist </li></ul></ul><ul><ul><li>anecdotal information </li></ul></ul><ul><ul><li>time-and motion log </li></ul></ul>Researcher
    23. 23. Chapter 7 Instrumentation <ul><ul><li>Rating scales </li></ul></ul><ul><ul><li>interview list </li></ul></ul><ul><ul><li>observation form </li></ul></ul><ul><ul><li>flow chart </li></ul></ul><ul><ul><li>behavior checklist </li></ul></ul><ul><ul><li>anecdotal information </li></ul></ul><ul><ul><li>time-and motion log </li></ul></ul>Researcher
    24. 24. Chapter 7 Instrumentation <ul><ul><li>Rating scales </li></ul></ul><ul><ul><li>interview list </li></ul></ul><ul><ul><li>observation form </li></ul></ul><ul><ul><li>flow chart </li></ul></ul><ul><ul><li>behavior </li></ul></ul><ul><ul><li>checklist </li></ul></ul><ul><ul><li>anecdotal information </li></ul></ul><ul><ul><li>time-and motion log </li></ul></ul>Researcher
    25. 25. Chapter 7 Instrumentation <ul><ul><li>Rating scales </li></ul></ul><ul><ul><li>interview list </li></ul></ul><ul><ul><li>observation form </li></ul></ul><ul><ul><li>flow chart </li></ul></ul><ul><ul><li>behavior checklist </li></ul></ul><ul><ul><li>anecdotal information </li></ul></ul><ul><ul><li>time-and motion log </li></ul></ul>Researcher
    26. 26. Chapter 7 Instrumentation <ul><li>7.7 Examples of Data-Collection Instruments </li></ul><ul><ul><li>7.7.1 The researcher himself </li></ul></ul><ul><ul><ul><li>Rating scales, interview list, observation form, flow chart, behaviour checklist, anecdotal information, time-and motion log </li></ul></ul></ul><ul><ul><li>7.7.2 The subject himself </li></ul></ul><ul><ul><ul><li>Questionnaire, behaviour checklist, attitude scale, performance, competence test, projective test, socio-gram </li></ul></ul></ul>
    27. 27. Chapter 7 Instrumentation <ul><li>7.7.3 Examples of Data-Collection Instruments </li></ul><ul><ul><li>The research himself </li></ul></ul><ul><ul><ul><li>Rating scales, interview list, observation form, flow chart, behavior checklist, anecdotal information, time-and motion log </li></ul></ul></ul><ul><ul><li>The subject himself </li></ul></ul><ul><ul><ul><li>Questionnaire, behavior checklist, attitude scale, performance, competence test, projective test, socio-gram </li></ul></ul></ul>Teachers’ unions should be abolished. Strongly Strongly agree Agree Undecided Disagree disagree (5) (4) (3) (2) (1) Subject
    28. 28. Chapter 7 Instrumentation <ul><li>Examples of Data-Collection Instruments </li></ul><ul><ul><li>The subject himself </li></ul></ul><ul><ul><ul><li>Questionnaire, behavior checklist , attitude scale, performance, competence test, projective test, socio-gram </li></ul></ul></ul>Subject
    29. 29. Chapter 7 Instrumentation <ul><li>Examples of Data-Collection Instruments </li></ul><ul><ul><li>The subject himself </li></ul></ul><ul><ul><ul><li>Questionnaire, behavior checklist, attitude scale, performance , competence test, projective test, socio-gram </li></ul></ul></ul>Subject
    30. 30. Chapter 7 Instrumentation <ul><li>7.8 Item-format </li></ul><ul><ul><li>True-false </li></ul></ul><ul><ul><li>Matching </li></ul></ul><ul><ul><li>Interpretative exercises </li></ul></ul><ul><ul><li>Multiple Choice </li></ul></ul><ul><ul><li>Short-answer items </li></ul></ul><ul><ul><li>Essay questions </li></ul></ul>selection supply
    31. 31. Chapter 7 Instrumentation <ul><ul><li>7.9 Unobtrusive Measures </li></ul></ul><ul><ul><li>To eliminate the reactive effect, data collection procedure that involve no intrusion into the naturally occurring course of events. </li></ul></ul>
    32. 32. Chapter 7 Instrumentation <ul><li>7.10 Types of Scores </li></ul><ul><ul><li>Raw </li></ul></ul><ul><ul><li>Derived scores: percentile ranks, standard scores </li></ul></ul><ul><li>7.11 Norm-referenced vs Criterion-referenced instruments </li></ul><ul><ul><li>Norm: score compared with a norm group </li></ul></ul><ul><ul><li>Criterion: score compared with a goal or target that each learner achieves </li></ul></ul>
    33. 33. Chapter 7 Instrumentation <ul><li>Norm-referenced vs Criterion-referenced instruments </li></ul><ul><ul><li>Norm: score compared with a norm group </li></ul></ul><ul><ul><li>Criterion: score compared with a goal or target that each learner achieves </li></ul></ul>He solved at least 75% of the problems and scored above 90 percent of all the students in the class . Criterion-referenced Norm-referenced
    34. 34. Chapter 7 Instrumentation <ul><li>7.12 Measurement Scales </li></ul><ul><ul><li>Nominal </li></ul></ul><ul><ul><li>Ordinal </li></ul></ul><ul><ul><li>Interval </li></ul></ul><ul><ul><li>Ratio </li></ul></ul>
    35. 35. Chapter 7 Instrumentation <ul><li>Measurement Scales </li></ul><ul><ul><li>Nominal </li></ul></ul><ul><ul><li>Ordinal </li></ul></ul><ul><ul><li>Interval </li></ul></ul><ul><ul><li>Ratio </li></ul></ul>Measurement Scale Characteristics Nominal Groups and labels data only; reports frequencies or percentages. Ordinal Ranks data; uses numbers only to indicate ranking. Interval Assumes that equal differences between scores really mean equal differences in the variable used. Ratio All of the above, plus true zero point.
    36. 36. Nominal: Ordinal: No logical order Ranked or ordered Chapter 7 Instrumentation
    37. 37. Chapter 7 Instrumentation <ul><li>Measurement Scales </li></ul><ul><ul><li>Nominal </li></ul></ul><ul><ul><li>Ordinal </li></ul></ul><ul><ul><li>Interval </li></ul></ul><ul><ul><li>Ratio </li></ul></ul>
    38. 38. Chapter 8 Validity and Reliability <ul><li>8.1 Basic concepts </li></ul><ul><ul><li>Validity: appropriate, meaningful, correct, useful </li></ul></ul><ul><ul><li>Reliability: the consistency of one item/instrument to another </li></ul></ul><ul><ul><li>Objectivity: </li></ul></ul>
    39. 39. Chapter 8 Validity and Reliability <ul><li>8.2 Validity </li></ul><ul><li>Do the results of the assessment provide useful information about the topic or variable being measured? </li></ul><ul><ul><li>8.2.1 Content-related evidence of validity </li></ul></ul><ul><ul><li>8.2.2 Criterion-related evidence of validity </li></ul></ul><ul><ul><li>8.2.3 Construct-related evidence of validity </li></ul></ul>
    40. 40. Chapter 8 Validity and Reliability <ul><li>8.2 Validity </li></ul><ul><ul><li>8.2.1 Content-related evidence of validity </li></ul></ul><ul><ul><ul><li>Content and format of the instrument </li></ul></ul></ul>
    41. 41. Chapter 8 Validity and Reliability <ul><li>8.2 Validity </li></ul><ul><ul><li>Content-related evidence of validity </li></ul></ul><ul><ul><ul><li>Content and format of the instrument </li></ul></ul></ul><ul><ul><li>8.2.2 Criterion-related evidence of validity </li></ul></ul><ul><ul><ul><li>Relationship between scores obtained using the instrument and scores obtained from other instruments  predictive / concurrent </li></ul></ul></ul>
    42. 42. Chapter 8 Validity and Reliability <ul><li>8.2 Validity </li></ul><ul><ul><li>Content-related evidence of validity </li></ul></ul><ul><ul><ul><li>Content and format of the instrument </li></ul></ul></ul><ul><ul><li>Criterion-related evidence of validity </li></ul></ul><ul><ul><ul><li>Relationship between scores obtained using the instrument and scores obtained  predictive / concurrent </li></ul></ul></ul><ul><ul><li>8.2.3 Construct-related evidence of validity </li></ul></ul><ul><ul><ul><li>Psychological construct being measured by the instrument </li></ul></ul></ul>
    43. 43. Chapter 8 Validity and Reliability <ul><li>8.3 Reliability </li></ul>Validity Instrument A Instrument B
    44. 44. Chapter 8 Validity and Reliability <ul><li>8.3 Reliability </li></ul>Instrument A Instrument A Week1 Week2 <ul><ul><li>Can be reliable, but not valid </li></ul></ul><ul><ul><li>If unreliable, must not be valid </li></ul></ul>Validity
    45. 45. Chapter 8 Validity and Reliability <ul><li>8.3 Reliability </li></ul><ul><ul><li>8.3.1 Errors of Measurement </li></ul></ul><ul><ul><li>8.3.2 Reliability Coefficient </li></ul></ul><ul><ul><ul><ul><li>Test-retest </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Equivalent-form method </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Internal-consistency method </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Split-half procedure </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Kuder-Richardson Approaches </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Cronbach’s α </li></ul></ul></ul></ul></ul>
    46. 46. Chapter 8 Validity and Reliability <ul><li>8.3 Reliability </li></ul><ul><ul><li>Can be reliable, but not valid </li></ul></ul><ul><ul><li>If unreliable, must not be valid </li></ul></ul><ul><ul><li>8.3.1 Errors of Measurement </li></ul></ul><ul><ul><li>8.3.2 Reliability Coefficient </li></ul></ul><ul><ul><li>8.3.3 Some rules of thumb of Reliability Coefficient </li></ul></ul><ul><ul><ul><ul><li>>.80 </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Attitude scale: >.60 </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Apitude test: >.80 </li></ul></ul></ul></ul><ul><ul><ul><ul><li>High stakes: >.95 </li></ul></ul></ul></ul>
    47. 47. Chapter 9 Internal Validity <ul><li>9.1 Internal Validity </li></ul><ul><ul><li>Observed differences on the dependent variable are directly related to the independent variable </li></ul></ul><ul><li>9.2 Threats to Internal Validity </li></ul><ul><ul><li>Subject Characteristics </li></ul></ul><ul><ul><li>Mortality: loss of subjects </li></ul></ul><ul><ul><li>Location: </li></ul></ul><ul><ul><li>Instrumentation: Instrument decay, data collector characteristics or bias </li></ul></ul>
    48. 48. Chapter 9 Internal Validity <ul><li>Internal Validity </li></ul><ul><ul><li>Observed differences on the dependent variable are directly related to the independent variable </li></ul></ul><ul><li>Threats to Internal Validity </li></ul><ul><ul><li>Subject Characteristics </li></ul></ul><ul><ul><li>Mortality: loss of subjects </li></ul></ul><ul><ul><li>Location: </li></ul></ul><ul><ul><li>Instrumentation: Instrument decay, data collector characteristics or bias </li></ul></ul>
    49. 49. Chapter 9 Internal Validity <ul><li>9.1 Internal Validity </li></ul><ul><ul><li>Observed differences on the dependent variable are directly related to the independent variable </li></ul></ul><ul><li>9.2 Threats to Internal Validity </li></ul><ul><ul><li>Subject Characteristics </li></ul></ul><ul><ul><li>Mortality: loss of subjects </li></ul></ul><ul><ul><li>Location: </li></ul></ul><ul><ul><li>Instrumentation: </li></ul></ul><ul><ul><ul><li>Instrument decay </li></ul></ul></ul><ul><ul><ul><li>data collector characteristics </li></ul></ul></ul><ul><ul><ul><li>data collector bias </li></ul></ul></ul>
    50. 50. Chapter 9 Internal Validity <ul><li>Internal Validity </li></ul><ul><ul><li>Observed differences on the dependent variable are directly related to the independent variable </li></ul></ul><ul><li>9.2 Threats to Internal Validity </li></ul><ul><ul><li>Subject Characteristics </li></ul></ul><ul><ul><li>Mortality: loss of subjects </li></ul></ul><ul><ul><li>Location: </li></ul></ul><ul><ul><li>Instrumentation: </li></ul></ul><ul><ul><ul><li>Instrument decay </li></ul></ul></ul><ul><ul><ul><li>data collector characteristics </li></ul></ul></ul><ul><ul><ul><li>data collector bias </li></ul></ul></ul>
    51. 51. Chapter 9 Internal Validity <ul><li>9.2 Threats to Internal Validity </li></ul><ul><ul><li>Subject Characteristics </li></ul></ul><ul><ul><li>Mortality: loss of subjects </li></ul></ul><ul><ul><li>Location: </li></ul></ul><ul><ul><li>Instrumentation: Instrument decay, data collector characteristics or bias </li></ul></ul><ul><ul><li>Testing: e.g. pretest </li></ul></ul><ul><ul><li>History: </li></ul></ul><ul><ul><li>Maturation: aging or experiences of the subjects change </li></ul></ul><ul><ul><li>Attitudes: Hawthorne effect; John Henry </li></ul></ul><ul><ul><li>Regression: laag  hoog, hoog  laag </li></ul></ul><ul><ul><li>Implementation: </li></ul></ul>
    52. 52. Chapter 9 Internal Validity <ul><li>9.2 Threats to Internal Validity </li></ul><ul><ul><li>Testing: e.g. pretest </li></ul></ul><ul><ul><li>History: </li></ul></ul><ul><ul><li>Maturation: aging or experiences of the subjects change </li></ul></ul><ul><ul><li>Attitudes: Hawthorne effect; John Henry </li></ul></ul><ul><ul><li>Regression: laag  hoog, hoog  laag </li></ul></ul><ul><ul><li>Implementation: </li></ul></ul>
    53. 53. Chapter 9 Internal Validity <ul><li>9.2 Threats to Internal Validity </li></ul><ul><ul><li>Testing: e.g. pretest </li></ul></ul><ul><ul><li>History: </li></ul></ul><ul><ul><li>Maturation: aging or experiences of the subjects change </li></ul></ul><ul><ul><li>Attitudes: Hawthorne effect; John Henry </li></ul></ul>Hawthorne effect John Henry effect
    54. 54. Chapter 9 Internal Validity <ul><li>9.3 How to minimize these threats </li></ul>
    55. 55. Today’s course <ul><li>7 Instrumentation </li></ul><ul><ul><li>Data </li></ul></ul><ul><ul><li>Data collection </li></ul></ul><ul><li>8 Validity and Reliability </li></ul><ul><li>9 Internal Reliability </li></ul>
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×