An Investigation of Educators’ Data Habit of Mind <br />A thesis submitted in conformity with the requirements for the deg...
Rationale<br />External pressures for educators to use data from large- scale assessments<br />Very little is known about ...
Literature Review<br />Proposed Use<br />Validity<br />Score Report Design<br />3<br />
Data Habit of Mind <br />4<br />
5<br />Conceptual Framework <br />
Research Questions <br />RQ1:How are educators proficient in statistical literacy?<br />RQ2:How do educators interpret and...
Methods <br />Designed instruments to measure Statistical Literacy and Score Report Interpretation (2 mock score reports)<...
Cognitive Interview Protocol<br />Statistical Literacy Tasks<br />Questions about background & comfort levels with informa...
Protocol Analysis Procedure <br />Stage 1: Recording Verbalizations<br />Stage 2: Transcribing Interviews <br />Stage 3: S...
A Snap Shot of the Sample <br />All elementary (Gr. 1-8) teaching experience with provincial testing <br />13 Female, 7 ma...
11<br />RQ1: How are educators proficient in statistical literacy?<br />
12<br />Statistical Literacy <br />
Results: Statistical Literacy<br />13<br />
14<br />RQ2: How do educators interpret and propose to use test score reports?<br />
Score Report Interpretation<br />15<br />
Results: Score Report Interpretation<br />16<br />
RQ3: What are the relationships between statistical literacy and score report interpretations to model Data Habit of Mind?...
Results: Data Habit of Mind<br />18<br />
Categories Based on Data Habit of Mind Skills<br />19<br />
Example: Questioning <br />Questioning Patterns for Cooper’s Fitness Test Score Report<br />20<br />
Example: Questioning <br />21<br />Questioning Patterns for Jabberwocky Test Score Report<br />
RQ4: How may an educator’s educational history and level of comfort with information, mathematics, and statistics contribu...
Results: Educational History & Comfort Levels<br />Expected that educational history and comfort levels (information, math...
Educator Beliefs About the Use of Assessment Data<br />Participant E18: “Assessment is probably, for me, has always been i...
Participant E13:“I’ve done a few workshops on DRA and reading, in math, like effective instruction. I’ve done a lot of rea...
Educator Beliefs About the Use of Assessment Data<br />Participant E2:“Ha ha, very high … Well, I think it’s kind of the p...
Educator Beliefs About the Use of Assessment Data<br />Participant E11: “Assessment to me ... it’s not always like paper a...
Discussion <br />This thesis provided an initial framework to understand how educators are interpreting and proposing to u...
Limitations <br />Statistical Literacy tasks were designed for children <br />Score Report Interpretation needs further re...
Implications <br />Professional <br />Developers <br />Measurement<br /> Specialists <br />B.Ed. Programs<br />30<br />
Future Research <br />More research needed to examine categories <br />Double coding or multiple raters <br />Make links w...
Thank You <br />32<br />
Upcoming SlideShare
Loading in...5
×

Saad Chahine Thesis

713

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
713
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Saad Chahine Thesis

  1. 1. An Investigation of Educators’ Data Habit of Mind <br />A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy, Graduate Department of Human Development and Applied Psychology, OISE/UT <br />Saad Chahine <br />
  2. 2. Rationale<br />External pressures for educators to use data from large- scale assessments<br />Very little is known about the process by which educators interpret and make judgments based on reports from large-scale testing<br />Potential compromise in validity when educators interpret and make judgments about students from large-scale assessment reports<br />2<br />
  3. 3. Literature Review<br />Proposed Use<br />Validity<br />Score Report Design<br />3<br />
  4. 4. Data Habit of Mind <br />4<br />
  5. 5. 5<br />Conceptual Framework <br />
  6. 6. Research Questions <br />RQ1:How are educators proficient in statistical literacy?<br />RQ2:How do educators interpret and propose to use test score reports?<br />RQ3: What are the relationships between statistical literacy and score report interpretations to model Data Habit of Mind?<br />RQ4: How may an educator’s educational history and level of comfort with information, mathematics, and statistics contribute to Data Habit of Mind?<br />6<br />
  7. 7. Methods <br />Designed instruments to measure Statistical Literacy and Score Report Interpretation (2 mock score reports)<br />Conducted pilot test with 12 pre-service students and refined instruments and interview procedure<br />Recruited educators through advertising<br />Conducted cognitive interviews to examine 20 educators’ data habit of mind<br />Used protocol analysis procedure to examine cognitive interview data (i.e., educators’ cognitive processing when interacting with score reports)<br />7<br />
  8. 8. Cognitive Interview Protocol<br />Statistical Literacy Tasks<br />Questions about background & comfort levels with information, statistics & mathematics <br />Cooper’s Score Report<br />Jabberwocky Score Report<br />Educators were asked to verbalize responses and explain their reasoning.<br />8<br />
  9. 9. Protocol Analysis Procedure <br />Stage 1: Recording Verbalizations<br />Stage 2: Transcribing Interviews <br />Stage 3: Segmenting Interviews <br />Stage 4: Encoding Episodes <br />Stage 5: Analysis of the codes <br />Stage 6: Categorizing Educators <br />9<br />
  10. 10. A Snap Shot of the Sample <br />All elementary (Gr. 1-8) teaching experience with provincial testing <br />13 Female, 7 male <br />1-21+ years of experience<br />17 had undergrad focus in Humanities or Social Science and 3 had undergraduate focus in Sciences<br />Majority had B.Ed. (13); others had higher degrees (1 Ph.D., 6 M.Ed.)<br />10<br />
  11. 11. 11<br />RQ1: How are educators proficient in statistical literacy?<br />
  12. 12. 12<br />Statistical Literacy <br />
  13. 13. Results: Statistical Literacy<br />13<br />
  14. 14. 14<br />RQ2: How do educators interpret and propose to use test score reports?<br />
  15. 15. Score Report Interpretation<br />15<br />
  16. 16. Results: Score Report Interpretation<br />16<br />
  17. 17. RQ3: What are the relationships between statistical literacy and score report interpretations to model Data Habit of Mind?<br />17<br />
  18. 18. Results: Data Habit of Mind<br />18<br />
  19. 19. Categories Based on Data Habit of Mind Skills<br />19<br />
  20. 20. Example: Questioning <br />Questioning Patterns for Cooper’s Fitness Test Score Report<br />20<br />
  21. 21. Example: Questioning <br />21<br />Questioning Patterns for Jabberwocky Test Score Report<br />
  22. 22. RQ4: How may an educator’s educational history and level of comfort with information, mathematics, and statistics contribute to Data Habit of Mind?<br />22<br />
  23. 23. Results: Educational History & Comfort Levels<br />Expected that educational history and comfort levels (information, math, and stats) would be meaningfully related to Data Habit of Mind<br />However, found no meaningful relationships…<br />Examination of interview transcripts gives us glimpse of insight into what other factors may be meaningfully related to Data Habit of Mind… <br />23<br />
  24. 24. Educator Beliefs About the Use of Assessment Data<br />Participant E18: “Assessment is probably, for me, has always been in my annual learning goals...my life-long learning goals in my formal years of education. It has been a constant pursuit of mine to understand, advance and implement assessment. We are a very data-driven school and I didn’t have to come to a data-driven school to be hungry for data. It already was in anything that I did...You keep yourself alive by having data. It just goes hand in hand. We are a very…we are a school that is constantly pushing for data.” <br />24<br />Educator with Higher Statistical Literacy and Higher Score Report Interpretation<br />
  25. 25. Participant E13:“I’ve done a few workshops on DRA and reading, in math, like effective instruction. I’ve done a lot of reading myself there and we have had a lot of in-service training. That being said, it is an area that is still a little bit hazy because we are going along in our lessons and I don’t necessarily assess them. We do the evaluation side to see what we can get in marks. That’s why I’m interested in the whole literacy thing because we assess the kids in the reading and see where we need to go from there.”<br />25<br />Educator Beliefs About the Use of Assessment Data<br />Educator with Higher Statistical Literacy and Lower Score Report Interpretation.<br />
  26. 26. Educator Beliefs About the Use of Assessment Data<br />Participant E2:“Ha ha, very high … Well, I think it’s kind of the package, frankly. I have my reading specialist, I have my religion specialist, which is essential for the Catholic school board. I do have a Master’s degree. I also think the fact that I’ve taught all three divisions helps a lot. Um, I am, in fact, I was an adjunct professor, and you have to mentor the young teachers... I’ve gone anywhere from basal readers to whole language to comprehensive literacy. I’ve kind of seen everything. That goes for math education, as well. <br />26<br />Educator with Lower Statistical Literacy and Higher Score Report Interpretation.<br />
  27. 27. Educator Beliefs About the Use of Assessment Data<br />Participant E11: “Assessment to me ... it’s not always like paper and a test, a hard assessment that way. Sometimes I think you can get enough of an assessment by watching, observing and talking to the students. Some might not be able to articulate on paper and pencil what they do by talking.”<br />27<br />Educator with lower Statistical Literacy and Lower Score Report Interpretation.<br />
  28. 28. Discussion <br />This thesis provided an initial framework to understand how educators are interpreting and proposing to use score reports. <br />Findings showed that majority of educators have adequate statistical literacy, and can describe and summarize score reports. <br />However, there was greater variability in educators’ questioning and proposed application of score reports.<br />Educators’ questioning and proposing an application of a report varied across the two score reports and suggests the importance of score report design.<br />Educators’ Data Habit of Mind maybe more related to values and beliefs than level of education, or comfort with information. <br />This framework may potentially be used in building validity arguments. <br />28<br />
  29. 29. Limitations <br />Statistical Literacy tasks were designed for children <br />Score Report Interpretation needs further refining <br />Empirical literature on graphic design and typography lacking <br />Reports were presented on paper <br />No double coding<br />29<br />
  30. 30. Implications <br />Professional <br />Developers <br />Measurement<br /> Specialists <br />B.Ed. Programs<br />30<br />
  31. 31. Future Research <br />More research needed to examine categories <br />Double coding or multiple raters <br />Make links with medical diagnostic methods <br />Include more interactive reporting methods <br />Make links with cognitive human-computer Interface work (e.g., Ware’s work) <br />Conduct large-scale study of reporting systems and improvements<br />31<br />
  32. 32. Thank You <br />32<br />
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×