• Save
Saad Chahine Thesis Presentation Final
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Saad Chahine Thesis Presentation Final

on

  • 615 views

This is my PHD Defense presentation. Successfully defended July 28th, 2011

This is my PHD Defense presentation. Successfully defended July 28th, 2011

Statistics

Views

Total Views
615
Views on SlideShare
615
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Quick 1) Thank committee members for their support and guidance
  • Quick Great deal of pressures on school and educators to use data from large scale assessment & We know very little about Potentially there is a compromise to our new notions of validity because we dot understanding how educators interpret and use score reports
  • QuickThis literature review started with the ideas of validity and the lack of investigation in educators interpretation and proposed use of score reports. I then examined what reports ought to look like to facilitate understanding and relate it to the initial intended proposes. Once you examine those, it is then left to the educator to interpret the score reports. The ways in which an educator proposes to use test score reports depends on their Data Habit of Mind.
  • Medium Talk about Dat Keating & Robert SternbergI conceptualized data habit of mind in this thesis as a combination of having a general fluency with data and information, that what I call statistical literacy and the ways in which an educator would read interpret and potentially use score report. Initially I wanted to be able to plot educators for example you may have an educator who is very proficient in stats like a mathematics teacher but does not use the results from students assessment. However Data habit of mind does not exist in a vacuum and it is a metaphor that was coined by Keating and Sternberg to help explain the habits we have in every day life. The problem is there are contexts in which these habits exist.
  • Quick Data habit of mind does not exist in a vacuum,In this model I belive that datahabit of mind would be directly related to the ways in which educator in reali life use the information which will in turn improve school and student achievement This is relationship potenttially mediated by culture of of the school which is often influeced by political pressures and the different types of reporting that may happen
  • Quick About statistical literacy Score Report intretrpationFinding a relationship between the two Trying to relate the relatiohship to some background variables
  • Quick Requretmrnt not sampling or valuenter Educators
  • Quick & refer to hand outs for reports
  • Very Quick
  • Quick
  • Longer & Explain coding & Thinking levels based on the SOLO taxonomy Horizontal are the different concepts f statistical literacy, a person myabe high on describing and low when it comes to representing for example
  • Quick Each response was coded to be either Idiosyncratic, Transitional, Quantitative, Analytical Describing: Educators had an easy time Organizing and Reducing: A little more difficult because of the word typical to describe central tendency Representing: Educators always tried to push the bar graphAnalyzing and Interpreting: Was mixed – some educators had difficulty while others found it easier
  • Longer & Explain Coding Horizontal are the different concepts of score report interpretation literacy, a person maybe high on describing and low when it comes to questioning for example
  • Longer Talks about both reports at the same time Describing: Educators were coded very high for this in both reports Summarizing: Educators were able to summarize the reports very well Questioning: Great deal of variability observedProposing an application: Great deal of variability observed
  • Longer Refer to hand outsLower Statistical Literacy and Lower Questioning & Proposed Application: E 11 -Higher Statistical Literacy and Lower Questioning & Proposed Application: E4, E5, E7, E13
  • Longer Educators with overall statistical literacy scores below 2 were placed in the Idiosyncratic category, between 2 and 3 were placed in the Transitional category, between 3 and 3.5 were placed in the Quantitative category, and above 3.5 were placed in the Analytical category.
  • Longer Questioning for coopers – say your only going to show one example, though the thesis describes this in much more detail -Lower Statistical Literacy and Lower Questioning & Proposed Application: E 11 -Higher Statistical Literacy and Lower Questioning & Proposed Application: E4, E5, E7, E13 -All other educators coding of questioning and proposed application differed with the different reports -25% of sample were not coded differently based on the reports.
  • Longer Educators shifted back and forth while some remained the same Of the ones who remained the same quadrant who remained the same in both proposed application and Questions between the two This tells us that only 5 educators have the same codes when it came to questioning and proposed application irrelevant f the report, While other educators would change there level of questioning and proposed application based on the report-Lower Statistical Literacy and Lower Questioning & Proposed Application: E 11 -Higher Statistical Literacy and Lower Questioning & Proposed Application: E4, E5, E7, E13 -All other educators coding of questioning and proposed application differed with the different reports -25% of sample were not coded differently based on the reports.
  • Though one might expect to see a relationship I did not find meaningful relationshipsNeed to Be at 12 Min Mark!!!
  • Quick
  • Quick
  • Quick
  • Quick
  • Longer This thesis provided an initial framework to study the ways in which educators may use and inepter score reportsWhen statistical literacy educators are okay When it comes to describing and summarising a score report educators are also okay Variability is more prevalent in questioning and proposing an application of score reports – this was not only based on the educators but also based on the design of the reports. In attempting to relate the relationship between statistical literacy to score report design, values and belifes were more viable than education or years of experence for example. I believe that one of the major contributions of this research is to put educators interpretation and use of score reports at the forefront when we are designing our validity arguments, without a claim about educators use then our arguments towards validity are weak. Must be at the 17 mark here Add more here How does it relate to my findings
  • Quick
  • Quick
  • Longer More research needed to examine categories Talk here about the 4 educators who were higher stats and lower score report interpretation

Saad Chahine Thesis Presentation Final Presentation Transcript

  • 1. An Investigation of Educators’ Data Habit of Mind
    A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy, Graduate Department of Human Development and Applied Psychology, OISE/UT
    Saad Chahine
  • 2. Rationale
    External pressures for educators to use data from large- scale assessments
    Very little is known about the process by which educators interpret and make judgments based on reports from large-scale testing
    Potential compromise in validity when educators interpret and make judgments about students from large-scale assessment reports
    2
  • 3. Literature Review
    Proposed Use
    Validity
    Score Report Design
    3
  • 4. Data Habit of Mind
    4
  • 5. 5
    Conceptual Framework
  • 6. Research Questions
    RQ1:How are educators proficient in statistical literacy?
    RQ2:How do educators interpret and propose to use test score reports?
    RQ3: What are the relationships between statistical literacy and score report interpretations to model Data Habit of Mind?
    RQ4: How may an educator’s educational history and level of comfort with information, mathematics, and statistics contribute to Data Habit of Mind?
    6
  • 7. Methods
    Designed instruments to measure Statistical Literacy and Score Report Interpretation (2 mock score reports)
    Conducted pilot test with 12 pre-service students and refined instruments and interview procedure
    Recruited educators through advertising
    Conducted cognitive interviews to examine 20 educators’ data habit of mind
    Used protocol analysis procedure to examine cognitive interview data (i.e., educators’ cognitive processing when interacting with score reports)
    7
  • 8. Cognitive Interview Protocol
    Statistical Literacy Tasks
    Questions about background & comfort levels with information, statistics & mathematics
    Cooper’s Score Report
    Jabberwocky Score Report
    Educators were asked to verbalize responses and explain their reasoning.
    8
  • 9. Protocol Analysis Procedure
    Stage 1: Recording Verbalizations
    Stage 2: Transcribing Interviews
    Stage 3: Segmenting Interviews
    Stage 4: Encoding Episodes
    Stage 5: Analysis of the codes
    Stage 6: Categorizing Educators
    9
  • 10. A Snap Shot of the Sample
    All elementary (Gr. 1-8) teaching experience with provincial testing
    13 Female, 7 male
    1-21+ years of experience
    17 had undergrad focus in Humanities or Social Science and 3 had undergraduate focus in Sciences
    Majority had B.Ed. (13); others had higher degrees (1 Ph.D., 6 M.Ed.)
    10
  • 11. 11
    RQ1: How are educators proficient in statistical literacy?
  • 12. 12
    Statistical Literacy
  • 13. Results: Statistical Literacy
    13
  • 14. 14
    RQ2: How do educators interpret and propose to use test score reports?
  • 15. Score Report Interpretation
    15
  • 16. Results: Score Report Interpretation
    16
  • 17. RQ3: What are the relationships between statistical literacy and score report interpretations to model Data Habit of Mind?
    17
  • 18. Results: Data Habit of Mind
    18
  • 19. Categories Based on Data Habit of Mind Skills
    19
  • 20. Example: Questioning
    Questioning Patterns for Cooper’s Fitness Test Score Report
    20
  • 21. Example: Questioning
    21
    Questioning Patterns for Jabberwocky Test Score Report
  • 22. RQ4: How may an educator’s educational history and level of comfort with information, mathematics, and statistics contribute to Data Habit of Mind?
    22
  • 23. Results: Educational History & Comfort Levels
    Expected that educational history and comfort levels (information, math, and stats) would be meaningfully related to Data Habit of Mind
    However, found no meaningful relationships…
    Examination of interview transcripts gives us glimpse of insight into what other factors may be meaningfully related to Data Habit of Mind…
    23
  • 24. Educator Beliefs About the Use of Assessment Data
    Participant E18: “Assessment is probably, for me, has always been in my annual learning goals...my life-long learning goals in my formal years of education. It has been a constant pursuit of mine to understand, advance and implement assessment. We are a very data-driven school and I didn’t have to come to a data-driven school to be hungry for data. It already was in anything that I did...You keep yourself alive by having data. It just goes hand in hand. We are a very…we are a school that is constantly pushing for data.”
    24
    Educator with Higher Statistical Literacy and Higher Score Report Interpretation
  • 25. Participant E13:“I’ve done a few workshops on DRA and reading, in math, like effective instruction. I’ve done a lot of reading myself there and we have had a lot of in-service training. That being said, it is an area that is still a little bit hazy because we are going along in our lessons and I don’t necessarily assess them. We do the evaluation side to see what we can get in marks. That’s why I’m interested in the whole literacy thing because we assess the kids in the reading and see where we need to go from there.”
    25
    Educator Beliefs About the Use of Assessment Data
    Educator with Higher Statistical Literacy and Lower Score Report Interpretation.
  • 26. Educator Beliefs About the Use of Assessment Data
    Participant E2:“Ha ha, very high … Well, I think it’s kind of the package, frankly. I have my reading specialist, I have my religion specialist, which is essential for the Catholic school board. I do have a Master’s degree. I also think the fact that I’ve taught all three divisions helps a lot. Um, I am, in fact, I was an adjunct professor, and you have to mentor the young teachers... I’ve gone anywhere from basal readers to whole language to comprehensive literacy. I’ve kind of seen everything. That goes for math education, as well.
    26
    Educator with Lower Statistical Literacy and Higher Score Report Interpretation.
  • 27. Educator Beliefs About the Use of Assessment Data
    Participant E11: “Assessment to me ... it’s not always like paper and a test, a hard assessment that way. Sometimes I think you can get enough of an assessment by watching, observing and talking to the students. Some might not be able to articulate on paper and pencil what they do by talking.”
    27
    Educator with lower Statistical Literacy and Lower Score Report Interpretation.
  • 28. Discussion
    This thesis provided an initial framework to understand how educators are interpreting and proposing to use score reports.
    Findings showed that majority of educators have adequate statistical literacy, and can describe and summarize score reports.
    However, there was greater variability in educators’ questioning and proposed application of score reports.
    Educators’ questioning and proposing an application of a report varied across the two score reports and suggests the importance of score report design.
    Educators’ Data Habit of Mind maybe more related to values and beliefs than level of education, or comfort with information.
    This framework may potentially be used in building validity arguments.
    28
  • 29. Limitations
    Statistical Literacy tasks were designed for children
    Score Report Interpretation needs further refining
    Empirical literature on graphic design and typography lacking
    Reports were presented on paper
    No double coding
    29
  • 30. Implications
    Professional
    Developers
    Measurement
    Specialists
    B.Ed. Programs
    30
  • 31. Future Research
    More research needed to examine categories
    Double coding or multiple raters
    Make links with medical diagnostic methods
    Include more interactive reporting methods
    Make links with cognitive human-computer Interface work (e.g., Ware’s work)
    Conduct large-scale study of reporting systems and improvements
    31
  • 32. Thank You
    32