The Doctoral Dissertation Literature Review: An Exploration of the Boote & Beile Literature Review Scoring Rubric
Dept...
Welcome
Why Should We Care?
 Education research doctoral programs sometimes fall short in preparing future scholars
 To improve ed...
So What’s the Problem?
 The dissertation provides insight into PhD candidates’ preparation for independent work
 There is ...
Why use the Boote & Beile (2005) Literature Review Scoring Rubric?
 What is the rubric?
 Who is using it?
 How is it b...
How Do These Studies Address the Problem in Unique Ways? 
Article I: Replication & Reliability
 Article II:  Validity ...
Assessing the quality of Doctoral dissertation literature reviews in Instructional Technology
Article 1
Problem Statement
The Boote & Beile (2005) rubric was created to evaluation dissertation lit. reviews in education res...
Purpose & Significance of Study
 Applies the Literature Review Scoring Rubric to a more narrow field of inquiry within...
Findings
 Mean score for the Instructional Technology (N=27) dissertations was 19.96 (SD =3.16) out of 37
Little differenc...
Article II
The Boote & Beile Literature Review Scoring Rubric:  A content validity study
Problem Statement
 The Literature Review Scoring Rubric (Boote & Beile, 2005) is a performance assessment for doctoral...
Purpose & Significance of Study
 Will contribute to the validity evidence of the Literature Review Scoring Rubric
 Ins...
Expected Findings
Will have a high content validity index for the 12 criteria
 Content validity index for performance leve...
Training Future Education Scholars: Literature Review Skills, Textbooks,  and the Doctoral Student
Article III
Problem Statement
Studies using the Literature Review Scoring Rubric reveal problems with quality of lit. reviews
 Underst...
Purpose & Significance of Study
A direct answer to Boote & Beile’s (2005) call to extend research on how doctoral ...
Expected Findings
 Based on the pilot study and anecdotal reports, low overall scores for textbooks are expected
 Search t...
Questions
Upcoming SlideShare
Loading in …5
×

M fitt dissertationproposaldefense_redux

465 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
465
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

M fitt dissertationproposaldefense_redux

  1. 1. The Doctoral Dissertation Literature Review: An Exploration of the Boote & Beile Literature Review Scoring Rubric Dept. of Instructional Technology & Learning Sciences Utah State University M. Harrison Fitt, B.S., M.S.
  2. 2. Welcome
  3. 3. Why Should We Care? Education research doctoral programs sometimes fall short in preparing future scholars To improve education research, future scholars need to be taught the habits of the mind, heart, and hand (Shulman, 2005) unique to their discipline. Lit. review skills embody a scholar’s ability to contribute to the field in an interactive, reflective, and generative manner Poor lit. reviews frequently lead to poorly constructed research; good reviews reduce methodological errors & duplication of former research Initial research points to low overall quality of dissertation literature reviews in education research
  4. 4. So What’s the Problem? The dissertation provides insight into PhD candidates’ preparation for independent work There is a small but growing body of research examining the dissertation literature review As part of that body of research, the Literature Review Scoring Rubric (Boote & Beile 2005) is an assessment tool for dissertation literature reviews The rubric is being used as a pedagogical tool & has been included in education research textbooks However, Boote & Beile’s initial research has not been replicated nor is there adequate reliability and validity evidence to warrant such widespread adaptation
  5. 5. Why use the Boote & Beile (2005) Literature Review Scoring Rubric? What is the rubric? Who is using it? How is it being used? Why is it a useful framework?
  6. 6. How Do These Studies Address the Problem in Unique Ways? Article I: Replication & Reliability Article II: Validity & Refinement Article III: First foray into a training needs analysis
  7. 7. Assessing the quality of Doctoral dissertation literature reviews in Instructional Technology Article 1
  8. 8. Problem Statement The Boote & Beile (2005) rubric was created to evaluation dissertation lit. reviews in education research It is being used widely in a variety of fields for a variety of purposes However, in their initial study, the rubric was only applied to the literature review chapter of quantitative dissertations We don’t know if the rubric works well for specific fields of study or for other types of dissertations Additionally, no reliability data is available
  9. 9. Purpose & Significance of Study Applies the Literature Review Scoring Rubric to a more narrow field of inquiry within education research In addition to Chapter 2, examines the dissertation as a whole Examines dissertations that incorporate a range of methodologies Analyzes the inter-rater reliability of the rubric
  10. 10. Findings Mean score for the Instructional Technology (N=27) dissertations was 19.96 (SD =3.16) out of 37 Little difference exists between the scores for chapter 2 and the overall dissertation when examined as a whole Low inter-rater reliability (.344)
  11. 11. Article II The Boote & Beile Literature Review Scoring Rubric: A content validity study
  12. 12. Problem Statement The Literature Review Scoring Rubric (Boote & Beile, 2005) is a performance assessment for doctoral dissertation literature reviews It is gaining widespread use across disciplines and research methods However, only one study reports reliability and no studies examine validity evidence More empirical research about the rubric & the constructs it measures is needed to justify its broadening use
  13. 13. Purpose & Significance of Study Will contribute to the validity evidence of the Literature Review Scoring Rubric Insights and recommendations will be applied to a revised version of the Literature Review Scoring Rubric May provide valuable information to aid others in the decision to use the rubric May contribute to the understanding of the performance expectations of the dissertation literature review
  14. 14. Expected Findings Will have a high content validity index for the 12 criteria Content validity index for performance levels will be lower These expectations are reasonable as faculty seem to disagree on the important of a dissertation literature review (Lovitts, 2006; Zaporozhetz, 1987) Findings will be applied to a revised form of the rubric Results will be submitted to Research in Higher Education
  15. 15. Training Future Education Scholars: Literature Review Skills, Textbooks, and the Doctoral Student Article III
  16. 16. Problem Statement Studies using the Literature Review Scoring Rubric reveal problems with quality of lit. reviews Understanding how those skills are being currently taught is an important part of improving quality Nearly all doctoral students in the US have to take a research methods course in which they are exposed to literature review skills in their textbooks Examining these textbooks provides an entrance point into understanding what students are being taught about dissertation literature reviews
  17. 17. Purpose & Significance of Study A direct answer to Boote & Beile’s (2005) call to extend research on how doctoral students are taught literature review skills To provide an understanding of what literature review skills students are being exposed to in their education research textbooks Will be the first time textbooks are empirically investigated using specific performance criteria of dissertation literature reviews May yield useful information for instructors who are selecting textbooks
  18. 18. Expected Findings Based on the pilot study and anecdotal reports, low overall scores for textbooks are expected Search techniques will be covered Less attention will be paid to the rhetorical devices of literature reviews
  19. 19. Questions

×