Aace assessment of teacher candidate reflection in e-portfolios
Assessment of Teacher CandidateReflection in E-PortfoliosA paper presented at the2013 AACE EdMedia Conference PresentationDaihong ChenAndrew LumpeDan Bishop
Teacher Reflection Reflection: an active, persistent, and careful consideration of any beliefor supposed form of knowledge in light of the grounds supporting it andfuture conclusions, to which it tends. (Dewey, 1933, p.43) Teacher reflection: spontaneous critical scrutiny of teachers’ beliefsand knowledge pertaining to teaching and learning, as well as practice andeffects elicited by those beliefs and knowledge. (Sung, Chang, Yu, Chang, 2009) Teacher reflection is significantly correlated with effective instructionalpractice, classroom organization, professional development, and teachingefficacy. (Giovanelli, 2003; Sparks-Langer et al. 1990; York-Barr et al. 2001) Schön’s Two Time Frames (1983)Reflection-in-Action: reflective action occurs simultaneously duringthe teaching and learning. Modification is made immediately.Reflection-on-Action: reflective action takes place before or after theaction according to a retrospection of the teacher.
E-portfoliosA promising and progressive approach for teacher preparation(Granberg, 2010; Green, 2008; Kim, Ng, & Lim, 2010; Zawacki-Richter, Hanft, & Baecker; 2011)Four pillars of e-portfolios (Barbera, 2009; Black & Willliam, 1998):1. Metacognition2. Authentic tasks3. Contextual feedback4. Student responsibilityThree main purposes of using e-portfolios in teacher education:1. Contribute to constructivist learning(Mair, 2012; Maher & Gerbic, 2009; Meeus, Questier, & Derks, 2006; Ritzhaupt, Ndoye, & Parker, 2010);1. Provide summative and formative assessment(Barrett, 2007; Granberg, 2010; Luchoomun, McLuckie & van Wesel, 2010; Ritzhaupt, Ndoye, & Parker, 2010);2. Showcase teacher candidates’ achievements and competencies(Granberg, 2010; Johnson-Leslie, 2009; Lumsden, 2007).
E-portfolios & Teacher Reflection Reflective writing in e-portfolios is proposed as acontributor for preservice teachers’ constructivist learningand professional preparation (Tzeng & Chen, 2012). Web 2.0 portfolios may serve as effective vehicles forfostering critical reflection (Lumpe & Wicks, 2011; Tan, 2006; Godwin-Jones, 2008; Bartlett-Bragg, 2003; Oner, & Adadan, 2011; Pechone, Pigg, Chung, andSouviney, 2005). Levels of quality in reflection vary and most reflectivewriting remains at a low level (Ayan, & Seferoglu, 2011; Bauer & Dunn,2003; Lai & Calandra, 2010; Parkes & Kajder, 2010; Seng, 2004). The depth of reflectivity in e-portfolios remains to beexamined and developed (Carney, 2006; Delandshere &Arens 2003).
Instruments for classifying reflectivitySparks-Langer et al. (1990, p.27) :(1) no descriptive language(2) a simple, layperson description(3) events labeled with appropriate terms(4) explanations with traditional or personal preferences given as the rationale(5) explanation with principle or theory given as the rationale(6) explanation with principle/theory and consideration of contextual factors(7) explanation with consideration of ethical, moral or political issuesHatton and Smith (1995) Kember et al. (2000, 2008) Ward and McCotter (2004)descriptive writingdescriptivereflectiondialogic reflectioncritical reflectionhabitual actionunderstandingreflectioncritical reflectionroutinetechnicaldialogictransformative
Current Study- Context Graduate Teacher Preparation Program One year period All teaching candidates maintained a bPortfolio onwww.wordPress.com Random sample of 50% of bPortfolios (n=41)6
Using bPortfolios Initial set up of bPortfolio and training Reflective posts made during courses/internships Posts linked to standards via categories Artifacts include text files, A/V, or web links Meta-reflections served as summative posts Peer and instructor feedback via comments link Summative evaluation by faculty at end of program7
Tags forIndividualPostCategories(typically programstandards)ReflectivePost forWeeklyModuleStudentGeneratedTitleOptionalGraphics,Links, etc. Comments forPeer or TeacherFeedback
“Doing bPortfolios” 10 Questions about bPortfolios Assessing bPortfolios Executive Summary of bPortfolios Sloan C Effective Practice Award 2011 Sample student bPortfolios http://hamiltonlauren.wordpress.com/ http://rollis1.wordpress.com10
Research Questions What is the level of reflection in preservice teachercandidates’ e-portfolios? What text analysis variables predict the level ofreflection?Methods Blog coding for level of reflection Text analytics via www.semantria.com Descriptive statistics Multiple regression
Coding Blog Reflection LevelTwo coders coded all posts. Reached 85% interrater agreement.Adapted from Kember, 2000, 2009; and Lai & Calandra, 201012• An activity is performed automatically1 – Habitual Action• Learning occurs without personal/practical application2 - Understanding• Self-centered concerns3 - Routine Reflection• Response to a situation without change4 - Technical Reflection• Focus on student learning5 - Dialogic Reflection• Focus on change in professional practice6 - TransformativeReflection• A major shift in personal perspective/beliefs over time7 - Critical Reflection
Text Analytics of Blog Text Form of learner analytics Used www.semantria.com Excel Plug-in Based on semantic linguistic algorithms fromhttp://www.lexalytics.com/ Text submitted to Semantria servers via API All blog text analyzed for Themes- noun phrases with contextual relevance scores. “Whatare they writing about?” Document Sentiment – positive or negative tone of text Facets – “Meta Themes” which rely on subject-verb-object parsing.13
14.05.010.015.020.025.030.01 - habitualaction2- understanding 3 - routinereflection4 - technicalreflection5 - dialogicreflection6 -transformativereflection7 - criticalreflectionPercent 10.4 26.5 24.3 14.9 9.3 11.7 2.9Reflection Level of Coded Blog Posts• 36.9% of posts were non-reflective (orange bars)• 63.1% were reflective (red bars)
Findings and Discussion Higher level reflections occurred in the later writings. Blogs may promotereflection. The teacher candidates’ reflection tends to focus on self-centered concerns (e.g.time management issues, workload, personal emotion, and recognition forpersonal success). Teacher candidates’ were not giving much attention to the learning of students. The candidates spent little time writing about changes or major shifts inprofessional practice. Most teacher candidates posted two entries per week during their internship. Theaverage number of blog entries was 78 with a range of 35 -122. The average number of unique facets (meta-themes) was 17.8 per candidate with arange of 15-33. The overall score of bPortfolio document sentiment or emotional tone was +.26 ona scale of -1.0 to +1.0 (mostly positive).
Findings and Discussion of Text AnalysisF = 2.8, p = .03. The adjusted R square was .29, with 29% of the variance in blogreflection explained by the model variables.Variable B Std.ErrorBeta t Sig.Blog Themes-.002 .001 -.494 -2.426 .023Blog Facets.000 .000 .544 2.645 .014WEST-E certification contenttest -.026 .008 -.566 -3.225 .004Blog Sentiment-3.468 2.2 -.282 -1.56 .131Table 1: Variables in the Regression Model Predicting Reflection Level in the bPortfolios
Conclusions Blogging portfolios may be an effective tool forfostering professional reflective practice. Teacher candidates’ blogs displayed relatively lowlevels of reflection. Preservice teachers may not develop well-groundedbeliefs and concepts of student-centered learning. Assessment and analysis of teacher reflection in e-portfolios could be an effective means to revealpreservice teachers’ beliefs. There is a need for training on reflective writing andongoing assessment and support. 17
Recommendations Professional preparation programs should consider using web-based blogging portfolios to enhance reflective practice. Professionals utilizing such portfolios should be givenstructured training on reflective writing via blogs. Efforts on exploring attributors influencing reflective writingand strategies for promoting reflective writing arerecommended in future study. Provide effective strategies to transfer teacher candidates’reflection from self-centered concerns to professionalcompetencies directly related to student learning.18
Comments or Questions?This work is licensed under the Creative Commons Attribution-Share Alike 3.0 United States License.To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/3.0/us/ or send aletter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA.