Sloan c et4_o_2012_study for improving student reflections electronic portfolios david denton

•

1 like•398 views

Presentation sharing results of study for improving student reflections in electronic portfolios. SLOAN-C ET4Online 2012 by Dr. David W. Denton

Improving Student Reflections in
Electronic Portfolio Environments
Evidence-based Learning, Writing
Instruction, Reflection

Dr. David Denton

Questions
What instructional practices improve students’
electronic portfolio entries?

Does metadata (tags and categories) predict
performance?

Writing
“writing ability is a source of construct-irrelevant
variance” (Mislevy et la., 2004)

Evidence-Based Learning
• Artifacts
• Program Standards

Method
• Context and Participants
– Convenience Sample
– 11 Undergraduates
• Mostly females

Method
• Repeated measures
– Three portfolio entries

First Entry Second Entry Third Entry

8 months 1 week
before before During
intervention intervention intervention

Method
• Measure:
– Writing Quality Rubric
• Adapted from AACU VALUE rubrics
• .82 inter-rater reliability

t-test
First Entry Second Entry Third Entry

8 months
1 week During
before before
intervention intervention
intervention

5

Method

Number of Tag
Cloud Terms

Writing Quality Score

Results
• Statistically significant improvement

M = 0.55 M = 2.36 M = 5.82

t(10) = 4.99,
p < .001,
d = 3.16,
95% CI = 1.91 to 5.00

Results
• Statistically significant correlation
between
– Tag cloud terms –writing quality
– Total portfolio entries – writing quality

Discussion
• Intervention improved writing quality on third
entry
• Use of metadata to predict general writing
quality

1 Intervention
• Explicit directions on content and format
– Prompt
– Instructions

2 Intervention
• Communication of Assessment Criteria
– Rubric

3 Intervention
• Evaluating evidence
– Questions

4 Intervention
• Instructor and peer feedback
– Strengths and weaknesses
• Submitted to instructor
• Read aloud to peer

5 Intervention
• Revising
– Initial submission
• Instructor feedback
• Peer feedback
– Final submission

References
Yao, Y., Aldrich, J., Foster, K., & Pecina, U. (2009). Preservice teachers’ perceptions of an electronic portfolio as a tool for

reflection and teacher certification. Journal of Educational Research & Policy Studies, 9, 25-43.

Yao, Y., Thomas, M., Nickens, N., Downing, J., Burkett, R. S., & Lamson, S. (2008). Validity evidence of an electronic portfolio

for preservice teachers. Educational Measurement: Issues and Practice, 27, 10-24.

Mislevy, R. J., Almond, R. G., Lukas, & J. F. (2004). A brief introduction to evidence-centered design. CSE Report 632. U.S.

Department of Education. Los Angeles, CA: National Center for Research on Evaluation

Ayan, D., & Seferoglu, G. (2011). Using electronic portfolios to promote reflective thinking in language teacher education.

Educational Studies, 37(5), 513-521.

This webinar discussed how schools can use surveys to listen to the voice of parents and strengthen parent-school relationships. It covered designing a Voice of Parent program using Qualtrics surveys to collect, analyze, and act on parent feedback over multiple waves throughout the school year. Attendees learned about asking targeted questions to segment the parent community, such as asking a single check-in question regularly. Standards and frameworks were presented to evaluate culture and performance based on domains like achievement, relationships, and reputation. Analyzing trends in the data can guide schools' strategic planning and community engagement efforts.

7.1 ealta guidelines

Jesús Ángel González López

The European Association for Language Testing and Assessment (EALTA) aims to promote understanding of language testing principles and improve testing practices across Europe. EALTA's guidelines provide best practices for those involved in teacher training, classroom assessment, and test development. The guidelines stress respect, responsibility, fairness, reliability, and validity. They also recommend clarifying purposes and ensuring appropriateness, accuracy, feedback, and stakeholder involvement in the assessment process. EALTA encourages engagement with decision makers to enhance quality of assessment systems.

Assessment and individual differences

Sullivan Turner

The document discusses key concepts in psychometrics including reliability, validity, standardized testing, and individual differences in assessment. It notes that reliability refers to the consistency of test scores and is improved by increasing the number of test items. Validity concerns whether a test accurately measures the intended construct. Standardized tests provide norm-referenced scores based on a normal distribution curve. Finally, the document outlines theories of cognitive styles, learning styles, and group differences that influence assessment.

Using GradeMark to improve feedback and involve students in the marking process

Sara Marsham

This document discusses a project to improve feedback and involve students in the marking process using the online platform GradeMark. The project had four main aims: 1) Develop effective marking criteria specific to assignments, 2) Engage students in using criteria before and after assignments, 3) Provide feedback directly linked to criteria, and 4) Use GradeMark's comment libraries to provide feedback like a dialogue. The project trialled this on coursework from three courses. Students found the online feedback easier to access and more positive and detailed. Staff found it reduced work while providing more detailed comments. The project aims to further develop criteria and help students engage with assessment.

Validity, reliablility, washback

Maury Martinez

This document discusses validity, reliability, and washback in language testing. Validity refers to a test measuring what it intends to measure, which includes content validity (testing relevant skills and concepts) and criterion-related validity (how test results agree with other assessment results). Reliability means a test is repeatable, which can be measured through reliability coefficients. Washback refers to how a test influences teaching and learning, with the goal of achieving positive washback that encourages effective preparation. Ensuring validity, reliability, and beneficial washback requires careful test construction and use of techniques like setting test specifications, direct testing of objectives, and providing clear scoring criteria.

Test Usefulness

sara_galastarxy

The document discusses test usefulness and proposes a model with six qualities that contribute to a test's usefulness: reliability, construct validity, authenticity, interactiveness, impact, and practicality. It defines each quality and provides examples to illustrate how the qualities of authenticity and interactiveness can vary in different testing situations. The overall usefulness of a test is maximized when an appropriate balance is achieved among all six qualities for the specific testing purpose, test takers, and language domain being assessed.

Evaluation.2011intro

KAthy Cea

The document discusses key concepts in language testing and evaluation, including validity, reliability, practicality, types of tests, purposes of evaluation, key terms, and types of testing items. It defines important terms like validity, reliability, assessment, evaluation, discrete point testing, and direct/indirect testing items. The document emphasizes that evaluation measures student learning and teaching effectiveness to improve instruction.

Chapter 4 b

Ukhti Farizah

The document outlines the key steps in the test construction process: 1. Defining the test purpose and what construct it aims to measure. 2. Selecting an appropriate scaling method such as nominal, ordinal, interval or ratio scales. 3. Constructing initial test items that sample different cognitive domains and difficulty levels. 4. Testing items through analysis to evaluate item difficulty, reliability, validity, and discrimination. 5. Revising the test based on item analysis and feedback. 6. Publishing the finalized test along with manuals for administration and interpretation.

This document discusses principles for designing effective tests, including practicality, reliability, validity, authenticity, and washback. Tests should be practical to administer, reliable in scoring consistently, valid in measuring the intended ability or knowledge, authentic in mimicking real-world tasks, and provide washback or feedback to students on their competence. Validity is the most complex criterion, and there are three types: content, face, and construct validity.

Laos Session 3: Principles of Reliability and Validity (EN)

NEQMAP

Testing for Language TeachersArthur Hughes

Rajputt Ainee

Testing is done for various purposes such as verifying that a product meets requirements, managing risk, and assessing knowledge or skills. The main purposes of testing are to verify that specifications are met and to manage risks. Tests can have negative effects if not aligned with learning objectives, and inaccuracies can arise from flawed test content or unreliable scoring techniques. Effective testing requires quality assurance and validation to catch errors before public release. Assessment includes formative assessment for immediate feedback and summative assessment for end-of-period evaluation. Teachers can help improve testing by writing better tests, educating others, and advocating for testing improvements.

Chapter 2: Principles of Language Assessment

Hamid Najaf Pour Sani

This document discusses key principles of language assessment, including reliability, validity, practicality, authenticity, and washback. It provides definitions and explanations of these principles in 3-7 sentences each. Reliability refers to a test producing consistent scores and being error-free. Validity is the correspondence between a test's content and the material being tested. Practicality balances the resources required to design, develop, and use a test with the available resources. Authenticity is the similarity between test tasks and real-life language use. Washback describes the influence of a test on teaching and learning, which can be positive or negative.

Validity, reliabiltiy and alignment to determine the effectiveness of assessment

Mirea Mizushima

The document discusses the importance of validity, reliability, and alignment in determining the effectiveness of assessments. It defines validity as measuring what is intended, reliability as consistency, and alignment as connecting objectives, activities, and assessments. The document provides details on factors affecting and types of validity, reliability, and strategies for developing effective assessments aligned to standards through higher-order skills, critical abilities, international benchmarks, and instructionally sensitive tasks.

Ietcpresentation

Dr. Popat Tambade

1. The document discusses a study assessing the effectiveness of multimedia in teaching physics concepts to undergraduate students. 2. Two groups of students were given pre-tests and post-tests on oscillations concepts, with one group receiving traditional lectures and the other receiving additional computer simulations and discussions. 3. Analysis of the results found that the experimental group that received the multimedia instruction showed significantly higher normalized learning gains compared to the control group, indicating that computer-aided instruction can help improve students' understanding of physics principles.

Root Cause Analysis

Carlo Magno

The document provides guidance on conducting a root cause analysis to identify the underlying factors that led to an undesirable outcome or problem in order to determine corrective actions. It outlines a 10-step process for defining the problem, gathering evidence, identifying contributing factors and root causes, determining solutions, and ensuring the effectiveness of implemented recommendations to prevent future recurrence. The goal of root cause analysis is to transform a reactive culture into a proactive one by solving problems before issues escalate.

Fulcher standardized testing

Melikarj

Standardized testing can take two forms: norm-referenced which compares test takers to each other, and criterion-referenced which determines if an individual has achieved a specified standard. Norm-referenced testing aims to discriminate between test takers in order to distribute scarce resources like university places. It became popular during WWI when psychological testing was used to contribute to the war effort. Proponents viewed testing as a scientific process of quantifying and measuring abilities. However, others argue that defining and measuring constructs like traits is problematic. Test scores are distributed along a normal curve and take on meaning based on their position within that distribution compared to other test takers. Reliability ensures test scores are consistent over time without instruction.

7.1 assessment and the cefr (1)

Jesús Ángel González López

This document discusses key concepts in language assessment including validity, reliability, and feasibility. It defines validity as the accuracy of a test in measuring the intended proficiency. There are different types of validity including content, criterion-related, and construct validity. Reliability refers to a test producing consistent results, which can be measured using methods like test-retest. Feasibility means a test is practical to administer. The document also discusses types of language tests, how to improve validity and reliability, and item analysis. Chapters from a book on language testing techniques are assigned for discussion.

Good measuring tools

Rupa Gupta

Assessment Presentation

Nakendra

This document categorizes and describes different types of literacy assessments: screening, diagnostic, progress monitoring, and outcome measurements. Screening assessments are administered to all students to identify those needing additional support, are quick, and provide minimal instructional guidance. Diagnostic assessments follow up on students who perform poorly on screenings and assess specific literacy skills and needs. Progress monitoring assessments periodically measure student response to interventions. Outcome measurements are standardized, summative assessments used to compare student and school performance.

Evaluation.2011intro

KAthy Cea

The document discusses key concepts in language testing and evaluation. It defines important terms like validity, reliability, and practicality that are characteristics of a good test. It also distinguishes between different types of tests like placement, diagnostic, progress, and proficiency tests. Furthermore, it explains the differences between discrete point testing versus integrative testing and direct versus indirect test items. The overall purpose of evaluation is to measure student learning, teaching effectiveness, and provide feedback to improve instruction.

Principles of language assessment ( evaluation of language teaching)

Alfi Suru

This document discusses principles for evaluating existing tests, including practicality, reliability, validity, authenticity, and washback. It describes practicality as a test being inexpensive, time-constrained, easy to administer and having clear evaluation procedures. Reliability refers to a test's consistency and lack of factors like variability between students, raters, or test administrations. Validity is the most important principle and involves analyzing content, criteria, construct-related, and consequential evidence as well as face validity. Authenticity means a test's tasks closely resemble real-world tasks. Washback refers to how testing influences teaching and learning, and can enhance language acquisition when used to provide score specifications to students.

The Reliability Programme: Leading the way to better tests and assessments

Ofqual Slideshare

Characteristics of a good measuring tool

Forum of Blended Learning

The document outlines the key characteristics of a good measuring tool, including validity, reliability, practicability, measurability, and objectivity. Validity refers to a test measuring what it claims to measure. Reliability is the consistency of a measurement under the same conditions. Practicability considers ease of use, administration, and cost. Measurability is measuring intended objectives. Objectivity means results are obtained the same by different scorers.

Criteria of a good language test

International advisers

This document discusses the criteria for a good language test, including that it should have reasonable cost and time requirements, be simple to administer and score, and demonstrate validity and reliability. Specifically, it outlines that a good test must stay within budget and have appropriate time limits, not be too complex to conduct or score, accurately assess students' language ability, and produce consistent results. Teachers should consider these six key principles when creating or evaluating language tests.

DEVELOPMENT AND EVALUATION OF SCALES/INSTRUMENTS IN PSYCHIATRY

Pawan Sharma

Assessment & feedback for learning module induction

Neil Currant

This document outlines an assessment module that uses a problem-based learning (PBL) approach. It includes the following: - The module focuses on assessment and feedback theories and practices in higher education. - Students will participate in 3 PBL scenarios over the semester in small groups facilitated by a tutor. - Assessment includes two group reports analyzing PBL scenarios and an individual report and reflection. - The PBL process involves 5 steps: exploring the problem, discovering knowns/unknowns, research, application, and presentation. - Scenarios provide an introduction and issues for groups to research and propose solutions for in their reports.

Improving Student Outcomes Through Data Dashboards

Ohio Education Research Center

The document discusses improving student outcomes through data dashboards. It describes a Higher Education Compact in Greater Cleveland that uses data to track the educational journeys of Cleveland Metropolitan School District (CMSD) students. The Compact aims to increase CMSD student college readiness, access, and persistence. Student-level data is collected from 17 higher education institutions to identify factors impacting student success, such as high school GPA, ACT scores, and first semester college GPA. The Compact seeks to increase CMSD student graduation rates and prepare more students for college and careers.

Cengage Learning Webinar: Improving Student Learning Outcomes

Cengage Learning

Abad21

Rosdan Nasar

2014 Student Learning Outcomes

Purdue University Division of Student Affairs

What's hot

Testing

anahianahianahi

Laos Session 3: Principles of Reliability and Validity (EN)

NEQMAP

Testing for Language TeachersArthur Hughes

Rajputt Ainee

Chapter 2: Principles of Language Assessment

Hamid Najaf Pour Sani

Validity, reliabiltiy and alignment to determine the effectiveness of assessment

Mirea Mizushima

Ietcpresentation

Dr. Popat Tambade

Root Cause Analysis

Carlo Magno

Fulcher standardized testing

Melikarj

7.1 assessment and the cefr (1)

Jesús Ángel González López

Good measuring tools

Rupa Gupta

Assessment Presentation

Nakendra

Evaluation.2011intro

KAthy Cea

Principles of language assessment ( evaluation of language teaching)

Alfi Suru

The Reliability Programme: Leading the way to better tests and assessments

Ofqual Slideshare

Characteristics of a good measuring tool

Forum of Blended Learning

Criteria of a good language test

International advisers

DEVELOPMENT AND EVALUATION OF SCALES/INSTRUMENTS IN PSYCHIATRY

Pawan Sharma

What's hot (17)

Testing

Laos Session 3: Principles of Reliability and Validity (EN)

Testing for Language TeachersArthur Hughes

Chapter 2: Principles of Language Assessment

Validity, reliabiltiy and alignment to determine the effectiveness of assessment

Ietcpresentation

Root Cause Analysis

Fulcher standardized testing

7.1 assessment and the cefr (1)

Good measuring tools

Assessment Presentation

Evaluation.2011intro

Principles of language assessment ( evaluation of language teaching)

The Reliability Programme: Leading the way to better tests and assessments

Characteristics of a good measuring tool

Criteria of a good language test

DEVELOPMENT AND EVALUATION OF SCALES/INSTRUMENTS IN PSYCHIATRY

Viewers also liked

Assessment & feedback for learning module induction

Neil Currant

Improving Student Outcomes Through Data Dashboards

Ohio Education Research Center

Cengage Learning Webinar: Improving Student Learning Outcomes

Cengage Learning

Abad21

Rosdan Nasar

2014 Student Learning Outcomes

Purdue University Division of Student Affairs

Improving student learning through assessment and feedback in the new higher ...

Learning Development Centre

The document summarizes key points from a book about improving student learning through assessment and feedback. It describes a case study of a program that had many innovative coursework assignments but students did not put in much effort or find the feedback useful. The program lacked formative assessment, had too much assessment variety, and provided feedback too slowly. The document recommends focusing assessment, increasing formative tasks, reducing variety, separating feedback from marks, and ensuring consistency across courses to improve the student experience and learning.

Improving student learning using information technologies

joaoppinto

This document discusses improving student learning through the use of information technologies. It begins by outlining reasons to use e-learning, including enhancing teaching quality, meeting student needs, increasing access and flexibility, and improving cost-effectiveness. It then discusses different models of e-learning on a continuum from no technology to fully online. Key decisions are identified, such as where a course should fall on this continuum and what content is best suited for online or face-to-face delivery. Considerations for students such as demographics, technology access, and learning styles are also outlined. The document concludes by discussing how new technologies can help develop the skills needed for a knowledge-based society and mobilize student-created content.

Web 2.0 Tools in Your Classroom

Mark Woolley

Action research

Hasan BİLOKCUOGLU

Looking for learning in 21st century classrooms - 2010

Justin Medved

This document discusses how technology is changing 21st century classrooms and offers questions for school leaders to consider when evaluating classroom instruction and student learning. Key questions focus on how technology is used, whether the physical classroom supports collaboration, how students are guided in research and accountability for their own learning, and ensuring technology leverages deeper understanding rather than just making tasks easier. The document advocates measuring schools based on their ability to provide students with new ways of learning not otherwise possible with technology.

Lifelong Learning Institution as a Means Of Creating Age Friendly Environment

guestd57072

The document discusses the establishment of a lifelong learning institution called the Tuymazy Folk School in Tuymazy, Russia to address challenges facing older adults. It aimed to provide social engagement opportunities for older adults through courses taught by volunteers. From 2007-2008, 350 classes were offered with 120 older adults participating. Benefits included increased social participation, volunteering opportunities, intergenerational communication, participation in decision making, and an improved community image of aging. The program helped address isolation, lack of recognition, and limited activities for older adults in the area.

Assessment Of Student Learning2015

mbanning

Assessment is an ongoing process aimed at understanding student learning through multiple methods. It serves diagnostic, formative, and summative functions to provide feedback to students and faculty. Authentic assessment observes students' ability to apply learning to real-world tasks, using work samples, observations, and student conferences. Informal assessments like questions and discussions are easy to individualize but require teacher skill. Portfolios and rubrics are tools to systematically evaluate student work over time based on defined criteria.

Viewers also liked (12)

Assessment & feedback for learning module induction

Improving Student Outcomes Through Data Dashboards

Cengage Learning Webinar: Improving Student Learning Outcomes

Abad21

2014 Student Learning Outcomes

Improving student learning through assessment and feedback in the new higher ...

Improving student learning using information technologies

Web 2.0 Tools in Your Classroom

Action research

Looking for learning in 21st century classrooms - 2010

Lifelong Learning Institution as a Means Of Creating Age Friendly Environment

Assessment Of Student Learning2015

Similar to Sloan c et4_o_2012_study for improving student reflections electronic portfolios david denton

TESTA Masterclass

Tansy Jessop

The document summarizes key points from a TESTA masterclass on using research tools to understand student assessment. The masterclass covered: - Defining formative and summative assessment - Auditing a program's assessment using a 10-step guide - Administering and analyzing data from the Assessment Experience Questionnaire (AEQ) - Conducting and analyzing focus groups on student experiences - Triangulating data from audits, AEQs, and focus groups to understand assessment in a program - Effectively presenting findings to program teams to facilitate positive changes to assessment practices

Improving student learning through programme assessment

Tansy Jessop

This document summarizes an interactive masterclass on improving student learning through programme assessment using the TESTA framework. The masterclass covered: 1. Discussing participants' highs and lows of assessment and feedback. 2. Explaining the TESTA approach which takes a holistic view of assessment across a degree programme. 3. The benefits of a programme approach over individual modules, including improved student perceptions of assessment and feedback and a better staff experience.

Effects of Technological Interventions for Self-regulation: A Control Experi...

Hassan Khosravi

The benefits of incorporating scaffolds that promote strategies of self-regulated learning (SRL) to help student learning are widely studied and recognised in the literature. However, the best methods for incorporating them in educational technologies and empirical evidence about which scaffolds are most beneficial to students are still emerging. In this paper, we report our findings from conducting an in-the-field controlled experiment with 797 post-secondary students to evaluate the impact of incorporating scaffolds for promoting SRL strategies in the context of assisting students in creating novel content, also known as learnersourcing. The experiment had five conditions, including a control group that had access to none of the scaffolding strategies for creating content, three groups each having access to one of the scaffolding strategies (planning, externally-facilitated monitoring and self-assessing) and a group with access to all of the aforementioned scaffolds. The results revealed that the addition of the scaffolds for SRL strategies increased the complexity and effort required for creating content, were not positively assessed by learners and led to slight improvements in the quality of the generated content. We discuss the implications of our findings for incorporating SRL strategies in educational technologies.

TESTA to FASTECH Presentation

Tansy_Jessop

This document provides an overview of an evidenced-informed approach to enhancing program-wide assessment called TESTA to FASTECH. It discusses the TESTA research methodology which triangulates data from program audits, assessment experience questionnaires, and focus groups. Key findings from the TESTA data are presented, such as high levels of summative assessment and variability in assessment patterns across programs. The document then introduces the FASTECH project which aims to use readily available technologies to improve feedback and assessment in a way that benefits student learning. It discusses the goals of FASTECH to enhance transparency, student participation, and the use of peer learning and assessment.

Assessment 2.0

Luís Tinoca

This document discusses assessment 2.0 and the challenges of e-assessment. It defines e-assessment as any technology-enabled assessment activity where student activities like completing, presenting, and submitting work must be mediated by technology. The document outlines several dimensions of e-assessment including authenticity, consistency, transparency, and practicability. It provides examples for each dimension and discusses how e-assessment can contribute to a new assessment culture with benefits like greater variety, improved engagement, and efficient marking. Overall, the document frames e-assessment in terms of its ability to authentically assess competencies consistently, transparently, and practically.

Dodson kelly symposium

Kelly Dodson

This document summarizes a study on the impact of training on school administrators' communication competencies and attitudes regarding external stakeholders. The study found that administrators who received training scored significantly higher on knowledge, application, and attitude assessments compared to a control group. Interviews also revealed administrators had more positive attitudes following positive interactions with the media compared to unpleasant interactions. The study recommends more communication training for administrators and further use of assessment tools to evaluate training impact.

Collaborative Examination Item Review Process in a Team-Taught Course

ExamSoft

Presented by Laurel Sampognaro,Clinical Associate Professor, David Caldwell, Director of Professional Affairs, and Adam Pate, Assistant Professor all from University of Louisiana Monroe School of Pharmacy This presentation will describe a process to improve examination item quality by educating and involving course instructors in an item review process using evidence based guidelines and describe application of this process to multiple courses. In this interactive session presenters will discuss personal experiences and barriers to implementation of a collaborative exam item review process involving 21 faculty members from 2 departments in 3 different courses. Attendees will be exposed to a review of item writing guidelines, a discussion of common errors in item writing, and effects of item writing on test performance. A post exam process to objectively categorize test items based on item statistics will also be outlined

Self-, peer-, and instructor-assessment from Bloom’s perspective

dutra2009

- The study examined differences between self, peer, and instructor assessments of students' work using Bloom's taxonomy. - Students completed tasks involving different cognitive levels from Bloom's taxonomy and then assessed their own and peers' work. Instructors also assessed the work. - Results found that at the knowledge level, self and peer assessments converged more with instructors' scores compared to comprehension level tasks, where peer assessments diverged more from self and instructor scores. - The results suggest Bloom's taxonomy may help explain differences between assessment methods by accounting for cognitive demand, and that assessment design should consider cognitive level.

Validity and reliability

randoparis

L1.PILATUÑA.PAMBI.SISA.assessment1.8234.pptx

SisaPilatua

The document discusses key concepts related to evaluation methods: practicality, reliability, validity, authenticity, and washback effect. It provides definitions and examples for each concept. Practicality refers to how well a test meets practical constraints like time and budget. Reliability is the consistency of results. Validity is whether a test accurately measures the intended objectives. Authenticity focuses on real-world application of skills. Washback effect describes how testing influences teaching and learning. The document concludes with a bibliography of references on assessment principles and authentic assessment.

Validity of test

Sarat Rout

This document discusses different types of validity including content validity, criterion validity, and construct validity. It provides definitions and steps for establishing each type of validity. Specifically, it explains that content validity determines if a test adequately measures the intended content area. Criterion validity compares test scores to an external outcome measure concurrently or predictively. Construct validity establishes if a test measures a theoretical construct through examining correlations between various measures of that construct. The document also notes factors that can impact a test's validity such as length, ability range, and ambiguous directions. Overall, the document provides an overview of establishing and interpreting different aspects of test validity.

Learning_activity#1_Sánchez_Jhon.NRC_18235.pptx

Jhon Sánchez Medina

1. The document discusses key concepts in language assessment including terminology, different types of assessments, and principles of effective assessment. 2. It describes formative and summative assessment, with formative used to identify student strengths and weaknesses and summative used to formally evaluate learning at the end of a period. 3. Important principles of language assessment discussed are practicality, reliability, validity, authenticity, and washback effect. Reliability and validity are important for assessments to accurately and consistently measure what they are intended to.

Keynote: 7th eSTEeM Annual Conference Critical discussion of Student Evaluati...

Bart Rienties

1) The document discusses a study examining the relationship between student satisfaction, learning design, and academic performance using data from over 111,000 students across 150+ modules. 2) The study found that student satisfaction had a limited relationship with learning outcomes, while learning design strongly influenced student engagement, satisfaction, and performance. Constructivist and socio-constructivist learning designs positively predicted satisfaction and completion rates. 3) The conclusions recommend improving understanding of students through communication, and interpreting student evaluations as a developmental tool to identify strengths and areas for improvement.

Feedback, Agency and Analytics in Virtual Learning Environments – Creating a ...

Diogo Casanova

The project comprises of a review of the literature and current technical provision of assessment and feedback in Virtual Learning Environments (VLEs); and data collected from ‘Sandpits’ with students and lecturers in two HEIs in the UK. A ‘Sandpit’ is a type of creative design-thinking focus group where participants are stimulated by a narrative of a scenario around the use of a product, object or artefact and are encouraged to critique, discuss and re-design it (Frohlich, Lim and Ahmed, 2014; Casanova and Mitchell, 2017). These ‘Sandpits’ look to clarify the role of VLEs in assessment and feedback, through understanding students’ perceptions of feedback and how they are being addressed and understanding teachers’ perceptions of the constraints they face. We are exploring what is available, looking to improve interface designs and features, and present these to VLE product designers.

New item analysis

sibi rachel alex

Item analysis is a process used to evaluate test questions and assess the quality of a test. It involves both qualitative and quantitative procedures. Quantitatively, it examines the difficulty index, discrimination index, and distractor power of each question. The difficulty index indicates how many students answered correctly, the discrimination index shows if a question distinguishes between high- and low-scoring students, and distractor power evaluates the effectiveness of incorrect answer options. Conducting item analysis helps improve the validity and reliability of assessments by identifying high- and low-quality questions.

Validity in performance appraisal

MOHAMMED SAQIB

An evidence based model

Jisc

MMU TESTA Keynote

Tansy Jessop

The document discusses TESTA (Transforming the Experience of Students Through Assessment), a programme-level approach to assessment and feedback. It identifies four key problems with current assessment practices: (1) a "knee-jerk" reaction to student feedback without meaningful change, (2) modular curriculum design not considering the student experience, (3) an "evidence-to-action gap" where data is collected but not used to improve learning, and (4) student confusion about learning goals and assessment standards. The TESTA approach aims to address these by shifting perspective to the whole programme, increasing formative assessment, providing ongoing feedback conversations, and helping students internalize goals and criteria. Several case studies showed positive impacts of TESTA

Journal raganit santos

essayumiyuri

The document discusses a study that was conducted to validate test papers used at Saint Paul School of Business and Law and relate the validity of the test papers to student performance. 50% of test papers from the previous term were analyzed by experts using a checklist. The validity of test papers was found to have a moderately small positive correlation with student performance. Based on the results, guidelines for standardized test construction were formulated to improve the quality of assessment at the institution. The guidelines differentiate requirements for theory-based versus skill-based subjects. The study aims to establish best practices and standards for test development and administration at the school.

RDAP14 Poster: Evaluation of research data services: What things should we ev...

ASIS&T

Similar to Sloan c et4_o_2012_study for improving student reflections electronic portfolios david denton (20)

TESTA Masterclass

Improving student learning through programme assessment

Effects of Technological Interventions for Self-regulation: A Control Experi...

TESTA to FASTECH Presentation

Assessment 2.0

Dodson kelly symposium

Collaborative Examination Item Review Process in a Team-Taught Course

Self-, peer-, and instructor-assessment from Bloom’s perspective

Validity and reliability

L1.PILATUÑA.PAMBI.SISA.assessment1.8234.pptx

Validity of test

Learning_activity#1_Sánchez_Jhon.NRC_18235.pptx

Keynote: 7th eSTEeM Annual Conference Critical discussion of Student Evaluati...

Feedback, Agency and Analytics in Virtual Learning Environments – Creating a ...

New item analysis

Validity in performance appraisal

An evidence based model

MMU TESTA Keynote

Journal raganit santos

RDAP14 Poster: Evaluation of research data services: What things should we ev...

More from David Denton

Closing The 2-Sigma Gap Eight Strategies to Replicate One-to-One Tutoring in ...

David Denton

The document discusses eight strategies for replicating the benefits of one-to-one tutoring in blended learning courses. The strategies include: 1) improving instructional materials by increasing quantity of instruction and providing cues/explanations, 2) enhancing peer interactions through cooperative learning and establishing a supportive class environment, 3) considering student differences with tutorial instruction and feedback, and 4) engaging higher mental processes like metacognitive training and setting goals. Each strategy is explained and examples are given for how instructors can implement the strategies in blended learning courses.

Interviewing spu 05 2013

David Denton

This document provides tips and guidelines for effective interviewing. It emphasizes the importance of preparation, including knowing yourself, the organization, and practicing responses. During the interview, remain calm and focused, speak clearly, ask informed questions, and follow up with a thank you note. The interviewer will be evaluating communication skills, self-confidence, willingness to take responsibility, leadership abilities, and more. Answering behavioral questions with specific stories about past experiences that highlight strengths is key. Proper preparation, a positive attitude, and demonstrating fit for the role and organization are essential for interview success.

MAT program orientation 2013

David Denton

This document outlines the orientation for a Master of Arts in Teaching (MAT) program cohort from 2013-2015. It includes an agenda covering an introduction to Seattle Pacific University's School of Education, the MAT program requirements and standards, course sequences, internship information, assessments, resources and academic policies. Students are provided details on creating an online portfolio, registering for classes, and next steps to complete before the start of the program. The orientation aims to prepare students for the MAT program and certification requirements.

ARC MTMS Program Orientation 2013

David Denton

This document provides an orientation for students in the 2013 ARC-MTMS cohort at Seattle Pacific University. It outlines the agenda for the orientation, which includes getting to know each other, learning about the university and teacher education program, certification requirements, academic policies, and campus resources. The alternative routes certification program is developmental in nature and blends theoretical and practical studies. It consists of summer courses, a 10-month internship, and additional graduate classes over 3 quarters to earn a Master's degree and teaching certification.

Denton presentation implementing electronic portfolios through social media

David Denton

This document discusses implementing electronic portfolios through social media platforms. It notes that while paper-based portfolios have been used in teacher education for decades, institutions are now migrating to electronic portfolios as they provide advantages over paper. Early electronic portfolio platforms required special technical skills or fees, but newer platforms use social media applications that are easy to use and free. The document outlines steps for implementing social media portfolios and promoting quality student entries through questions and prompts.

David W Denton SPU Retreat Technology 2012

David Denton

Denton presentation implementing electronic portfolios through social media

David Denton

This document discusses implementing electronic portfolios through social media platforms and provides recommendations. It summarizes that while portfolios have been used in education for decades, institutions are now migrating to electronic formats which provide advantages but initially required technical skills. The newest iteration uses social media applications that are easy to use, free, and sustainable. However, literature on implementing portfolios through social media and promoting quality entries is still limited. The document provides steps for implementation, including defining scope and purpose, selecting a platform, creating a model, and instructing on appropriate use. It also provides recommendations for using questions and prompts to promote quality portfolio entries.

Intervention for improving electronic portfolio entries sloan emerging 2012

David Denton

This document discusses improving student reflections in electronic portfolios. It provides directions and examples of reflection prompts based on professional teaching standards to help students connect concepts and engage in critical thinking. The document also discusses using rubrics and feedback models to evaluate student reflections and help students revise their work based on instructor and peer feedback in the electronic portfolio environment.

Edu 6132 presentation interest attention and motivation

David Denton

This document discusses factors that influence student interest, attention, and motivation. It describes how attention is promoted through unusual, unpredictable, and distinctive stimuli. Emotionally engaging content is more memorable. Motivation decreases from elementary to high school due to impersonal environments. Self-efficacy can be improved through appropriate challenges, models, feedback, and choices. Cooperative learning promotes problem-solving when groups have interdependence and accountability. Interest is sparked by meaningful choices, relevance, prior knowledge, and active learning.

More from David Denton (9)

Closing The 2-Sigma Gap Eight Strategies to Replicate One-to-One Tutoring in ...

Interviewing spu 05 2013

MAT program orientation 2013

ARC MTMS Program Orientation 2013

Denton presentation implementing electronic portfolios through social media

David W Denton SPU Retreat Technology 2012

Denton presentation implementing electronic portfolios through social media

Intervention for improving electronic portfolio entries sloan emerging 2012

Edu 6132 presentation interest attention and motivation

Recently uploaded

Your Skill Boost Masterclass: Strategies for Effective Upskilling

Excellence Foundation for South Sudan

How to deliver Powerpoint Presentations.pptx

HajraNaeem15

B. Ed Syllabus for babasaheb ambedkar education university.pdf

BoudhayanBhattachari

What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...

GeorgeMilliken2

BBR 2024 Summer Sessions Interview Training

Katrina Pritchard

Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx

siemaillard

Bed Making ( Introduction, Purpose, Types, Articles, Scientific principles, N...

Leena Ghag-Sakpal

ANATOMY AND BIOMECHANICS OF HIP JOINT.pdf

Priyankaranawat4

Leveraging Generative AI to Drive Nonprofit Innovation

TechSoup

NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx

iammrhaywood

The History of Stoke Newington Street Names

History of Stoke Newington

clinical examination of hip joint (1).pdf

Priyankaranawat4

BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...

Nguyen Thanh Tu Collection

How to Setup Warehouse & Location in Odoo 17 Inventory

Celine George

BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...

Nguyen Thanh Tu Collection

Hindi varnamala | hindi alphabet PPT.pdf

Dr. Mulla Adam Ali

हिंदी वर्णमाला पीपीटी, hindi alphabet PPT presentation, hindi varnamala PPT, Hindi Varnamala pdf, हिंदी स्वर, हिंदी व्यंजन, sikhiye hindi varnmala, dr. mulla adam ali, hindi language and literature, hindi alphabet with drawing, hindi alphabet pdf, hindi varnamala for childrens, hindi language, hindi varnamala practice for kids, https://www.drmullaadamali.com

MARY JANE WILSON, A “BOA MÃE” .

Colégio Santa Teresinha

C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx

mulvey2

Film vocab for eal 3 students: Australia the movie

Nicholas Montgomery

Liberal Approach to the Study of Indian Politics.pdf

WaniBasim

Recently uploaded (20)

Your Skill Boost Masterclass: Strategies for Effective Upskilling

How to deliver Powerpoint Presentations.pptx

B. Ed Syllabus for babasaheb ambedkar education university.pdf

What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...

BBR 2024 Summer Sessions Interview Training

Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx

Bed Making ( Introduction, Purpose, Types, Articles, Scientific principles, N...

ANATOMY AND BIOMECHANICS OF HIP JOINT.pdf

Leveraging Generative AI to Drive Nonprofit Innovation

NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx

The History of Stoke Newington Street Names

clinical examination of hip joint (1).pdf

BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...

How to Setup Warehouse & Location in Odoo 17 Inventory

BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...

Hindi varnamala | hindi alphabet PPT.pdf

MARY JANE WILSON, A “BOA MÃE” .

C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx

Film vocab for eal 3 students: Australia the movie

Liberal Approach to the Study of Indian Politics.pdf

Sloan c et4_o_2012_study for improving student reflections electronic portfolios david denton

1. Improving Student Reflections in Electronic Portfolio Environments Evidence-based Learning, Writing Instruction, Reflection Dr. David Denton

2. Concept Map Overview

3. Study

4. Questions What instructional practices improve students’ electronic portfolio entries? Does metadata (tags and categories) predict performance?

5. Writing

6. Writing “writing ability is a source of construct-irrelevant variance” (Mislevy et la., 2004)

7. bPortfolio

8. Evidence-Based Learning • Artifacts • Program Standards

9. Example

10. Method • Context and Participants – Convenience Sample – 11 Undergraduates • Mostly females

11. Method • Repeated measures – Three portfolio entries First Entry Second Entry Third Entry 8 months 1 week before before During intervention intervention intervention

12. Method • Measure: – Writing Quality Rubric • Adapted from AACU VALUE rubrics • .82 inter-rater reliability

13. t-test First Entry Second Entry Third Entry 8 months 1 week During before before intervention intervention intervention 5

14. Method

15. Method Number of Tag Cloud Terms Writing Quality Score

16. Results • Statistically significant improvement M = 0.55 M = 2.36 M = 5.82 t(10) = 4.99, p < .001, d = 3.16, 95% CI = 1.91 to 5.00

17. t-test First Entry Second Entry Third Entry 8 months 1 week During before before intervention intervention intervention 5

18. Results • Statistically significant correlation between – Tag cloud terms –writing quality – Total portfolio entries – writing quality

19. Discussion • Intervention improved writing quality on third entry • Use of metadata to predict general writing quality

20. 1 Intervention • Explicit directions on content and format – Prompt – Instructions

21. 2 Intervention • Communication of Assessment Criteria – Rubric

22. 3 Intervention • Evaluating evidence – Questions

23. 4 Intervention • Instructor and peer feedback – Strengths and weaknesses • Submitted to instructor • Read aloud to peer

24. 5 Intervention • Revising – Initial submission • Instructor feedback • Peer feedback – Final submission

25. References Yao, Y., Aldrich, J., Foster, K., & Pecina, U. (2009). Preservice teachers’ perceptions of an electronic portfolio as a tool for reflection and teacher certification. Journal of Educational Research & Policy Studies, 9, 25-43. Yao, Y., Thomas, M., Nickens, N., Downing, J., Burkett, R. S., & Lamson, S. (2008). Validity evidence of an electronic portfolio for preservice teachers. Educational Measurement: Issues and Practice, 27, 10-24. Mislevy, R. J., Almond, R. G., Lukas, & J. F. (2004). A brief introduction to evidence-centered design. CSE Report 632. U.S. Department of Education. Los Angeles, CA: National Center for Research on Evaluation Ayan, D., & Seferoglu, G. (2011). Using electronic portfolios to promote reflective thinking in language teacher education. Educational Studies, 37(5), 513-521.

Sloan c et4_o_2012_study for improving student reflections electronic portfolios david denton

Recommended

Recommended

More Related Content

What's hot

What's hot (17)

Viewers also liked

Viewers also liked (12)

Similar to Sloan c et4_o_2012_study for improving student reflections electronic portfolios david denton

Similar to Sloan c et4_o_2012_study for improving student reflections electronic portfolios david denton (20)

More from David Denton

More from David Denton (9)

Recently uploaded

Recently uploaded (20)

Sloan c et4_o_2012_study for improving student reflections electronic portfolios david denton