This document discusses measuring program objectives for a new Master's in Marketing degree via online assessment. It outlines creating program objectives and mapping them to two direct and two indirect measures. Direct measures include entrance and exit exams, with a goal of 80% of students scoring over 73%. Indirect measures include mid-term and end-of-program surveys, with a goal of 80% of students agreeing or strongly agreeing that they can apply program learning outcomes. The document then discusses using case analysis as one direct measure, outlining four cases and central questions. It provides results from the first exam that were skewed and adjustments made going forward. Finally, it covers setting up the case analysis in an LMS like Blackboard, including why cases
Retiring Exam Questions? How to Use These Items in Formative AssessmentsExamSoft
Presented by: Dan Thompson, Manager of Instructional Design and Educational Technology, Oklahoma State University Center for Health Sciences
As semesters come and go, new exams are created and used items are retired, never to be used again. While moving on to new exam items due to poor performing statistics or to ensure exam security are sound practices, these retired items can still positively impact student learning. Retired summative assessment items can serve new purposes as formative assessment items that engage students with course content while preparing them for course exams. This presentation addresses the process of using retired exam items in creative ways to develop formative assessments to increase student self-awareness of their content knowledge and improve student retention.
Presentation by Jean-Claude Callens, Vives University at the 2018 European Distance Learning Week's third day webinar on "Innovative Education – Case Studies" - 7 November 2018
Recording of the discussion is available: https://eden-online.adobeconnect.com/pynq0w4ku2b1/
Psychometrics 201: Putting assessment data into actionExamSoft
Presented by: Dan Thompson, Sr. Manager of Education and Assessment, ExamSoft Worldwide
At the conclusion of each exam, a wealth of student assessment data is immediately available to faculty. The short term value of this data is evident - evaluate individual question performance. However, that’s only the beginning. Assessment data provides opportunities for in-depth analysis on course, curricular, and instructional method performance. Furthermore, specific performance data can be distributed to students to drive their self-assessment and focused study plans. This presentation addresses the process of using assessment data to make tangible changes to improve instructional and student outcomes.
Helping Students Get the Most Out of ExamSoft Longitudinal ReportsExamSoft
Presented by Dr. Melinda E. Lull, Assistant Professor of Pharmaceutical Sciences, Wegmans School of Pharmacy, St. John Fisher College
While students are able to view their own assessment data and longitudinal reports from ExamSoft, they can easily become lost in a sea of numbers and categories. In order to best benefit student performance, students must understand both the interpretation of and the benefit from ExamSoft reports. This session will discuss ways to provide assessment data to students and aid them interpreting and using their results.
Districts have an abundance of data but often do not know how or when to use it. Take an inventory of your existing assessments, decide if they are being used correctly and then eliminate overlaps and fill in gaps. Now you are ready to develop an assessment plan based on need.
Retiring Exam Questions? How to Use These Items in Formative AssessmentsExamSoft
Presented by: Dan Thompson, Manager of Instructional Design and Educational Technology, Oklahoma State University Center for Health Sciences
As semesters come and go, new exams are created and used items are retired, never to be used again. While moving on to new exam items due to poor performing statistics or to ensure exam security are sound practices, these retired items can still positively impact student learning. Retired summative assessment items can serve new purposes as formative assessment items that engage students with course content while preparing them for course exams. This presentation addresses the process of using retired exam items in creative ways to develop formative assessments to increase student self-awareness of their content knowledge and improve student retention.
Presentation by Jean-Claude Callens, Vives University at the 2018 European Distance Learning Week's third day webinar on "Innovative Education – Case Studies" - 7 November 2018
Recording of the discussion is available: https://eden-online.adobeconnect.com/pynq0w4ku2b1/
Psychometrics 201: Putting assessment data into actionExamSoft
Presented by: Dan Thompson, Sr. Manager of Education and Assessment, ExamSoft Worldwide
At the conclusion of each exam, a wealth of student assessment data is immediately available to faculty. The short term value of this data is evident - evaluate individual question performance. However, that’s only the beginning. Assessment data provides opportunities for in-depth analysis on course, curricular, and instructional method performance. Furthermore, specific performance data can be distributed to students to drive their self-assessment and focused study plans. This presentation addresses the process of using assessment data to make tangible changes to improve instructional and student outcomes.
Helping Students Get the Most Out of ExamSoft Longitudinal ReportsExamSoft
Presented by Dr. Melinda E. Lull, Assistant Professor of Pharmaceutical Sciences, Wegmans School of Pharmacy, St. John Fisher College
While students are able to view their own assessment data and longitudinal reports from ExamSoft, they can easily become lost in a sea of numbers and categories. In order to best benefit student performance, students must understand both the interpretation of and the benefit from ExamSoft reports. This session will discuss ways to provide assessment data to students and aid them interpreting and using their results.
Districts have an abundance of data but often do not know how or when to use it. Take an inventory of your existing assessments, decide if they are being used correctly and then eliminate overlaps and fill in gaps. Now you are ready to develop an assessment plan based on need.
Topic: Report Test Result to Administration
Student Name: Rooha Shaikh
Class: B.Ed. (Hons) Elementary
Project Name: “Young Teachers' Professional Development (TPD)"
"Project Founder: Prof. Dr. Amjad Ali Arain
Faculty of Education, University of Sindh, Pakistan
By Jennifer Spohrer, Bryn Mawr College for the e-Learning 2.0 Conference, March 29, 2012. In this presentation, Jennifer Spohrer addresses this research question: Can we use a blended learning approach to improve learning outcomes in introductory STEM courses?
Classroom will be observed through following aspects:
Physical Setting, Context and Organization, Communication Skills, Questioning Skills, Rapport with Students, Teaching Methods, Overview Assessment, Classroom Management
Essential Skills for Elementary Teachers. Practice and master the Positive Behavioral Interventions and Supports skills you need to improve students’ behavior and support academic success. 7 step strategy for managing behavior expectations.
Classroom Management in Action is a practical research-based training course for mastering the nuts and bolts of classroom management. Both instructional and entertaining, it provides educators of all skill and experience levels strategies that decrease behavior problems and increase student social and academic achievement.
This is NOT my original work. This was created by Janet Holmshaw and Jeff Sapiro of Middlesex University, London. I have simply uploaded it for use in one of my college courses.
Using Assessment Data to Engage Students in RemediationExamSoft
Presented by Dr. Sarah Zahl, Director of Educational Assessment, Marian University
Remediation is the process of partnering with students to make corrections when they have moved off course or are unable to perform to standards or competencies. This process involves self-remediation as well as facilitation by an advisor, faculty member, and/or course director. In this webinar, we will discuss strategies to use assessment data to engage students in the remediation process at various points in the curriculum:
• when a student is struggling with specific concepts
• after a student fails an assessment
• after a course failure
The presenter will share best practices and specific strategies to develop student-centered remediation initiatives.
Topic: Report Test Result to Administration
Student Name: Rooha Shaikh
Class: B.Ed. (Hons) Elementary
Project Name: “Young Teachers' Professional Development (TPD)"
"Project Founder: Prof. Dr. Amjad Ali Arain
Faculty of Education, University of Sindh, Pakistan
By Jennifer Spohrer, Bryn Mawr College for the e-Learning 2.0 Conference, March 29, 2012. In this presentation, Jennifer Spohrer addresses this research question: Can we use a blended learning approach to improve learning outcomes in introductory STEM courses?
Classroom will be observed through following aspects:
Physical Setting, Context and Organization, Communication Skills, Questioning Skills, Rapport with Students, Teaching Methods, Overview Assessment, Classroom Management
Essential Skills for Elementary Teachers. Practice and master the Positive Behavioral Interventions and Supports skills you need to improve students’ behavior and support academic success. 7 step strategy for managing behavior expectations.
Classroom Management in Action is a practical research-based training course for mastering the nuts and bolts of classroom management. Both instructional and entertaining, it provides educators of all skill and experience levels strategies that decrease behavior problems and increase student social and academic achievement.
This is NOT my original work. This was created by Janet Holmshaw and Jeff Sapiro of Middlesex University, London. I have simply uploaded it for use in one of my college courses.
Using Assessment Data to Engage Students in RemediationExamSoft
Presented by Dr. Sarah Zahl, Director of Educational Assessment, Marian University
Remediation is the process of partnering with students to make corrections when they have moved off course or are unable to perform to standards or competencies. This process involves self-remediation as well as facilitation by an advisor, faculty member, and/or course director. In this webinar, we will discuss strategies to use assessment data to engage students in the remediation process at various points in the curriculum:
• when a student is struggling with specific concepts
• after a student fails an assessment
• after a course failure
The presenter will share best practices and specific strategies to develop student-centered remediation initiatives.
Collecting Information Please respond to the followingUsi.docxmary772
"Collecting Information" Please respond to the following:
Using your evaluation plan, describe it briefly and discuss the appropriateness, benefits, and limitations of using two of the following designs: (a) case study, (b) time-series, (c) causal –pre- and posttest, (d) comparison.
"Evaluation Designs" Please respond to the following:
Since it is usually impossible to evaluate the whole population of a large program, evaluators must select samples. Using your evaluation plan, discuss the possible benefits and limitations of selecting a random sample or using purposive sampling to obtain the target population.
THIS IS THE PROGRAM EVALUATION
Program Evaluation Approach for Education
Student`s Name
Instructor
Institutional Affiliation
Course
Date
The program evaluation is a viable mechanism that is used in schools that seek to strengthen the quality ofeducation that they offer as well as improving the outcomes of the students. Today, many approaches that are used in the evaluation focus on education and especially about the key features of the program that will be evaluated. This paper will seek to describe the planned approach as it applied in education as well as the rationale for the strategy, description of the question areas and their rationale and finally the stockholders and the reasons they should be involved as well as the ways that can be used in obtaining their involvement.
Description
and Rationale
the Program Evaluation Approach
The Tylerian evaluation approach usually has a significant influence on both evaluation and education. His theory foresaw the concepts that will be used in today`s world in the improvement and multiple as the means of assessment. He defined the objectives as a way for the teachers to explain what they wanted to teach in the classes(Posavac, 2015). Through stating the goals in terms of what the students should do, Tyler believed that the teachers should plan more on their curricula so that they can be able to achieve more. Tyler eventually defined the program evaluation approach as a process of determining how best one is achieving its objectives (Jacobs, 2017). In the evaluation process, one should consider the following steps; establishment of the broad goals as well as the objectives, classifying the goals, define the objectives in terms of behavior, finding situation in which the achievement of the targets can be shown, development of the required measurement techniques, collecting the performance data and eventually compare the performance of the data with the behaviors that have been stated in the objectives.
Description of the
Question and their Rationale
Some of the description questions that can be asked on the process are why is there a discrepancy?
The discrepancy in education is the model that is usually used in the determination of whether a child is eligible for education. It usually refers to the mismatch between the child`s intellectual ability and their progres.
This is the second Part of CTET Mathematics Pedagogy. In this part we will discuss Problem of Teaching Mathematics, Error Analysis, Diagnostic and Remedial Teaching.
1. Measuring Program Objectives
Via Online Assessment
Julia Cronin-Gilmore, Ph.D
Tracy Gies
Bellevue University
International Assembly for Collegiate Business Education
9. Create Program Objectives
• Demonstrate problem solving and critical
thinking skills in order to be successful in
the business world
• effective communication skills and
professionalism when working with others
in a business setting.
• Demonstrate effective interpersonal skills
and an appreciation of business ethics.
• Demonstrate the ability to apply content
knowledge to create value as business
leaders in an organization.
10. Two Direct & Two Indirect
Measures
Direct measures:
Entrance & Exit Exams:
80% of students will
score at least 73% or
greater, which is at a
level of proficiency.
11. Two Direct & Two Indirect
Measures
Indirect Measures:
Mid-term and End of
Program Surveys: At least
80% of the students' will
note the ability to apply
program learning outcomes
as noted by their responses
of "Agree" or "Strongly
Agree".
12. Measure Program Objectives
Two Direct Measures
• In other programs, we used a
business plan and a group of
professors would evaluate their
specialty area (marketing, finance,…).
• Found to be a time consuming
process.
• Needed something manageable for
one person
• Decided to try analyzing a case study
13. Case Analysis
Four Cases Total
• All four are equal in length – about ¾
page.
• Each has multiple things going on
Three Central Questions
• What are the relevant facts of the
case?
• What are the ethical issues?
• What are the ethical alternatives?
15. Test Results
Test Results were Skewed…
• Students are used to answering once
on a multiple choice exam.
• There were several answers that were
correct.
• If students answered correctly, they
received a point
• If students missed a correct answer, it
deducted points from ones they
earned.
16. Going Forward
• Questions reflect how many correct
answers are possible.
• Students will earn one point per
correct answer
• Students will still loose points earned
for incorrect answers (couldn’t fix in
the system).
17. Test Results
Test Results were Skewed…
• Exam #1 was extra credit
• Eliminated the issue with it being a
major portion of their grade or
• Something I had to prepare them for.
• Most were encouraged to take
• Not every student completed the
exam.
19. Agenda
• Why use cases online?
• Mapping
• Creating the tests in the LMS
20. Why Use Cases
• Authenticity (Suskie, 2009)
• Effectiveness and student
acceptance of CBL (Barnes et al, 1994;
Srinivasan et al, 2007)
• Program curriculum favors CBL
learning
• CBL in assessment good tie-in
21. Issues
• Grading responses to cases can be
very time consuming
• There can be a wide range of correct
answers
• Setting up auto graded tests that
capture a wide range of correct
answers can be difficult
22. Our Approach
• Use Multiple Answer questions
• Try to capture as broad of a range as
possible of correct answers in terms
of ethical considerations, and
resolutions
23. What We’ve Learned So Far
• Generally well received by students
• Refine instructions
24. Mapping
• Provides a common document to prompt
discussion for all parties involved in course
design, development and review
• Keeps the two main components of learning in a
program (program outcomes and learning
objectives) a the forefront of design and
development discussions
• Helps ensure alignment of program outcomes
and course objectives
25. How to use the Excel Mapping Tool
• Some cells are linked so you can enter info on
the Data Entry worksheet, and it will fill the
appropriate cells on the other worksheets.
• Demo
26. Setting up the Test
• Blackboard 9.1.201404.160205
• Fairly standard Multiple Answer set
up
• Partial Credit
• Group cases and MC questions
Authenticity: Authenticity in educational assessment relates to assessing students in a manner that more closely resembles the way in which students will employ the skills. CBL learning encourages students to think like a marketing professional operating under conditions that require action while working with problems that are similar to problems they will face on the job.
Effectiveness and acceptance: CBL learning turns the classroon into learning laboratory, in which students are active participants rather than passive observers. Studies have shown that students prefer the more active experience
Program Curriculum: The Marketing program curriculum uses cases heavily throughout as learning activities
Tie in: Therefore, it makes sense to use cases in assessment of program outcomes
Responses to cases typically take the form of essays in which there are a wide range of possible correct responses. Because of this wide range of possible correct responses it is difficult to set up tests that capture this range