• Save
Online Course Assessment Part 2
Upcoming SlideShare
Loading in...5
×
 

Online Course Assessment Part 2

on

  • 4,198 views

Online Course Assessment Part 2

Online Course Assessment Part 2

Statistics

Views

Total Views
4,198
Views on SlideShare
4,094
Embed Views
104

Actions

Likes
5
Downloads
0
Comments
0

8 Embeds 104

http://www.scoop.it 51
http://moodle1.wilkes.edu 20
https://elearning.bgsu.edu 15
http://www.slideshare.net 12
http://internshippreparation.blogspot.com 3
url_unknown 1
http://internshippreparation.blogspot.co.nz 1
https://bb.wpunj.edu 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Veronica
  • Patricia
  • Veronica
  • Patricia
  • Patricia - Most of online learning is independent in the sense that the learners are never physically together. Collective learning experiences, particularly asynchronous ones, are difficult to capture. Although social presence and interaction increases engagement, caution must be taken to make sure that it results in learning.
  • Veronica
  • Veronica -Make sure you communicate this clearly to students in an online environment
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Michaelsen, L. K. (1992). Team learning: a comprehensive approach for harnessing the power of smallgroups in higher education. To Improve the Academy, 11, pp. 107-122Rafiq, Y., & Fullerton, H. (1996). Peer assessment of group projects in civil engineering. Assessment &Evaluation in Higher Education, 21, pp. 69-81.Somervell, H. (1993). Issues in assessment, enterprise and higher education: the case for self-, peer andcollaborative assessment. Assessment & Evaluation in Higher Education, 18, pp. 221-233.
  • Patricia
  • Veronica
  • Patricia - Assessments and feedback can be conducted in real time or in off time. For example, synchronous activities (chats, VOIP, etc.) can be assessed by using checklists or rubrics and this information conveyed to the learner. Asynchronous feedback and assessments more closely mirror traditional classroom assessments in that instructor or peer takes time to score against pre-determined criteria.
  • Patricia - In an online course, you may assess things you would not assess in a classroom because of the isolation of the learner, the developmental phase of the learner, and the requirement to engage the learner while documenting their successful participation in the course. Oftentimes online courses limit assessment to assignments, quizzes and tests, because these are the tools built into course management systems. However, the value of practice, activity, and interaction is just as important in an online course as in a traditional classroom. Unlike a classroom students may not know when they are achieving these successful. Assessment is one tactic to give them feedback about their performance. You can make a choice about whether this assessment ‘counts’ as a grade or not.
  • Patricia
  • Patricia
  • Patricia - Grading against a set of criteria is how we typically view assessment in education. However, ungraded assessment may be expedient and provide reassuring feedback to students just knowing that they have done “enough” can motivate and give students confidence.
  • Patricia
  • Patricia
  • Veronica
  • Veronica
  • Patricia
  • Patricia
  • Patricia
  • Patricia
  • Patricia
  • Patricia
  • Patricia – projects require instructional decisions that relate to the appropriate type of assessmentVirtual projects may be documented by a project, report or presentation for which criteria are pre-established. Field-base projects may require corroboration by a third party or collection or documentation of experience.Cooperative Learning – individuals work together where each person contributed a piece of the whole’ requires interdependence to achieve a goal.Collaborative Learning - individuals work independently to contribute to a goal.Progressive projects may be completed piecemeal throughout the course rather than a single activity or assignment.
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • V
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica - Formative assessment is generally carried out throughout a course or project. Formative assessment, also referred to as educative assessment, is used to aid learning. In an educational setting, formative assessment might be a teacher (or peer) or the learner, providing feedback on a student's work, and would not necessarily be used for grading purposes.
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Patricia
  • Patricia
  • Patricia
  • Patricia
  • Patricia
  • Patricia
  • Patricia
  • Patricia and Veronica – making assessment that is learner-centered, progressive, relevant, and aligned with objectives and effective practices will help you ensure a reliable and valid set of strategies that contributes to the successful design and implementation of an online course.

Online Course Assessment Part 2 Online Course Assessment Part 2 Presentation Transcript

  • Designing and Developing Online Course Assessments
    Day 2: Strategies and Resources
    Dr. Veronica Diaz, Maricopa Community Colleges
    Dr. Patricia McGee, The University of Texas at San Antonio
  • Day 2 Agenda
    Welcome
    Part 1: What are the types of effective assessments in an online course?
    Part 2: How can we select the appropriate assessment strategy?
    Q&A
  • Introduction
  • Assessment and
    Learning
  • Assessment Toolkit
    P. McGee
  • What are the types of effective assessments in an online course?
  • Effective uses of rubrics: Content, processes, attitudes, etc.
  • Rubrics
    Specifically state the criteria for evaluating student work
    Are more specific, detailed, and disaggregated than a grade and can help students to succeed before a final grade
    Can be created from
    Language in assignments
    Comments on students’ papers, or
    Handouts intended to help students complete an assignment
  • Development Steps
    Identify what you are assessing (e.g., critical thinking, writing, process, participation)
    Identify the characteristics/behavior of what you are assessing (e.g., presenting, problem-solving)
    Decide what kind of scales you will use to score the rubric (e.g. checklists, numerical, qualitative, or numerical-qualitative)
    Describe the best work you could expect using these characteristics: top category
    Describe the worst acceptable product using these characteristics: lowest category
  • More Steps
    Develop descriptions of intermediate-level products and assign them to intermediate categories:
    1-5: unacceptable, marginal, acceptable, good, outstanding
    1-5: novice, competent, exemplary
    Other meaningful set
    Test it out with colleagues or students by applying it to some products or behaviors and revise as needed to eliminate ambiguities
  • Rubric Tips
    Develop the rubric with your students
    Use same rubric that was used to grade
    Use examples to share with students, so they can begin to understand what excellent, good, and poor work looks like
    Have students grade sample products using a rubric to help them understand how they are applied
    In a peer-review process, have students apply the rubric to each other’s work before submitting it for official grading
  • Benefits
    Allows assessment to be more objective and consistent
    Focuses instruction to clarify criteria in specific terms
    Clearly shows the student how their work will be evaluated and what is expected
    Promotes student awareness of about the criteria to use in assessing peer evaluation
  • Rubric Resources
    http://www.uwstout.edu/soe/profdev/rubrics.shtml
    http://www.calstate.edu/AcadAff/SLOA/links/rubrics.shtml
    http://www.iuk.edu/~koctla/assessment/rubrics.shtml
    http://www.csupomona.edu/~uwc/faculty/CSU-EPTScoringGuide.shtml
    http://condor.depaul.edu/~tla/html/assessment_resources.html
    http://www.winona.edu/AIR/rubrics.htm
    http://www.engin.umich.edu/teaching/assess_and_improve/handbook/direct/rubric.html
    http://www.seattleu.edu/assessment/rubrics.asp
    http://wsuctproject.wsu.edu/ctr.htm
    http://www.utexas.edu/academic/diia/assessment/iar/students/report/rubrics-types.php
    Rubric Template: http://edweb.sdsu.edu/triton/july/rubrics/Rubric_Template.html
  • Activity
    Consider how you might use or modify one of these for your course
    Share in chat
  • Peer Assessment
  • Contributes to…
    Ownership and deeper learning
    Motivation
    Independent and creative thinking
    Self awareness of learning as compared to that of others
  • Peer Assessment….
    Strategies
    Participation in peer assessment scales, processes
    Rating
    Peer Critique/Review
    Mentoring/tutoring/coaching
    Rubrics: for drafts, in conjunction with instructor assessment
    Tools
    Wiki or Google Docs
    CMS rating function
    Rubric, generic survey
    IM, Twitter, Facebook
  • Peer Review
    Example: peer review in a research methods course
    Google Docs
  • Developing Good Feedback
  • Frequency of Feedback
    Pedagogical Timing
    After practice
    At completion/achievement of objective
    Calendar Schedule
    Indirect – weekly
    Direct - bi-monthly
  • Where and how…
    From http://www.french-in-aude.com/pages/skype.htm
    From http://www.thethinkingstick.com/?p=604
  • Student Self-Assessment
  • Involving the learner…
    Incorporate meta-cognitive assessments
    Provide a strategy for self-assessment and progress
    Track progress
    Compare work to that of others
    Grade own performance
  • Progressive Benchmarks
  • Self-Assessment
    Techno CATs
    Before/after unit
    Reflection
    Feedback on design
    Feedback on technology
    Increases accountability
    Engages students
    Starts discussions
    Early alert
    Practice
  • Examples
    What was the one most useful thing you learned in this assignment, unit or module?
    What suggestions would you give other students on ways to get the most out of this assignment, unit or module?
    In what area did you learn or understand the most? Least?
    List three ways you think you have developed or grown as a result of this assignment, unit or module?
    What did you learn about writing, research, (or any other skill) from this assignment, unit or module?
    What problems did you encounter in this assignment, project, unit, or tool that was used?
    What unit/module of this course was your best work and why?
  • Cases
  • Case-based instruction
    Well-bounded cases are presented to students as a focus for discussion and analysis.
    One situation or case becomes the focal point for an instructional sequence.
    Cases can illustrate a real world situation that requires application of learned course content.
    Cases can be provided in segments, as learners become prepared to address different components of the case.
  • Case Assessment
    Teams
    Final outcome against pre-determined criteria
    Completion
    Performance against other teams
    Individual
    Contribution
    Objective assessments
  • Case-based Assessment
    Student Generated
    Pre-determined and communicated context, focus, format
    Peer review/critique
    Expert review/critique
    Contest
    Instructor Provided
    Case is focus of formal assessment
    Solution-resolution-outcome is withheld
    Students complete case
  • Projects
  • Project-based learning
    A long term instructional activity in which students work as a group as they focus on a question, problem, event or interest, investigate and negotiate understanding, and produce a product that represents their understanding.
    (Brown & Campione, 1994)
  • Considerations for Online Collaborative Projects
    Virtual or field?
    Team or individual?
    Cooperative or collaborative?
    Progressive?
  • Types of Projects
    Research
    Debate
    Presentation
    Teach/mentor/coach
    Design-Develop
    Simulation (virtual worlds)
  • Possible Tools
    Presentation (such as Voicethread, Bubbleshare, Yawnbuster)
    Videocasts (Such as Viddler, Videostream)
    Social networking (such as Ning, Epsilen)
    Virtual Worlds (such as Second Life)
  • Helpful techniques
    Set benchmarks
    Use peer review/assessments
    Provide regular and informative feedback if not formal assessment
    Set criteria for performance, completion, scope, and achievement
  • Student Portfolio
  • What are they?
    A way to organize, summarize, and share artifacts, information, and ideas about [you decide]
    A sampling of the breadth and depth of a person's work conveying the range of abilities, attitudes, experiences, and achievements
  • Uses
    Reflection
    Reference
    Progressive
    Evaluation
    Sharing
    Employer oriented
    Course, unit, program based
    Repository
  • Tools & Considerations
    Tool Ideas
    Internal
    CMS
    Web pages
    External
    Wikis
    Google Sites
    Blogs
    Considerations
    Interactive
    Support multimedia
    Security and privacy
    Allows assessment (scoring and data aggregation based on a rubric)
    Portable (exportable)
    Storage space
  • How can we select the appropriate assessment strategy?
  • Team/Group Assessment
    46
  • POLL: Which best describes your experience with student teams?
    Have used them successfully (students like them and I do too)
    Have used them, but students don’t like them
    Have never used them
    I’d use them more often if I could use them effectively
    They don’t fit well with my courses
    47
  • Using Teams
    Based on the work of Larry Michaelsen (University of Oklahoma)
    http://teambasedlearning.apsc.ubc.ca/
    3 Keys
    Promoting ongoing accountability
    Using linked and mutually reinforcing assignments
    Adopting practices that stimulate idea exchange
    48
  • Promoting Ongoing Accountability
    Require pre-group work
    Require group members to express individual opinions and monitor via another member
    Include peer evaluation in grading
    Readiness Assurance Process
    Test over readings
    Group: Test, discuss, reach consensus and retest
    Provide information for peer feedback process
    49
  • Using linked and Mutually Reinforcing Assignments
    50
  • Adopting Practices that Stimulate Idea Exchange
    Use of assignments that create conditions that foster give-and-take interaction
    Diversity of opinion, ideas, and perspectives
    • Not too easy
    • Not too much writing
    • Employ, select, apply concepts from the course
    • Assign roles
    • Use permanent groups
    • Size: 4-7
    51
  • Team Teaching Tips
    Outline learning goals
    Teach team skills
    Clear and detailed instructions
    Rubric
    Stages of team development
    Forming - polite but untrusting
    Storming - testing others
    Norming - valuing other types
    Performing - flexibility from trust
    52
  • Cooperation and Collaboration
    COOPERATION – Each team member takes on a specific and unique role, e.g., editor, task manager, researcher, etc.
    The instructor assign tasks, monitors accountability, sets a clear procedure, devices a strategy for oversight of parts as relate to the whole
    COLLABORATION – Team members contribute perform same role and compile contributed pieces, e.g. all students research, interview, collect, examine, etc.
    Assign roles, accountability, clear procedure
  • Team Contracts
    Purpose, goals, and missions
    Expectations
    Roles
    Conflict resolution strategies
    Meetings
    Communication
    Decision-making policy
    Agendas
    Record-keeping
    54
  • Other Resources
    Team Based Learning (Michaelsen)
    http://teambasedlearning.apsc.ubc.ca/?page_id=9
    Video Demonstrations
    http://teambasedlearning.apsc.ubc.ca/v/michaelsenvid.html
    55
  • 56
  • 4 Questions
    What do I want students to be able to DO after this unit of instruction (behavioral outcomes)
    What will students have to KNOW to do XYZ (learning outcomes)
    How can I ASSESS whether or not students have successfully mastered key course concepts? (feedback)
    How can I tell if students will be able to USE their knowledge of key course concepts? (application)
    57
  • Polls/Surveys
  • Poll/Survey Options
    Poll/Survey Options
    Functions
    Strategy Ideas
    • Engage – collect data – generate conversation
    • Informal assessment of understanding/comprehension
    • Formal assessment of critical thinking
    • Random questions for points
    • Drill & practice activities with participation points (online quizzes)
    • Student constructed – collaborative/cooperative
    • Matrix survey
    http://elearningtools.wetpaint.com/page/Polling%2C++Surveying
  • Individual Assessments
  • Individual Classroom Assessment
    A method used to inform you on ….
    Students learning
    Effectiveness of course content
    Effectiveness of teaching methods
  • Individual Assessment Benefits
    Learner-centered
    Teacher-prompted
    Mutually beneficial
    Formative
    Fast to administer and interpret
    Non threatening
    Ongoing
    Foster trust between student and instructor
  • Basic Development Steps
    Choose a learning goal to assess
    Choose an assessment technique
    Apply the technique
    Analyze the data and share the results with students
    Respond to the data, i.e., make modifications as necessary
  • 5 Suggestions for Online CATs
    Customize to your specific needs and learning environment
    Should be consistent with your instructional philosophy
    Test out a CAT and assess their effectiveness
    Allow extra time to carry out and respond to the assessment
    Let students know what you learn from their feedback and how you and they can use that information to improve learning
  • Online CAT Examples
    Chain notes
    One-sentence summary
    Application cards
    Student-generated test questions
    Can be easily modified or converted to an online environment
  • CATS as Formative Assessments
    CATs (most can be easily converted)
    Classroom Assessment in Web-Based Instructional Environment: http://pareonline.net/getvn.asp?v=9&n=7
    Teaching Tips: http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/teachtip.htm#assessment
  • Assessment Resources
    http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-2.htm
    http://www.ntlf.com/html/lib/bib/assess.htm
    http://www.ntlf.com/html/sf/vc75.htm
    http://technologysource.org/article/classroom_assessment_techniques_in_asynchronous_learning_networks/
    http://www4.nau.edu/assessment/main/research/webtools.htm
  • Exercise
    Review the CATs and pick one
    Identify the goal for your CAT
    Explain why this CAT is helpful/necessary in this particular area of the course
    How and when will students receive feedback on the CAT
  • Discussion Assessments
  • Posting vs. processing
    But I posted six messages??
    You didn’t tell me I had to respond to other posts
    I didn’t have anything to say!
    I didn’t understand the instructions!!!
  • Start with..
    General discussion criteria in the syllabus
    Reiterate criteria and instructions as first message in discussion thread
    Provide examples of an appropriate post
    Facilitate with public feedback to the group and private feedback to the individual
  • Possible discussion criteria
    Number of posts (least effective)
    Number of words
    Type of responses:
    Answer a question
    Pose a question
    Respond to another post
    Provide new information, examples, or evidence
    Quality of response
  • Chat Assessments
  • Participation vs. contribution
    Did the learner arrive on time?
    Did the learner remain in chat room the entire time?
    Did the learner contribute?
    Did the learner interact with others?
    Did learner follow chat conventions?
    What was frequency of contribution?
    Where chat expectations communicated?
    How did student meet chat expectations?
    How was chat assessed?
  • Chat strategies
    Breakout rooms with specific outcomes
    Presentations with pere-determined criteria (Rubric)
    Debates (winner)
    Discussion (participation and contribution)
    Learner generated conventions
  • Other chat assessments?
    [respond in chat]
  • Q&A
  • Assessment Toolkit
    P. McGee
  • Thank you!
    drvdiaz@gmail.com
    Patricia.mcgee@utsa.edu