Designing and Developing Online Course Assessments  Day 1:  Online Strategies for Assessment Design Dr. Veronica Diaz, Mar...
Day 1 Agenda <ul><li>Welcome </li></ul><ul><li>Part 1: What are the foundations of online course assessment?  </li></ul><u...
What are the foundations of online course assessment?
POLL: Which best describes your experience with assessments?  <ul><li>Developed my own assessments </li></ul><ul><li>Used ...
 
Purposes of Assessment
Principles of Sound Assessment
Focus on …. <ul><li>The learner </li></ul><ul><li>Quality and relevant feedback </li></ul>
Assessment Principles for Effective Design <ul><li>Valid  and Reliable </li></ul><ul><li>Actionable </li></ul><ul><li>Effi...
Online Assessment  Principles
Align objectives – activities – assignments – assessments Objective Activity Assignment Criteria Given everyday objects, t...
Provide multiple and alternative modes of assessments
Integrate strategies to prevent cheating <ul><li>Randomized objective quizzes/tests </li></ul><ul><li>Timed and authentica...
Provide accurate and  timely feedback for performance assessments <ul><li>Detailed expectations, e.g., rubrics </li></ul><...
Evaluate your assessment practices regularly <ul><li>Are their persistent problems? </li></ul><ul><li>Is there an area of ...
Online mid-semester and/or end-of-semester course evaluations <ul><li>Does the student know what they are accomplishing? <...
What other assessment principles have you applied? <ul><li>[respond in chat] </li></ul>
Classroom vs. Online Assessment
Face-to-Face vs. Online <ul><li>Campus Classroom </li></ul><ul><li>One to many </li></ul><ul><li>Just in time assessment <...
<ul><li>One to many </li></ul><ul><li>Just in time assessment </li></ul><ul><li>Immediacy </li></ul><ul><li>Focus on metho...
Differences? <ul><li>Time  </li></ul><ul><li>Security  </li></ul><ul><li>Integrity </li></ul><ul><li>Resources </li></ul><...
Progressive Assessment
Transfer of learning Objectives? Outcomes? Monitor and Adjust What Students Know transfer of learning What students can do...
Progressive Assessment: Instructor POV (from Gardner, 2006, p. 105 adapted from Harlen, 2000)
Progressive Assessment: Series of Events (from Gardner, 2006, p. 105 adapted from Harlen, 2000)
Online Progressive Assessment
Online Progressive Assessment
Formative Assessment  <ul><li>Informs the learner of what they have learned </li></ul><ul><li>Informs the instructor of ne...
Formative: What to keep in mind  <ul><li>What kinds of issues are involved? </li></ul><ul><li>How does the learner know wh...
Ideas for formatively assessing learning… <ul><li>[respond in chat] </li></ul>
Summative Assessment <ul><li>Documents completion & achievement </li></ul><ul><li>Provides barometer against benchmark </l...
Summative: What to keep in mind <ul><li>What kinds of issues are involved? </li></ul><ul><li>Where do benchmarks come from...
Example: benchmarks
Example: benchmarks
Example: Periodic Assessments
 
 
WHAT IS THE PROCESS OF DEVELOPING ASSESSMENT?
Where do you start when you are developing an assessment? <ul><li>[respond in chat] </li></ul>
What portion of your assessments do you design yourself? <ul><li>[respond in chat] </li></ul>
POLL: Select the 2 most common assessments you’ve used <ul><li>Multiple choice tests </li></ul><ul><li>Written exams </li>...
Determining what needs to be assessed
Assessment Development  <ul><li>Specify educational objectives/intended outcomes. </li></ul><ul><li>Select assessment meas...
Role of Objectives  <ul><li>Review course objectives </li></ul><ul><ul><li>Ask, are these for skills, knowledge, or abilit...
Objective ABCDs <ul><li>Audience (the learners)  </li></ul><ul><ul><li>Identify who it is that will be doing the performan...
Objectives Activity <ul><li>Write a research report with abstract, introduction, procedure, results, discussion, conclusio...
Alignment of course activities & outcomes
Assessment and  Learning
<ul><li>Example: recall of factual knowledge </li></ul><ul><ul><li>Use multiple choice or short response assessment  </li>...
What needs to be assessed? <ul><li>Course goals  </li></ul><ul><li>Module objectives </li></ul><ul><li>Activity objectives...
Performance vs  Objective Assessments <ul><li>Via another activity </li></ul><ul><ul><li>Cases  </li></ul></ul><ul><ul><li...
Do your… <ul><li>Learning objectives  </li></ul><ul><li>Resource and materials </li></ul><ul><li>Learner interaction and a...
Assessment Alignment  <ul><li>Objective is to “design a research study related to a professional field,” but the assessmen...
 
Design vs Selection <ul><li>Design </li></ul><ul><li>High stakes assessments </li></ul><ul><li>Customized for your course ...
Bo sure to… <ul><li>Be explicit in directions </li></ul><ul><li>Don’t get off task </li></ul><ul><li>Use checklists  </li>...
Q and A <ul><li>[respond in chat] </li></ul>
WHAT SHOULD GO INTO AN ASSESSMENT TOOLKIT?
P. McGee
Determining an appropriate strategy For learning outcomes
Learners should know…. <ul><li>That an assessment is coming </li></ul><ul><ul><li>A word about helping students plan </li>...
Tools  <ul><li>Assessments  </li></ul><ul><li>Technologies  </li></ul><ul><li>Methodology or learning framework  </li></ul...
Technology and the Cycle
Direct Assessments  <ul><li>Classroom Assessment Techniques </li></ul><ul><li>Multiple choice tests </li></ul><ul><li>Embe...
Indirect Assessments <ul><li>Exit interviews or student self-report surveys </li></ul><ul><li>Final course grade </li></ul...
Assessment Considerations <ul><li>Provides guidance to further develop conceptual framework </li></ul><ul><li>Continuous p...
Suggestions for Better Assessment <ul><li>Build multiple “check point” assessments </li></ul><ul><li>Allow open note/book ...
Bloom’s Taxonomy  Focus on learner Focus on measure of learning
Bloom’s & Web 2.0 Processes Tools Attributes Remember Recognizing, recalling Visual/Text/Audio stimuli, selecting, feedbac...
Selecting assessment criteria
Assessment criteria should be… <ul><li>Measureable </li></ul><ul><li>Align with outcomes </li></ul><ul><li>Achievable with...
For example <ul><li>Chat – content learning </li></ul><ul><li>Answer questions correctly </li></ul><ul><li>Ask questions t...
Outcome + Criteria = Assessment <ul><li>How are you communicating that to learners? </li></ul><ul><li>How will you documen...
Selecting appropriate tools
Course Management Systems <ul><li>Privacy and security  </li></ul><ul><li>Familiarity  </li></ul><ul><li>Integration  </li...
CMS to Web 2.0 and Assessment <ul><li>http://web20-toolkit.wetpaint.com/ </li></ul><ul><li>http://elearningtools.wetpaint....
Web 2.0 Considerations <ul><li>Cost  </li></ul><ul><li>Accounts  </li></ul><ul><li>Technical requirements  </li></ul><ul><...
Web 2.0 Considerations <ul><li>Tools not necessarily developed for an educational audience </li></ul><ul><li>No obligation...
FERPA and Web 2.0 (handout) <ul><li>This [insert name of technology tool] is for academic use of [insert name of course], ...
Q&A
Take-Aways <ul><li>Keep the learner in the forefront </li></ul><ul><li>Focus on timely feedback and benchmarks </li></ul><...
Thank you! [email_address] [email_address]
Upcoming SlideShare
Loading in...5
×

Online Course Assessment Part 1

2,585

Published on

Online Course Assessment - Part 1

Published in: Education, Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,585
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • Veronica – Welcome and Agenda
  • Patricia
  • Patricia – Assessment is aligned with evaluation process but serve different purposes. While assessment measures the achievement and progress of the individual, evaluation indicates the successful implementation of a course of instruction and how learning is transferred to other contexts.
  • Patricia – Assessments are conducted for a variety of purposes. In an online environment these are particularly crucial since evidence may not be readily available or evident to the course instructor.
  • Patricia
  • Patricia - Valid: directly reflects the learning outcome being assessed Reliable: inter-rater reliability when subjective judgments are made Actionable: results help faculty identify what students are learning well and what requires more attention Efficient and cost-effective in time and money Engaging to respondents, so they’ll demonstrate the extent of their learning Interesting to stakeholders, care about results and are willing to act on them Triangulation: multiple lines of evidence point to the same conclusion
  • Patricia Match learning objectives with assessments Take precautions to limit the possibility of cheating Communicate assessment tasks clearly Use formative assessment to promote deeper learning; consider alternative forms of assessment such as portfolios Use self-assessments to improve learning and self-awareness Have students conduct peer-assessments (may be particularly effective when used in conjunction with group work) Multiple sources including http://www.sjc.cc.nm.us/pages/2852.asp
  • Patricia Match learning objectives with assessments Take precautions to limit the possibility of cheating Communicate assessment tasks clearly Use formative assessment to promote deeper learning; consider alternative forms of assessment such as portfolios Use self-assessments to improve learning and self-awareness Have students conduct peer-assessments (may be particularly effective when used in conjunction with group work) Multiple sources including http://www.sjc.cc.nm.us/pages/2852.asp
  • Patricia
  • Patricia Performance feedback is challenging in an online environment because performance must be re-defined. Whether performance involved actions and productions that can be easily relayed online, feedback is key to the learner understanding “how they did.” Depending upon the type of course and the numbers of students there are a variety of strategies to provide feedback.
  • Patricia
  • Patricia Evaluations can inform you if the assessment are helping students understand what they are accomplishing. They can also identify other barriers or ‘invisible’ problems.
  • Patricia
  • Patricia
  • Patricia -When considering differences between face-to-face and online contexts for learning, several factors are in critical when designing, developing, and implementing online assessments. In a campus classroom the instructor is typically delivering instruction to a group of students, while similar in an online classroom, the students are separated and are realistically in a classroom on one. Students can ‘feel’ that instruction is one-on-one and the instructor is much more in tune with what they are experiencing. Students can rely on the instructor to keep track of everything they do, while in a classroom the instructor is more group focused. In a classroom assessments are scheduled whereas in an online classroom, assessments may be needed if there are disparate responses, lack of achievement, etc. In a classroom the instructor knows and seeks out ways of assessing student understanding in real time through multiple modalities and strategies. In a.n online class there is much ambiguity about understanding. In a classroom courses are typically designed around teaching methods: lecture, presentation, research, etc. In an online course, content is typically chunked into segments where the focus is more strategy-based: discussions, chats, group work, etc.
  • When considering differences between face-to-face and online contexts for learning, several factors are in critical when designing, developing, and implementing online assessments. In a campus classroom the instructor is typically delivering instruction to a group of students, while similar in an online classroom, the students are separated and are realistically in a classroom on one. Students can ‘feel’ that instruction is one-on-one and the instructor is much more in tune with what they are experiencing. Students can rely on the instructor to keep track of everything they do, while in a classroom the instructor is more group focused. In a classroom assessments are scheduled whereas in an online classroom, assessments may be needed if there are disparate responses, lack of achievement, etc. In a classroom the instructor knows and seeks out ways of assessing student understanding in real time through multiple modalities and strategies. In a.n online class there is much ambiguity about understanding. In a classroom courses are typically designed around teaching methods: lecture, presentation, research, etc. In an online course, content is typically chunked into segments where the focus is more strategy-based: discussions, chats, group work, etc.
  • Patricia
  • Patricia
  • Patricia - In thinking about assessment we need to start by thinking about what we expect students to know. Undergraduate students particularly are focused on the acquisition of foundational knowledge that are typically measured by objective assessments which work fairly well in an online environment. If a course is designed to have the learner transfer what they have learned into application, then performance measures and assessments must be designed to allow them authentic ways to perform. In a traditional classroom, I would argue, performance assessments are easier to implement because you have control over the environment and variables, and you can see the processes students use. In an online course, these processes are often invisible and if students are novices to higher education or distance learning, then their lack of sophistication and expertise may present challenges.
  • Patricia – progressive assessment is important in online contexts because the student may not always be aware of where they stand in accomplishing course objectives. In a ftf classroom, teachers typically view progressive assessment as a series of learning events, collection and interpretation of evidence judged against pre-determined criteria, culminating in a report. References   Caldwell, B. &amp; Carter, E. M. A. (1993). The return of the mentor: Strategies for workplace learning . Routledge   Gardner, J. (2006). Assessment and learning. SAGE.
  • Patricia- Progressive assessment can also be viewed as a series of events. Here we may start with goals from which student either provide or generate evidence before or during a class. Then students and instructor determine criteria are used to judge achievement. The outcome of this judgment results in next steps for learning References   Caldwell, B. &amp; Carter, E. M. A. (1993). The return of the mentor: Strategies for workplace learning . Routledge   Gardner, J. (2006). Assessment and learning. SAGE. .
  • Patricia – in an online environment, progressive assessment in learner-centered; learners keep track, participant in, discuss, and communicate their understanding, mastery, deficits, etc. with each other and the instructor.
  • Patricia – In an online environment, progressive assessment allows the learner to document and recognize their achievement of learning objectives
  • Patricia
  • Patricia
  • Patricia
  • Patricia
  • Patricia
  • Patricia - The relationship between formative and summative assessment can be viewed in several ways. Some course designs support a semester long progression of assignments the lead to a final, summative product, project, or final evaluative event – an exam, performance, presentation, etc.. In this model, formative assessments determine whether the learner has satisfactory met the criteria and is prepared to proceed.
  • Patricia - An example of such an approach is a writing or research-based course.
  • Patricia - Another approach is to have periodic assessment that measure student learning but do not indicate a progression towards a final summative event. In this design students can proceed through the materials once they have completed the assessment.
  • Patricia - One example of this approach in an online course design is where students take different approached to the assignment and proceed through activities without intervention if they haven’t mastered content.
  • Patricia
  • Veronica
  • Veronica
  • Do in chat
  • Veronica
  • Veronica
  • Veronica - Note: if objectives are poorly written, the assessment will be difficult to construct
  • Veronica
  • Veronica - Let’s go through each and in the chat make suggestions on how it could be improved, then suggest an an assessment tool that could be used to assess
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica - Alignment
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Patricia and Veronica
  • Patricia
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica
  • Veronica- Students or others report their perception of how well a given learning outcome has been achieved
  • Veronica
  • Veronica
  • Veronica
  • Patricia - handout
  • Patricia – Selecting criteria is important for either framework
  • Patricia – Selecting criteria is important for either framework
  • Patricia
  • Patricia Look at the objective (s) Think about the criteria for assessment
  • Veronica
  • Veronica
  • Veronica
  • Veronica See handout
  • Veronica
  • Veronica- See handout
  • Veronica and Patricia
  • Transcript of "Online Course Assessment Part 1"

    1. 1. Designing and Developing Online Course Assessments Day 1: Online Strategies for Assessment Design Dr. Veronica Diaz, Maricopa Community Colleges Dr. Patricia McGee, The University of Texas at San Antonio
    2. 2. Day 1 Agenda <ul><li>Welcome </li></ul><ul><li>Part 1: What are the foundations of online course assessment? </li></ul><ul><li>Part 2: What is the process of developing assessment? </li></ul><ul><li>Part 3: What should go into an Assessment Toolkit? </li></ul><ul><li>Q&A </li></ul>
    3. 3. What are the foundations of online course assessment?
    4. 4. POLL: Which best describes your experience with assessments? <ul><li>Developed my own assessments </li></ul><ul><li>Used publisher-created assessments </li></ul><ul><li>Worked with others to create assessments </li></ul><ul><li>Used assessments developed by someone other than publisher </li></ul>
    5. 6. Purposes of Assessment
    6. 7. Principles of Sound Assessment
    7. 8. Focus on …. <ul><li>The learner </li></ul><ul><li>Quality and relevant feedback </li></ul>
    8. 9. Assessment Principles for Effective Design <ul><li>Valid and Reliable </li></ul><ul><li>Actionable </li></ul><ul><li>Efficient and Cost-effective </li></ul><ul><li>Engaging and Interesting </li></ul>Source: Allen, M. J., October 2007
    9. 10. Online Assessment Principles
    10. 11. Align objectives – activities – assignments – assessments Objective Activity Assignment Criteria Given everyday objects, the learner will identify three out of four objects with masses relatively equivalent to 1 cubic foot of water. (1) View Water flow and force. (2) Complete water displacement simulation <ul><li>Read CH 3 </li></ul><ul><li>Complete CH 3 worksheet </li></ul>(1) Complete 5 out of 6 challenge problems correctly and turned in on time
    11. 12. Provide multiple and alternative modes of assessments
    12. 13. Integrate strategies to prevent cheating <ul><li>Randomized objective quizzes/tests </li></ul><ul><li>Timed and authenticated </li></ul><ul><li>Multiple ways to demonstrate understanding </li></ul><ul><ul><li>Choices </li></ul></ul><ul><ul><li>Duplication </li></ul></ul><ul><ul><li>Repetition </li></ul></ul><ul><ul><li>Revise and resubmit </li></ul></ul><ul><li>Small group assessments for accountability </li></ul>
    13. 14. Provide accurate and timely feedback for performance assessments <ul><li>Detailed expectations, e.g., rubrics </li></ul><ul><li>Respond within a pre-designated timeframe </li></ul><ul><li>Peer critique review before final submission </li></ul>
    14. 15. Evaluate your assessment practices regularly <ul><li>Are their persistent problems? </li></ul><ul><li>Is there an area of the course that gets “stuck” and remediation or review is needed? </li></ul><ul><li>Is a lot of time spent on answering questions or clarifying? </li></ul>
    15. 16. Online mid-semester and/or end-of-semester course evaluations <ul><li>Does the student know what they are accomplishing? </li></ul><ul><li>Are their “invisible” factors that are impacting assessment? </li></ul>
    16. 17. What other assessment principles have you applied? <ul><li>[respond in chat] </li></ul>
    17. 18. Classroom vs. Online Assessment
    18. 19. Face-to-Face vs. Online <ul><li>Campus Classroom </li></ul><ul><li>One to many </li></ul><ul><li>Just in time assessment </li></ul><ul><li>Immediacy </li></ul><ul><li>Focus on methods </li></ul>
    19. 20. <ul><li>One to many </li></ul><ul><li>Just in time assessment </li></ul><ul><li>Immediacy </li></ul><ul><li>Focus on methods </li></ul><ul><li>One to one </li></ul><ul><li>Just in need assessment </li></ul><ul><li>Ambiguity </li></ul><ul><li>Focus on strategies </li></ul>
    20. 21. Differences? <ul><li>Time </li></ul><ul><li>Security </li></ul><ul><li>Integrity </li></ul><ul><li>Resources </li></ul><ul><li>Learner autonomy </li></ul><ul><li>Lack of just in need responses </li></ul><ul><li>Getting, keeping, and directing attention </li></ul><ul><li>Clarity of instructions and expectations </li></ul>
    21. 22. Progressive Assessment
    22. 23. Transfer of learning Objectives? Outcomes? Monitor and Adjust What Students Know transfer of learning What students can do Facts Procedures Principles Strategies Critical thinking Problem finding/solving Creative thinking Typically measured by objective assessments Typically measured by performance assessments Homework, quizzes, tests, exams Projects, cases, problems, designs, experiments
    23. 24. Progressive Assessment: Instructor POV (from Gardner, 2006, p. 105 adapted from Harlen, 2000)
    24. 25. Progressive Assessment: Series of Events (from Gardner, 2006, p. 105 adapted from Harlen, 2000)
    25. 26. Online Progressive Assessment
    26. 27. Online Progressive Assessment
    27. 28. Formative Assessment <ul><li>Informs the learner of what they have learned </li></ul><ul><li>Informs the instructor of next steps </li></ul><ul><li>Can be informal or formal </li></ul>
    28. 29. Formative: What to keep in mind <ul><li>What kinds of issues are involved? </li></ul><ul><li>How does the learner know where they are on their progression towards competency and mastery? </li></ul><ul><li>How can we document/illustrate/measure? </li></ul><ul><li>Has the learner practiced? </li></ul><ul><li>Other? </li></ul>
    29. 30. Ideas for formatively assessing learning… <ul><li>[respond in chat] </li></ul>
    30. 31. Summative Assessment <ul><li>Documents completion & achievement </li></ul><ul><li>Provides barometer against benchmark </li></ul><ul><li>Formal </li></ul>
    31. 32. Summative: What to keep in mind <ul><li>What kinds of issues are involved? </li></ul><ul><li>Where do benchmarks come from? </li></ul><ul><li>How do we illustrate progress against benchmarks? </li></ul><ul><li>Other? </li></ul>
    32. 33. Example: benchmarks
    33. 34. Example: benchmarks
    34. 35. Example: Periodic Assessments
    35. 38. WHAT IS THE PROCESS OF DEVELOPING ASSESSMENT?
    36. 39. Where do you start when you are developing an assessment? <ul><li>[respond in chat] </li></ul>
    37. 40. What portion of your assessments do you design yourself? <ul><li>[respond in chat] </li></ul>
    38. 41. POLL: Select the 2 most common assessments you’ve used <ul><li>Multiple choice tests </li></ul><ul><li>Written exams </li></ul><ul><li>Problem-based projects </li></ul><ul><li>Research papers </li></ul><ul><li>Case studies </li></ul><ul><li>Short answer </li></ul><ul><li>Large projects </li></ul><ul><li>Essay questions </li></ul>
    39. 42. Determining what needs to be assessed
    40. 43. Assessment Development <ul><li>Specify educational objectives/intended outcomes. </li></ul><ul><li>Select assessment measures and techniques. </li></ul><ul><li>Specify assessment criteria. </li></ul><ul><li>Evaluate student performance on exams/projects for course grades. </li></ul><ul><li>Evaluate student performance on course measures to assess effectiveness. </li></ul>
    41. 44. Role of Objectives <ul><li>Review course objectives </li></ul><ul><ul><li>Ask, are these for skills, knowledge, or abilities </li></ul></ul><ul><li>Are they measurable </li></ul><ul><ul><li>Demonstrate (authentic) or </li></ul></ul><ul><ul><li>Reflect on knowledge/experience </li></ul></ul><ul><li>What do they say about student outcomes </li></ul><ul><li>What is the goal of assessment, what do you want the student to be able to do or demonstrate they can do </li></ul><ul><ul><li>Use verbs to describe this </li></ul></ul><ul><ul><li>Share with students </li></ul></ul>
    42. 45. Objective ABCDs <ul><li>Audience (the learners) </li></ul><ul><ul><li>Identify who it is that will be doing the performance (not the instructor). </li></ul></ul><ul><li>Behavior (Performance): </li></ul><ul><ul><li>Make sure it is something that can be seen or heard. </li></ul></ul><ul><li>Condition (under which the learners must demonstrate their mastery of the objective): </li></ul><ul><ul><li>What will the learners be allowed to use? What won't the learners be allowed to use? </li></ul></ul><ul><li>Degree (HOW WELL the behavior must be done): </li></ul><ul><ul><li>Common degrees include: Speed, Accuracy, Quality </li></ul></ul>
    43. 46. Objectives Activity <ul><li>Write a research report with abstract, introduction, procedure, results, discussion, conclusions, and recommendations. </li></ul><ul><li>You will learn to analyze and interpret data. </li></ul><ul><li>You will learn about about theoretical issues underlying communicative and task-based language teaching. </li></ul><ul><li>Describe the political and social factors impacting the implementation of biotechnology in agriculture. </li></ul>
    44. 47. Alignment of course activities & outcomes
    45. 48. Assessment and Learning
    46. 49. <ul><li>Example: recall of factual knowledge </li></ul><ul><ul><li>Use multiple choice or short response assessment </li></ul></ul><ul><li>Example: complex learning outcomes, such as reasoning, communication, teamwork </li></ul><ul><ul><li>a performance assessment is likely to be appropriate </li></ul></ul>
    47. 50. What needs to be assessed? <ul><li>Course goals </li></ul><ul><li>Module objectives </li></ul><ul><li>Activity objectives </li></ul><ul><li>Content Knowledge: declarative, intellectual skills (concept, principle, procedure, problem solving) </li></ul><ul><li>Thinking Skills </li></ul><ul><li>Inter/Intrapersonal Skills: collaboration, cooperation </li></ul><ul><li>Technology Literacy/Proficiency </li></ul><ul><li>Other </li></ul>
    48. 51. Performance vs Objective Assessments <ul><li>Via another activity </li></ul><ul><ul><li>Cases </li></ul></ul><ul><ul><li>Projects </li></ul></ul><ul><ul><li>Presentations </li></ul></ul><ul><ul><li>Debates/negotiations </li></ul></ul><ul><ul><li>Research paper </li></ul></ul><ul><li>Be the activity itself </li></ul><ul><ul><li>Quiz </li></ul></ul><ul><ul><li>Multiple-choice test </li></ul></ul><ul><ul><li>Simulation </li></ul></ul>Learning can be assessed…
    49. 52. Do your… <ul><li>Learning objectives </li></ul><ul><li>Resource and materials </li></ul><ul><li>Learner interaction and activities </li></ul><ul><li>Course technologies </li></ul><ul><li>Support student performance? </li></ul>
    50. 53. Assessment Alignment <ul><li>Objective is to “design a research study related to a professional field,” but the assessment is a short answer test </li></ul><ul><li>Objective is to “understand particular statistical methods for various types of research,” but the assessment is to write an essay </li></ul>
    51. 55. Design vs Selection <ul><li>Design </li></ul><ul><li>High stakes assessments </li></ul><ul><li>Customized for your course </li></ul><ul><li>Process-oriented </li></ul><ul><ul><li>Cases </li></ul></ul><ul><ul><li>PBL </li></ul></ul><ul><ul><li>Projects </li></ul></ul><ul><li>Classroom assessments (CATs) for an online environment </li></ul><ul><li>Select </li></ul><ul><li>Self assessments </li></ul><ul><li>Short, quick checkpoints </li></ul><ul><li>Repeatable with large question bank </li></ul><ul><li>Low stakes </li></ul>
    52. 56. Bo sure to… <ul><li>Be explicit in directions </li></ul><ul><li>Don’t get off task </li></ul><ul><li>Use checklists </li></ul><ul><li>Provide examples </li></ul>
    53. 57. Q and A <ul><li>[respond in chat] </li></ul>
    54. 58. WHAT SHOULD GO INTO AN ASSESSMENT TOOLKIT?
    55. 59. P. McGee
    56. 60. Determining an appropriate strategy For learning outcomes
    57. 61. Learners should know…. <ul><li>That an assessment is coming </li></ul><ul><ul><li>A word about helping students plan </li></ul></ul><ul><li>What happens after the assessment </li></ul><ul><li>How to prepare for an assessment </li></ul><ul><li>What participating in the assessment will look like </li></ul><ul><li>Practice </li></ul>
    58. 62. Tools <ul><li>Assessments </li></ul><ul><li>Technologies </li></ul><ul><li>Methodology or learning framework </li></ul><ul><li>Deploying the method or framework </li></ul><ul><li>Disciplinary considerations and requirements </li></ul>
    59. 63. Technology and the Cycle
    60. 64. Direct Assessments <ul><li>Classroom Assessment Techniques </li></ul><ul><li>Multiple choice tests </li></ul><ul><li>Embedded Test Questions </li></ul><ul><li>Pre and Post Tests </li></ul><ul><li>Discussion Threads </li></ul>
    61. 65. Indirect Assessments <ul><li>Exit interviews or student self-report surveys </li></ul><ul><li>Final course grade </li></ul><ul><li>Tracking of student data </li></ul><ul><li>Focus groups </li></ul>
    62. 66. Assessment Considerations <ul><li>Provides guidance to further develop conceptual framework </li></ul><ul><li>Continuous process (formative) </li></ul><ul><li>Guides the student to mastery of the learning outcomes </li></ul><ul><li>Assessment strategy becomes foundation for developing the instructional design of the online course </li></ul>
    63. 67. Suggestions for Better Assessment <ul><li>Build multiple “check point” assessments </li></ul><ul><li>Allow open note/book timed tests </li></ul><ul><li>Have in class quizzes and tests as well </li></ul><ul><li>Project-based assessments </li></ul><ul><li>Portfolio assessments </li></ul><ul><li>Assign group work in a wiki area that tracks student participation </li></ul><ul><li>Provide best practice guidelines for taking exams online (See handouts) </li></ul>
    64. 68. Bloom’s Taxonomy Focus on learner Focus on measure of learning
    65. 69. Bloom’s & Web 2.0 Processes Tools Attributes Remember Recognizing, recalling Visual/Text/Audio stimuli, selecting, feedback Understand Interpreting, classifying, comparing, summarizing, explaining Sorting, tagging, labeling, entering, selecting Apply Executing, implementing Manipulating, entering, feedback Analyze Differentiating, organizing, attributing Selecting, grouping, altering, tagging, labeling Evaluate Checking, critiquing Commenting, entering, responding Create Generating, planning, producing Adding, generating, combining, publishing
    66. 70. Selecting assessment criteria
    67. 71. Assessment criteria should be… <ul><li>Measureable </li></ul><ul><li>Align with outcomes </li></ul><ul><li>Achievable within online format </li></ul><ul><li>Documentable </li></ul><ul><li>Clear, articulated expectations </li></ul><ul><li>Does not require additional technology skills </li></ul>
    68. 72. For example <ul><li>Chat – content learning </li></ul><ul><li>Answer questions correctly </li></ul><ul><li>Ask questions that </li></ul><ul><ul><li>Helped others learn </li></ul></ul><ul><ul><li>Extended discussion </li></ul></ul><ul><li>Make contributions to discussion that extended materials </li></ul><ul><li>Chat - performance </li></ul><ul><li>Participate throughout discussion </li></ul><ul><li>Do not interrupt </li></ul><ul><li>Respond when called upon </li></ul><ul><li>When appropriate, come to chat prepared. </li></ul>
    69. 73. Outcome + Criteria = Assessment <ul><li>How are you communicating that to learners? </li></ul><ul><li>How will you document the learning ? </li></ul><ul><li>Do you expect learner to learner a new tool for assessment ? </li></ul><ul><li>Have you provided for practice before assessment? </li></ul>
    70. 74. Selecting appropriate tools
    71. 75. Course Management Systems <ul><li>Privacy and security </li></ul><ul><li>Familiarity </li></ul><ul><li>Integration </li></ul><ul><li>Support </li></ul><ul><li>Limited tool options </li></ul><ul><li>Some web 2.0 options </li></ul>
    72. 76. CMS to Web 2.0 and Assessment <ul><li>http://web20-toolkit.wetpaint.com/ </li></ul><ul><li>http://elearningtools.wetpaint.com/?t=anon </li></ul><ul><li>http://maricopatech.wetpaint.com/ </li></ul><ul><li>http://web2educationuk.wetpaint.com/ </li></ul>
    73. 77. Web 2.0 Considerations <ul><li>Cost </li></ul><ul><li>Accounts </li></ul><ul><li>Technical requirements </li></ul><ul><li>Is the tool generative? If so, how will product generated by learner be accessed by others? </li></ul><ul><li>Exportable product </li></ul><ul><li>Technical skills required </li></ul><ul><li>Support </li></ul><ul><li>Can instructor embed or attach instructions within tool? </li></ul><ul><li>Shared/selective viewing </li></ul><ul><li>Public viewing </li></ul>
    74. 78. Web 2.0 Considerations <ul><li>Tools not necessarily developed for an educational audience </li></ul><ul><li>No obligation to users </li></ul><ul><li>Ever-changing </li></ul><ul><li>Require separate logins/accounts/fragmentation </li></ul><ul><li>No centralized institutional support (usually) </li></ul><ul><li>Reliance on internet connection (high speed) </li></ul><ul><li>Lack of security </li></ul><ul><li>Learning curve </li></ul><ul><li>Variety of use and selection of tools could overwhelm students; lack of a common experience across courses </li></ul>
    75. 79. FERPA and Web 2.0 (handout) <ul><li>This [insert name of technology tool] is for academic use of [insert name of course], [insert term/year]. It is open to the public for the purpose of sharing our work with the larger Internet community. To use this [insert name of technology tool] responsibly please observe all laws and university policy that are incorporated into the Codes of Conduct and Academic Integrity. Some specific aspects of law and policy that might be well to remember are prohibitions against copyright infringement, plagiarism, harassment or interference with the underlying technical code of this software. </li></ul><ul><li>As a student using this [insert name of technology tool], certain rights accrue to you. Any original work that you make tangible belongs to you as a matter of copyright law. You also have a right to the privacy of your educational records as a matter of federal law. Your contributions to this [insert name of technology tool] constitute an educational record. By contributing to this [insert name of technology tool], and not taking other options available to you in this course equivalent to this assignment that would not be posted publicly on the Internet and not available for the editing by others, you consent to the collaborative use of this material as well as to the disclosure of it in this course and potentially for the use of future courses. </li></ul>
    76. 80. Q&A
    77. 81. Take-Aways <ul><li>Keep the learner in the forefront </li></ul><ul><li>Focus on timely feedback and benchmarks </li></ul><ul><li>Align objectives – activities - assessments </li></ul>
    78. 82. Thank you! [email_address] [email_address]

    ×