Iterative research and development of teacher training in learning design
Continuous assessment design case
1. DESIGNING A SELF- AND PEER-
ASSESSMENT METHOD TO GRADE Micah Gideon
EQUITABLY AND REDUCE SOCIAL Modell
LOAFING IN GROUPS
2. WHAT IS A DESIGN CASE?
• Description of a designed artifact or experience
• Common in many design fields
• Accumulated by expert designers
• Serve as inspiration or precedent
For information and examples, refer to the International Journal of Designs for
Learning (http://scholarworks.iu.edu/journals/index.php/ijdl)
3. COMPONENTS OF THE DESIGN
•Online system
•Training for
students
•Email reminders
•Reports for
instructor
4. BACKGROUND
• Corporate Experience
• Instructional Design
• Software Development
• Masters in Instructional Design,
Development & Evaluation
• Inspired by Vygotsky
5. BACKGROUND
• Game design course
• Internal group conflicts
• Lack of clarity on contributions to product
• Reflection papers
• Valuable for students
• Less valuable for the instructor
• Other methods exist
6. THE CONTEXT
• Asked to teach two courses
• Full (re)design necessary
• Intimidating technical content
• Graduates likely to work in groups
• 30 students in one of the classes
7. THE PROBLEM
• Equitably allocate credit
• Address concerns of students
• Credit for work done
• Student voice
8. INFLUENCES
(PRECEDENT)
• Tucker & Reynolds’ (2006) work
• Enterprise software development (web
applications)
• Human-Computer Interaction Design
• Red Hat performance-based assessments
9. THE CONSTRAINTS
• 3 weeks to delivery
• Security of confidential data
• Modular reusability (for class)
• Low impact on students
11. IMPLEMENTATION
• How exactly would I use this?
• Formative in addition to summative
• Look for trends (not spikes )
• Is all effort the same?
• Categories of effort
• Work
• Creativity
• Group Dynamic
16. THE EXPERIENCE
• Indicated group issues
• High rate of completion
• Very little ongoing support was necessary
17. FUTURE
• Research
• What does the data mean?
• Categories/behaviours
• Longitudinal research
• Reporting
• Proactive/formative
• Summative calculations
• Administration
• Support for organization structure
18. REFERENCES
• Boling, E. (2010). The Need for Design Cases : Disseminating Design
Knowledge. International Journal of Designs for Learning, 1(1), 1–8.
Retrieved from http://scholarworks.iu.edu/journals/index.php/ijdl/
index
• Driscoll, M. P. (2005). Psychology of learning for instruction. (3rd
ed.). New York, NY: Pearson Education, Inc. Retrieved from http://
eric.ed.gov/ERICWebPortal/recordDetail?accno=ED369772
• Vygotsky, L. S. (1978). Mind in society. (M. Cole, S. Scribner, V. John-
Steiner, & E. Souberman, Eds.) (p. 159). Harvard University Press.
www.PeerAssess.info
It is descriptive\nnot a report on that research\nhow this tool came to be.\n\nFor this case, I have used my own recollections, version control records, emails and IRB-covered student-respondent data to render this report.\n
I would probably have reached out to him/her or this group.\nI never even built the summative calculations -> v2\n\n\n
Vygotsky (as explained by Driscoll)\nCollaboration with others =>ZPD\n\n\n
Students reported difficulties with their group\nThe groups were black boxes – I could not see inside or tell who had done what.\nReflection is valuable to student (Schon, Cowan, Davis )\n- Too late for a learning experience\nHis word against hers\nUnlikely to reflect the entirety of a long group project\n\nAsk: what other methods have you used?\n
audience was non-technical\nGraduates likely to work in groups\nPast: 10-15 students per class.\n
Ask: how many have used group projects?\nAsk: how do students respond when they learn of the group project?\n
Tucker & Reynolds: Weekly Self- and Peer-Assessment\nGroups of architecture students\nSemi-engineered groups (personality tests)\nJ2EE: I knew what was possible\nHCI/d: user-centered approach / experience design\nRH: no stranger to building complex assessment tools\n
Low impact – self-imposed based upon HCI/d study\nUser/student-centric approach\n
Formative capabilities seem obvious now, but I didn’t see it at the beginning\n\nNew method would require training and support\nDeveloped an introductory script \nFormative: “you can have a bad week, but you can’t have a bad semester”\nSummative: I reserve the right to reallocate points to those who put in effort\n
Low Load\n Reminder emails\n Simple\n \n\n\n
Assessment Screen\n Matrix of values\n Validation of math\n Feedback on math\n\n\n
Assessment Screen\n Matrix of values\n Validation of math\n Feedback on math\n\n\n
I would probably have reached out to him/her or this group.\nI never even built the summative calculations -> v2\n\n\n
89.2% (51)\nI found it useful and valuable\nSome errors were identified and pointed out by students:\nMinor errors\nNo one seemed annoyed\n
