This document discusses different types of assessment questions that can be asked to evaluate educational games and online courses. It provides examples of questions related to learning outcomes, processes, goals, comparisons, satisfaction, students, and student experiences. Traditional methods for collecting data to answer these questions are also outlined, such as exams, surveys, interviews and tracking systems. The document poses discussion questions about how to collect this evaluation data using games or other non-traditional methods.

1. Games and
Assessment
Week 6

2. Asking the Right Question

3. Effective systematic inquiry
processes begin with an appropriate
question.
Asking the right question is often
the most difficult part of
assessment.

4. Learning Outcome related questions
• Start with a specific, measurable, time-bound
learning outcome.
• Example: After unit 3, students will be able to
list the 4 fun keys to create important
emotions.

5. Learning Outcome related questions
• Example: After unit 3, students will be able to
list the 4 fun keys to create important
emotions.

6. Answer:
The 4 Fun Keys create games' four most
important emotions:

7. 1. Hard Fun: Fiero - in the moment personal
triumph over adversity
2. Easy Fun: Curiosity
3. Serious Fun: Relaxation and excitement
4. People Fun: Amusement

8. Traditional methods for data
collection:
• Exams, online or in class
• Quizzes, online or in class
• Use grading rubrics

9. Discussion Question: How could you
collect this information using games
or other non-traditional methods?

10. Process Evaluation related
questions

11. • Do our processes work well? Could they work
better?
• Do our policies help or hinder student
success?
• To what extend is our program being
implemented according to plan?
• Do we have the resources we need?

12. • What are the barriers or challenges to
implementing our processes?
• How well did students understand our
process?
• Are faculty/instructors sufficiently trained?
• Are the directions clear?

13. Traditional methods for data
collection:
• Follow-up survey
• Point-of-service interview (catch student right
after they created an account on the game
site…ask for their experiences)
• Secret shopper (have a neutral third party go
through the process and give you feedback)

14. Discussion Question: How could you
collect this information using games
or other non-traditional methods?

15. Net Effect related questions

16. • What is the effect of completing 30-credits
online at FRCC?
• Can the net effects be attributed to the course
games versus maturation?

17. Traditional methods for data
collection:
• Longitudinal tracking systems
• Set up experiments or quasi-experiments
• Ex post facto (after the fact)/post-experience surveys

18. Discussion Question: How could you
collect this information using games
or other non-traditional methods?

19. Goal Based questions
These are based on S.M.A.R.T.
(Specific, Measurable, Attainable,
Realistic, Time-bound) goals.

20. Example question(s):
• Did we meet or exceed our goal?
• If not, what could we have done differently?
• Why weren’t people signing up?
• Were we being unrealistic?

21. Traditional methods for data
collection:
• Institutional Research
• Tracking systems you have purchased or
developed (count the number of students who
signed up)
• Further qualitative methods (surveys,
interviews, focus groups, etc.) to understand
the reasons why or why not.

22. Discussion Question: How could you
collect this information using games
or other non-traditional methods?

23. Comparison questions

24. Examples:
• Do students who played Game A perform
better on the final exam than those who
played Game B?
• How much would students practice if they
practiced on the game FreeRice versus a quiz
for points?

25. Traditional methods for data
collection:
Quasi-experimental or Ex Post Facto
research methods

26. Discussion Question: How could you
collect this information using games
or other non-traditional methods?

27. Satisfaction related questions

28. Example: How satisfied are students
with ______________?

29. Traditional methods for data
collection:
• Satisfaction Surveys
• Focus groups
• Interviews

30. Discussion Question: How could you
collect this information using games
or other non-traditional methods?

31. Who Are Our Students questions

32. Example:
What type of student prefers using a
game to learn versus other modes?

33. Traditional methods for data
collection:
• Focus groups
• Interviews
• Surveys
– Paper-pencil
– Electronic
– Smart phones (surveys can now be done via smart phone)

34. Discussion Question: How could you
collect this information using games
or other non-traditional methods?

35. What Are Our Students’
Experiences questions

36. Examples:
• What do students view as engaging in an
online course?
• HOW did the game help students learn?
• What is it about the game that enhanced
learning?

37. Traditional methods for data
collection:
• Think-alouds (sit with a student while he/she
is working a problem or playing a game, have
the student talk aloud while going through the
process)

38. Discussion Question: How could you
collect this information using games
or other non-traditional methods?

39. Game Component questions

40. Examples:
What features should my game
include in order to engage students?

41. Discussion Question: What are
other game related questions?

42. MAJOR DISCUSSION QUESTION
• How would YOU evaluate this MOOC?
• What different types of questions would you
ask?
• How would you collect data to help answer
each question?