By Maggie Seah & Yong Zi Fong KEIO-NUS CUTE CENTER In collaboration with NUS Dentistry

1. User Research
(Biostatistics in Dentistry)

2. CONTENT PAGE
1. GENERAL OVERVIEW
2. STAKEHOLDERS
3. gOALS
4. pRE-KNOWLEDGE AND POST-KNOWLEDGE
5. SERIOUS GAMES
6. SERIOUS GAMES AND PRE/POST-KNOWLEDGE
7. DISCUSSION’S COMMENTS
8. SUGGESTIONS
9. CONCLUSION
10. APPENDIX

3. GENERAL OVERVIEW
Overarching Stakeholder Map
In this user study, we (Cute Center) are doing ethnographic research for an educational game-based tool that be-
tween the two main stakeholders: Prof Hsu and Current students. This tool is aimed to ease the teaching and learning
of Biostatistics in Faculty of Dentistry, National University of Singapaore. We will also be interviewing Dentistry alumni
to gain more insights into the opportunity space.
Empathy Map
Not a priority
Biostats is
important
Provides
testimony
Stepping in
to design a
game-based tool

4. Stakeholders
Current Curriculum Structure (Student’s perspective)
Desired Curriculum Structure (Student’s perspective)
Dr Hsu
• Planned minimal mathematical concepts in syllabus
• Quizzes has scenarios and data
• Aim to assess student’s critical thinking
• From teacher’s perspective, one of the top challenges is teaching the concept of probability.
Current Students
• “Don’t understand or care about Biostats”
• Most understanding of Biostatistics comes from self-reading and revolves around basic familiarity with
terminologies and definitions
• Students can grasp new information, but retention is low
• What they hear from alumni confirms their belief that biostatistics is not essential in becoming a success-
ful practictioner
• Need more content and scenarios; lack of exposure to situational application and exercises
- Generation of data -> Running the data -> Analysis of data
- Reading papers
• Improving SPSS skills
- Not knowing what to click on SPSS (would be good if there was a tutorial that students can work
alongside)
• Improving Graphical knowledge
- Do not understand graphs cannot make the judgement as to whether a new treatment is effective
Would like to learn more about how these graphs are formed and how to read these graphs/data.
Scenarios
Scenarios
Graphical knowledge
Graphical knowledge
Technical knowledge (eg. SPSS)
Technical knowledge (eg. SPSS)
Terminologies/Definitions
Terminologies/Definitions

5. Graduated Students
Recognise value to Biostatistics before graduation in:
• Reading Research Papers
- Reading scenarios and analysing limitations
- Assess articles more before graduation, to see if the statistical methods are applied correctly.
• Evidence-based practice
- Evidence based practice vs Dogma | Important to design a learning environment where we can pick
up some skills to think critically instead of just following dogma.
- What does evidence based practice entail: When new trends/products/literature comes out:
Treatment vs Prevention. What is new now that there is a lot more preventative? When do we use
preventative? In this patient’s particular case, is using preventive treatment appropriate?
• Running own research
- What Elim learn for FYP:: Choosing what tests to run for my research project. It is good that we learn
before our final projects, this helps us know how to choose our samples and whether our samples
normally distributed.
- However, had to self-learnt Biostatistics because knowledge was insufficient (not enough informa
tion to ascertain the reasons: Low retention? Low understanding? Inadequate syl labus?)
Cute Center
• Have limited capabilities in understanding in-depth biostatistics content
• Have the technical know-how but resources are dedicated to AR
• Feels that indirect vision does not help in the learning of biostatistics

6. GOALS
Goal of Biostatistics in Dentistry
Goal of the Game (Overview)
From our interviews, we understand that Dr Hsu wants students to be able to apply the knowledge from Biostatistics
into their other modules in school like Community Health and UROP and further on in their clinical career. This Bio-
statistical knowledge can also be broken down into Pre-knowledge, understanding of concepts and Post-knowledge,
spontaneous and intuitive application of the knowledge. (Refer to Appendix I)
From the interview, we also understand that the overt goal is to achieve Statistical Literacy and the hidden agenda is
Joy. The game also has three criteria:
1) An ability to allow teachers and students to teach and learn Biostatistics with ease.
2) Engagement with students
3) Dental Relevance
(Refer to Appendix I)
Overt goal:
Statistical Literacy
Hidden goal:
Joy
Pre-Knowledge Post-Knowledge
OTHER MODULES clinical careerBiostats module
Evidence-based
dentistry
Community Health
UROP
(Final Year Project)
Clinical Tutorials

7. PRE-KNOWLEDGE AND POST-KNOWLEDGE
Pre-knowledge
(Acquisition of prior knowledge)
Current syllabus:
• A short amount of time allocated to learn a lot of
content (1 hour lesson a week)
• Content: T-test (One sample/Two Sample), Spearman,
Pearsons correlation tests
• Notes has minimal mathematical concepts (Prof Hsu:
to avoid overloading the students)
• Collaborative learning: Instead of just doing ques-
tions like in JC, the students are currently challenged
to come up with their own questions for group proj-
ects
• Understand the HOW TO (summarise the data) and
the WHAT (is probability), not so much to remember.
Post-knowledge
(Familiarization and application)
Assessment: To assess student’s critical thinking
• Non-examinable
• Quizzes (Some scenarios, some data)
• Lit Reviews (No JC-like questions, more like lit re-
views. Know what kind of tests to use. If there is final
data, run relevant tests, see the p-values, we find out
if we accept or reject null hypothesis; or perhaps find
the possible limitations of these tests)
• They usually check for accuracy by running
through with Prof Hsu.
Other application methods that the faculty has been
considering:
• Game-based learning - Board game
• Flipped classroom lectures (no tutorials)
Bloom’s Taxonomy was mentioned in the discussion, however Dr Hsu states that the concept is not clinically sound.
However, we highlighted the levels that were relevant, in respect to the discussion.

8. SERIOUS GAMES
A serious game or applied game is a game designed for a primary purpose other than pure entertainment, in this
case, it is also aim to fulfil educational goals. To understand serious games better, we broke down the various types
of serious games currently present in the market. We have also placed them on a market positioning map to evaluate
the level of density content (Low to High Density Content) against their method of game-play (Hands-on to Thinking).
“Hands-on” refers to action-driven gameplay and “Thinking” refers to turn-based cognitive gameplay.
1. Gamified 2. Strategy
3. Simulation

9. serious games and pre/post knowledge
We categorised the different types of gameplay according to our intepretation of its functionality in educating stu-
dents for Non-Biostatistics, Pre-knowledge and Post-knowledge Biostatistics.
Post knowledge: Games in the high-density - thinking quadrant may be better at accomodating scenario-driven
gameplay, hence familiarization and application of knowledge will be needed.
Pre-knowledge: Games in the low density quadrant are able to engage students in teaching singular concepts in
Biostatistics, hence allowing acqusition of knowledge
Non-biostatistics: Games in the high-density - hands-on quadrant are more appropriate for practical clinical situa-
tions, instead of conceptual subjects like Biostatistics.

10. discussion’s comments
We collected suggestions from the discussion and match it with the type of gameplay to see if it matches the goal of
the Biostatistics in Dentistry. (Refer to Appendix II for the full suggestions)
“Good if the game can incorporate which data to use, giving you many scenarios and how to
gather those data, to make it more intuitive.”
“...this seems to be the kind of game where you use something that is hands on and might inter-
est dental students and then ask them how biostats can be applied to it. Their interest might be
transferred into understanding the lesson. Monte carlo was seen as a flip coin sort of thing. They
will participate in it so that they can engage in the lesson. Then there there will report the data.
This kind of game definitely helps you to gain the knowledge when you first encounter Biostatis-
tics.”

11. “...Situation would play a bigger role for the post knowledge aspect of learning.”
(Using a game to generate a data set) then “My intention is to stimulate the students to think
about their statistics after they score/collection of data. But again I see that it is not spontane-
ous.”

12. Module Aim Suggestion
1 Probability To be able to estimate
that probability and
make a decision based
on that estimation
Gambling/Shooting
Yilin: E.g. The Cross eye you look through is much bigger then
when you look through you don’t know where you are shooting. Or
some bows are better than others etc. Make the game more proba-
bility centered. Different factors will affect different things.
Disease Risk Assessment
e.g. the risk of my mother to have a second stroke in the next 12
years. The clinician must be able to estimate the risk of e.g. tooth
decay depending on the patient profile. The character you choose
etc. Once they hit a certain score they are given a status
2 Formulating
concept of data
summary
distribution
To familiarise with not
just mean, but also dis-
persion
Probe game
3 Comparison
(t-test)
To determine rejection
of null hypothesis
-
4 Correlation /
Association
Finding the degree of
relationship between
two variables.
-
5 Post Knowledge Help to firm up acquired
concepts
Using existing data, consolidate from all
students and discuss.
Eg. Utilize phone data
Mandi’s feedback on 1st game : Maybe we can use something more
close to home (Instead of bow and arrow or gambling) Like their
phone data that already exists and be used for discussion. If its just
giving them context for information maybe we don’t an app to do it
because everyone has a data set somewhere, e.g. how many times
do you log into facebook. If this generates interest maybe this
could be a good start to increase their engagement.
Dental product advertisements
Q&A covering all content
Pop out questions
SUGGESTIONS

13. HIGH DENSITY CONTENT
LOW DENSITY CONTENT
HANDS-ON THINKING
POST KNOWLEDGE
PRE KNOWLEDGE
GAMBLING/SHOOTING
DISEASE RISK
ASSESSMENT
PROBE GAME
POP OUT QUESTIONS
PHONE DATA
E.G. VISITS TO FACEBOOK
DENTAL PRODUCT
ADVERTISEMENTS Q&A COVERING
ALL CONTENT
We then placed the suggestions on our positioning map to see where they can best contribute - Pre-knowledge or
Post-knowledge.

14. The suggested game format of mini-games lies in the low-density content quardrants of the
product positioning map, and lean more toward thinking (as opposed to hands-on) gameplay.
Minigames are more feasible in terms of immediate production, but is limited in the amount of
content that it can contain. It is important to think about the long-term costs of producing mini
games to cover the entire syllabus, or value of only focusing on a few topics/minigames.
Mini games are optimal for piqueing interest at the beginning of acquiring pre-knowledge, but
may have less value as the same questions or gameplay is repeated each week. This may be
mitgated by generating level-based platforms which correspond to the students week-by-week
exposure to the content in class. However, it will incur greater costs.
Mini-games might not be the most apt method to support scenario-driven gameplay. This means
less opportunity for students to apply their knowledge in a realistic and spontaneous context.
Hence, mini-games may be limited in its ability to fulfil Post-knowledge, which was stated to be
a goal of the Biostatistics game.
opportunity space

15. conclusion

16. APPENDIX

17. APPENDIX I
Goal of Biostatistics in Dentistry
Ability to evaluate research papers
Critical thinking/analysis of research articles
- Evaluate if the method been applied correctly
- Assess critical thinking: This is harder to measure
entistry
Ability to make (spontaneous) and in-
formed decisions in clinical work
Biostats to not be merely short term knowledge for
the students
- How to extend these short-term knowledge to
long-term knowledge
Embody the information rather than just memoris-
ing it
- In fact this concept (probability) will affect them down the
road. When a patient comes in, it’s not black and white (text-
book disease). In clinical situations, we have to gauge (risk)
and this might involve probability concepts.
In-depth understanding beyond JC level, to be
able to see the dispersion in data in real life situ-
ations
- “Living creatures have undulating readings. Like BP its
highly dependent on the mood of the person and the state
at the time it is taken. When we look at the BP we cant just
view it as a number. We need to know SD and potential error
range. Number alone doesn’t mean that its a lot of information.
So next time when salesman comes to the door they will be
able to ask, the numbers look nice but what is the dispersion?
So I’m hoping that the game we are making will help them
develop this. The truth is not just in one number but in the
distribution. “
Goal of Game (Overview)
Must be engaging
Must be relevant to Dentistry
Must be able to allow students to learn
Biostatistics
Game more for application of knowledge
- Not really transfer of knowledge from expert
(teacher) to student.
- Will be good if the game can increase confi-
dence level so that they can apply biostatistics in
the future
Goal of Biostatistics in Dentistry vs.
Goal of the Game (Overview)

18. APPENDIX Iia
Goal of the Game/Biostatistics vs
Suggested format of Biostatistics game
Prof Hsu
“And I think gambling is a great way to (because you
have to) estimate, and it comes at a cost, so there’s an
emotional attachment to your decision. So if there is a
way to develop a game that is geared toward/ expose
them to understand probability…”
“So instead of developing complex games maybe we
can develop this in a modular format.”
“Maybe first game just helps them to understand
probability. And another game is the one that uses the
mirror and probe to help formulate the concept of data
summary and distribution.”
“Distribution is related to some… and if we were not
forced to summarise the data, we would not need this
curve. When we ask them are you above or below this
average, they still have this linear concept of mean,
being representative of (inaudible). Mean is a very
intuitive way of summarising data. But going through
this module, we are hoping that they will add anoth-
er dimension, which is dispersion, the curve. You can
summarize the data, then the comparison will make
sense because we are not just comparing A & B (two
data points). Some of the companies want to bluff us…”
“Hopefully the third and fourth game will require them
to spontaneously exercise this (post) knowledge. Like
the ball game.”
“To enhance learning”

19. YiLin (Graduate Student)
“Good if the game can incorporate which data
to use, giving you many scenarios and how to
gather those data, to make it more intuitive. It
would definitely be good if we could incorporate
biostats into the game itself.”
“Dental advertisements”
“How a lot of dental schools taught biostats (at
least in the 60s and 70s) was through the Monte
Carlo Simulation, they get the students to play it
to teach probability. From what I’m hearing, and
what Prof has shared with me, this seems to be
the kind of game where you use something that
is hands on and might interest dental students
and then ask them how biostats can be applied to
it. Their interest might be transferred into under-
standing the lesson.”
“Monte carlo was seen as a flip coin sort of thing.
They will participate in it so that they can en-
gage in the lesson. Then there there will report
the data. This kind of game definitely helps you
to gain the knowledge when you first encounter
Biostatistics. So I imagine it to be useful if you did
not know. Like when you see the odds of some-
thing, typically when you see it, if it’s a higher
number, you would assume it represents worse
or better odds, but playing the game would give
you a more academic understanding of why is it
that this number… why is it that the odds of land-
ing on a 6 is as such… etc. So this would help with
the pre-knowledge lessons. Situation would play
a bigger role for the post knowledge aspect of
learning.”
APPENDIX IiB

20. Dr Hsu
“I’m trying to ask them, when they identify e.g. mean of
7 data set, x of a score of (0-100). If Yilin’s average is 60,
and standard deviation is 5, then when we give them the
class curve and the class has a mean, based on probabil-
ity, where would yilin be? Somehow he will know that his
name game score probably will be somewhere (gestures).
Bringing in the concept that this is a class, and this is the
distribution of the class, where I am. To be average is to
belong to the majority (Mean +/- 2sd) Do u think you are
an average student?”
Dr Hsu’s thought process: Stimulate the students to think
about their statistics after they score/collection of data.
But again I see that it is not spontaneous.
Mandi (in response)
“This kind of game will most likely end up in a QnA format.
We can make it more fun in a gamified way with positive
and negative feedback, but ultimately be more in a QnA
format. Less of a game where the act of playing it will be
the process of learning, like where in our prototype the
use of the mirror trained indirect vision.This is the best
case scenario for game-learning.”
Best case scenario should be where the is involved in for
post-knowledge and not pre-knowledge - Q and A format
APPENDIX IiC