This document discusses the role of academic research in educational technology (edtech). It aims to understand how Australian edtech entrepreneurs perceive and experience accessing and applying academic research. It notes there has been a long-standing promise that technology can solve education's problems, but this transformation has failed to materialize. Some issues identified are a lack of understanding among edtech developers of how people learn with technology, and edtech research being too focused on "what works" rather than deeper issues. The study aims to understand how edtech entrepreneurs engage with academic research to help address these issues and better inform the development of edtech products and services.

1. CRLI
Learning and Innovation Research Fest
29 November 2018
University of Sydney, Australia
#CRLIFest
Centre for Research on
Learning and Innovation (CRLI)

2. RESEARCH POSTER PRESENTATION DESIGN © 2015
www.PosterPresentations.com
We were interested in improving the learning
design of problem‐based learning (PBL) in
medical education.
In particular, our aim was to modify the "close"
phase of PBL to include more direct feedback
and an opportunity to compare and contrast
different student understandings1. There is
strong evidence that minimal guidance alone is
not effective for learning.2,3,4
We drew upon the advantages of the
consolidation phase of the productive failure
technique as part of the theoretical basis for this
redesign.5
RESEARCH AIMS AND OVERVIEW
Participants were 29 students and their four
tutors in their second year of the University of
Newcastle’s Medical Education program
(Callaghan campus) in May 2016.
Participants were randomly assigned to one of
four tutorial groups – two in the Traditional PBL
Group and two in the Productive Failure PBL
Group. The learning portion of the study ran for
four weeks.
PARTICIPANTS
KNOWLEDGE SURVEY QUESTION EXAMPLES
Before the intervention, there were not
significant differences:
• between the participating tutorial groups, nor
• between participants and their non‐
participating peers, using scores in two
recently completed courses as indicators.
The Productive Failure PBL Group performed
significantly better than the Traditional PBL
Group for:
• Week 3, Question 10 (t = 2.486, df = 26, p = .011,
one‐tailed, Cohen's d = .94).
• Week 3, total score (t = 2.042, df = 26, p = .026,
one‐tailed, Cohen's d = .773).
The Traditional PBL Group had a particularly
difficult time explaining their answers.
• Week 3, Question 5: 58% of participants in
the Traditional PBL group scored a 0/5 when
explaining the MCQ question that they got
correct, while this only occurred for 20% of
the Productive Failure PBL group.
• This difference was significant (U = 55.5,
N1 =15, N2= 12, p = .049, one‐tailed).
Students in the Productive Failure PBL Group
expressed a desire to change their concept
maps at the end of the tutorial in more
substantial ways compared to the Traditional
PBL Group (Week 3: t = 2, df = 17.951, p =.031, one‐
tailed, d = .805; Week 4: t = 2.691, df = 16.93,
p = .008, one‐tailed, d = 1.108).
KEY RESULTS
REFERENCES
1. Rittle‐Johnson, B., & Star, J. R. (2007). Does comparing
solution methods facilitate conceptual and procedural
knowledge? An experimental study on learning to solve
equations. Journal of Educational Psychology, 99, 561‐574.
2. Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why
minimal guidance during instruction does not work: An
analysis of the failure of constructivist, discovery, problem‐
based, experiential, and inquiry‐based teaching. Educational
Psychologist, 41, 75‐86.
3. Mayer, R. E. (2004). Should there be a three‐strikes rule
against pure discovery learning? The case for guided
methods of instruction. American Psychologist, 59, 14‐19.
4. Wijnia, L., Loyens, S. M., van Gog, T., Derous, E., &
Schmidt, H. G. (2014). Is there a role for direct instruction in
problem‐based learning? Comparing student‐ constructed
versus integrated model answers. Learning and Instruction,
34, 22‐ 31.
5. Kapur, M. (2012). Productive failure in learning the
concept of variance. Instructional Science, 40, 651‐672.
[alisha.portolese@sydney.edu.au] 1The University of Sydney; 2The University of Newcastle
Alisha Portolese1,, Michael J. Jacobson1, Robbert Duvivier2, & Lina Markauskaite1
Redesigning Problem‐Based Learning (PBL) in Medical Education:
Improving Learning and Consolidation
CRLI:
Centre for Research
on Learning and
Innovation
Problem
Trigger
(Open)
Self‐
Directed
Learning
Close
METHODS
All participants completed :
• the open phase of their PBL each week as per
their regular practice.
• their self‐directed learning phase as normal,
with the addition of a one‐page concept map
that summarised their learning for the week.
• a knowledge survey (opportunity to explain
desired changes to concept map, multiple‐
choice and short‐answer questions,
confidence rankings, and opinion/feedback
questions).
Participants in the Traditional PBL Group
completed their close phase of PBL as per their
regular practice. Participants in the Productive
Failure PBL Group followed a redesigned close
phase that we titled the Integrated Feedback
Approach, using their concept maps as an anchor
for comparing and contrasting understanding.
Some participants participated in a short
interview. We also collected data on participants'
and non‐participant (cohort summary) scores on
their regularly scheduled examinations.
Week 4, Question 1
Which of the following is considered to be
one of the three main areas in which
abnormalities can lead to thrombus
formation?
a) Blood oxygen deficiency
b) Disruption to blood vessel walls
c) Abnormal cell production in the bone
marrow
d) A decrease in platelet activating
factors
Week 2, Question 7
How would you explain the process of
normal coagulation to a ten‐year‐old?
__________________________________
__________________________________
__________________________________
Week 3, Question 10
What might be a technology that could
prevent leukemia? Explain why.
__________________________________
__________________________________
__________________________________
Week 3, Question 5
Explain the reasons behind your answer
to Question (3) above.
__________________________________
__________________________________
__________________________________
WORK IN PROGRESS
• Analysing student and tutor interviews and
responses to optional written feedback on
knowledge surveys.
Week 4, Question 10
If you were designing a new way to test if
a patient had VTE, what body
mechanism(s) would you need to make
sure that your test assessed? Explain why.
__________________________________
__________________________________
__________________________________
Week 3, Question 3
What genetic disorder may predispose an
individual to leukaemia?
a) Cystic fibrosis
b) Down syndrome
c) Huntington’s Disease
d) Sickle Cell Anemia
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Week 3
Question 3
Correct
Week 3
Question 5,
Score > 0
Traditional PBL
Productive Failure
PBL

3. From memorising facts to constructing categories:
Promoting deeper learning through the construction of relational
categories in an online chess‐learning platform
Memory is a crucial component
of learning. So what are the best
strategies to ensure we
remember what we learn?
1.
Testing effect:
‐ Instead of passively re‐
reading/watching/listening
‐ Test yourself (otherwise
known as active‐recall; e.g.
using flashcards)
2.
Spacing effect:
‐ Instead of ‘cramming’ all your
practice before a test
‐ Space out practice sessions
over time
As simple and effective active recall and spaced repetition are, we must be
weary of the ‘tail wagging the dog’. In other words, the goal of learning is
more than just remembering lots of things: we want learners to deeply
understand what they remember too.
A number of educational software
applications are informed by these
ideas. One such example is
www.chessable.com, which provides a
platform for learners to learn and train
the game of chess.
We are excited to be collaborating with
Chessable, and plan to run experiments
with their users on how people learn,
and how to design for learning in an
online software application.
1. Learners are shown chess
positions/puzzles, and are prompted to
find the best move
2. If the learner chooses the wrong
move, they are then shown the right
one
LEARNING ON
3. Learners then review positions
they’ve learned (over subsequent
days/weeks) according to an adaptive
spaced‐repetition type design. In
other words, they are re‐tested on
positions they get wrong more
frequently, and less frequently re‐test
positions they consistently get right.
What does ‘deep understanding’ mean?
Shallow understanding Deep understanding
• Shallow understanding is just remembering specifics
• Deep understanding is perceiving relations between
specifics, allowing abstraction and generalization.
Courtney Hilton1,2, David Kramaley3, GM Alex Colovic4, Micah Goldwater2
1 Centre for Research on Learning and Innovation (CRLI); 2 School of Psychology;
3www.chessable.com; 4Chess grandmaster, teacher, and author
Two mindsets?
When learners are practicing these chess
positions, we posit two possible ‘mindsets’ the
learner could bring to the exercise:
• Memorization mindset: attention is focused
on remembering the ‘right move’ (surface‐
features/specifics), and will play this as soon
as she remembers it.
• Relational/category mindset: attention is
focused on categorizing the relations that
define what makes the ‘right move’ correct.
Playing this move is of secondary importance.
How can we support learners in adopting a relational/category mindset?
1. Before learners solve a problem, they
tag any categories that are relevant to
the solution of the puzzle.
Here: the ‘Deflection’ category is relevant
white to play and win
2. Then, learners map this category
tag to schematic arrows on the
board.
‘Deflection’ is defined
as a move (often a
sacrifice) that deflects a
piece from the defense
of a key square. Here,
allowing the pawn to
turn into a Queen.
What’s next?: We will be running experiments to test
this idea on Chessable in 2019. Specifically, seeing
whether this ‘tag and map’ approach allows learners
to transfer and generalize what they learn more
flexibly than learners who just ‘drill’ positions for
memorization.
Why is this important for you?: Chess is a fantastic
‘model domain’ to explore how people learn for transfer.
However, expertise in almost all domains relies on having
deep relational understanding. Therefore, we hope that
insights from this research can inform learning design in
other domains including mathematics, science, and
music, to name a few.
courtney.hilton@sydney.edu.au

4. UNDERSTANDING THE IMPACT OF
FLEXIBLE LEARNING ENVIRONMENTS
ON STUDENTS’ WELLBEING
INDOOR
ENVIRONMENTAL
QUALITY
RESEARCH QUESTIONS
What’s the impact on students’ satisfaction,
concentration and incidental physical activity levels
due to the introduction of mobility observed in FLE
when compared to non-mobile, traditional schools?
How much does the IEQ within flexible spaces
differ from traditional classrooms?
What are the quantifiable benefits, if any, to
students’ satisfaction, concentration and incidental
physical activity levels arising from the design of
flexible learning environments?
RESEARCH METHODOLOGY
Comparative analysis of flexible and traditional
learning environments
Analysis to be based on field studies, including
objective and subjective measurements, conducted
pre and post-relocation, from a traditional to a
flexible environment
Measurement parameters would include indoor
environmental quality indicators (i.e. temperature,
air quality, humidity, lighting, acoustics) as well as
mobility patterns (measured as incidental physical
activity).
INTRODUCTION
Classrooms and school building designs have
come a long way since their inception, in terms of
pedagogical approach, curriculum, technology as
well as building design. The term “classroom” has
started to fade away, giving way to “learning
environments”. The driving force for this significant
transition is “flexibility”, that is required to
accommodate the needs of the 21st century
learners, mainly creativity and collaboration.
AUTHOR:
Diksha Vijapur, PhD Candidate
School of Architecture, Design and Planning

5. REFERENCES
[1] Selwyn, N. (2017). Education and technology: Key issues and debates (Second ed.). London;New York, NY;: Bloomsbury Academic, an imprint of Bloomsbury Publishing Plc.
[2] Castañeda, L., & Selwyn, N. (2018). More than tools? making sense of the ongoing digitizations of higher education. International Journal of Educational Technology in Higher Education, 15(1),
[3] Bulfin, S., Henderson, M., Johnson, N. F., & Selwyn, N. (2014). Methodological capacity within the field of “educational technology” research: An initial investigation: Methodological capacity within educational technology. British Journal of Educational Technology,
45(3), 403-414.
[4] Veletsianos, G., & Moe, R. (2017). The Rise of Educational Technology as a Sociocultural and Ideological Phenomenon. Educause Review. Retrieved on Apr 10, 2017 from http://er.educause.edu/articles/2017/4/the-rise-of-educational-technology-as-a
-sociocultural-and-ideological-phenomenon
[5] Amiel, T., & Reeves, T. C. (2008). Design-based research and educational technology: Rethinking technology and the research agenda. Journal of Educational Technology & Society, 11(4), 29-40.
[6] Marton, F. (1986). Phenomenography — A Research Approach to Investigating Different Understandings of Reality. Journal of Thought, 21(3), 28‐49.
The Role of Academic Research in Edtech
How Australian Educational Technology Entrepreneurs Perceive and
Experience Accessing and Applying Academic Research
Dwayne Ripley
University of Sydney [dwayne.ripley@sydney.edu.au]
CENTRE FOR RESEARCH ON LEARNING AND INNOVATION
SYDNEY SCHOOL OF EDUCATION & SOCIAL WORK
CRLI
Supervisor: Lina Markauskaite
BACKGROUND
There has been a long-standing promise for technology to solve education’s
problems. However, the promise of a technology-led transformation of educational
processes and practices has continually failed to materialize [1]. Academic
researchers taking a critical perspective on technology use in education have
noted issues with both those developing edtech products and services and those
researching educational technology. Some key issues include a lack of edtech
developers’ understanding of how people learn with technology [2], and issues
with edtech research being too focused on ‘what works’ [3]. It has been suggested
that if academic researchers and edtech developers continue their separate
efforts and do not increase collaboration, the promise of technology to solve
education’s problems will never materialize [4]. However, any suggested paths
forward must take into account the perspective of edtech entrepreneurs as well as
those of academics.
RESEARCH QUESTIONS
1. What are Edtech entrepreneurs’ conceptions of accessing and applying
academic research?
2. What have been Edtech entrepreneurs’ experiences in accessing
academic research?
3. What have been Edtech entrepreneurs’ experiences in applying academic
research?
RESEARCH DESIGN
Design Framework: Open-ended semi-structured interviews
Research Site: Locations convenient to the edtech entrepreneurs
Sample: Purposeful sampling was used. Sixteen founders of
Australian edtech companies volunteered for the study.
AIM
This study builds upon
research identifying the need
for increased collaboration
between academic
researchers and those who
develop edtech products and
services (entrepreneurs). It
does so by exploring the
perspectives and experiences
METHODOLOGY
Phenomenography, a qualitative research method which maps the different
ways people perceive or experience a phenomenon [6] was used for the study.
The study maps the variance in ways the phenomenon of accessing and
applying academic research was perceived and experienced. The data were
analysed through a non-linear, iterative and comparative process of sorting and
resorting which resulted in the emergence of a hierarchically structured
‘outcome space’ shown in the table on the right.
of edtech entrepreneurs accessing and applying academic research. There is a
notable gap in empirical knowledge of the role that academic research plays in
edtech entrepreneurs’ businesses. The knowledge gained can be useful to
academic researchers aspiring to collaborate and co-create knowledge together
with edtech entrepreneurs. This study also aims to contribute an entrepreneurial
perspective to identify benefits of collaboration which extend to academic
research in addition to previously identified benefits for the development of edtech
products and services.
FINDINGS
Findings suggest that entrepreneurs conceive of academic research as both
documented knowledge and the expertise of academics which is illustrated in
the two parallel columns of the chart below. Edtech entrepreneurs view
academic research as useful, but they encounter many barriers when accessing
and applying it to their businesses. Data was sorted into nine hierarchical
categories which include knowledge flows which support both unidirectional and
bidirectional knowledge transfer, knowledge translation, as well as knowledge
co-creation. Edtech entrepreneurs also view academic research in terms of how
it can benefit and evolve from edtech knowledge and expertise and increased
collaboration across academic-entrepreneurial boundaries.
------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ --------------------------------------------------------------
Research as knowledge Research as expertise
CONCLUSION
Entrepreneurs access and apply academic research for multiple purposes, with
varying degrees of success. These findings identify not only barriers to access,
application and collaboration, but also identify opportunities for collaboration
and learning across boundaries. The need for the development of thinking
across boundaries (epistemic fluency) is also identified as a potential area
of focus for improving academic-entrepreneurial collaboration.
Researchers should be “actively
engaging with practitioners in
constructing what constitutes
valuable research in order to help
direct technological development
rather than react to it”
[5] Amiel & Reeves, 2008

7. USING VR TO CREATE EDUCATIONAL
EXPERIENCES
Emerging Ideas Series: http://crlionline.net/emerging-ideas
CENTRE FOR RESEARCH ON LEARNING AND INNOVATION
SYDNEY SCHOOL OF EDUCATION & SOCIAL WORK
CRLI
What people thought
From exploring the solar system to practising surgical procedures; virtual
reality presents an opportunity for engaging with spaces and
experiences that would otherwise be impossible or impractical.
This potential, however, is far from being fully realised. How do we get it
there? And does this supposed potential really live up to all the hype? And
are there risks? For example, some research suggests that immersive VR is
not suitable for children under 13 years.
We created an online discussion and also surveyed people on their opinions
about how virtual reality (VR) technologies can be used in education.
Comments were both positive and negative, with some providing specific
recommendations for how to think about and approach VR and education.
“To make educational VR work, the VR environment and activities have to be well‐designed
(can justify there is a need to use VR) otherwise it'd just be a gimmick.”
“VR has even more possibilities thanks to be able to immerse students into environments
that are not usually possible and the factor of collaboration in a remote manner where
people can interact with each other in a virtual room without having to be sitting physically
in the same room.”
“As with all innovation VR will be held back by the slow crawl of education bureaucracy.
However it has huge value in immersing children in diverse contexts.”
“VR is a pointless technologists dream. It is a tool and has no pedagogical significance.
Higher resolution models are not superior to lower resolution models for learning.”
If you haven't already filled out the survey, we encourage you to do so here ‐
http://crlionline.net/node/381
Virtual reality (VR) is being heralded as a game‐changing technology in many
industries, with an estimated market impact of ~20billion USD by 2020. But how
might it impact education?
Overall, the results show an optimism for how VR might support, extend,
and enable new forms of learning, and how these technologies may
become more ubiquitous in the next ten years.

8. CHANGING WHAT AND HOW WE LEARN:
THE FUTURE OF AI AND EDUCATION
Emerging Ideas Series: http://crlionline.net/emerging-ideas
CENTRE FOR RESEARCH ON LEARNING AND INNOVATION
SYDNEY SCHOOL OF EDUCATION & SOCIAL WORK
CRLI
CHANGING WHAT WE LEARN
In the time that you can solve a simple math problem (solve for x: 2x + 4 = 10), a
computer can solve billions. So, are computing machines just that much smarter
than you? Thankfully not. For the time being, AI conforms to something known as
Moravec’s paradox: what humans find hard, computers find easy, and vice versa. In
more concrete terms: robots can crunch billions of numbers, but will struggle to
pick up a mug of coffee.
So while barista work is for the time being safe, AI has a deeper weakness. While
the predominant paradigm of modern AI, Deep Learning, has gotten good at
classifying things, the extent to which it 'understands' what it learns is far from
deep. In fact, Shallow Learning may be apter ('deep' refers to a technical aspect of
this technology, rather than being conceptually 'deep'). As AI pioneer Judea Pearl
puts it: current AI may be able to classify, but it can’t understand why.
So in imagining a future where humans and intelligent machines coexist, we ought
to think about how we can cooperate; instead of replacing our mortal mind AI
pioneer Terrance Sejnowski refers to this usage of AI as 'cognitive appliances' in his
new book ‘The Deep Learning Revolution’.
So what is the future of work in a workplace infused with intelligent machines? And
what is the role of education in preparing us for such a world?
AI and automation are predicted to result in up to 800 million jobs disappearing by
2030‐‐at the same time, new jobs will be created on a massive scale. New technologies
have always been disruptive, but the predicted impact of AI is unprecedented. Google
CEO Sundar Pichai has gone as far as to say that AI “is more profound than … electricity
or fire.” From Siri to medical diagnosis tools , AI is already in the building and it's here
to stay.
And rebelling against the then predominant behaviourist psychology of time,
Pressey stressed the importance of engaging with misconceptions and using
incorrectly answered questions as opportunities for "cognitive clarification",
rather than the behavourist approach of "rote reinforcings of bit of learnings".
In the context of AI and education, Pressey's great insight was that the problem
of instruction could be decomposed into tasks suited to 1) for scalable machine
intelligence, and 2) flexible human intelligence. And although Pressey's approach
to teaching wasn't perfect, this general heuristic is still useful today.
In more modern times, the idea of Intelligent Tutoring Systems became popular
in the 1980s, making use of the incrementally growing powers of what we now
call 'old‐school AI'. This movement sought to further decouple learning from the
classroom, and increasingly automate instruction. The Intelligent Tutoring
Systems movement still has considerable influence today, being the intellectual
foundation of educational software such as the popular language‐learning
platform Duolingo.
What is the next frontier of machine‐augmented education? Today, as the
powers of AI continue to grow, it is likely we will be able to find more and more
aspects of learning that can be automated by machines. For example, Natural
Language Processing (the ability of machines to understand day‐to‐day language)
has seen rapid improvement in recent years. This may one‐day allow AI
augmented assessment in schools and universities. Further, AI systems could
one‐day help with the generation of learning materials by dynamically producing
questions tailored to a specific learner. More generally, schools and universities
are often bogged down bureaucratically. Could machines help by automated
aspects of this too, freeing up the time of academics and teachers? And yes,
there may be robots in the classroom soon to help students with special needs.
But we must also be wary of ethical issues in the use of AI. As AI applications
become increasingly ubiquitous, we are likely to become increasingly unaware of
their presence and the power of influence they might exert, or the unchecked
weaknesses that may bring. And perhaps more worryingly, one of the challenges
in modern 'neural network' style AI architectures is the lack of transparency in
how they operate, even to their creators. Related to this, such neural
network systems can fall victim to any number of ethically questionable biases
without their creators intending for this to happen. There is some progress in
addressing these issues in AI design, but there is still a way to go.
Join the discussion:
http://crlionline.net/node/393
Complete the survey:
http://crlionline.net/node/476
CHANGING HOW WE LEARN
So machines may influence what we learn, but can they change how we learn? In the
1920s, Sidney Pressey, an early cognitive psychologist, built one of the first ever
examples of a 'teaching machine': a machine that implemented multiple‐choice
questioning. Pressey offered the following justification of how such a machine
might augment standard teaching:
"The procedures in mastery of drill and informational material were in many instances
simple and definite enough to permit handling of much routine teaching by mechanical
means. The average teacher is woefully burdened by such routine of drill and
information‐fixing. It would seem highly desirable to lift from her shoulders as much as
possible of this burden and make her freer for those inspirational and thought‐
stimulating activities which are, presumably, the real function of the teacher"

9. CENTRE FOR RESEARCH ON COMPUTER SUPPORTED LEARNING AND
COGNITION
FACULTY OF EDUCATION AND SOCIAL WORK
CoCo
1. Example diagram
1. Example diagram1. Example diagram
Research Aims
Over recent years, teams have emerged as a crucial vehicle for doing
various projects. Teamwork offers both organisations and individuals
the ability to become more familiar with each other, learn new skills and
draw on other team members’ talents, experiences and perspectives.
Learning collaborative teamwork and understanding the skills involved
in the team-working environment are important factors in the
obtainment of a productive team activity.
The research investigates one of the biggest challenges in team
collaboration for group projects and provides a collaborative working
model to increase individual performance when participating in group
discussions.
Theoretical Framework
Harkins (1987) proposes that students’ motivation towards
group work depend on the potential for evaluation. In his
explanation, social loafing is considered to cause loss of
motivation in groups. As he said: “opportunity for comparison
may have led participants to believe that their outputs could be
evaluated, and it was this potential for evaluation, not only
identifiability, that motivated performance”.
Johnson and Johnson (1989) argue that social interdependence
emerges when team members’ behaviours or actions can
influence other individual’s team members. Social
interdependence has two different types, namely, positive
(cooperation) and negative (competition).
Research Design
• Case study
• Aimed at understanding the mechanism
of the peer facilitation process
• Conducted with postgraduate students
at The University of Sydney
• Qualitative and quantitative data were
collected from one group of students
References
Harkins, S. (1987). Social loafing and social facilitation. Journal of Experimental Social
Psychology, 23, 1-18.
Johnson, D. W., & Johnson, R. (1989). Cooperation and competition: Theory and research.
Edina, MN: Interaction Book Co.
Motivating team players to work closer and
harder by using a tracking model in
facilitation practicesJason Leung1,
1 University of Sydney
Abstract: Given the growing demands for collaborative teamwork, it has been suggested that facilitation is a vital skill in both the workplace and classroom in the
21st century. Research has found that strong facilitation skills would be critical for group decision-making and problem-solving. This project aims to explore how to
motivate team members to work harder and closer by using a tracking model, which is a tool in managing future projects in both the classroom and workplace
environment.
Implications
The current team-working mode as well as
its assessment may need to be
redeveloped in order to satisfy current
needs.
Facilitation is helpful for teamwork, but it
would be better to include tracking to
ensure an equal-shared contribution by
team members.Results
• Social loafing is severe and prevalent in
teamwork.
• Teamwork is to obtain knowledge and to
develop communication, collaboration and
leadership skills.
• Facilitation should emphasize how to
motivate team members to work.
• Facilitation and teamwork are losing their
value in helping students
CENTRE FOR RESEARCH ON LEARNING AND INNOVATION
CRLI
SCHOOL OF EDUCATION AND SOCIAL WORK
Supervisor: Prof. Peter Reimann
Auxiliary supervisor: A/Prof. Lina Markauskaite
Discussion
Lecturers are unable to attend after-class
discussions and to monitor the teamwork
process. On the other hand, a facilitator
usually fails to motivate team members to
work unless he/ she has a ‘motivator’ to
achieve it. Therefore, a way to monitor
students’ teamwork process and to secure
the quality of teamwork is needed.

11. REFERENCES
EEG/ERP RESULTS
INTRODUCTION
METHODS
BEHAVIOURAL RESULTS
DISCUSSION
NEUROSCIENCE AND EDUCATION SIG
CENTRE FOR RESEARCH ON LEARNING AND INNOVATION
SYDNEY SCHOOL OF EDUCATION & SOCIAL WORK
CRLI
Enhancing young children’s empathetic learning through a tablet game
Combining microgenetic method with EEG
Ling Wu1 & Minkang Kim 1
1 Neuroscience and Education SIG, CRLI, The University of Sydney
Participants:
Typically developing preschool-age children (n=26) between the ages of 43
and 62 months (M = 51.51 months, SD = 5.51; n=8 girls) from one
preschool located in Sydney south participated in the study. All
participating children attended preschool 3 days per week in two separate
classrooms. One classroom served as an intervention classroom (n =12, 3
girls); the other classroom served as a control group (n=14, 5 girls).
Children’s parents (n=26) and teachers (n=4) also participated in the study.
Electroencephalogram (EEG) Measure:
All participating children completed the Chicago Moral Sensitivity Task
(CMST) that investigates changes in perceptual sensitivities to actions that
might lead to positive or negative emotions at a neurodynamic level.
Pre- and Post-Test Questionnaire Measures:
Parents for all participating children filled out two questionnaires – the 23-
item Griffith Empathy Measure (GEM, Dadds et al., 2008) and the 28-item
Interpersonal Reactivity Index (IRI, Davis, 1983).
Teachers in both groups completed the Empathy Questionnaire (EmQue,
English Version for teacher with 20 items)
Game Play Experience:
The 12 experimental group children interacted with the game for
approximately 25 minutes each week over a 10-week period. The control
group (n=14) children, not using the game, were otherwise exposed to
typical Australian early childhood curriculum.
ERP Results:
Consistent with previous studies, the findings of our study seem to show changed characteristics of P2 component where greater amplitude
modulation was observed in the difference waveforms of experimental group children, reflecting an increased sensitivity to harming
situations. This finding suggests that these pre-schoolers’ ability to perceive social cues may have been sensitised and modulated by the
game play experience, during which the children’s ability to recognise distressful cues displayed in social actions and facial expressions was
heightened and intensively practised at this developmentally critical period.
Findings on component EPN (on F4), LPP (on F4) and LLPP (on C3) also show a consistent pattern where the amplitude of difference
waveforms show a decrease in the experimental group children while an increase is observed in control group children.
Questionnaire and ERP Results Correlation:
Changes in teacher’s ratings on Attention to Others’ Feelings before and after the intervention period had a strong negative correlation with
changes in P2diff at Cz (r=.814, p<.01). This negative relationship was greater in the experiment group (β=-.76, R2=.484) compared to the
control group (β=-.3, R2=.334).
Main Design Mechanism:
The game invited children to engage in three main learning mechanisms
that may enhance empathy learning:
1. attend to and perceive emotionally salient events in a story,
2. actively share the emotions of the characters identified, and
3. take others’ perspectives, reasoning why a given emotion arises within
the context.
Game Components and images:
All educational elements are in picture format, clickable and accompanied
with a real human voice recorded from a fully trained female teacher. If
children click and select an element, an ‘if-then’ follow up sub-scene will
emerge and give the player feedback.
Images:
1. Theme map with locations of new stories; 2. An example of perception
scene in first theme. The ‘glow effect’ will appear when a child clicks on
the element and the voice will be played simultaneously; 3. Emotion
component demonstrating a basic emotion (sadness), the cartoon faces
are options for children; 4. Perception component showing increased
social complexity in School Theme; 5. Emotion component demonstrating
a complex emotion (loneliness), with the optional emotions being real
human faces 6. Reasoning component with reasons of why the character
felt lonely displayed for children to listen to and select.
Behaviour results from the EmQue (Rieffe, Ketallar & Wiefferink, 2010)
completed by teachers showed significant Time x Group effects (F(1,
24)=22.893; p<.001, partial η2 =.49) in the ‘Attention to Other’s Feelings’
subscale compared with control group, a time significance was not
observed. Increase in prosocial behaviour was observed by the teachers
in both groups (shown in ‘Prosocial Actions’ subscale) with a Time main
effect (F(1, 24)=9.620; p<.01, partial η2 =.286), but no significant Time x
Group effect was observed. As for subscale ‘Emotion Contagion’, neither
Time nor Time x Group effect was observed from the data.
Data Analysis:
EEG data were analysed in Brain Vision
Analyzer (Brain Products, Germany),
following effective procedures validated
through previous research (Cowell &
Decety, 2015) before statistical tests using
ANOVA. Game play data were analysed
quantitatively (log data) and qualitatively
(conversations), and questionnaire data
were analysed mainly through ANOVA and
correlated with EEG.
Cowell, J., & Decety, J. (2015). The neuroscience of implicit moral evaluation and its relation to generosity in early childhood. Current Biology, 25(1), 93-97.
doi:10.1016/j.cub.2014.11.002
Cowell, J. M., & Decety, J. (2015). Precursors to morality in development as a complex interplay between neural, socioenvironmental, and behavioral facets. Proceedings of the National
Academy of Sciences of the United States of America, 112(41), 12657-12662. doi:10.1073/pnas.1508832112
Dadds, M. R., Hunter, K., Hawes, D. J., Frost, A. D. J., Vassallo, S., Bunn, P., Masry, Y. E. (2008). A measure of cognitive and affective empathy in children using parent ratings. Child
Psychiatry and Human Development, 39(2), 111-122.
Davis, M. H. (1983). Measuring individual differences in empathy: Evidence for a multidimensional approach. Journal of personality and social psychology, 44(1), 113.
Eisenberg, N., Guthrie, I. K., Murphy, B. C., Shepard, S. A., Cumberland, A., & Carlo, G. (1999). Consistency and development of prosocial dispositions: A longitudinal study. Child
development, 70(6), 1360-1372.
Granic, I., Lobel, A. M., & Engels, R. C. M. E. (2014). The benefits of playing video games. American Psychologist, 69(1), 66-78. doi:10.1037/a0034857
Preston, S. D., & de Waal, Frans B. M. (2002). Empathy: Its ultimate and proximate bases. Behavioral and Brain Sciences, 25(1), 1-20. doi:10.1017/S0140525X02000018
Rieffe, C., Ketelaar, L., & Wiefferink, C. H. (2010). Assessing empathy in young children: Construction and validation of an empathy questionnaire (EmQue). Personality and Individual
Differences, 49(5), 362-367. doi:10.1016/j.paid.2010.03.046
Research on child development reveals that empathy starts to develop as
early as child birth and that some aspects such as perception, perspective
taking and cognitive empathy require resources and effortful learning early
in ontogeny (Cowell & Decety, 2015a; Eisenberg et al., 1999; Preston & de
Waal, 2002). While a rapidly growing body of research confirms
observable neurophysiological change in developmental time in children
across different age groups, it is less clear how specific learning can
influence and support this process. What is becoming clear is that tablet
technology, when appropriately implemented, can contribute to the
learning and emergence of change in empathy related skills in older
children (Granic et al., 2014). How might neuroscientific findings,
combined with the usefulness of mobile technology, be translated into the
early years education to enhance young children’s empathy is yet to be
fully discovered. The above mentioned ideas contributed to the foundation
of this PhD study. Based on extended literature review, a set of crucial
design principles were identified, on which a tablet game was designed
and developed. The study implemented the game as part of the Early
Childhood Education curriculum in one Australian preschool, aiming at
evaluating its developmental impact by combining a microgentic method
with pre- and post EEG, while gathering behavioural observation data on
children from their parents and teachers.
THE EMPATHY GAME Figure 2. P2 Difference Waveforms (top) and Range-Scaled Voltage Spline Map of the scalp distribution of the P200diff (right) at two
time-windows
Post
Pre
Control Experimental
Above are difference waveforms of harming and helping conditions (grand averaged
ERP waveforms of harming conditions were subtracted from helping conditions) at
Fz, Cz and Pz from both experimental and control group children, with negative
values plotted up. On the right are mapping views of pre-test control group children
(top-left) and experimental group children (top-right) and post-test difference
waveforms on children from the control (bottom-left) and experimental group
(bottom-right).
Components of Interest and Significant Interactions:
EPN (100 - 175 ms): The Time x Group interaction was significant only at F4 (F(1,
18)=5.381; p<.05, partial η2 =.23).
P200 (150 - 350 ms) : Time x Group significance was found on all central electrodes
(Fz (F(1, 18) =5.089; p<.05, partial η2 =.22); Cz (F(1, 18) = 13.795; p<.01, partial η2
=.43); and Pz (F(1, 18)=11.989; p<.01, partial η2 =.40)
N200 (200 - 400 ms): Time x Group interaction significance at F4 F4 (F(1,
18)=5.381; p<.05, partial η2 =.23
LPP (400 - 600 ms): The Time x Group interaction was significant at F4 (F(1,
18)=4.532; p<.05, partial η2 =.20).
Late LPP (600 – 800 ms) and Slow Wave (800 - 1,000 ms): No significant effects
found.

12. CENTRE FOR RESEARCH ON COMPUTER SUPPORTED LEARNING AND COGNITION
FACULTY OF EDUCATION AND SOCIAL WORKCoCo
Example 1
Previous Studies
• Explicit instructions to point and trace with the index finger
enhance learning (Aghostino et al., 2015; Ginns et al., 2015;
Hu et al., 2015; Macken & Ginns, 2014).
• 44 Year 5 & 6 students from NSW public primary school
studied the learning booklet and integrated poster for 16 mins
and 4 mins, respectively (Diagrams 1 & 2)
• Tracing group (n=22) outperformed non-tracing group (n=22)
in: recall and transfer tests.
• Tracing effect improves learning performance.
Recall test (t(42) = 2.45, p=.019, d=.74)
Transfer test (U=105, p=.001, d=1.11)
Current Experiment
“How do cognitive processes and gestures align to
support learning?”
• Hypothesis: When instructed to gesture, at least two cognitive
processing will be activated: attention regulation(c.f. de Koning
et al., 2009) and information packaging (Alibali et al., 2000).
• 9 Year 5 & 6 students were randomly assigned into tracing and
non-tracing conditions.
• The same material as Experiment 1.
• Participants were asked to verbalise their thoughts, i.e. Think
Aloud (Ericsson & Simon, 1993), and video recorded.
Induced Gesturing
Behaviours
• Pretest: no difference was found in
frequency of gestures made and time
spent on gesturing.– gesturing was a
naturally occurring behaviour.
• Learning phase: tracing group gestured
significantly more frequently and spent
more time gesturing than non-tracing
group, on top of instructed gestures.–
induced more gesturing behaviours in
learning phase.
• Test Phase: more frequent gesturing and
longer gesturing time were observed in
tracing group than non-tracing group. –the
effect of tracing instruction was carried
over into test phase.
Gestures have
meanings
• Deictic gestures were found to replace
words and/or phrases, reducing mental
effort.
• Iconic and metaphoric gestures were
found to represent items/location and
movements of water in the water cycle.
• In Example 1, gesturally replaced terms
were placed back into written text to
demonstrate the mental burden without
the use of gesturing. – reduced mental
burden when gestures were used.
Benefits of Gesturing
• Reduced mental burden, freeing up
working memory space for better schema
building.
• Inducing further gesturing during learning
phase. Increased gesturing allows more
free space for information processing.
References
Agostino, S., Tindall-Ford, S., Ginns, P., Howard, S., Leahy, W., & Paas, F. (2015). DOI: 10.1007/s10648-015-9315-5
Alibali, M. W., Kita, S., & Young, A. J. (2000). DOI:10.1080/016909600750040571
De Koning, B. B., Tabbers, H. K., Rikers, R. M., & Paas, F. (2009). DOI: 10.1007/s10648-009-9098-7
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis. Cambridge, MA: MIT press.
Ginns, P., Hu, F.-T., Byrne, E., & Bobis, J. (2015). DOI: 10.1002/acp.3171
Hu, F. T., Ginns, P., & Bobis, J. (2015). DOI: 10.1016/j.learninstruc.2014.10.002
Macken, L., & Ginns, P. (2014). DOI: 10.3109/0142159X.2014.899684
Tracing while learning:
a Think Aloud protocol study.
Michael Tang, Paul Ginns, and Michael Jacobson
University of Sydney [ mtan7870@uni.sydney.edu.au, paul.ginns@sydney.edu.au]
Abstract: Previous experiment of the study has demonstrated the effectiveness of tracing over reading on tests for knowledge retention and
transfer knowledge. At present, there is a lack of understanding about the cognitive processes underpinning the tracing effect. The current
experiment aimed to elucidate these underlying cognitive processes activated by tracing via verbal protocol analysis. Focusing on the learning
phase of the experiment, video analysis have revealed features of gesturing, including unspoken information and induced behaviours, which may
have contributed to better learning performance.
Diagram 1
Diagram 2
Table 1
Exp 2: Frequency and number of times gestures were made in pretest,
learning, and posttest phases.
Tracing Group Non-Tracing Group
(n = 6) (n = 3)
M SD M SD
Pretest
Freq. (#) 12.5 8.41 7.33 10.21
Time (sec) 42.65 26.15 35.8 53.17
Learning Phase
Instructed
Freq. (#) 18.83 8.26
Time (sec) 126.47 65.11
Uninstructed
Freq. (#) 88.83 45.90 4.33 4.04
Time (sec) 500.52 296.48 7.83 7.14
Test Phase
Freq. (#) 20.6 6.53 9.67 9.61
Time (sec) 87.9 46.97 27.4 30.16
Table 2
Future Studies
• More than pointing and tracing – beat
(tapping), commonly observed as induced
gesturing in this experiment.
• Larger sample size.
• Individual differences– e.g. spatial ability.
Participant’s speech:
“…these [p] suck up the
water [p]… pulls them
up here [p]… and when
it [p] gets too heavy.
These [p] are sucked [p]
too much, this [p] is
taking too much, it
drops [p] back down,
and it [p] restarts.”
Participant’s speech
‘translated’ :
“…The trees[p] suck up
the water [p]… pulls the
water up to the air [p]…
and when the water [p]
gets too heavy. The
plants [p] are sucked [p]
too much, the cloud [p]
is taking too much, the
water drops [p] back
down, and the water
cycle [p] restarts.”
CENTRE FOR RESEARCH ON LEARNING AND INNOVATION
School of Education and Social WorkCRLI
Exp1: Means (M), Standard Deviations (SD), and Anderson-Darling
Test p-value (A-D) for Water Cycle Pre-test; Self-reported Cognitive
Load ratings; and Posttest Scores.
Non-Tracing Group Tracing Group
(n = 18) (n = 20)
M SD A-D M SD A-d
Prior Knowledge (/20) 10 2.31 0.178 11 2.11 0.506
Test Phase
Recall (/20) 6.91 1.99 0.498 10.09 1.83 0.778
Transfer (/25) 2.32 2.19 0.004 5.23 3.25 0.155

13. Co-constructing Epistemic Environments
A Sociomaterial Inquiry into Complex Problem Solving
in Higher Education
Natasha Arthars
University of Sydney [Natasha.Arthars@Sydney.edu.au]
CENTRE FOR RESEARCH ON LEARNING AND INNOVATION
SYDNEY SCHOOL OF EDUCATION & SOCIAL WORK
CRLI
Supervisor: Lina Markauskaite
BACKGROUND
Faced with an increasingly complex world of work, situated within what many refer to as the ‘knowledge age’, learners need to enter the workforce equipped to
collaboratively solve problems and create new knowledge. The complexity of contemporary problems has led to claims that we have in fact moved beyond the
‘knowledge age’ to the ‘conceptual age’ in which both creative and complex problem solving skills are of prime importance [1]. In order to solve these complex
problems, learners require the ability and the agency to co-construct their epistemic environment [2].
RESEARCH QUESTIONS
1. What elements of the epistemic environment are provided to learners to
assist them to solve complex problems?
2. How do groups choose which (if any) affordances of the provided
environment to utilize?
3. In what ways do groups reconfigure and build upon the epistemic
environment over time and why?
RESEARCH DESIGN
Design Framework: Mini-ethnographic case studies
Research Site: University of Sydney
Setting: Units of study containing collaborative, complex
problem solving as part of curriculum
Sample: Six groups (1-3 per unit of study)
Groups will be offered use of the Design Studio (Figure 1) for group meetings.
The Design Studio has two writable whiteboards, a smartboard and three
projectors.
DATA COLLECTION & ANALYSIS
AIM
This research project will examine cases in higher education where learners
work collaboratively in groups to co-construct their epistemic environment while
engaging in complex problem solving tasks.
EPISTEMIC ENVIRONMENT
The epistemic environment is a complex and dynamic assemblage of
material, social and conceptual arrangements that collectively afford
epistemic activity.
The epistemic environment offers affordances and constraints that interact to
support (or constrain) complex problem solving tasks. Research on education
and professional learning has only recently begun to discuss and consider
epistemic environments. There remains a gap in our understanding of how
these environments are constructed, particularly in the context of university
students tasked with collaboratively solving complex, real world problems.
Research Question 1 2 3
Data collection
methods
• Teacher
interviews
• Observation
• Group interviews
• Observation
Data collection
instruments/tools
• Audio recorded semi-structured face to face
interviews
• Video recorded
observations of group
meetings (during and/or
outside of classes)
• Photographs
• Electronic copies of
artefacts created
• Electronic
copies of
resources
provided
Data analysis • thematic analysis
REFERENCES
[1] Pink, D. H. (2005). A whole new mind: moving from
the information age to the conceptual age. Crows Nest,
N.S.W: Allen & Unwin.
[2] Markauskaite, L., & Goodyear, P. (2017). Epistemic
fluency and professional education: innovation,
knowledgeable action and actionable knowledge.
Dordrecht: Springer.
Figure 1: Design Studio
Table 1: Data collection and analysis
Figure 2: Collaborative group work

14.
How do business students employ conversation to create and utilise learning opportunities when negotiating consensus in an ethics case
study in a face‐to‐face learning environment?
Material Business case studies around ethical decision
making
Considering the reliance on the case study method for ethics education in business, there appears to be a distinct lack of studies that
qualitatively explore how students learn when engaging with this method. Frameworks for ways to teach with cases have been put
forward (Bridgman, 2011; McWilliams & Nahavandi, 2006; Sims, 2004; Sims & Felton Jr., 2006; Singer, 2013) and quantitative studies
have proven links between culture and students’ interaction with cases (Jonson, McGuire, & O’Neill, 2015) and the use of cases and
increased networked thinking (Pilz & Zenner, 2017). However, to our knowledge, only Thomson (2011) have analysed students written
online responses to gain an understanding of how an ethical decision-making model was implemented.
Participants 31 postgraduate Commerce students across six
groups
Except for the aspect of gender, the participants in the study were representative of the larger cohort of students enrolled in the unit
of the study. Furthermore, the participants also represented the countries where most of Australia’s overseas enrolments are currently
from (Nabi, 2017). 23 of the participants were Chinese, four were Australian and two were Indian. The remaining two participants
were from Vietnam and Bangladesh, respectively. Seven participants were native speakers of English or had first language proficiency
while the remaining 24 were non‐native speakers with varying levels of competency in English.
Data analysis Video recorded group sessions were analysed
following an analysis scheme similar to Wasson’s
(2016, p. 384) work combining Conversation
Analysis (CA) and Issue Framing (IF)
Step 1: CA transcriptions in ELAN (ten Have, 2007)
Step 2: Identification of Issue‐framing and Decision‐making Speech Acts (Wasson, 2016)
Step 3: Coding of speech acts
Step 4: Identification of positions adopted by group members (Barnes, 2004)
see laptop
for findings & examples
Barnes, M. (2004). The use of positioning theory in studying student participation in collaborative learning activities. Paper presented at the Australian Association for Research in Education, Melbourne, Australia. http://www.aare.edu.au/publications‐
database.php/4082/The‐use‐of‐Positioning‐Theory‐in‐studying‐student‐participation‐in‐collaborative‐learning‐activities
Bridgman, T. (2011). Beyond the manager’s moral dilemma: Rethingking the ‘ideal‐type’ business ethics case. Journal of Business Ethics, 94, 311‐322. doi:10.1007/s10551‐011‐0759‐3
Jonson, E. P., McGuire, L. M., & O’Neill, D. (2015). Teaching ethics to undergraduate business students in Australia: Comparison of integrated and stand‐alone approaches. Journal of Business Ethics, 132, 477‐491. doi:10.1007/s10551‐014‐2330‐5
McWilliams, V., & Nahavandi, A. (2006). Using live cases to teach ethics. Journal of Business Ethics, 67(4), 421‐433. doi:10.1007/sl0551‐006‐9035‐3
Nabi, Z. (2017). Most international students come to Australia from these countries. SBS, 2018. Retrieved from SBS website: https://www.sbs.com.au/yourlanguage/urdu/en/article/2017/08/29/most‐international‐students‐come‐australia‐these‐countries
Pilz, M., & Zenner, L. (2017). Using case studies in business education to promote networked thinking: findings of an intervention study. Teaching in Higher Education, 1‐18.
Sims, R. R. (2004). Business ethics teaching: Using conversational learning to build an effective classroom learning environment. Journal of Business Ethics, 49(2), 201‐211.
Sims, R. R., & Felton Jr., E. L. (2006). Designing and delivering business ethics teaching and learning. Journal of Business Ethics, 63, 297‐312. doi:10.1007/s10551‐005‐3562‐1
Singer, A. E. (2013). Teaching ethics cases: a pragmatic approach. Business Ethics: A European Review, 22(1), 16‐31. doi:10.1111/beer.12004
ten Have, P. (2007). Doing Conversation Analysis (Second edition ed.). Cornwall: Sage.
Thomson, G. S. (2011). Good conversations: An enhanced model to teach business ethics. Journal of International Education Research, 7(1), 73‐80.
Sanri le Roux (Ph.D candidate)
Prof Peter Reimann (Supervisor)
Dr Kelly Freebody (Associate Supervisor)