A methodology for determining relationships
between cognitive processes and the
knowledge dimension when implementing
tasks in virtual worlds.
Michael Vallance.
Future University
Hakodate, Japan.
&
Stewart Martin.
Teesside University,
UK.
1

purpose of research
• To develop a framework for designing
effective tasks in virtual worlds
• NOT directly replicate real world tasks
• BUT utilise the uniqueness of virtual
world tools and communication

tasks in Second Life

tasks in Second Life

robot to follow circuit instructions
tasks in Second Life

robot to follow circuit instructions
tasks in Second Life

robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution

robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution

robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution

robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution
test the program

robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution
test the program

robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution
further collaboration
test the program

robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution
further collaboration
test the program

robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution
further collaboration end result
test the program

data collection
• Japan group (n=4) and UK group (n=4)
• 12 tasks: 1 per day over 3 weeks
• task given by researchers to one group:
program LEGO robot to follow a pre-set circuit
• group A then teach group B: communication
through Second Life using text and graphical
objects (prims) as desired
• video capture of both groups
• 60 hours to transcribe and analyse

Bloom’s Taxonomy
Instrument
Anderson, L.W., Krathwohl, D.R., Airasian, P.W., Cruicshank, K.A., Mayer, R.E., Pintrich, P.R., Raths, J. & Wi@rock,
M.C. (2001) A taxonomy for learning, teaching and assessing: A revision of Bloom’s taxonomy of
educa:onal objec:ves (Abridged ed.). New York: Longman.

TRANSANA for transcribing
and linking video clips

TAMS ANALYZER
for data analysis

* Bloom’s coded analysis * Share at Google Docs.
exported from TAMS to Excel.
* Care with activity number
(which group, which task)

Motion chart: pioneered by Hans Rosling of Gapminder

Motion chart: pioneered by Hans Rosling of Gapminder

Motion chart

Motion chart

Motion chart: our data

Motion chart: our data

Excel charts

Why dip in APPLY? Bloom’s definition: carry out a procedure in a
given situation.
Compare to ANALYZE. Bloom’s definition: determine how parts
relate to one another and to an overall purpose.
Decision early on: (1) only choose ONE cognitive process and ONE
knowledge dimension per communicative act. (2) Try to work within
definitions as objectively as humanly possible.
Observation: if looking at a particular communicative act in isolation
then it may be deemed APPLY but within the context of what came
before some of the communication was obviously more than ‘carrying
out a given procedure’
There was evidence in the communication and looking at the video
(in TRANSANA) of ANALYZE or other cognitive process categories.

initial observations
observation: as days went by for conceptual knowledge the amount
of analyzing, evaluating and creating increase.
observation: procedural knowledge completely unrelated to
remembering.
There are links in applying and evaluating though.
observation: our tasks are limited in communicating evidence about
metacognitive knowledge.
observation: we have proven that the development of knowledge does
not necessarily occur as task challenge increases.
observation: components of the cognitive process and knowledge
domain can be developed based upon the specifics of the task
rather than simply increasing task complexity.
observation: just making that same task harder does not necessarily
engage in further occurrences of same components of the cognitive
process and knowledge domain.
observation: witnessed a challenge of human-robot interface and the
challenge of communication, i.e. that human intent must be
expressed in robot control values; this is a source of interaction

questions
question: Is it simply about task design? Can we say task
difficulty increases the likelihood of engaging, say, conceptual
knowledge?
question: What about the data we used?
Our data was drawn from transcriptions.
question: Would it be better to set up buttons to push at regular
intervals for students to acknowledge what they are doing at
that moment. See presentation by Prof. Paul van Schaik at 3 PM on
Tuesday:- Measuring flow experience in an immersive virtual
environment for collaborative learning.
question: Another method?
This is our Design Experiment (the iterative development and
collection of data analysed via a mixed methods approach).
How could you better measure how students are Applying or
Analyzing, for instance?

our
resources
references
• Vallance, M., Martin, S., Wiz, C. & van
Schaik, P. (2010). Designing effective
• Transana (www.transana.com) spaces, tasks and metrics for
communication in Second Life within the
context of programming LEGO NXT
• TAMS Analyzer (http://tamsys.sourceforge.net/)
Mindstorms™ robots. International Journal
of Virtual and Personal Learning
• LEGO Mindstorms 9797 (http://www.active- Environments. Vol. 1 (1), pp. 20-37.
robots.com/products/mindstorms4schools/lego- January - March 2010. DOI: 10.4018/jvple.
education-nxt.shtml) 2010091703.
• Gapminder (http://www.gapminder.org)
• Vallance, M., Martin, S., Wiz, C. & van
Schaik, P. (2009). LEGO Mindstorms for
• Google Motion Graphs (http://www.google.com/ig/ informed metrics in virtual worlds. In
directory?url=www.google.com/ig/modules/ Proceedings of Human Computer
motionchart.xml) Interaction (HCI) 2009 - People and
Computers XXIII. Cambridge University,
UK. pp. 159-162.