The document describes a study that developed a framework for designing effective tasks in virtual worlds. Researchers had one group program a LEGO robot to follow a circuit and then teach another group how to do it by communicating through Second Life. Student interactions were video recorded and analyzed using Bloom's Taxonomy to code cognitive processes and knowledge dimensions. The analysis found that conceptual knowledge tasks engaged higher-order cognitive processes over time, and that task design, not just difficulty, impacted learning.

1. A methodology for determining relationships
between cognitive processes and the
knowledge dimension when implementing
tasks in virtual worlds.
Michael Vallance.
Future University
Hakodate, Japan.
&
Stewart Martin.
Teesside University,
UK.
1

2. purpose of research
• To develop a framework for designing
effective tasks in virtual worlds
• NOT directly replicate real world tasks
• BUT utilise the uniqueness of virtual
world tools and communication

4. tasks in Second Life

5. tasks in Second Life

6. robot to follow circuit instructions
tasks in Second Life

7. robot to follow circuit instructions
tasks in Second Life

8. robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution

9. robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution

10. robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution

11. robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution
test the program

12. robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution
test the program

13. robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution
further collaboration
test the program

14. robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution
further collaboration
test the program

15. robot to follow circuit instructions
tasks in Second Life
develop and communicate a solution
further collaboration end result
test the program

18. data collection
• Japan group (n=4) and UK group (n=4)
• 12 tasks: 1 per day over 3 weeks
• task given by researchers to one group:
program LEGO robot to follow a pre-set circuit
• group A then teach group B: communication
through Second Life using text and graphical
objects (prims) as desired
• video capture of both groups
• 60 hours to transcribe and analyse

19. Bloom’s Taxonomy
Instrument
Anderson, L.W., Krathwohl, D.R., Airasian, P.W., Cruicshank, K.A., Mayer, R.E., Pintrich, P.R., Raths, J. & Wi@rock,
M.C. (2001) A taxonomy for learning, teaching and assessing: A revision of Bloom’s taxonomy of
educa:onal objec:ves (Abridged ed.). New York: Longman.

20. TRANSANA for transcribing
and linking video clips

21. TAMS ANALYZER
for data analysis

22. * Bloom’s coded analysis * Share at Google Docs.
exported from TAMS to Excel.
* Care with activity number
(which group, which task)

23. Motion chart: pioneered by Hans Rosling of Gapminder

24. Motion chart: pioneered by Hans Rosling of Gapminder

25. Motion chart

26. Motion chart

27. Motion chart: our data

28. Motion chart: our data

29. Excel charts

30. Why dip in APPLY? Bloom’s definition: carry out a procedure in a
given situation.
Compare to ANALYZE. Bloom’s definition: determine how parts
relate to one another and to an overall purpose.
Decision early on: (1) only choose ONE cognitive process and ONE
knowledge dimension per communicative act. (2) Try to work within
definitions as objectively as humanly possible.
Observation: if looking at a particular communicative act in isolation
then it may be deemed APPLY but within the context of what came
before some of the communication was obviously more than ‘carrying
out a given procedure’
There was evidence in the communication and looking at the video
(in TRANSANA) of ANALYZE or other cognitive process categories.

31. initial observations
observation: as days went by for conceptual knowledge the amount
of analyzing, evaluating and creating increase.
observation: procedural knowledge completely unrelated to
remembering.
There are links in applying and evaluating though.
observation: our tasks are limited in communicating evidence about
metacognitive knowledge.
observation: we have proven that the development of knowledge does
not necessarily occur as task challenge increases.
observation: components of the cognitive process and knowledge
domain can be developed based upon the specifics of the task
rather than simply increasing task complexity.
observation: just making that same task harder does not necessarily
engage in further occurrences of same components of the cognitive
process and knowledge domain.
observation: witnessed a challenge of human-robot interface and the
challenge of communication, i.e. that human intent must be
expressed in robot control values; this is a source of interaction

32. questions
question: Is it simply about task design? Can we say task
difficulty increases the likelihood of engaging, say, conceptual
knowledge?
question: What about the data we used?
Our data was drawn from transcriptions.
question: Would it be better to set up buttons to push at regular
intervals for students to acknowledge what they are doing at
that moment. See presentation by Prof. Paul van Schaik at 3 PM on
Tuesday:- Measuring flow experience in an immersive virtual
environment for collaborative learning.
question: Another method?
This is our Design Experiment (the iterative development and
collection of data analysed via a mixed methods approach).
How could you better measure how students are Applying or
Analyzing, for instance?

33. our
resources
references
• Vallance, M., Martin, S., Wiz, C. & van
Schaik, P. (2010). Designing effective
• Transana (www.transana.com) spaces, tasks and metrics for
communication in Second Life within the
context of programming LEGO NXT
• TAMS Analyzer (http://tamsys.sourceforge.net/)
Mindstorms™ robots. International Journal
of Virtual and Personal Learning
• LEGO Mindstorms 9797 (http://www.active- Environments. Vol. 1 (1), pp. 20-37.
robots.com/products/mindstorms4schools/lego- January - March 2010. DOI: 10.4018/jvple.
education-nxt.shtml) 2010091703.
• Gapminder (http://www.gapminder.org)
• Vallance, M., Martin, S., Wiz, C. & van
Schaik, P. (2009). LEGO Mindstorms for
• Google Motion Graphs (http://www.google.com/ig/ informed metrics in virtual worlds. In
directory?url=www.google.com/ig/modules/ Proceedings of Human Computer
motionchart.xml) Interaction (HCI) 2009 - People and
Computers XXIII. Cambridge University,
UK. pp. 159-162.