The meme of the physical ‘uni-versity’ is changing and moving swiftly, due mostly to virtual technological developments, towards the ‘multi-versity’ where the Higher Education Institute will exist in both the real world and the virtual space. The concern for researchers though is the need to produce metrics that provide evidence of learning in these augmented futures of the virtual institute. This paper will summarise the theoretical and technical progress of two years of research in the development of metrics for evidencing the processes of learning (witnessed as measurements of six cognitive processes and four knowledge dimensions) of participants (N=8) programming robots within a virtual world. The paper will explain the research, its innovative usage of technologies, and how metrics for learning are being uniquely recorded, analysed and interpreted. These are the slides presented at eCASE&eTECH Tokyo in January 2011 and similarly at NIE, Nanyang Technological University, Singapore in March 2011. Further information may be found on the final slide. Feedback welcome :-) Michael Vallance Department of Media Architecture, Future University Hakodate. Stewart Martin School of Social Sciences and Law Teesside University,

1. Augmented education
in the futures university.
Dr.Michael Vallance.
A collaboration between
Future University Hakodate, Japan
& Teesside University, UK.
1

2. Hakodate
in Hokkaido,Japan
2

3. Augmentation-
education -futures
3

4. Cuban (1992): Informed ICT use necessitates a move from first order
change (replication of existing practices) to second order change
(unique pedagogical affordances offered by emerging technologies).
deFreitas (2008): metrics for evaluating virtual world learning
experiences essential.
4

5. REAL VIRTUAL
Virtual communication
Pranav Mistry, SixthSense
Virtual dinos in a real museum
b 5

6. OpenSim virtual space
6

7. Real world tasks
7

8. Tasks:program robots
8

9. to follow specific
circuits
9

10. Avatars (UK & JPN)
collaborate for a solution
10

11. Why robot programming?
• Provides closed, highly defined tasks.
• Level of difficulty can be quantified.
• Task difficulty = the minimum number of discrete maneuvers (action +
direction) required to successfully navigate a given maze (Barker and Ansorge,
2007).
• Tasks can be replicated (same level of difficulty but different maneuvers).
• Provoke behaviors and communicative exchanges which could be located on a
framework for analysis.
• Science university expectations and funding opportunities
11

12. 12

13. 13

14. 14

15. 15

16. LEGO robot 8527: same
configuration in UK & JPN
16

17. 17

18. Example from pilot study.
Students compare programs.
18

19. 19

20. demo movie
20

21. data: TRANSANA for transcribing and dynamic linking video to transcript
21

22. BLOOM’s revised taxonomy
The language found in many commonly used assessment structures and marking schemes
in Higher Education institutions reflects the revised hierarchy within Bloom's Taxonomy.
Whilst the terminology varies somewhat from one institution or programme to another, the
marking schemes and guidelines we have found exhibit at the very least a significant
congruence with the revised Bloom's sequence of 'remember', 'understand', 'apply',
'analyze', 'evaluate' and 'create'.
In HEI assessment structures there is an assumption that ordered structures of cognitive
descriptors for assessment in such hierarchies map the sequence of students’ cognitive
development.
Bloom’s also offers a visualization between cognitive process and knowledge domains.
This may make virtual worlds and tasks more accessible to educators.
It may not provide a framework of learning but for learning.
22

23. data: TAMS ANALYZER for coding transcripts using Bloom’s revised taxonomy
23

24. coded transcripts then re-imported back to TRANSANA for analysis
24

25. data: cooperate across globe and input data to a GOOGLE Doc (spreadsheet) and
Export to Excel
25

26. Data analysis
Actual data of number of times each cognitive process was tagged
per knowledge dimension in each task
here, series =
task number
!
actual data 26

27. data - not real! - hypothetical graph
Example: PROCEDURAL KNOWLEDGE
Number of occurrences per task converted as a percentage of the total
$!!"#
,!"#
+!"#
*!"#
)!"#
897D#$#
(!"#
'!"# 897D#%#
&!"# 897D#&#
%!"#
897D#'#
$!"#
!"# 897D#(#
4#
#
4#
4#
#
4#
34
34
23
23
B3
123
<=2
B
36
<=>
A9
.9
0.
;
89
9
:;
C1
9<
./
:3
17
?@
6.
/
-.
53
27

28. actual data: procedural
Number of occurrences per task converted as a percentage of the total
!
$!!"#
,!"#
+!"#
*!"#
)!"#
897D#$#
(!"#
'!"# 897D#%#
&!"# 897D#&#
%!"#
897D#'#
$!"#
!"# 897D#(#
4#
4#
#
4#
4#
#
34
34
123
23
23
B3
<=2
B
36
<=>
A9
.9
0.
;
89
9
:;
C1
9<
./
:3
17
?@
6.
/
-.
53
28

29. actual data: conceptual
Number of occurrences per task converted as a percentage of the total
!
$!!"#
,!"#
+!"#
*!"#
)!"#
897D#$#
(!"#
'!"# 897D#%#
&!"# 897D#&#
%!"#
897D#'#
$!"#
!"# 897D#(#
4#
4#
#
4#
4#
#
34
34
123
23
23
B3
<=2
B
36
<=>
A9
.9
0.
;
89
9
:;
C1
9<
./
:3
17
?@
6.
/
-.
53
29

30. GOOGLE MOTION GRAPH- not real! - hypothetical
30

31. GOOGLE MOTION GRAPH- actual data - procedural
31

32. Observation 1: increase in task complexity, the amount of analyzing,
evaluating and creating increased.
Observation 2: procedural knowledge less related to remembering as
expected. More applying and evaluating though.
Observation 3: we have proven that the development of knowledge does
not necessarily occur as task challenge increases. Learning is not linear
as might be asserted by university metrics for under-graduate and
post-graduate education.
Observation 4: components of the cognitive process and knowledge
domain need to be developed based upon the specifics of the task
rather than simply increasing task complexity.
Observation 5: just making the same task harder does not necessarily
engage in more occurrences of same components of the cognitive
process and knowledge domain.
32

33. Next ...
33

34. 34

35. data: from Bloom’s iPad to csv server then export to Excel
35

36. bypass
Mindstorms s/w
to connect
LEGO robot
directly
Virtual telemetry kit
by Reaction Grid
36

37. Virtual spaces and real world tasks for
augmented futures in Higher Education.
Preparing effective tasks and
assessment metrics.
Please join us: http://www.iverg.com
37

38. http://tinyurl.com/6ynexc8
Acknowledgements: I wish to acknowledge the contributions of fellow researchers: Takushi Homma of Future
University Hakodate, Japan, Stewart Martin and Paul van Schaik of Teesside University UK, and Charles Wiz of
Yokohama National University, Japan.
The research is supported by the Japan Advanced Institute of Science and Technology kakenhi grant 00423781 and the
UK Prime Minister’s Initiative (Science Direct).
Also, many thanks to the participating students at Future University Hakodate, Yokohama National University, and
Teesside University.
Please join us:http://www.iverg.com
38

39. Issues that keep arising when UK & JPN researchers meet.
Can you advise?
Question 1: How do robot researchers/academics determine degrees of
complexity in robots?
Question 2: We need quantitative evidence of learning specifically
applied to the tasks in our virtual world. We use Bloom’s for the reasons
stated. What other taxonomy can we use in the process of conducting
tasks which would facilitate quantitative evidence?
Paul van Schaik is looking at Flow (Csikszentmihalyi, 1990):
flow dimensions being independent predictors of learning task
performance.
http://tinyurl.com/6ynexc8
http://www.iverg.com
39