This document discusses the use of virtual and collaborative virtual environments for education, with a focus on students with special needs. It describes several projects led by Sue Cobb at the University of Nottingham to develop VEs and CVEs using participatory design methods. Evaluation of the projects found that students were engaged with the technologies and they showed potential for supporting collaboration, communication skills, and perspective taking. However, more work is needed to improve realism and robustness for use in classroom settings.

1. Tell me what you want and I’ll show you what you can have
Who drives design of technology for learning?
Sue Cobb
Human Factors Research Group, University of Nottingham, UK

2. Structure of presentation
•Introduction
•My background and research
•Description of VE/CVE technology
•Why we wanted to apply this technology for student learning
•Example projects and Design/development approach: UCD/PD
•Outcomes and Lessons learned

3. Sue Cobb
•1992: Virtual Reality Applications Research Team (VIRART), Human Factors Research Group, University of Nottingham
•Head of HFRG, Faculty of Engineering
•Background - Psychology and Human Factors
•Research interest – Applications of VR technology in education
–Specific interest from special educational needs community in how VR technology could bring the world of learning into the classroom
–Need to work with technology that could be placed in schools (desktop VR systems)
•Research approach = Methods for user-centred design and evaluation of VR applications
•www.hfrg.nottingham.ac.uk

4. Virtual and mixed reality technologies
Virtual
Real

6. User-Centred Design (UCD)
•Design centred around the user
•Understanding the needs of the user in order to inform design
–Involve users as much as possible (recognise limitations)
–Integrate knowledge and expertise from different disciplines
–Iterative testing to check design meets user requirements

7. Participatory Design
•Originated in Scandinavia 1970‟s
•Established PD methods
–System prototypes
–User evaluation
–Mutual learning between active participants
–All partners are equal: Users, managers, systems designers, human factors experts

8. Working with the User Group

9. How can VR support learning?
•Constructivist learning
–„direct experience‟
–Multi-sensory cues
–3D immersion
–Frames of reference
•Constructionist learning
–Students create or modify VE
–Collaborative learning
•Situated learning
–Interactive role play in CVEs
Moshell and Hughes (2002)
•Computers are fun!
•Motivational
•Increased learning opportunities
•Self-directed, active learning
•Self-paced
•Learning by doing
•Safe space for learning
•Scaffolded
Neale, Brown, Cobb and Wilson (1999) Cobb, Neale and Stewart (2001).

10. •The Virtual City Project
•Practice of procedural steps in everyday tasks
•3 weeks of training improved task familiarisation
•Student-led activity in real world Brown et al.(1998)
Life skills education

11. Main findings
•Students found it easier to use VEs than mentors believed they would
•Shift of control-action and decision-making shift from teacher initiated to student initiated
•Anecdotal reports of benefit to performance
•Student feedback on UCD method positive
•Usability issues raised

12. VEs for Social Skills Training
AS Interactive  3yr project: 2000-2003
 Funded by Shirley Foundation
 University of Nottingham
 Human Factors/VIRART
 Computer Science
 Psychology
 National Autistic Society
www.virart.nott.ac.uk/asi

13. Why Virtual Environments (VEs)?
•Ability to control input levels e.g. number of people present, auditory inputs, non-verbal cues
•Practice skills safely without experiencing real world problems (role play)
•Shared (visual) features between virtual and real worlds may facilitate generalisation of skills from former to latter
Parsons & Mitchell (2002)

14. Participatory design
•Idea generation
–Discussion and brainstorming
–Low tech prototyping
–Bodystorming
•Design modelling
–Mock-ups (“Wizard of Oz”)
•Prototype evaluation
–User testing
–Observation
–User notes and comments

15. Text is displayed here
VE content and viewpoints
Interface design

16. •What type of VEs?
•How do we replicate social situations?
•How do users understand and interpret these VEs?
•How can we encourage social interaction?
•How do we measure social skills?
•How can we support generalisation?
Research Questions

17. Single User Virtual Environments (SVEs)
A single user is guided through a social interaction task and invited to make choices about what to do and what to say in specific situations
The user may be supported by a teacher or training advisor seated next to them at the PC

18. SVE Training Scenarios
Social Café
Catching a bus

19. •Use, understanding and learning (to a limited degree).
•Some indication of successful learning and potential generalisation to other media
•Learning potential not confined to those with stronger verbal abilities (Mitchell et al. 2007)
•However, limitations
–Visually „blocky and unrealistic‟
–Limited spontaneous interpersonal communication
•CVEs offer more flexibility for but technology not sufficiently robust for use in schools (Cobb, 2007)
Main findings

20. http://cospatial.fbk.eu
Objective: to develop specific interactive technologies for school settings that may help to promote learning and social competence in children on the autism spectrum.

21. Project structure
SoA Review
CBT Framework
SAS Technology
CVE Technology
Evaluation
Social competence training
Technology for ASC
Social competence skills:
• cooperation/collaboration
• social conversation
Application of CBT techniques:
• learning
• experience
• Design Process
• Usability
• Barriers to implementation
• Effectiveness

22. Collaborative Virtual Environments (CVEs)
• Several users, real time communication
• Open-ended, more natural
• Less anxiety provoking
• Exploration
• Dynamic
• Perspective taking
• Facilitation
• Supportive environment for rehearsing appropriate behaviours

23. Design method
•Design team: research group + teachers
•Design workshops –
evaluation & discussion of
design concepts
•School visits
•Concept designs
•Prototype building
•Methods for user testing and evaluation
•User trials

24. Participatory design

25. CBT aspects of scenario
What is the intervention focused general social goal?
Cooperation / Collaboration
What are goal’s main social components?
Choosing
Mutual planning
Mutual performance
What are goal’s sub social components?
Compromise
Which CBT learning techniques are most suitable to be implemented?
Indicate which are best implemented via software
Problem solving
Indicate which are best implemented via a human mediator
Concept clarification (collaboration/cooperation)
Which CBT experience techniques are most suitable to be implemented?
Indicate which are best implemented via software
Behavioural rehearsal (working together)
Feedback reinforcement (players are rewarded for completing the task)
Indicate which are best implemented via a human mediator
Modelling practice
Homework (practice working together on a task in a real world activity)

26. Teacher Involvement
•3 schools; 3 teachers in core design team
•5 schools; 40 teachers in stage 2 reviews
•4 schools intervention study (22 pupils)
•Dissemination event: 120 delegates, 1000+ hits online
Agile
Design
Team
WP5
Wider school setting (new schools & PD with ASD students)
Block Party
Block Party
Block Party
Catch
the Sheep
Catch the Sheep
Catch the Sheep
Snack Time

27. Design Concepts

28. Adapted methods: involving children with ASC in design
Millen, Cobb, Patel and Glover (2012).
Millen, Cobb, and Patel (2011)..

29. CVE software
Talk2U: Two children work with a teacher who explains social conversation and helps them to practise key parts. Different topics can be chosen and talked about by pressing the button in the room.
Block Challenge: Two children work together to build a tower of blocks. Each has a different target, and they must talk to each other to select blocks and rotate them so that the side facing each player matches their own target.

30. Main findings
•Suitable for supporting collaboration and communication
•Encourages children to consider others‟ perspective and task-related needs
•Facilitation can be tailored to the individual needs of children
•Engaging for all children but too easy for most
•Overall, game does what we wanted it to do!
Follow-on project: Shape digital stories. See outcomes relating to COSPATIAL in videos „Collaboration by Stealth‟, „Outside the Box‟, and „Working Party‟. http://www.birmingham.ac.uk/research/activity/education/shape/digital-stories/index.aspx

31. Why can’t you have what you want?
Lessons learned: how to improve the participatory design approach

32. CVE development process
Concept evaluation & prototyping
Formative evaluation & re-design
Background technology review
Concept elaboration
User centred design process.
Iterative review of design stages
Identification of needs & requirements + CBT

33. Presentation of requirements to different stakeholders

34. Multi-disciplinary research in development and application of interactive technology
 Multi-disciplinary design/development teams
 Stakeholders--- define objectives/interpret outcomes
 HF/HCI researchers --- interface design
 CS and technical research--- develop design prototypes
 Users --- inform and evaluate interface, usability
 Understanding user needs and requirements
 Task description
 Interviews and observation
 Focus groups
 Informing technology design and development
 UCD and PD methods
 Low tech prototyping
 Focus group review
 Usability studies
 Evaluation of outcomes
 User testing
 Observation and measurement of performance
 Evaluation of effectiveness
Inclusive design toolbox of methods (Neale, Cobb and Kerr, 2003)
Applying Participatory Design to Develop Technology for Autism Evaluation and Treatment. Online course in collaboration with Autism Speaks: http://pdtech.haifa.ac.il/login/index.php

35. The COSPATIAL sandwich
Top slice: theory-driven  from the literature and disciplinary custom
Filling: technology requirements  affordances, representations, control, input devices
Bottom slice: user-driven  participants, practices, processes
Parsons and Cobb (2014)

36. Lessons learned

37. Overall conclusions
•Development of VR for learning requires a multi- disciplinary team
•A participatory design approach ensures contribution from all stakeholders
•Development process difficult and time consuming
•Acknowledge tensions between design drivers -
•Utility still requires teacher-led facilitation
•Not easy for teachers to set up in schools  conduct design process within school context? (Newbutt, N. PhD Thesis 2013)

38. Acknowledgements
•Thanks to the students, parents, teachers and other professionals who participated in these projects.
•The Virtual City project was supported by a National Lotteries Grant.
•AS Interactive was supported by funding from the Shirley Foundation. Visit www.virart.nott.ac.uk/asi for information
•COSPATIAL was supported by the Seventh Framework Programme of the European Commission (Grant agreement no. 231266). Visit http://cospatial.fbk.eu/ for project information.

39. References
Brown, D. J., Neale, H. R., Cobb, S. V. G., & Reynolds, H. (1999). Development and evaluation of the virtual city. International Journal of Virtual Reality, 3(4), 27-38.
Cobb, SVG (2007). Virtual environments supporting learning and communication in special needs education. Topics in Language Disorders, 27 (3), 211-225.
Cobb, S.V.G., Hawkins, T., Millen, L., and Wilson, J.R. (2014) Design and Development of Virtual Environments for Special Educational Needs. Chapter 42 in: In K. Hale and K. Stanney (Eds.), Virtual Environment Handbook, 2nd Edition, Boca Raton: CRC Press, 1075-1108.
Cobb, S.V. G., Neale, H.R. and Stewart, D. (2001). Virtual Environments - Improving accessibility to learning? Proceedings of 1st International Conference on Universal Access and Human Computer Interaction, Aug 8-10, New Orleans, Lawrence Erlbaum Associates, 783-787.
Millen, L., Cobb, S.V.G. & Patel, H (2011). Participatory design approach with children with autism. International Journal on Disability and Human Development (IJDHD), 10(4), 289-294.
Millen, L., Cobb, S.V.G., Patel, H. and Glover, T. (2012). Collaborative virtual environment for conducting design sessions with students with autism spectrum conditions. Proc.9th Intl Conf. on Disability, Virtual Reality and Assoc. Technologies. P.M. Sharkey, E. Klinger (Eds), pp269-278. Laval, France, 10-12 Sept. 2012.
Neale, H., Cobb, S. and Kerr, S. (2003). An inclusive design toolbox for development of educational Virtual Environments. Presented at: Include2003, Royal College of Art, London, 25-28 March 2003
Parsons, S. and Mitchell, P. (2002) The potential of virtual reality in social skills training for people with autistic spectrum disorders. Journal of Intellectual Disability Research, 46, (5), 430-443.
Neale, H. R., Brown, D. J., Cobb, S. V. G., & Wilson, J. R. (1999). Structured evaluation of Virtual Environments for special needs education. Presence: Teleoperators and Virtual Environments, 8(3), 264-282.
Parsons, S. and Cobb, S. (2014). Reflections on the role of the „users‟: challenges in a multidisciplinary context of learner-centred design for children on the autism spectrum. International Journal of Research & Method in Education, DOI:10.1080/1743727X.2014.890584
Parsons, S. and Mitchell, P. (2002) The potential of virtual reality in social skills training for people with autistic spectrum disorders. Journal of Intellectual Disability Research, 46, (5), 430-443.