1. Virtual Worlds in Education Forum – March 2012
Panagiotis Ritsos, Robert Gittins and Jonathan C. Roberts
p.ritsos, r.gittins, j.c.roberts @bangor.ac.uk
The IVY project has been funded with support from the European Commission. This publication reflects the views
only of the authors, and the Commission cannot be held responsible for any use which may be made of the
information contained therein

2. THE PROJECT IVY GROUP
University of Surrey (UK)
Uniwersystet im. Adama Mickiewicza (Poland)
University of Cyprus (Cyprus)
Steinbeis GmbH & Co. KG für Tech-transfer (Germany)
University of Bangor (UK - Wales)
Eberhard Karls Universität Tübingen (Germany)
Bar Ilan University (Israel)

3. Presentation Outline
 To present purpose of IVY and the IVY Virtual Environment (IVY-VE)
 To present the strategic decisions, resulting design and
implementation progress to date, towards the creation of a prototype
 To provide an overview of the main features of our prototype
 To allow for discussion on future development and research oriented
exploitation

4. Project IVY – Scope
 The rise of migration and multilingualism in Europe requires
professional interpreters in business, legal, medical and many other
settings.
 Future interpreters need to master an ever broadening range of
interpreting skills and scenarios – training for which is often difficult to
achieve with traditional teaching methods.
 Project IVY employs 3D virtual environment technology to create an
virtual educational space that supports the acquisition and application
of skills required in interpreter-mediated communication.
 Project IVY Partners provide user interaction and interpreter
resources – audio and video material from previous video
conferencing research – ‘BACKBONE’.

5. Project IVY in a nutshell
 A dedicated 3D virtual environment for
 interpreting students and
 future clients of interpreters
 A range of virtual interpreting scenarios (e.g. ‘business meeting’) that can
be run in different working modes: simulation, activity/exercise mode,
exploration and live interaction mode;
 Multilingual video-/audio-based content for interpreting scenarios, by
 adapting and supplementing the corpora from the LLP project
BACKBONE (in EN, DE, ES, FR, PL, TR) and
 adding new corpora (GR and AR, EL or RU).
 Two sets of pedagogical material for interpreter students and (future)
‘clients’, e.g. awareness-raising and interpreting exercises, and
explanations.

6. Project IVY scenario form
Dialogue
Monologue

7. IVY Virtual Environment – Requirements
To provide an intuitive, easy to use interface to a Virtual World for
To allow access through that Virtual World to existing audio
To allow easy dialogue management – addition, modification,
To enable limited dialogue synthesis, resulting in different

8. IVY Virtual Environment – Technical Aspects - I
 Dialogues are assembled from mp3 files, corresponding to the scene
participants (denoted A & B) speech
 They do not have to be of a particular order – i.e. dialogues can
follow a sequence such as `ABBAAABAB’
 Audio sequences form a dialogue script…
 …which is accompanied from textual information – title, keywords
(domains), description and scene
 In the future we will be able to replace the audio files corresponding to
a language in a script with other ones, compliant with the latter…
 …allowing limited dialogue synthesis

9. IVY Virtual Environment – Technical Aspects - II
 VE Visitors need to be able to roam freely into the areas of Project
IVY, without any obtrusive GUI elements or VW noticeboards
 Ideally users need to be able to `jump’ from scenario to scenario
without needing to return to a `reception’ area
 Audio controls need to include stop, rewind, fast-forward etc.
 Main challenge is to `match’ audio events to environment events
and/or avatar expressions and gestures
 Dialogue management does not need to be performed from within the
VE

11. IVY Virtual Environment – Second Life
 Second Life was chosen as the VW for our first prototype
 Exploration of alternatives, such as OpenSim may follow in the future
 Second Life compared to alternatives (OpenSim, ActiveWorls etc) offers:
 Large community, offering various add-ons, plugins and examples of
customisations
 A platform for social interaction and education, used by numerous institutions,
colleges, universities – thus increasing chances of exposure
 Existing scenes build in Bangor Island, allowing faster scene development
 Does not require that you run the VW yourself, but can access public servers

12. IVY Virtual Environment – Scenarios
 Various Scenarios - Classroom, meeting room, shops, outdoor,
community centre etc.

13. IVY Virtual Environment – Scenarios
 Various Scenarios - Classroom, meeting room, shops, outdoor,
community centre etc.

14. IVY Virtual Environment – Scenarios
 Various Scenarios - Classroom, meeting room, shops, outdoor,
community centre etc.

15. IVY Virtual Environment – Scenarios
 Interpreter students at Surrey University have already trialled basic
meetings.

16. IVY Virtual Environment – Scenarios
 Courtroom settings are also being developed – multi purpose training
scenarios – interpreter and user of interpreters.

18. IVY Virtual Environment – Architecture

19. IVY Virtual Environment – Admin Console
 An Appfuse-based web application for dialogue and user management

20. IVY Virtual Environment – Menu
 Menu is a Heads-Up display (HUD) with same aesthetics to the SL
user interface build using jQuery, Flash and Javascript as part of
our web-app

21. IVY Virtual Environment – Dialogue HUD
 Simple, drill-down menu for
selecting form, language pair
and dialogue by title
 Information Pane and Launcher
 ‘Breadcrumb’ for navigation
 Individual per user
 Allows teleportation from
scenario to scenario without
requiring returning to a
reception area

22. IVY Virtual Environment – Player - Teleportation
 Once a dialogue is selected a player and a teleportation confirmation
window appear

23. IVY Virtual Environment – Player Detail
 Simple audio player with dialogue specific information
 Exiting returns user to dialogue selection menu

24. IVY Virtual Environment – Next Steps
 Integrate IVY HUD functionality with the virtual
world by:
 Controlling participant-avatars when speaking
through a telnet-based `bot’ server
 Each bot performs basic gestures for as long as
`he’ or `she’ talks
 Explore the possibility of using directional sound
 Integrate tighter with SL GUI elements such as
flashing voice indicators

25. IVY Virtual Environment – Research Element
 Where is the research element? Is it merely a development
project, mixing and matching existing techniques?
 The research element comes from the `service’ and not the
technology…
 Performance comparison to traditional methods used by interpreters
 Investigation on how the sense of immersion enhances the user
experience of IVY-VE participants
 Definition of metrics for the above

26. Thank you!