SlideShare a Scribd company logo
Virtual Learning Technologies, Bangor, Wales – October 2012
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




              Project IVY - Using Virtual Reality for
              Interpreter-mediated Communication and Training


                                        Panagiotis D. Ritsos1, Robert Gittins1 and Jonathan C. Roberts1


                                 School of Computer Science, Bangor University, UK - {p.ritsos, r.gittins, j.c.roberts}@bangor.ac.uk)




              Lifelong Learning Programme – Project 511862-LLP-2010-1-UK-KA3-KA3MP - The IVY project has been funded with support from the European
              Commission. This publication reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made
              of the information contained therein.
Presentation Outline

                                 To present purpose of project IVY – Interpreting in Virtual Environment and
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                  the IVY Virtual Environment (IVY-VE)


                                 To present the strategic decisions, resulting design and implementation progress
                                  to date, towards the creation of a prototype


                                 To provide an overview of the main features of our prototype


                                 Comment on preliminary evaluation and pinpoint identified limitation


                                 To allow for discussion on future development
Ysgol Gwyddorau Cyfrifiadurol   The Project IVY Consortium
School of Computer Science




                                University of Surrey                               (UK)

                                Uniwersystet im. Adama Mickiewicza                 (Poland)

                                University of Cyprus                               (Cyprus)

                                Steinbeis GmbH & Co. KG für Technologiertransfer   (Germany)

                                Bangor University                                  (UK)

                                Eberhard Karls Universität Tübingen                (Germany)

                                Bar Ilan University                                (Israel)
Project IVY – Scope

                                 The rise of migration and multilingualism in Europe requires professional
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                  interpreters in business, legal, medical and many other settings.


                                 Future interpreters need to master an ever broadening range of interpreting
                                  skills and scenarios – training for which is often difficult to achieve with
                                  traditional teaching methods.


                                 Project IVY employs 3D virtual world technology to create an educational
                                  space that supports the acquisition and application of skills required in
                                  interpreter-mediated communication.


                                 Project IVY uses existing interpreter resources – audio and video material from
                                  previous video conferencing research.
IVY VE – Requirements

                                 To provide an intuitive, easy to use interface within a Virtual World for
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                  accessing multimedia material created for interpreting training and simulation.


                                 To allow easy scenario management by users who often are not experts in
                                  commuting (i.e., can not/should not write code)...


                                 … meaning the creation and modification of existing scenarios in terms of their
                                  multimedia content, requiring basic CRUD functionality.


                                 To enable limited dialogue and monologue synthesis, resulting in the
                                  enrichment of the corpora with different language combinations of existing
                                  scenarios.
IVY Virtual Environment (IVY-VE) in a nutshell

                                 A dedicated, adaptive 3D virtual environment for
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                   o interpreting students and
                                   o future clients of interpreters


                                 Supports range of virtual interpreting scenarios (e.g., ‘business meeting’) that can
                                  be run in different modes:
                                   o Interpreting (& Learning Activity) mode, where students can practice using
                                     dialogues and monologues
                                   o Exploration mode, where clients can learn about interpreting
                                   o Live interaction mode, where both groups can engage in role plays


                                 Uses multilingual video/audio-based content for interpreting scenarios, by
                                  adapting existing multimedia corpora from the LLP project BACKBONE and the
                                  ELISA corpus, and creating three new corpora in Greek, Russian and Hebrew;

                                 Supported by two sets of pedagogical material for interpreter students and
                                  (future) ‘clients’, e.g. awareness-raising and interpreting exercises.
School of Computer Science
            Ysgol Gwyddorau Cyfrifiadurol   Project IVY – Scenario forms




                        Dialogue




Monologue
IVY Virtual World – Second Life

                                 Second Life was chosen as the Virtual World for our first prototype.
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                 Exploration of alternatives, such as OpenSim, WebGL and Unity will follow in
                                  the future.


                                 Second Life compared to alternatives (OpenSim, ActiveWorls etc) offers:

                                   o Large community, various add-ons, plugins and examples of customisations.

                                   o A platform for social interaction and education, used by numerous
                                     institutions, colleges, universities – thus increasing chances of exposure.

                                   o Accessibility via public servers and it does not require that you run the VW yourself.
IVY VE – Lack of Instancing & Scalability

                                 Due to limits on the number of primitives available to the IVY Island and the
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                  lack of instancing mechanisms in SL, IVE-VE uses a collection of unique, in-
                                  world locations for each type of scenario (e.g., Classroom, Meeting Office).


                                 Therefore, a scenario may share its location with another, being carried out in a
                                  similar setting.


                                 In order to maintain consistency in the virtual world only one scenario can be
                                  played per location at a given time.


                                 Upon a scenario launch by a user, all scenarios sharing the same location
                                  become unavailable for other users.


                                 Once the user exits the selected scenario, all scenarios sharing the same
                                  location become available again.
IVY VE – Design Strategies

                                 Script things in Second Life using LSL
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                   + Comparable aesthetics and better integration with the native SL GUI
                                   + Graphics appear crispier, clearer

                                   - Scripting can be fairly static and requires coding
                                   - Communication with DBs is fairly limited in terms of size of info that can be
                                      transmitted.



                                 Rely on web application technologies as interface creation, database
                                  connectivity and overall flexibility surpasses LSL. Therefore…


                                 …either embed SL in a web application
                                   o Popular notion, no web viewer supported from Linden Labs.
                                   o Only Canvas from Tripodean Technologies appears to exist at the moment.


                                 …or ‘embed’ the web application within SL, using HTML on a Prim.
IVY VE – Implementation

                                 Our chosen implementation strategy aspires to merge useful features from both
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                  alternatives, resulting into a hybrid solution.




                                 One module consists of a web application with two entry points. One entry point
                                  remains independent of SL and is used by content managers to manage scenarios
                                  and users’ information, offering basic CRUD functionality.


                                 The other end is viewable within Second Life, in the form of a ‘Heads-up-
                                  Display’ (HUD), populated from a database, showing available scenarios to the
                                  users, having a player functionality and initiating in-world teleport events.
IVY VE – User Classes and Characteristics

                                 Separating user roles allows controlled access to different parts of the application
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                  as well as means of monitoring scenario selection and execution.


                                 The system’s user classes and their respective role descriptions are:

                                   o Interpreters/Users, whose purpose is to explore, participate and exercise with the
                                     scenarios in Second Life.

                                   o Observers, whose purpose is to observe other users in Second Life.

                                   o Content Managers who are responsible for user and scenario management.
IVY VE – The IVY Island

                                 Locations created according to the corpus requirements, trying to keep prim-count
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                  to a minimum.


                                 Reception serves as a central hub and a HUD pick-up point.
IVY VE – Scenario Actors

                                 We populate the scenarios with additional ‘actors’. Currently these actors are
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                  manually placed in each location, using Second-life ‘robots’ (bots) from
                                  Pikkubot and Thoys.


                                 Bots are controlled either by using in-world chat to issue commands directly to
                                  the bots or through a dedicated server’s telnet prompt.


                                 We currently use animation overrides, to make them appear life-like.


                                 Our aspiration is to create a service that talks to the bots and relays scenario
                                  specific information and teleport commands.


                                 However… sound does not appear to originate from the bots.
IVY VE – Web Application

                                 The IVY web application is build using the Appfuse 2 open source project
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                  (appfuse.org).


                                 Appfuse, built on the Java platform, uses industry-standard features, such as
                                  Apache Maven integration, JPA support for database operations and popular
                                  web frameworks such as Spring and Apache Struts, employed in this example.


                                 Appfuse comes with out of the box features, needed in IVY-VE such as:

                                   o   Generic CRUD backend
                                   o   Authentication and authorization,
                                   o   User management
                                   o   Strong Internationalization support


                                 Our prototype is deployed using Apache Tomcat 6.x and uses the MySQL 5.x
                                  database.
IVY VE – Audio File Management

                                 The IVY-VE uses audio extracts (segments), in MPEG-2 Audio Layer III
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                  format, from the LLP project BACKBONE, wrapped in XSPF (XML Shareable
                                  Playlist Format) play-lists (scripts) and played within pre-fabricated scenes.


                                 Audio segments are uniquely named and can be interchanged — within each
                                  script — to derive further language combinations of that scenario.


                                 Actors may speak for more than one consecutive segments, allowing greater
                                  flexibility in creating scripts where one talks for extended periods.


                                 However, it is assumed that only one actor talks per audio segment and there is
                                  no overlap between actor’s speech.


                                 Each script has textual information associated with it, such as brief content
                                  information, scene description and domain keywords.
IVY VE – Heads Up Display I

                                 The HUD is built using the jQuery JavaScript library, displaying the list of
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                  scenarios in the database as a drill-down menu.


                                 It is normally attached to the bottom left corner of the user’s viewport.


                                 Audio is being played by means of a Flash player, parsing the XSPF playlists
                                  upon scenario selection.


                                 Navigation through the island is performed using slurls, providing direct url-
                                  like teleport links to locations within the virtual world.


                                 Each slurl is being called upon scenario launch, triggering the native SL-client
                                  teleport interface.
IVY VE – Heads Up Display II

                                 Login, form selection and language combination selection views
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                 Free Scenarios views – Scenario Info & Player




                                 Locked Scenarios views
IVY VE – Administrator’s Panel I

                                 The management console allows Content Administrators to easily populate the
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                  scenario database and create XSPF playlists.


                                 New scenarios are created through a form, where administrators provide textual
                                  information (title, language combos, participants gender etc.) as well as an
                                  ordered list of the audio tracks in each scenario.


                                 Scenario textual info is stored in the DB and XSPF playlists on a separate playlist
                                  inventory.


                                 A separate listing of all the scenarios in the system allows administrators to see
                                  which user is currently working on each scenario and also fire-up their SL client
                                  and teleport to that location.
IVY VE – Administrator’s Panel II

                                 Scenario Upload Form
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                 Scenario Listing
                                 (with Teleport functionality)
IVY VE – Preliminary Functional Evaluation

                                 A preliminary evaluation of the current prototype was done by nine
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                  interpreting and two virtual world experts, using talk-aloud, try-out sessions
                                  in Second Life, followed by a discussion with the assessor.


                                 The evaluation focused on the HUD functionality and in-world locations.


                                 The admin panel was not evaluated at this stage; however it has been
                                  successfully used, by content administrators, for the past four months to upload
                                  scenarios in our system.


                                 Overall opinion was quite positive and users with very limited experience in
                                  virtual worlds, gaming or similar environments felt comfortable using IVY-VE.


                                 However, a series of limitations of the current system where pinpointed…
IVY VE – Current Limitations

                                 Some users focused too much on the HUD, not paying attention to the world.
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                 No sound directionality and no visual cues on who is speaking.

                                 Interpreters were enthusiastic they have a tool – but does that generates bias?

                                 Lots of comments regarding aesthetics of locations – all participants observed
                                  and commented on the quality of locations using photorealistic graphics.


                                 No zoning, instancing and replication like in games.

                                 Limitation on available ‘prims’ affects world scalability

                                 … and consequently scenario management and execution, actor placement.

                                 Currently bots are manually placed to required scenarios – nowadays we use
                                  many bots and manage locations to cover all gender combinations in our corpus.

                                 Sound is heard only from the player controller and is not broadcast back to the
                                  world – hence observers do not hear it.
IVY VE – Future Work

                                 Currently undergoing two evaluation cycles, focused on interpreting students and
Ysgol Gwyddorau Cyfrifiadurol
School of Computer Science




                                  potential clients of interpreters during the autumn of 2012 and will attempt to get
                                  some feedback on the IVY-VE usability and resulting sense of immersion.

                                 Interpreting mode continuously enhanced with a supporting exercises, both
                                  generic to dialogues and monologues, as well as specific to particular scenarios
                                  which have an inherit challenge in interpreting practice.

                                 Use student/client feedback and experience from using IVY-VE to design a
                                  new bespoke system…

                                 ..while exploring alternative technologies that allow tighter integration with
                                  current web-based scenario system, e.g., Unity and WebGL.
Ysgol Gwyddorau Cyfrifiadurol    IVY VE – Forthcoming Events
School of Computer Science




                                 Exploiting Emerging Technologies to Prepare Interpreters and their Clients
                                  for Professional Practice, London, 23rd November 2012




                                  For more info visit:
                                  Consortium website                    http://www.virtual-interpreting.net/Seminar.html

                                  …or the Bangor IVY partner website    http://www.vmg.cs.bangor.ac.uk/IVY/
School of Computer Science
Ysgol Gwyddorau Cyfrifiadurol




                   Thank you!

More Related Content

What's hot

Artigo Educon2012
Artigo Educon2012Artigo Educon2012
Artigo Educon2012
Ana Margarida Maia
 
Bertini - Automatic Metadata Extraction in VidiVideo & im3i @EUscreen Mykonos
Bertini - Automatic Metadata Extraction in VidiVideo & im3i @EUscreen MykonosBertini - Automatic Metadata Extraction in VidiVideo & im3i @EUscreen Mykonos
Bertini - Automatic Metadata Extraction in VidiVideo & im3i @EUscreen Mykonos
EUscreen
 
Maya
MayaMaya
Maya
NASAPMC
 
Javier Diaz Presentacion Korea V4
Javier Diaz Presentacion Korea V4Javier Diaz Presentacion Korea V4
Javier Diaz Presentacion Korea V4
Javier Diaz
 
CICT's ICT Month Celebrations - HCDG Week
CICT's ICT Month Celebrations - HCDG WeekCICT's ICT Month Celebrations - HCDG Week
CICT's ICT Month Celebrations - HCDG Week
Mel Tan
 
Digital technologies
Digital technologiesDigital technologies
Digital technologies
jamberryxxx
 
Virtual Reality: Stereoscopic Imaging for Educational Institutions
Virtual Reality: Stereoscopic Imaging for Educational Institutions Virtual Reality: Stereoscopic Imaging for Educational Institutions
Virtual Reality: Stereoscopic Imaging for Educational Institutions
Rodrigo Arnaut
 

What's hot (7)

Artigo Educon2012
Artigo Educon2012Artigo Educon2012
Artigo Educon2012
 
Bertini - Automatic Metadata Extraction in VidiVideo & im3i @EUscreen Mykonos
Bertini - Automatic Metadata Extraction in VidiVideo & im3i @EUscreen MykonosBertini - Automatic Metadata Extraction in VidiVideo & im3i @EUscreen Mykonos
Bertini - Automatic Metadata Extraction in VidiVideo & im3i @EUscreen Mykonos
 
Maya
MayaMaya
Maya
 
Javier Diaz Presentacion Korea V4
Javier Diaz Presentacion Korea V4Javier Diaz Presentacion Korea V4
Javier Diaz Presentacion Korea V4
 
CICT's ICT Month Celebrations - HCDG Week
CICT's ICT Month Celebrations - HCDG WeekCICT's ICT Month Celebrations - HCDG Week
CICT's ICT Month Celebrations - HCDG Week
 
Digital technologies
Digital technologiesDigital technologies
Digital technologies
 
Virtual Reality: Stereoscopic Imaging for Educational Institutions
Virtual Reality: Stereoscopic Imaging for Educational Institutions Virtual Reality: Stereoscopic Imaging for Educational Institutions
Virtual Reality: Stereoscopic Imaging for Educational Institutions
 

Similar to Project IVY - Using Virtual reality for interpreter-mediated communication and training

Interpreting in Virtual Reality
Interpreting in Virtual RealityInterpreting in Virtual Reality
Interpreting in Virtual Reality
Panagiotis Ritsos
 
Project IVY- Virtual Environment Development
Project IVY- Virtual Environment  DevelopmentProject IVY- Virtual Environment  Development
Project IVY- Virtual Environment Development
Panagiotis Ritsos
 
Project IVY - Virtual Environment Development
Project IVY - Virtual Environment DevelopmentProject IVY - Virtual Environment Development
Project IVY - Virtual Environment Development
Panagiotis Ritsos
 
Wonderland @ Cattid - Sun's Virtual Workplace
Wonderland @ Cattid - Sun's Virtual WorkplaceWonderland @ Cattid - Sun's Virtual Workplace
Wonderland @ Cattid - Sun's Virtual Workplace
vincenzo de simone
 
Workshop3 d koenraad
Workshop3 d koenraadWorkshop3 d koenraad
Workshop3 d koenraad
Ton Koenraad
 
ICL 2011 ST.ART, AVATAR, EUROVERSITY
ICL 2011 ST.ART, AVATAR, EUROVERSITYICL 2011 ST.ART, AVATAR, EUROVERSITY
ICL 2011 ST.ART, AVATAR, EUROVERSITY
Cristina Stefanelli
 
Mpeg v-awareness event
Mpeg v-awareness eventMpeg v-awareness event
Mpeg v-awareness event
Marius Preda PhD
 
Challenges and requirements for a next generation service for video content s...
Challenges and requirements for a next generation service for video content s...Challenges and requirements for a next generation service for video content s...
Challenges and requirements for a next generation service for video content s...
Wesley De Neve
 
WCETR 2012 VWPfE presentation
WCETR 2012 VWPfE presentationWCETR 2012 VWPfE presentation
WCETR 2012 VWPfE presentation
Ridvan Ata
 
Exploiting metadata, ontologies and semantics to design/enhance new end-user ...
Exploiting metadata, ontologies and semantics to design/enhance new end-user ...Exploiting metadata, ontologies and semantics to design/enhance new end-user ...
Exploiting metadata, ontologies and semantics to design/enhance new end-user ...
Ahmet Soylu
 
AVATAR – The Course: Recommendations for Using 3D Virtual Environments for Te...
AVATAR – The Course: Recommendations for Using 3D Virtual Environments for Te...AVATAR – The Course: Recommendations for Using 3D Virtual Environments for Te...
AVATAR – The Course: Recommendations for Using 3D Virtual Environments for Te...
eLearning Papers
 
SUMA & T-MAESTRO
SUMA & T-MAESTROSUMA & T-MAESTRO
SUMA & T-MAESTRO
IDTV Laboratory
 
Visionaire project learning in 3D virtual worlds, enabling vacademia in cave
Visionaire project learning in 3D virtual worlds, enabling vacademia in caveVisionaire project learning in 3D virtual worlds, enabling vacademia in cave
Visionaire project learning in 3D virtual worlds, enabling vacademia in cave
Mikhail Fominykh
 
Nmc 2007 Publish
Nmc 2007 PublishNmc 2007 Publish
Nmc 2007 Publish
urauch
 
Repositories of community memory as visualized activities in 3D virtual worlds
Repositories of community memory as visualized activities in 3D virtual worldsRepositories of community memory as visualized activities in 3D virtual worlds
Repositories of community memory as visualized activities in 3D virtual worlds
Mikhail Fominykh
 
Gsic@decl icls2012 20120701
Gsic@decl icls2012 20120701Gsic@decl icls2012 20120701
Gsic@decl icls2012 20120701
Yannis
 
Cnie Projet Enjeux S Diaporama Banff 2008 Va
Cnie Projet Enjeux S  Diaporama Banff  2008 VaCnie Projet Enjeux S  Diaporama Banff  2008 Va
Cnie Projet Enjeux S Diaporama Banff 2008 Va
guest7e67ab
 
Mvc
MvcMvc
Things you should_know_about_future_trends
Things you should_know_about_future_trendsThings you should_know_about_future_trends
Things you should_know_about_future_trends
Cheryl Todd
 
AVATAR - Added Value of teaching in a virtual world
AVATAR - Added Value of teaching in a virtual worldAVATAR - Added Value of teaching in a virtual world
AVATAR - Added Value of teaching in a virtual world
Cristina Stefanelli
 

Similar to Project IVY - Using Virtual reality for interpreter-mediated communication and training (20)

Interpreting in Virtual Reality
Interpreting in Virtual RealityInterpreting in Virtual Reality
Interpreting in Virtual Reality
 
Project IVY- Virtual Environment Development
Project IVY- Virtual Environment  DevelopmentProject IVY- Virtual Environment  Development
Project IVY- Virtual Environment Development
 
Project IVY - Virtual Environment Development
Project IVY - Virtual Environment DevelopmentProject IVY - Virtual Environment Development
Project IVY - Virtual Environment Development
 
Wonderland @ Cattid - Sun's Virtual Workplace
Wonderland @ Cattid - Sun's Virtual WorkplaceWonderland @ Cattid - Sun's Virtual Workplace
Wonderland @ Cattid - Sun's Virtual Workplace
 
Workshop3 d koenraad
Workshop3 d koenraadWorkshop3 d koenraad
Workshop3 d koenraad
 
ICL 2011 ST.ART, AVATAR, EUROVERSITY
ICL 2011 ST.ART, AVATAR, EUROVERSITYICL 2011 ST.ART, AVATAR, EUROVERSITY
ICL 2011 ST.ART, AVATAR, EUROVERSITY
 
Mpeg v-awareness event
Mpeg v-awareness eventMpeg v-awareness event
Mpeg v-awareness event
 
Challenges and requirements for a next generation service for video content s...
Challenges and requirements for a next generation service for video content s...Challenges and requirements for a next generation service for video content s...
Challenges and requirements for a next generation service for video content s...
 
WCETR 2012 VWPfE presentation
WCETR 2012 VWPfE presentationWCETR 2012 VWPfE presentation
WCETR 2012 VWPfE presentation
 
Exploiting metadata, ontologies and semantics to design/enhance new end-user ...
Exploiting metadata, ontologies and semantics to design/enhance new end-user ...Exploiting metadata, ontologies and semantics to design/enhance new end-user ...
Exploiting metadata, ontologies and semantics to design/enhance new end-user ...
 
AVATAR – The Course: Recommendations for Using 3D Virtual Environments for Te...
AVATAR – The Course: Recommendations for Using 3D Virtual Environments for Te...AVATAR – The Course: Recommendations for Using 3D Virtual Environments for Te...
AVATAR – The Course: Recommendations for Using 3D Virtual Environments for Te...
 
SUMA & T-MAESTRO
SUMA & T-MAESTROSUMA & T-MAESTRO
SUMA & T-MAESTRO
 
Visionaire project learning in 3D virtual worlds, enabling vacademia in cave
Visionaire project learning in 3D virtual worlds, enabling vacademia in caveVisionaire project learning in 3D virtual worlds, enabling vacademia in cave
Visionaire project learning in 3D virtual worlds, enabling vacademia in cave
 
Nmc 2007 Publish
Nmc 2007 PublishNmc 2007 Publish
Nmc 2007 Publish
 
Repositories of community memory as visualized activities in 3D virtual worlds
Repositories of community memory as visualized activities in 3D virtual worldsRepositories of community memory as visualized activities in 3D virtual worlds
Repositories of community memory as visualized activities in 3D virtual worlds
 
Gsic@decl icls2012 20120701
Gsic@decl icls2012 20120701Gsic@decl icls2012 20120701
Gsic@decl icls2012 20120701
 
Cnie Projet Enjeux S Diaporama Banff 2008 Va
Cnie Projet Enjeux S  Diaporama Banff  2008 VaCnie Projet Enjeux S  Diaporama Banff  2008 Va
Cnie Projet Enjeux S Diaporama Banff 2008 Va
 
Mvc
MvcMvc
Mvc
 
Things you should_know_about_future_trends
Things you should_know_about_future_trendsThings you should_know_about_future_trends
Things you should_know_about_future_trends
 
AVATAR - Added Value of teaching in a virtual world
AVATAR - Added Value of teaching in a virtual worldAVATAR - Added Value of teaching in a virtual world
AVATAR - Added Value of teaching in a virtual world
 

Recently uploaded

RDBMS Lecture Notes Unit4 chapter12 VIEW
RDBMS Lecture Notes Unit4 chapter12 VIEWRDBMS Lecture Notes Unit4 chapter12 VIEW
RDBMS Lecture Notes Unit4 chapter12 VIEW
Murugan Solaiyappan
 
ASP.NET Core Interview Questions PDF By ScholarHat.pdf
ASP.NET Core Interview Questions PDF By ScholarHat.pdfASP.NET Core Interview Questions PDF By ScholarHat.pdf
ASP.NET Core Interview Questions PDF By ScholarHat.pdf
Scholarhat
 
Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...
Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...
Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...
Codeavour International
 
6. Physiological Disorder of fruits and vegetables.pptx
6. Physiological Disorder of fruits and vegetables.pptx6. Physiological Disorder of fruits and vegetables.pptx
6. Physiological Disorder of fruits and vegetables.pptx
UmeshTimilsina1
 
React Interview Question PDF By ScholarHat
React Interview Question PDF By ScholarHatReact Interview Question PDF By ScholarHat
React Interview Question PDF By ScholarHat
Scholarhat
 
Brigada Eskwela 2024 PowerPoint Update for SY 2024-2025
Brigada Eskwela 2024 PowerPoint Update for SY 2024-2025Brigada Eskwela 2024 PowerPoint Update for SY 2024-2025
Brigada Eskwela 2024 PowerPoint Update for SY 2024-2025
ALBERTHISOLER1
 
JavaScript Interview Questions PDF By ScholarHat
JavaScript Interview  Questions PDF By ScholarHatJavaScript Interview  Questions PDF By ScholarHat
JavaScript Interview Questions PDF By ScholarHat
Scholarhat
 
FIRST AID PRESENTATION ON INDUSTRIAL SAFETY by dr lal.ppt
FIRST AID PRESENTATION ON INDUSTRIAL SAFETY by dr lal.pptFIRST AID PRESENTATION ON INDUSTRIAL SAFETY by dr lal.ppt
FIRST AID PRESENTATION ON INDUSTRIAL SAFETY by dr lal.ppt
ashutoshklal29
 
11EHS Term 3 Week 1 Unit 1 Review: Feedback and improvementpptx
11EHS Term 3 Week 1 Unit 1 Review: Feedback and improvementpptx11EHS Term 3 Week 1 Unit 1 Review: Feedback and improvementpptx
11EHS Term 3 Week 1 Unit 1 Review: Feedback and improvementpptx
mansk2
 
5. Postharvest deterioration of fruits and vegetables.pptx
5. Postharvest deterioration of fruits and vegetables.pptx5. Postharvest deterioration of fruits and vegetables.pptx
5. Postharvest deterioration of fruits and vegetables.pptx
UmeshTimilsina1
 
C Interview Questions PDF By Scholarhat.pdf
C Interview Questions PDF By Scholarhat.pdfC Interview Questions PDF By Scholarhat.pdf
C Interview Questions PDF By Scholarhat.pdf
Scholarhat
 
Parkinson Disease & Anti-Parkinsonian Drugs.pptx
Parkinson Disease & Anti-Parkinsonian Drugs.pptxParkinson Disease & Anti-Parkinsonian Drugs.pptx
Parkinson Disease & Anti-Parkinsonian Drugs.pptx
AnujVishwakarma34
 
11. Post harvest quality, Quality criteria and Judgement.pptx
11. Post harvest quality, Quality criteria and Judgement.pptx11. Post harvest quality, Quality criteria and Judgement.pptx
11. Post harvest quality, Quality criteria and Judgement.pptx
UmeshTimilsina1
 
Lecture Notes Unit4 Chapter13 users , roles and privileges
Lecture Notes Unit4 Chapter13 users , roles and privilegesLecture Notes Unit4 Chapter13 users , roles and privileges
Lecture Notes Unit4 Chapter13 users , roles and privileges
Murugan146644
 
Node JS Interview Question PDF By ScholarHat
Node JS Interview Question PDF By ScholarHatNode JS Interview Question PDF By ScholarHat
Node JS Interview Question PDF By ScholarHat
Scholarhat
 
Java MCQ Questions and Answers PDF By ScholarHat
Java MCQ Questions and Answers PDF By ScholarHatJava MCQ Questions and Answers PDF By ScholarHat
Java MCQ Questions and Answers PDF By ScholarHat
Scholarhat
 
DepEd School Calendar 2024-2025 DO_s2024_008
DepEd School Calendar 2024-2025 DO_s2024_008DepEd School Calendar 2024-2025 DO_s2024_008
DepEd School Calendar 2024-2025 DO_s2024_008
Glenn Rivera
 
BỘ ĐỀ THI HỌC SINH GIỎI CÁC TỈNH MÔN TIẾNG ANH LỚP 9 NĂM HỌC 2023-2024 (CÓ FI...
BỘ ĐỀ THI HỌC SINH GIỎI CÁC TỈNH MÔN TIẾNG ANH LỚP 9 NĂM HỌC 2023-2024 (CÓ FI...BỘ ĐỀ THI HỌC SINH GIỎI CÁC TỈNH MÔN TIẾNG ANH LỚP 9 NĂM HỌC 2023-2024 (CÓ FI...
BỘ ĐỀ THI HỌC SINH GIỎI CÁC TỈNH MÔN TIẾNG ANH LỚP 9 NĂM HỌC 2023-2024 (CÓ FI...
Nguyen Thanh Tu Collection
 
Dot NET Interview Questions PDF By ScholarHat
Dot NET Interview Questions PDF By ScholarHatDot NET Interview Questions PDF By ScholarHat
Dot NET Interview Questions PDF By ScholarHat
Scholarhat
 
INSIDE OUT - PowerPoint Presentation.pptx
INSIDE OUT - PowerPoint Presentation.pptxINSIDE OUT - PowerPoint Presentation.pptx
INSIDE OUT - PowerPoint Presentation.pptx
RODELAZARES3
 

Recently uploaded (20)

RDBMS Lecture Notes Unit4 chapter12 VIEW
RDBMS Lecture Notes Unit4 chapter12 VIEWRDBMS Lecture Notes Unit4 chapter12 VIEW
RDBMS Lecture Notes Unit4 chapter12 VIEW
 
ASP.NET Core Interview Questions PDF By ScholarHat.pdf
ASP.NET Core Interview Questions PDF By ScholarHat.pdfASP.NET Core Interview Questions PDF By ScholarHat.pdf
ASP.NET Core Interview Questions PDF By ScholarHat.pdf
 
Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...
Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...
Codeavour 5.0 International Impact Report - The Biggest International AI, Cod...
 
6. Physiological Disorder of fruits and vegetables.pptx
6. Physiological Disorder of fruits and vegetables.pptx6. Physiological Disorder of fruits and vegetables.pptx
6. Physiological Disorder of fruits and vegetables.pptx
 
React Interview Question PDF By ScholarHat
React Interview Question PDF By ScholarHatReact Interview Question PDF By ScholarHat
React Interview Question PDF By ScholarHat
 
Brigada Eskwela 2024 PowerPoint Update for SY 2024-2025
Brigada Eskwela 2024 PowerPoint Update for SY 2024-2025Brigada Eskwela 2024 PowerPoint Update for SY 2024-2025
Brigada Eskwela 2024 PowerPoint Update for SY 2024-2025
 
JavaScript Interview Questions PDF By ScholarHat
JavaScript Interview  Questions PDF By ScholarHatJavaScript Interview  Questions PDF By ScholarHat
JavaScript Interview Questions PDF By ScholarHat
 
FIRST AID PRESENTATION ON INDUSTRIAL SAFETY by dr lal.ppt
FIRST AID PRESENTATION ON INDUSTRIAL SAFETY by dr lal.pptFIRST AID PRESENTATION ON INDUSTRIAL SAFETY by dr lal.ppt
FIRST AID PRESENTATION ON INDUSTRIAL SAFETY by dr lal.ppt
 
11EHS Term 3 Week 1 Unit 1 Review: Feedback and improvementpptx
11EHS Term 3 Week 1 Unit 1 Review: Feedback and improvementpptx11EHS Term 3 Week 1 Unit 1 Review: Feedback and improvementpptx
11EHS Term 3 Week 1 Unit 1 Review: Feedback and improvementpptx
 
5. Postharvest deterioration of fruits and vegetables.pptx
5. Postharvest deterioration of fruits and vegetables.pptx5. Postharvest deterioration of fruits and vegetables.pptx
5. Postharvest deterioration of fruits and vegetables.pptx
 
C Interview Questions PDF By Scholarhat.pdf
C Interview Questions PDF By Scholarhat.pdfC Interview Questions PDF By Scholarhat.pdf
C Interview Questions PDF By Scholarhat.pdf
 
Parkinson Disease & Anti-Parkinsonian Drugs.pptx
Parkinson Disease & Anti-Parkinsonian Drugs.pptxParkinson Disease & Anti-Parkinsonian Drugs.pptx
Parkinson Disease & Anti-Parkinsonian Drugs.pptx
 
11. Post harvest quality, Quality criteria and Judgement.pptx
11. Post harvest quality, Quality criteria and Judgement.pptx11. Post harvest quality, Quality criteria and Judgement.pptx
11. Post harvest quality, Quality criteria and Judgement.pptx
 
Lecture Notes Unit4 Chapter13 users , roles and privileges
Lecture Notes Unit4 Chapter13 users , roles and privilegesLecture Notes Unit4 Chapter13 users , roles and privileges
Lecture Notes Unit4 Chapter13 users , roles and privileges
 
Node JS Interview Question PDF By ScholarHat
Node JS Interview Question PDF By ScholarHatNode JS Interview Question PDF By ScholarHat
Node JS Interview Question PDF By ScholarHat
 
Java MCQ Questions and Answers PDF By ScholarHat
Java MCQ Questions and Answers PDF By ScholarHatJava MCQ Questions and Answers PDF By ScholarHat
Java MCQ Questions and Answers PDF By ScholarHat
 
DepEd School Calendar 2024-2025 DO_s2024_008
DepEd School Calendar 2024-2025 DO_s2024_008DepEd School Calendar 2024-2025 DO_s2024_008
DepEd School Calendar 2024-2025 DO_s2024_008
 
BỘ ĐỀ THI HỌC SINH GIỎI CÁC TỈNH MÔN TIẾNG ANH LỚP 9 NĂM HỌC 2023-2024 (CÓ FI...
BỘ ĐỀ THI HỌC SINH GIỎI CÁC TỈNH MÔN TIẾNG ANH LỚP 9 NĂM HỌC 2023-2024 (CÓ FI...BỘ ĐỀ THI HỌC SINH GIỎI CÁC TỈNH MÔN TIẾNG ANH LỚP 9 NĂM HỌC 2023-2024 (CÓ FI...
BỘ ĐỀ THI HỌC SINH GIỎI CÁC TỈNH MÔN TIẾNG ANH LỚP 9 NĂM HỌC 2023-2024 (CÓ FI...
 
Dot NET Interview Questions PDF By ScholarHat
Dot NET Interview Questions PDF By ScholarHatDot NET Interview Questions PDF By ScholarHat
Dot NET Interview Questions PDF By ScholarHat
 
INSIDE OUT - PowerPoint Presentation.pptx
INSIDE OUT - PowerPoint Presentation.pptxINSIDE OUT - PowerPoint Presentation.pptx
INSIDE OUT - PowerPoint Presentation.pptx
 

Project IVY - Using Virtual reality for interpreter-mediated communication and training

  • 1. Virtual Learning Technologies, Bangor, Wales – October 2012 Ysgol Gwyddorau Cyfrifiadurol School of Computer Science Project IVY - Using Virtual Reality for Interpreter-mediated Communication and Training Panagiotis D. Ritsos1, Robert Gittins1 and Jonathan C. Roberts1 School of Computer Science, Bangor University, UK - {p.ritsos, r.gittins, j.c.roberts}@bangor.ac.uk) Lifelong Learning Programme – Project 511862-LLP-2010-1-UK-KA3-KA3MP - The IVY project has been funded with support from the European Commission. This publication reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.
  • 2. Presentation Outline  To present purpose of project IVY – Interpreting in Virtual Environment and Ysgol Gwyddorau Cyfrifiadurol School of Computer Science the IVY Virtual Environment (IVY-VE)  To present the strategic decisions, resulting design and implementation progress to date, towards the creation of a prototype  To provide an overview of the main features of our prototype  Comment on preliminary evaluation and pinpoint identified limitation  To allow for discussion on future development
  • 3. Ysgol Gwyddorau Cyfrifiadurol The Project IVY Consortium School of Computer Science University of Surrey (UK) Uniwersystet im. Adama Mickiewicza (Poland) University of Cyprus (Cyprus) Steinbeis GmbH & Co. KG für Technologiertransfer (Germany) Bangor University (UK) Eberhard Karls Universität Tübingen (Germany) Bar Ilan University (Israel)
  • 4. Project IVY – Scope  The rise of migration and multilingualism in Europe requires professional Ysgol Gwyddorau Cyfrifiadurol School of Computer Science interpreters in business, legal, medical and many other settings.  Future interpreters need to master an ever broadening range of interpreting skills and scenarios – training for which is often difficult to achieve with traditional teaching methods.  Project IVY employs 3D virtual world technology to create an educational space that supports the acquisition and application of skills required in interpreter-mediated communication.  Project IVY uses existing interpreter resources – audio and video material from previous video conferencing research.
  • 5. IVY VE – Requirements  To provide an intuitive, easy to use interface within a Virtual World for Ysgol Gwyddorau Cyfrifiadurol School of Computer Science accessing multimedia material created for interpreting training and simulation.  To allow easy scenario management by users who often are not experts in commuting (i.e., can not/should not write code)...  … meaning the creation and modification of existing scenarios in terms of their multimedia content, requiring basic CRUD functionality.  To enable limited dialogue and monologue synthesis, resulting in the enrichment of the corpora with different language combinations of existing scenarios.
  • 6. IVY Virtual Environment (IVY-VE) in a nutshell  A dedicated, adaptive 3D virtual environment for Ysgol Gwyddorau Cyfrifiadurol School of Computer Science o interpreting students and o future clients of interpreters  Supports range of virtual interpreting scenarios (e.g., ‘business meeting’) that can be run in different modes: o Interpreting (& Learning Activity) mode, where students can practice using dialogues and monologues o Exploration mode, where clients can learn about interpreting o Live interaction mode, where both groups can engage in role plays  Uses multilingual video/audio-based content for interpreting scenarios, by adapting existing multimedia corpora from the LLP project BACKBONE and the ELISA corpus, and creating three new corpora in Greek, Russian and Hebrew;  Supported by two sets of pedagogical material for interpreter students and (future) ‘clients’, e.g. awareness-raising and interpreting exercises.
  • 7. School of Computer Science Ysgol Gwyddorau Cyfrifiadurol Project IVY – Scenario forms Dialogue Monologue
  • 8. IVY Virtual World – Second Life  Second Life was chosen as the Virtual World for our first prototype. Ysgol Gwyddorau Cyfrifiadurol School of Computer Science  Exploration of alternatives, such as OpenSim, WebGL and Unity will follow in the future.  Second Life compared to alternatives (OpenSim, ActiveWorls etc) offers: o Large community, various add-ons, plugins and examples of customisations. o A platform for social interaction and education, used by numerous institutions, colleges, universities – thus increasing chances of exposure. o Accessibility via public servers and it does not require that you run the VW yourself.
  • 9. IVY VE – Lack of Instancing & Scalability  Due to limits on the number of primitives available to the IVY Island and the Ysgol Gwyddorau Cyfrifiadurol School of Computer Science lack of instancing mechanisms in SL, IVE-VE uses a collection of unique, in- world locations for each type of scenario (e.g., Classroom, Meeting Office).  Therefore, a scenario may share its location with another, being carried out in a similar setting.  In order to maintain consistency in the virtual world only one scenario can be played per location at a given time.  Upon a scenario launch by a user, all scenarios sharing the same location become unavailable for other users.  Once the user exits the selected scenario, all scenarios sharing the same location become available again.
  • 10. IVY VE – Design Strategies  Script things in Second Life using LSL Ysgol Gwyddorau Cyfrifiadurol School of Computer Science + Comparable aesthetics and better integration with the native SL GUI + Graphics appear crispier, clearer - Scripting can be fairly static and requires coding - Communication with DBs is fairly limited in terms of size of info that can be transmitted.  Rely on web application technologies as interface creation, database connectivity and overall flexibility surpasses LSL. Therefore…  …either embed SL in a web application o Popular notion, no web viewer supported from Linden Labs. o Only Canvas from Tripodean Technologies appears to exist at the moment.  …or ‘embed’ the web application within SL, using HTML on a Prim.
  • 11. IVY VE – Implementation  Our chosen implementation strategy aspires to merge useful features from both Ysgol Gwyddorau Cyfrifiadurol School of Computer Science alternatives, resulting into a hybrid solution.  One module consists of a web application with two entry points. One entry point remains independent of SL and is used by content managers to manage scenarios and users’ information, offering basic CRUD functionality.  The other end is viewable within Second Life, in the form of a ‘Heads-up- Display’ (HUD), populated from a database, showing available scenarios to the users, having a player functionality and initiating in-world teleport events.
  • 12. IVY VE – User Classes and Characteristics  Separating user roles allows controlled access to different parts of the application Ysgol Gwyddorau Cyfrifiadurol School of Computer Science as well as means of monitoring scenario selection and execution.  The system’s user classes and their respective role descriptions are: o Interpreters/Users, whose purpose is to explore, participate and exercise with the scenarios in Second Life. o Observers, whose purpose is to observe other users in Second Life. o Content Managers who are responsible for user and scenario management.
  • 13. IVY VE – The IVY Island  Locations created according to the corpus requirements, trying to keep prim-count Ysgol Gwyddorau Cyfrifiadurol School of Computer Science to a minimum.  Reception serves as a central hub and a HUD pick-up point.
  • 14. IVY VE – Scenario Actors  We populate the scenarios with additional ‘actors’. Currently these actors are Ysgol Gwyddorau Cyfrifiadurol School of Computer Science manually placed in each location, using Second-life ‘robots’ (bots) from Pikkubot and Thoys.  Bots are controlled either by using in-world chat to issue commands directly to the bots or through a dedicated server’s telnet prompt.  We currently use animation overrides, to make them appear life-like.  Our aspiration is to create a service that talks to the bots and relays scenario specific information and teleport commands.  However… sound does not appear to originate from the bots.
  • 15. IVY VE – Web Application  The IVY web application is build using the Appfuse 2 open source project Ysgol Gwyddorau Cyfrifiadurol School of Computer Science (appfuse.org).  Appfuse, built on the Java platform, uses industry-standard features, such as Apache Maven integration, JPA support for database operations and popular web frameworks such as Spring and Apache Struts, employed in this example.  Appfuse comes with out of the box features, needed in IVY-VE such as: o Generic CRUD backend o Authentication and authorization, o User management o Strong Internationalization support  Our prototype is deployed using Apache Tomcat 6.x and uses the MySQL 5.x database.
  • 16. IVY VE – Audio File Management  The IVY-VE uses audio extracts (segments), in MPEG-2 Audio Layer III Ysgol Gwyddorau Cyfrifiadurol School of Computer Science format, from the LLP project BACKBONE, wrapped in XSPF (XML Shareable Playlist Format) play-lists (scripts) and played within pre-fabricated scenes.  Audio segments are uniquely named and can be interchanged — within each script — to derive further language combinations of that scenario.  Actors may speak for more than one consecutive segments, allowing greater flexibility in creating scripts where one talks for extended periods.  However, it is assumed that only one actor talks per audio segment and there is no overlap between actor’s speech.  Each script has textual information associated with it, such as brief content information, scene description and domain keywords.
  • 17. IVY VE – Heads Up Display I  The HUD is built using the jQuery JavaScript library, displaying the list of Ysgol Gwyddorau Cyfrifiadurol School of Computer Science scenarios in the database as a drill-down menu.  It is normally attached to the bottom left corner of the user’s viewport.  Audio is being played by means of a Flash player, parsing the XSPF playlists upon scenario selection.  Navigation through the island is performed using slurls, providing direct url- like teleport links to locations within the virtual world.  Each slurl is being called upon scenario launch, triggering the native SL-client teleport interface.
  • 18. IVY VE – Heads Up Display II  Login, form selection and language combination selection views Ysgol Gwyddorau Cyfrifiadurol School of Computer Science  Free Scenarios views – Scenario Info & Player  Locked Scenarios views
  • 19. IVY VE – Administrator’s Panel I  The management console allows Content Administrators to easily populate the Ysgol Gwyddorau Cyfrifiadurol School of Computer Science scenario database and create XSPF playlists.  New scenarios are created through a form, where administrators provide textual information (title, language combos, participants gender etc.) as well as an ordered list of the audio tracks in each scenario.  Scenario textual info is stored in the DB and XSPF playlists on a separate playlist inventory.  A separate listing of all the scenarios in the system allows administrators to see which user is currently working on each scenario and also fire-up their SL client and teleport to that location.
  • 20. IVY VE – Administrator’s Panel II  Scenario Upload Form Ysgol Gwyddorau Cyfrifiadurol School of Computer Science  Scenario Listing (with Teleport functionality)
  • 21. IVY VE – Preliminary Functional Evaluation  A preliminary evaluation of the current prototype was done by nine Ysgol Gwyddorau Cyfrifiadurol School of Computer Science interpreting and two virtual world experts, using talk-aloud, try-out sessions in Second Life, followed by a discussion with the assessor.  The evaluation focused on the HUD functionality and in-world locations.  The admin panel was not evaluated at this stage; however it has been successfully used, by content administrators, for the past four months to upload scenarios in our system.  Overall opinion was quite positive and users with very limited experience in virtual worlds, gaming or similar environments felt comfortable using IVY-VE.  However, a series of limitations of the current system where pinpointed…
  • 22. IVY VE – Current Limitations  Some users focused too much on the HUD, not paying attention to the world. Ysgol Gwyddorau Cyfrifiadurol School of Computer Science  No sound directionality and no visual cues on who is speaking.  Interpreters were enthusiastic they have a tool – but does that generates bias?  Lots of comments regarding aesthetics of locations – all participants observed and commented on the quality of locations using photorealistic graphics.  No zoning, instancing and replication like in games.  Limitation on available ‘prims’ affects world scalability  … and consequently scenario management and execution, actor placement.  Currently bots are manually placed to required scenarios – nowadays we use many bots and manage locations to cover all gender combinations in our corpus.  Sound is heard only from the player controller and is not broadcast back to the world – hence observers do not hear it.
  • 23. IVY VE – Future Work  Currently undergoing two evaluation cycles, focused on interpreting students and Ysgol Gwyddorau Cyfrifiadurol School of Computer Science potential clients of interpreters during the autumn of 2012 and will attempt to get some feedback on the IVY-VE usability and resulting sense of immersion.  Interpreting mode continuously enhanced with a supporting exercises, both generic to dialogues and monologues, as well as specific to particular scenarios which have an inherit challenge in interpreting practice.  Use student/client feedback and experience from using IVY-VE to design a new bespoke system…  ..while exploring alternative technologies that allow tighter integration with current web-based scenario system, e.g., Unity and WebGL.
  • 24. Ysgol Gwyddorau Cyfrifiadurol IVY VE – Forthcoming Events School of Computer Science  Exploiting Emerging Technologies to Prepare Interpreters and their Clients for Professional Practice, London, 23rd November 2012 For more info visit: Consortium website http://www.virtual-interpreting.net/Seminar.html …or the Bangor IVY partner website http://www.vmg.cs.bangor.ac.uk/IVY/
  • 25. School of Computer Science Ysgol Gwyddorau Cyfrifiadurol Thank you!

Editor's Notes

  1. Slide for SL Conference and VMG only – need to set the scene, something not necessary for our partners!
  2. Slide for SL Conference and VMG only – need to set the scene, something not necessary for our partners!