syntherella feedback synthesizer

Syntherella: A Feedback Synthesizer forEfficient Exploration of Virtual Worlds using a Screen Reader
BugraOktay and EelkeFolmer
Player-Game Interaction Lab

Outline
Intro
Problem
Syntherella
Discussion
Future Work

Introduction
Virtual World
Screen Reader

Introduction
Virtual World
TextSL

Problem
Densely Populated Environments
Lots of Information to consume
Easy with visual perception; overwhelming when transformed into text
overwhelming

Problem (cnt’d)
Command-line interface
“describe”
“describe objects”
Iterative Queries
“describe table”
Slow interaction
(even when there are very few objects)
“zzz… zzz…”
Consumable

Syntherella
There are 5 animals, 4 tools, a house and a bike.
There are 2 books, a cat, a dog and a house to your west; a dog, a bike … to your east; ... 2 cats are described as “Angora cat with blue eyes”. One of them is interactive and for sale for $20 ...

Syntherella: Scanning
Scanning objects inside a virtual circle around the user’s avatar where the radius is defined by the user.

Syntherella: Filtering
Filtering objects that do not contribute to the textual description of the environment (i.e. objects with non-descriptive or meaningless names).

Syntherella: Thresholding
Good enough
74
table
cat
laptop
book
30
Size / Distance /
Name / Description /
Interaction Capabilities
undescriptive
box
too small
stone
too far
cat
16

Syntherella: Grouping
book
book
cat
book
chair
cat
chair
table
chair
table
book
chair
cat
cat
chair
chair
chair
cat
cat
chair
cat, book, chair, chair, cat, table, chair, book, chair and cat.
3 cats, 2 books, 4 chairs and table.

Syntherella: Aggregation
There are
object
11 objects.
animal
vehicle
…
2 bikes and 9 animals.
mammal
reptile
ground vehicle
2 bikes, 8 mammals and a snake.
…
more specific
cat
dog
snake
bike
2 bikes, 3 cats, 5 dogs and a snake.

Syntherella: Detailing
There are 2 cats to your north; 1 cat to your south and 2 chairs to your east.
Level 2:3 cats are described as “Angora cat with blue eyes”. 2 chairs are described as “black leather office chair”.
Level 3:3 cats are described as “Angora cat with blue eyes”. 2 of them are interactive. 1 of them is not interactive. 2 chairs are described as “black leather office chair”. They are not interactive.
Level 4:3 cats are described as “Angora cat with blue eyes”. 1 of them is interactive but not for sale. 1 of them is interactive and for sale for L300. 1 of them is not interactive and not for sale. 2 chairs are described as “black leather office chair”. They are not interactive but for sale for L20.

Syntherella: Example
“describe objects”
There are 936 objects: window, object, prim, box_by_vi$boyz, chair, tree, flowerz, book v2.0, stone, dog, cat, book, window, window, object, object, cat, no_name, obj23, owl, table, window, object, notecard dispenser, SL educational places kiosk, landmark_3.4, buy_me_L34, object, window, table, chair, chair, car, chair, bicycle, rabbit, rose, chair, chalk, object, skidzPrimz, object4, box7, window, door, tableXY, object, object, paper clip v4.0, lcd screen with remote, voice activated robot helper, phone booth, object, skidzPrimz, object4, box7, window, door, tableXY, object, object, paper clip v4.0, lcd screen with remote, voice activated robot helper, phone booth, window, object, prim, box_by_vi$boyz, chair, tree, flowerz, book v2.0, stone, dog, cat, book, window, window, object, object, cat, no_name, obj23, owl, table, window, object, notecard dispenser, SL educational places kiosk, landmark_3.4, buy_me_L34, object, window, table, chair, chair, car, chair, bicycle, rabbit, rose, chair, chalk, object, skidzPrimz, object4, box7, window, door, tableXY, object, object, paper clip v4.0, lcd screen with remote, voice activated robot helper, phone booth, object, skidzPrimz, object4, box7, window, door, tableXY, object, object, paper clip v4.0, lcd screen with remote,
Pretty long

After scanning
There are 87 objects: frame, object, dog, step, object, object, object, cat, prim, box_by_vi$boyz, object, stone, tree, rose, chair, object, object, frame, nozd_jkie&&jack, stone, step, dog, object, object, object, object, chair, table, cat, object, prim, prim, tree, prim, box, Julia’s Gull, cat, chair, object, object, object, object, object, prim, prim, object, stone, step, object, chair, table, cat, object, prim, prim, tree, prim, box, cat, chair, object, object, object, object, object, prim, prim, object, stone and step.
After filtering
There are 26 objects: frame, dog, step, cat, stone, frame, tree, rose, chair, frame, stone, dog, chair, table, cat, cat, chair, stone, step, stone, tree, chair, stone, step, Julia’s Gull and frame.

After thresholding
There are 14 objects: dog, cat, tree, rose, chair, dog, chair, table, cat, cat, chair, tree, chair and Julia’s Gull.
After grouping
There are 14 objects: 2 dogs, 3 cats, 2 trees, rose, 4 chairs, table and Julia’s Gull.
After aggregation (word limit: 14)
There are 3 plants, 5 animals, 5 furniture and Julia’s Gull.
After aggregation (word limit: 10)
There are 13 known and 1 unknown objects.

After detailing
There are 2 dogs, 1 cat, 2 trees to your north; 4 chairs, 1 table, 1 rose to your west and 2 cats, Julia’s Gull to your south. 2 dogs are described as “black and brown Saint Bernard”. 3 cats are described as “Angora cat with blue eyes”. 1 tree is described as “large oak tree”. 1 tree is described as “green and tall palmae tree”. 4 chairs are described as “brown stylish dining chair”. Rose is described as “Red rose”. Table is described as “diner table for 4”. Julia’s Gull is described as “Laughing gull that can fly; designed by Julia Doe”.

Discussion & Future Work
http://ear.textsl.org
Information Source
Perceptional Challenges
User Studies
Performance Measurements
Widening the Scope (augmented reality, navigation, web accessibility etc.)
Goal/Intention Oriented Feedback Generation
Exploiting Command History

Discussion & Future Work
http://ear.textsl.org
Information Source
Perceptional Challenges
User Studies
Performance Measurements
Widening the Scope (augmented reality, navigation, web accessibility etc.)
Goal/Intention Oriented Feedback Generation
Exploiting Command History
bugra@cse.unr.edu

syntherella feedback synthesizer

by eelke folmer on Jul 11, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

syntherella feedback synthesizer — Presentation Transcript