Hci presentation v2.2

Never Stand Still Faculty of Engineering Computer Science and Engineering
Click to edit Present’s Name
Touchless, touch-based and Augmented
Reality-based interactions with bacterial
biofilm images
Mohammadreza Hosseini, Arcot Sowmya
mhosseini, sowmya@cse.unsw.edu.au
Tomasz Bednarz Tomasz.Bednarz@csiro.au

School of Computer Science and EngineeringPage 2
Introduction
• Image review, manipulation within sterile environments,
maintaining boundaries between sterile and non-sterile areas
of work environment, are essential in biology studies
• Remote control and visualization of biomedical images can
reduce direct exposure of researchers to viruses and bacteria
• Investigation on human user interfaces, via touchless
touch- based and Augmented Reality interfaces
• Also study their usability through series of user experiments

Touchless system design and image visualization

Touch based system design

User Experience Design
• 10 participants selected randomly from among scientists and
employees in CSIRO to participate in the experiments
• People from different backgrounds, nationalities and genders
• At the beginning of the experiments every participant introduced
individually to the system by an expert. They could observe the
expert interacting with two systems
• They get enough time to fill the survey forms
• They are also requested to indicate which interface felt more
natural

Survey form1: SUS
• System usability scale (SUS) is a ten-item Linkert scale) with a weighted
scoring range of 0-100, giving a global view of system usability.
• To calculate the SUS score, the score contributions from each item are
summed. Each item score contribution will range from 0 to 4. For items 1, 3,
5, 7, and 9, the score contribution is the scale position minus 1. For items 2, 4,
6, 8 and 10, the contribution is five minus the scale position. Multipling the
sum of the scores by 2.5 provides the overall value of SUS.
• The scores are then converted to a percentile rank using a process called
normalization. The SUS score percentile rank is usually referred as school
grade scale

Survey form2: Self assessment manikin
• Self Assessment Manikin (SAM) is a graphical figure to measure
feelings of pleasure, arousal and dominance
• SAM displays each dimension with a graphical character array along
a continuous nine-point centre movement scale
• For pleasure, SAM shows characters from smiley happy faces to
unhappy and sad faces
• For arousal, SAM displays figures that are very excited with eye
open down to sleepy and bored faces
• For dominance, a very large figure presenting feelings of strength
and being in control, and a small figure showing the feeling of being
controlled or submissive

0 1 2 3 4 5 6 7
F
D
C
B
A
Number of Participants
Grade
Touchless SUS grade scale score
Usability study results disclose that
majority of user found a touchlesss
interaction as an ok (E) or poor (F) form
of interaction

0 1 2 3 4 5
F
D
C
B
A
Number of Participants
Grade Touch based SUS grade scale score
majority of user initiate touch based
interaction as an excellent (A) choice for
interaction

0
10
20
30
40
50
60
70
80
90
100
p1
p2
p3
p4
p5
p6
p7
p8
p9
p10
Individual participant SUS score for two systems
Touchless
Touch based
all participants gave higher scores to touch-
based system in comparison to tuchless
interaction systems.

20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105
Touchless
Touch based
SUS Scores
third quartile and the overall average
Linear (Average)
The overall average score of 86 for
touch-based interaction compare to
62 for touchlesss interaction reveals
the higher usability of touch-based
interaction in comparison to touchless
interaction.

0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Happiness Arousal Dominance
SAM feeling average for both systems
Touchless
Touch based
people feel happier and more in control when
they are interacting with a touchbased
interactive system, but the excitement of using
touchless is higher

Conclusion
• Participant observations during user experiments reveal that moving the
entire hand is not an ideal way to communicate with the system.
• The feeling of tiredness that users experience when using hand gesture
explain why the touchless system is less pleasurable compared with a
touch-based system.
• working with large images where information content is very high is the
major cause of tiredness.
• the feeling of not being in control of the touchless interface is due to the
delay between the hand movement and pointer positioning on the screen.
• Every participant also thought that an iPad is a more natural screen than a
3-meter hemispherical dome.

Augmented reality as a new form of interaction
• To benefit from the capacities of both touchless and touch-based
unimodal interaction. AR is introduced
• AR interaction can enhance understanding of physical objects by
addition of digital information to captured video streams.

Tracking in AR
• Major challenge for every AR application is tracking
• Tracking is about locating position of an object in the video frame
and aligning the virtual information with the user field of view
• To detect the user’s field of view, it is assumed that the handheld
device camera direction is the same as the user field of view
• By detecting features in images received through the camera and
matching them with images stored in database, the field of view
can be computed

System Design

Proposed Tracking
Feature matching between
image from camera and stored
image in database is used to
find the corresponding tapped
bacterium in the image stored in
database
Corresponding bacterium

Information Retrieval
Information from the bacteria in
database is extracted and
translated back to the position of
corresponding bacterium on the
image from camera
Corresponding bacterium

Dimension: Length, Width
The selected bacterium highlighted and information from
database displayed on the screen of handheld device

Feature matching for tracking: Comparing different feature detectors
and descriptors on accuracy and real-time performance
Feature Detector Feature Descriptor
Fast : Using a window of 16 Pixels to
classify whether a pixel is actually a
corner
Brief : A simple comparison of pixel
pairs around feature points.
ORB : A FAST or Harris detector ORB: A BRIEF descriptor with some
hints about the key point orientation
SIFT : Using Scale-Space Pyramid
and DoG to detect feature points
SIFT: A 128 vector of orientation
histogram of pixels around the feature
point
SURF: Using a simplified version of
Laplacian of Gaussian
SURF: Wavelet response in horizontal
and vertical direction

Real-time Performance
• Different combination of feature detector and descriptor used for
designing the tracking algorithm.
• The application runs for 30 seconds and frame frame rates was
recorded for each different combination.

Real-Time Performance
Fast Brief Fast Sift
Fast Surf Orb ORB
SIFT SIFT SURF SURF

Real-Time performance
Detector Descriptor Frame rate
FAST BRIEF 30
FAST SIFT 2-5
FAST SURF 4
ORB ORB 8-10
SIFT SIFT 1
SURF SURF 1
WINNER

Accuracy
• The application accuracy is estimated by measuring the acceptable
range of device rotation.
• The acceptable range is the maximum rotation in every direction
before the application loses the bacterium position between two
consecutive taps
• This is carried out by comparing the positions extracted from inverse
homography of different matching methods with results from SURF
matching inverse homography method in different device
orientations.
• The reason for selecting SURF as the base model is because of its
rotation and scale invariant properties.

Accuracy
Feature
Detector
Feature
Descriptor
Accuracy
FAST BRIEF 15 degree in each direction
FAST SIFT 45 degree in each direction
FAST SURF 90 degree in each direction
ORB ORB 85 degree in each direction
WINNER

Conclusion
• The AR application can run in 30 frame per second using
FAST/BRIEF feature detector and descriptor
• The FAST/BRIEF combination has limited acceptable device
rotation, which drop usability of the application.
• There is a trade-off between accuracy and real-time performance for
AR application in high-dense environment
• Further research is necessary for developing more accurate feature
with real-time capability

Hci presentation v2.2

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (19)

Similar to Hci presentation v2.2

Similar to Hci presentation v2.2 (20)

Recently uploaded

Recently uploaded (20)

Hci presentation v2.2