• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
UsiGesture: an Environment for Integrating Pen-based Interaction in User Interface Development
 

UsiGesture: an Environment for Integrating Pen-based Interaction in User Interface Development

on

  • 972 views

— Several algorithms have been developed for pen-based gesture recognition. Yet, their integration in streamline engineering of interactive systems is bound to several shortcomings: they are hard to ...

— Several algorithms have been developed for pen-based gesture recognition. Yet, their integration in streamline engineering of interactive systems is bound to several shortcomings: they are hard to compare to each other, determining which one is the most suitable in which situation is a research problem, their performance largely vary depending on contextual parameters that are hard to predict, their fine-tuning in a real interactive application is a challenge. In order to address these shortcomings, we developed UsiGesture, an engineering method and a software support platform that accommodates multiple algorithms for pen-based gesture recognition in order to integrate them in a straightforward way into interactive computing systems. The method is aimed at providing designers and developers with support for the following steps: defining a dataset that is appropriate for an interactive system (e.g., made of com¬mands, symbols, characters), determining the most suitable gesture recognition algorithm depending on contextual variables (e.g., user, platform, environment), fine-tuning the parameters of this algorithm by multi-criteria optimization (e.g., system speed and recognition rate vs. human distinguishability and perception), and incorporation of the fine-tuned algorithm in an integrated development environment for engineering interactive systems.

Statistics

Views

Total Views
972
Views on SlideShare
970
Embed Views
2

Actions

Likes
0
Downloads
6
Comments
0

1 Embed 2

https://si0.twimg.com 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    UsiGesture: an Environment for Integrating Pen-based Interaction in User Interface Development UsiGesture: an Environment for Integrating Pen-based Interaction in User Interface Development Presentation Transcript

    • François Beuvens and Jean Vanderdonckt francois.beuvens@uclouvain.be Researcher at LiLab, http://www.lilab.be Université catholique de Louvain (Belgium)17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 1
    • - Context - An open platform for benchmarking gesture recognition algorithms - Including the results in user interfaces - Ongoing and future works17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 2
    • - Context - An open platform for benchmarking gesture recognition algorithms - Including the results in user interfaces - Ongoing and future works17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 3
    • - Scope: 2D pen-based gestures - Needs of pen-based gesture recognition - PDAs – Graffiti - Mouse gestures plug-in for Firefox - Games - …17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 4
    • - No « One fits all » algorithm - Needs of algorithms comparison - Same data and conditions - Data acquisition - Enhancements of existing algorithms17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 5
    • - Context - An open platform for benchmarking gesture recognition algorithms - Including the results in user interfaces - Ongoing and future works17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 6
    • 17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 7
    • - 30 participants - 16 men and 14 women with average age of 28 - 60 gestures to reproduce - 26 lower case letters, 10 digits, 16 action commands, 8 geometrical shapes - 10 reproductions of each gesture by participant - More information: https://sites.google.com/site/comparativeevaluation/data-collection17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 8
    • - Training types - User dependent, user independent (2 methods) - Gesture types - Letters, digits, actions, geometrical shapes - Algorithms - Rubine, One Dollar, Levenshtein, stochastic Levenshtein17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 9
    • - Training - Compute mean vector of features and weights for learning classes - Recognition - Compute vector of features - Scalar product of weights and vector - Best score is chosen17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 10
    • - Training - Preprocessing: resampling, rotation, rescaling, translation - Recognition - Preprocessing - Comparison to all training examples - Finding optimal angle - Evaluating point-to-point Euclidean distance - Smaller distance chosen17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 11
    • - Training - Transformation into strings - Recognition - Comparison to all training examples - Computing Levenshtein distance - Smaller distance chosen17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 12
    • - Training - Transformation into strings - Creation of a string pairs set - Optimization of the primitive edition costs - Expectation-maximization algorithm - Recognition - Comparison to all training examples - Evaluating distance based on probabilities - Smaller distance chosen17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 13
    • - Training types for Levenshtein on digits17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 14
    • - Gesture types for Rubine with user dependent training and user independent training17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 15
    • - Algorithms on digits and actions with user independent training17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 16
    • - Summary User-dep. User-indep. 1 User-indep. 2 Digits $1 $1 $1 Letters $1 $1 $1 Actions Sto. Lev. Rub. Sto. Lev. Shapes Rub. Rub. $1 - More Information: https://sites.google.com/site/comparativeevaluation/results17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 17
    • - Rubine - Preprocessings - One Dollar - kNN - Levenshtein - Preprocessings, kNN, normalisations, multi-strokes gestures, other features handling, costs matrix - Stochastic Levenshtein - Idem except costs matrix17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 18
    • - Preprocessings: resampling, rotation, rescaling17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 19
    • - kNN17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 20
    • - Normalizations, pre-processings, kNN, multi- strokes, pressure inclusion, costs17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 21
    • - Normalizations, preprocessings, kNN, multi- strokes, pressure inclusion17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 22
    • - Summary Rub. $1 Lev. Sto. Lev. Resampling + X ++ ++ Rescaling + X X X Rotation - X - - kNN X - - - Multi-stroke X X + = Multi-stroke + X X ++ ++ resampling Pressure X X -- - Normalization 1 X X + - Normalization 2 X X = - Normalization 3 X X = - Adapted costs matrix X X + X - More Information: https://sites.google.com/site/comparativeevaluation/results17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 23
    • - Context - An open platform for benchmarking gesture recognition algorithms - Including the results in user interfaces - Ongoing and future works17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 24
    • 17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 25
    • 17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 26
    • 17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 27
    • - Context - An open platform for benchmarking gesture recognition algorithms - Including the results in user interfaces - Ongoing and future works17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 28
    • - Theoretical - Conceptual modeling of gesture-based UIs - Methodological - Integration into UsiXML - Definition of development lifecycle (e.g. SPEM) - Empirical - User study on user satisfaction with gesture integration - Development benefits estimation17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 29
    • 17/05/2012 RCIS’12: Sixth International Conference on Research Challenges in Information Science 30