Vector-based, Structure Preserving Stroke Gesture Recognition
â˘Download as PPTX, PDFâ˘
0 likesâ˘109 views
Unistroke and multistroke gesture recognizers have always striven to reach some robustness with respect to all variations encountered when people issue gestures by hand on touch surfaces or with sensing devices. For this purpose, successful stroke recognizers rely on a gesture recognition algorithm that satisfies a series of invariance properties such as: stroke-order invariance, stroke-number invariance, stroke direction invariance, position, scale, and rotation invariance. Before initiating any recognition activity, these algorithms ensure these properties by performing several pre-processing operations. These operations induce an additional computational cost to the recognition process, as well as a potential error bias. To cope with this problem, we introduce an algorithm that ensures all these properties analytically instead of statistically based on a vector algebra. Instead of points, the recognition algorithm works on vectors between vectors. We demonstrate that this approach not eliminates the need for these preprocessing operations but also satisfies an entire structure preserving transformation. Paper available at https://dial.uclouvain.be/pr/boreal/en/object/boreal%3A217006
Recommended
More Related Content
Similar to Vector-based, Structure Preserving Stroke Gesture Recognition
Similar to Vector-based, Structure Preserving Stroke Gesture Recognition (20)
More from Jean Vanderdonckt
More from Jean Vanderdonckt (20)
Recently uploaded
Recently uploaded (20)
Vector-based, Structure Preserving Stroke Gesture Recognition
- 1. Vector-based, Structure Preserving Stroke Gesture Recognition DMSVIVAâ2019 (Lisbon, Portugal, July 8th-9th, 2019)
- 2. Vector-based, Structure Preserving Stroke Gesture Recognition DMSVIVAâ2019 (Lisbon, Portugal, July 8th-9th, 2019) Nathan Magrofuoco1, Paolo Roselli2, Jorge Luis Perez-Medina1,3, Jean Vanderdonckt1, Santiago Villarreal1 1LouRIM, UniversitĂŠ catholique de Louvain, Belgium 2UniversitĂ degli Studi di Roma âTor Vergataâ, Roma, Italy 3Universidad de Las AmĂŠricas, Quito, Ecuador
- 3. 3 Background on Stroke Gesture Recognition ⢠Two families of approaches ⢠Specific: tied to a particular gesture set ⢠Machine Learning, SVM, Hidden Markov Models, Neural networks,⌠⢠Two main limitations: ⢠Need to re-train, re-model if gesture set is modified ⢠Overfitting problem ⢠Generic: independent of any gesture set ⢠Nearest-Neighbor-Classification (NNC) ⢠Pattern matching ⢠Two advantages ⢠No need to re-train, re-model is gesture set is modified ⢠No overfitting
- 4. ⢠Invariance properties ⢠Variation of strokes => stroke clustering Stroke invariance = independence of any combination of strokes ⢠Variation of directions => direction interpretation Direction invariance = independence of any direction in gesture recognition
- 5. ⢠Invariance properties ⢠Variation of sampling => re-sampling needed Sampling invariance = independence of any sampling Source: J.O. Wobbrock, A.D. Wilson, Y. Li, Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes, Proc. of UISTâ2007
- 6. ⢠Invariance properties ⢠Variation of scale => different sizes depending on the surface of the platform: rescaling is perhaps needed Scale invariance = independence of any size/scale
- 7. ⢠Invariance properties ⢠Variation of location => different locations depending on the surface of the platform: translation is perhaps needed Translation invariance = independence of any location
- 8. ⢠Invariance properties ⢠Variation of angle => different orientations depending on position Rotation invariance = independence of any rotation
- 9. 9 Nearest-Neighbor-Classification (NNC) ⢠Pre-processing steps to ensure invariance ⢠Re-sampling ⢠Points with same space between: isometricity ⢠Points with same timestamp between: isochronicity ⢠Same amount of points: isoparameterization ⢠Re-Scaling ⢠Normalisation of the bounding box into [0..1]x[0..1] square ⢠Rotation to reference angle ⢠Rotate to 0° ⢠Re-rotating and distance computation ⢠Distance computed between candidate gesture and reference gestures (1-NN)
- 10. 10 Nearest-Neighbor-Classification (NNC) ⢠Two families of approaches ⢠âBetween pointsâ distance ⢠$-Family recognizers: $1, $3, $N, $P, $P+, $V, $Q,⌠⢠Variants and optimizations: ProTractor, Protactor3D,⌠⢠âVector between pointsâ distance ⢠PennyPincher, JackKnife,⌠⢠A third new family of approaches ⢠âVector between vectorsâ distance: this paper!
- 11. 11 ⢠Definition of a basic gesture as a vector ⢠From p1 to p2, create vector đ˘ ⢠From p2 to p3, create vector đŁ ⢠By derivation, create vector đ = đ˘ + đŁ ⢠Note that â(đ˘ + đŁ) is the inverse vector đŁ đ˘ p1 p2 p3 đ˘ đŁ b = đ˘ + đŁ đ = â(đ˘ + đŁ) đŁ đ˘
- 12. 12 ⢠Local Shape Distance between 2 triangles based on similarity This is the only simple formula to compute đ đ đ˘ đŁđ + đ đ˘ + đŁ
- 13. 13 ⢠Step 1. Vectorization for each pair of vectors between three consecutive points p1 p2 p3 p4 p5 p6 q1 q2 q3 q4q5 q6 Training gesture Candidate gesture
- 14. 14 ⢠Step 1. Vectorization for each pair of vectors between three consecutive points p1 p2 p3 p4 p5 p6 q1 q2 q3 q4q5 q6 p1 p2 p3 p4 p5 p6 q4q5 q6 Training gesture Candidate gesture q1 q2 q3
- 15. 15 ⢠Step 2. Mapping candidateâs triangles onto training gestureâs triangles p1 p2 p3 p4 p5 p6 q1 q2 q3 q4q5 q6 p1 p2 p3 p4 p5 p6 q1 q2 q3 q4q5 q6 Training gesture Candidate gesture p1 p2 p3 p2 p3 p4 p3 p4 p5 p4 p5 p6
- 16. 16 ⢠Step 2. Mapping candidateâs triangles onto training gestureâs triangles p1 p2 p3 p4 p5 p6 q1 q2 q3 q4q5 q6 p1 p2 p3 p4 p5 p6 q1 q2 q3 q4q5 q6 Training gesture Candidate gesture p1 p2 p3 p2 p3 p4 p3 p4 p5 p4 p5 p6
- 17. 17 ⢠Step 3. Computation of Local Shape Distance between pairs of triangles p1 p2 p3 p4 p5 p6 q1 q2 q3 q4q5 q6 p1 p2 p3 p4 p5 p6 q1 q2 q3 q4q5 q6 p1 p2 p3 p3 p4 p5 p2 p3 p4 p4 p5 p6 Training gesture Candidate gesture ďp1p2p3, ďq1q2q3 (N)LSD ( ) ( =0.02 ďp2p3p4, ďq2q3q4 ) =0.04 (ďp3p4p5, ďq3q4q5 ) =0.0001 ) ďp4p5p6, ďq3q4q5 ( =0.03
- 18. 18 ⢠Step 4. Summing all individual figures into final one ⢠Step 5. Iterate for every training gesture p1 p2 p3 p4 p5 p6 q1 q2 q3 q4q5 q6 p1 p2 p3 p4 p5 p6 q1 q2 q3 q4q5 q6 p1 p2 p3 p3 p4 p5 p2 p3 p4 p4 p5 p6 Training gesture Candidate gesture ďp1p2p3, ďq1q2q3 (N)LSD ( ) ďp2p3p4, ďq2q3q4 ) ( ) ďp4p5p6, ďq3q4q5 ((ďp3p4p5, ďq3q4q5 ) =0.02 =0.04 =0.0001 =0.03 =0.02+0.04+0.0001+0.03=0.0901 (indicative figures)
- 19. 19 ⢠Normalized Local Shape Distance between 2 triangles based on similarity ⢠LSD is not symmetric => anisotropic distance ⢠NLSD is symmetric => isotropic distance đđżđđˇ đ, đ , đ, đ = đżđđˇ đ đ , đ đ , đ đ , đ đ
- 20. 20 Discussion ⢠Advantages ⢠!FTL is aligned with $P, the state-of-the-art recognizer in terms of recognition rate on this gesture set ⢠!FTL(LSD), resp. (NLSD), is 4, resp. 3 times faster than $P ⢠Position, Scale, Rotation invariances ⢠Are algebraically guaranteed due to vector-based approach ⢠Can be controlled on-demand ⢠No need to perform pre-processing such as ⢠Normalization ⢠Re-scaling ⢠Re-rotation
- 21. Thank you very much for your attention