Declarative Gesture Spotting UsingInferred and Refined Control PointsDepartment of Computer Science, Vrije Universiteit Br...
Gesture Classification                          template                         sample segment             e.g. Dynamic T...
Gesture Classification                          template                         sample segment             e.g. Dynamic T...
Gesture Classification                         4
Gesture Classification                         5
Gesture Classification                         6
Gesture Classification                         7
Gesture Classification                         8
Gesture Classification             Too Complex                           9
Gesture ClassificationStart?                 We need gesture spotting                        End?                         ...
Gesture ClassificationStart?                 We need gesture spotting               reduce amount of work for gesture     ...
Gesture SpottingGoal:   - High Recall   - High PrecisionPopular techniques:   - Hidden Markov Models   - Continuous Dynami...
Our Declarative Gesture Spotting Approach     1. Automatically infer control points           - from one representative sa...
Automatically Infer Control PointsOne representative sample                                        Z gesture              ...
Automatically Infer Control PointsOne representative sample - start with the first point                                  ...
Automatically Infer Control PointsOne representative sample - start with the first point - search for the smallest angle -...
Automatically Infer Control PointsOne representative sample - start with the first point - search for the smallest angle -...
Automatically Infer Control PointsOne representative sample - start with the first point - search for the smallest angle -...
Comprehensible Code GenerationBased on the inferred control points:                                            19
Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2   ?p1 (Point2D) there   is a star...
Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2   ?p1 (Point2D)3   ?p2 (Point2D) ...
Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2   ?p1 (Point2D)3   ?p2 (Point2D) ...
Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2   ?p1 (Point2D)3   ?p2 (Point2D) ...
Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2   ?p1 (Point2D)3   ?p2 (Point2D) ...
Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2   ?p1 (Point2D)3   ?p2 (Point2D) ...
Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2   ?p1 (Point2D)3   ?p2 (Point2D) ...
Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2   ?p1 (Point2D)3   ?p2 (Point2D) ...
Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2   ?p1 (Point2D)3   ?p2 (Point2D) ...
Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2   ?p1 (Point2D)3   ?p2 (Point2D) ...
Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2   ?p1 (Point2D)3   ?p2 (Point2D) ...
A State Machine is Not Enough          template                    continuous stream                                 State...
A State Machine is Not Enough          template               continuous stream                                       Stat...
A State Machine is Not Enough          template               continuous stream                                       Stat...
A State Machine is Not Enough          template               continuous stream                                           ...
A State Machine is Not Enough          template               continuous stream                                           ...
A State Machine is Not Enough          template               continuous stream                                           ...
A State Machine is Not Enough          template               continuous streamState 1              State 2               ...
A State Machine is Not Enough          template               continuous streamState 1              State 2State 3        ...
A State Machine is Not Enough          template               continuous streamState 1              State 2               ...
A State Machine is Not Enough          template                    continuous streamState 1              State 2          ...
Incremental Evaluation of Every Possible Combination        1 (defrule symbol_z        2   ?p1 (Point2D)                  ...
Incremental Evaluation of Every Possible Combination        1 (defrule symbol_z        2   ?p1 (Point2D)        3   ?p2 (P...
Incremental Evaluation of Every Possible Combination            1 (defrule symbol_z            2   ?p1 (Point2D)        1 ...
Incremental Evaluation of Every Possible Combination            1 (defrule symbol_z            2   ?p1 (Point2D)        1 ...
Incremental Evaluation of Every Possible Combination            1 (defrule symbol_z            2   ?p1 (Point2D)          ...
Incremental Evaluation of Every Possible Combination            1 (defrule symbol_z            2   ?p1 (Point2D)          ...
Incremental Evaluation of Every Possible Combination            1 (defrule symbol_z            2   ?p1 (Point2D)          ...
Incremental Evaluation of Every Possible Combination            1 (defrule symbol_z            2   ?p1 (Point2D)          ...
Incremental Evaluation of Every Possible Combination ?p1                                                  49
Incremental Evaluation of Every Possible Combination       ?p1 ?p1                                                  50
Incremental Evaluation of Every Possible Combination   ?p1         ?p1 ?p1                                                ...
Incremental Evaluation of Every Possible Combination   ?p1         ?p1 ?p1                                                ...
Incremental Evaluation of Every Possible Combination   ?p1         ?p1 ?p1                                                ...
Incremental Evaluation of Every Possible Combination               ?p1   ?p1         ?p1 ?p1                              ...
Incremental Evaluation of Every Possible Combination                             ?p2               ?p1   ?p1         ?p1 ?...
Incremental Evaluation of Every Possible Combination                                ?p2               ?p1                 ...
Incremental Evaluation of Every Possible Combination                                 ?p2                     ?p1          ...
Incremental Evaluation of Every Possible Combination                                 ?p2                     ?p1          ...
Incremental Evaluation of Every Possible Combination                                 ?p2                     ?p1          ...
Incremental Evaluation of Every Possible Combination                                       ?p2                     ?p1    ...
Incremental Evaluation of Every Possible Combination                                             ?p2                     ?...
Incremental Evaluation of Every Possible Combination                                            ?p2                    ?p1...
Incremental Evaluation of Every Possible Combination                                            ?p2                    ?p1...
Incremental Evaluation of Every Possible Combination                                          ?p2                  ?p1    ...
Incremental Evaluation of Every Possible Combination                                         ?p2                 ?p1      ...
Incremental Evaluation of Every Possible Combination                                       ?p2               ?p1          ...
Incremental Evaluation of Every Possible Combination                                       ?p2               ?p1          ...
Incremental Evaluation of Every Possible Combination                                       ?p2               ?p1          ...
Incremental Evaluation of Every Possible Combination                                       ?p2               ?p1          ...
Incremental Evaluation of Every Possible Combination                                       ?p2               ?p1          ...
Incremental Evaluation of Every Possible Combination                                       ?p2               ?p1          ...
Incremental Evaluation of Every Possible Combination                                       ?p2               ?p1          ...
Incremental Evaluation of Every Possible Combination                                       ?p2               ?p1          ...
Extending the ‘Flick Right’ Definition       Reducing false positives using negation        ?p1              ?p2          ...
Extending the ‘Flick Right’ Definition       Reducing false positives using negation        ?p1              ?p2          ...
Extending the ‘Flick Right’ Definition          ?p1          ?p2         ?p3                                         76
Extending the ‘Flick Right’ Definition                   ?p1               ?p2                   ?p3(not   (and     (Point...
Extending the ‘Flick Right’ Definition                   ?p1               ?p2                ?p3(not   (and     (Point2D ...
Extending the ‘Flick Right’ Definition          y                   ?p1                ?p2                ?p3          y(n...
Evaluation22   77.50   52.10   78.75   56.5024   83.13   47.16   84.38   52.5326   90.63   42.40   91.25   46.7928   93.75...
Future WorkScale / rotational            Visual 3D gestureinvariant spotting            definitions                       ...
Summary                lhoste@vub.ac.be bderooms@vub.ac.be   bsigner@vub.ac.beAuto infer control points        Generate Sp...
Lode Hoste, Brecht De Roomsand Beat Signer, DeclarativeGesture Spotting UsingInferred and Refined ControlPoints, Proceedin...
Upcoming SlideShare
Loading in …5
×

Declarative Gesture Spotting Using Inferred and Refined Control Points

1,089 views

Published on

Presentation given at ICPRAM 2013, International Conference on Pattern Recognition, Barcelona, Spain, February 2013

ABSTRACT: We propose a novel gesture spotting approach that offers a comprehensible representation of automatically inferred spatiotemporal constraints. These constraints can be defined between a number of characteristic control points which are automatically inferred from a single gesture sample. In contrast to existing solutions which are limited in time, our gesture spotting approach offers automated reasoning over a complete motion trajectory. Last but not least, we offer gesture developers full control over the gesture spotting task and enable them to refine the spotting process without major programming efforts.

Paper:http://www.academia.edu/2157798/Declarative_Gesture_Spotting_Using_Inferred_and_Refined_Control_Points

Published in: Science
  • Be the first to comment

Declarative Gesture Spotting Using Inferred and Refined Control Points

  1. 1. Declarative Gesture Spotting UsingInferred and Refined Control PointsDepartment of Computer Science, Vrije Universiteit Brussel (VUB) Lode Hoste - lhoste@vub.ac.be Brecht De Rooms - bderooms@vub.ac.be Beat Signer - bsigner@vub.ac.be 1
  2. 2. Gesture Classification template sample segment e.g. Dynamic Time Warping 2
  3. 3. Gesture Classification template sample segment e.g. Dynamic Time Warping 3
  4. 4. Gesture Classification 4
  5. 5. Gesture Classification 5
  6. 6. Gesture Classification 6
  7. 7. Gesture Classification 7
  8. 8. Gesture Classification 8
  9. 9. Gesture Classification Too Complex 9
  10. 10. Gesture ClassificationStart? We need gesture spotting End? 10
  11. 11. Gesture ClassificationStart? We need gesture spotting reduce amount of work for gesture classification End? 11
  12. 12. Gesture SpottingGoal: - High Recall - High PrecisionPopular techniques: - Hidden Markov Models - Continuous Dynamic Programming 12
  13. 13. Our Declarative Gesture Spotting Approach 1. Automatically infer control points - from one representative sample - one shot learning 2. Generate declarative codeSpotted? - understandable and extensible spotting definitions 3. Call a gesture classification algorithm 13
  14. 14. Automatically Infer Control PointsOne representative sample Z gesture 14
  15. 15. Automatically Infer Control PointsOne representative sample - start with the first point Z gesture 15
  16. 16. Automatically Infer Control PointsOne representative sample - start with the first point - search for the smallest angle - within a given time-frame Z gesture 16
  17. 17. Automatically Infer Control PointsOne representative sample - start with the first point - search for the smallest angle - within a given time-frame Z gesture 17
  18. 18. Automatically Infer Control PointsOne representative sample - start with the first point - search for the smallest angle - within a given time-frame - refine control points (optionally) Z gesture 18
  19. 19. Comprehensible Code GenerationBased on the inferred control points: 19
  20. 20. Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2 ?p1 (Point2D) there is a start point ?p1 20
  21. 21. Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2 ?p1 (Point2D)3 ?p2 (Point2D) there is a second point ?p1 ?p2 21
  22. 22. Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2 ?p1 (Point2D)3 ?p2 (Point2D) ?p1 ?p24 (test (< ?p1.time ?p2.time)) temporal constraint: that comes after the start point 22
  23. 23. Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2 ?p1 (Point2D)3 ?p2 (Point2D) ?p1 ?p24 (test (< ?p1.time ?p2.time))5 (test (inside_control_point ?p1 ?p2 185 4 76)) spatial constraint: 23
  24. 24. Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2 ?p1 (Point2D)3 ?p2 (Point2D) ?p1 ?p24 (test (< ?p1.time ?p2.time))5 (test (inside_control_point ?p1 ?p2 185 4 76)) 4 185 spatial constraint: .. a circular area with centerpoint x: 185 y:4 (pixels) 24
  25. 25. Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2 ?p1 (Point2D)3 ?p2 (Point2D) ?p1 ?p24 (test (< ?p1.time ?p2.time))5 (test (inside_control_point ?p1 ?p2 185 4 76)) 4 185 spatial constraint: .. a circular area with centerpoint x: 185 y:4 (pixels) and radious 76 25
  26. 26. Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2 ?p1 (Point2D)3 ?p2 (Point2D) ?p1 ?p24 (test (< ?p1.time ?p2.time))5 (test (inside_control_point ?p1 ?p2 185 4 76)) 4 185 26
  27. 27. Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2 ?p1 (Point2D)3 ?p2 (Point2D) ?p1 ?p24 (test (< ?p1.time ?p2.time))5 (test (inside_control_point ?p1 ?p2 185 4 76)) 46 ?p3 (Point2D) 1857 (test (< ?p2.time ?p3.time))8 (test (inside_control_point ?p1 ?p3 15 178 76))9 ?p4 (Point2D)10 (test (< ?p3.time ?p4.time))11 (test (inside_control_point ?p1 ?p4 197 175 76)) ?p3 ?p4 27
  28. 28. Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2 ?p1 (Point2D)3 ?p2 (Point2D) ?p1 ?p24 (test (< ?p1.time ?p2.time))5 (test (inside_control_point ?p1 ?p2 185 4 76))6 ?p3 (Point2D)7 (test (< ?p2.time ?p3.time))8 (test (inside_control_point ?p1 ?p3 15 178 76))9 ?p4 (Point2D)10 (test (< ?p3.time ?p4.time))11 (test (inside_control_point ?p1 ?p4 197 175 76)) each point expressed relative to ?p1 ?p3 ?p4 28
  29. 29. Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2 ?p1 (Point2D)3 ?p2 (Point2D) ?p1 ?p24 (test (< ?p1.time ?p2.time))5 (test (inside_control_point ?p1 ?p2 185 4 76))6 ?p3 (Point2D)7 (test (< ?p2.time ?p3.time))8 (test (inside_control_point ?p1 ?p3 15 178 76))9 ?p4 (Point2D)10 (test (< ?p3.time ?p4.time))11 (test (inside_control_point ?p1 ?p4 197 175 76)) temporal constraints: the points should be ordered ?p3 ?p4 29
  30. 30. Comprehensible Code Generation Based on the inferred control points:1 (defrule symbol_z2 ?p1 (Point2D)3 ?p2 (Point2D) ?p1 ?p24 (test (< ?p1.time ?p2.time))5 (test (inside_control_point ?p1 ?p2 185 4 76))6 ?p3 (Point2D)7 (test (< ?p2.time ?p3.time))8 (test (inside_control_point ?p1 ?p3 15 178 76))9 ?p4 (Point2D)10 (test (< ?p3.time ?p4.time))11 (test (inside_control_point ?p1 ?p4 197 175 76))12 =>13 (call DynamicTimeWarping14 (select-between ?p1.time ?p4.time)15 (gesture-set “CharacterZ”)) ?p3 ?p4 call an external algorithm for classification 30
  31. 31. A State Machine is Not Enough template continuous stream State 1State 1 State 2State 3 State 4 31
  32. 32. A State Machine is Not Enough template continuous stream State 2State 1 State 2State 3 State 4 32
  33. 33. A State Machine is Not Enough template continuous stream State 2State 1 State 2 reset state machine?State 3 State 4 33
  34. 34. A State Machine is Not Enough template continuous stream State 1State 1 State 2 reset state machine?State 3 State 4 34
  35. 35. A State Machine is Not Enough template continuous stream State 1State 1 State 2State 3 State 4 35
  36. 36. A State Machine is Not Enough template continuous stream State 1State 1 State 2State 3 State 4 36
  37. 37. A State Machine is Not Enough template continuous streamState 1 State 2 gesture overshot  missed gestureState 3 State 4 37
  38. 38. A State Machine is Not Enough template continuous streamState 1 State 2State 3 State 4 38
  39. 39. A State Machine is Not Enough template continuous streamState 1 State 2 refused by gesture classificationState 3 State 4 39
  40. 40. A State Machine is Not Enough template continuous streamState 1 State 2 refused by gesture classificationState 3 State 4 overlapping gesture  missed gesture 40
  41. 41. Incremental Evaluation of Every Possible Combination 1 (defrule symbol_z 2 ?p1 (Point2D) 41
  42. 42. Incremental Evaluation of Every Possible Combination 1 (defrule symbol_z 2 ?p1 (Point2D) 3 ?p2 (Point2D) 4 (test (< ?p1.time ?p2.time)) 5 (test (inside_control_point ?p1 ?p2 185 4 76)) 42
  43. 43. Incremental Evaluation of Every Possible Combination 1 (defrule symbol_z 2 ?p1 (Point2D) 1 (defrule symbol_z 2 ?p1 (Point2D) 3 ?p2 (Point2D) 4 (test (< ?p1.time ?p2.time)) 5 (test (inside_control_point ?p1 ?p2 185 4 76)) 43
  44. 44. Incremental Evaluation of Every Possible Combination 1 (defrule symbol_z 2 ?p1 (Point2D) 1 (defrule symbol_z 2 ?p1 (Point2D) 3 ?p2 (Point2D) 4 (test (< ?p1.time ?p2.time)) 5 (test (inside_control_point ?p1 ?p2 185 4 76)) 44
  45. 45. Incremental Evaluation of Every Possible Combination 1 (defrule symbol_z 2 ?p1 (Point2D) 1 (defrule symbol_z 2 ?p1 (Point2D) 3 ?p2 (Point2D) 1 (defrule symbol_z 2 ?p1 (Point2D) 3 ?p2 (Point2D) 4 (test (< ?p1.time ?p2.time)) 5 (test (inside_control_point ?p1 ?p2 185 4 76)) 45
  46. 46. Incremental Evaluation of Every Possible Combination 1 (defrule symbol_z 2 ?p1 (Point2D) 1 (defrule symbol_z 2 ?p1 (Point2D) 3 ?p2 (Point2D) 1 (defrule symbol_z 2 ?p1 (Point2D) 1 (defrule symbol_z 3 ?p2 (Point2D) 2 ?p1 (Point2D) 3 ?p2 (Point2D) 4 (test (< ?p1.time ?p2.time)) 5 (test (inside_control_point ?p1 ?p2 185 4 76)) 46
  47. 47. Incremental Evaluation of Every Possible Combination 1 (defrule symbol_z 2 ?p1 (Point2D) 1 (defrule symbol_z 2 ?p1 (Point2D) 3 ?p2 (Point2D) 1 (defrule symbol_z 2 ?p1 (Point2D) 1 (defrule symbol_z 3 ?p2 (Point2D) 2 ?p1 (Point2D) 3 ?p2 (Point2D) 4 (test (< ?p1.time ?p2.time)) 5 (test (inside_control_point ?p1 ?p2 185 4 76)) 47
  48. 48. Incremental Evaluation of Every Possible Combination 1 (defrule symbol_z 2 ?p1 (Point2D) 1 (defrule symbol_z 2 ?p1 (Point2D) 3 ?p2 (Point2D) 1 (defrule symbol_z 2 ?p1 (Point2D) 1 (defrule symbol_z 3 ?p2 (Point2D) 2 ?p1 (Point2D) 3 ?p2 (Point2D) 4 (test (< ?p1.time ?p2.time)) 5 (test (inside_control_point ?p1 ?p2 185 4 76)) RETE engine takes care of caching intermediate calculations 48
  49. 49. Incremental Evaluation of Every Possible Combination ?p1 49
  50. 50. Incremental Evaluation of Every Possible Combination ?p1 ?p1 50
  51. 51. Incremental Evaluation of Every Possible Combination ?p1 ?p1 ?p1 51
  52. 52. Incremental Evaluation of Every Possible Combination ?p1 ?p1 ?p1 52
  53. 53. Incremental Evaluation of Every Possible Combination ?p1 ?p1 ?p1 53
  54. 54. Incremental Evaluation of Every Possible Combination ?p1 ?p1 ?p1 ?p1 54
  55. 55. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p1 55
  56. 56. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p1 ?p1 56
  57. 57. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p1 ?p3 ?p1 57
  58. 58. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p1 ?p3 ?p1 58
  59. 59. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p1 ?p3 ?p1 59
  60. 60. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p1 ?p1 ?p3 ?p1 60
  61. 61. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p2 ?p1 ?p1 ?p3 ?p1 61
  62. 62. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p2 ?p1 ?p1 ?p3 ?p3 ?p1 62
  63. 63. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p2 Limit calculations to a sliding window ?p1 ?p1 ?p3 ?p3 ?p1 63
  64. 64. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p2 Limit calculations to a sliding window ?p1 ?p1 ?p3 ?p3 64
  65. 65. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p2 Limit calculations to a sliding window ?p1 ?p3 ?p3 65
  66. 66. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p2 Limit calculations to a sliding window ?p1 ?p3 ?p1 ?p3 66
  67. 67. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p2 Limit calculations to a sliding window ?p1 ?p3 ?p1 ?p4 ?p3 67
  68. 68. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p2 Limit calculations to a sliding window ?p3 ?p1 ?p4 ?p3 68
  69. 69. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p2 Limit calculations to a sliding window ?p3 ?p1 ?p4 ?p3 69
  70. 70. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p2 Limit calculations to a sliding window ?p3 ?p1 ?p4 ?p3 70
  71. 71. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p2 Limit calculations to a sliding window ?p2 ?p3 ?p1 ?p4 ?p3 71
  72. 72. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p2 Limit calculations to a sliding window ?p2 ?p3 ?p1 ?p4 ?p3 72
  73. 73. Incremental Evaluation of Every Possible Combination ?p2 ?p1 ?p1 ?p1 ?p2 Limit calculations to a sliding window ?p2 ?p3 ?p1 ?p4 ?p3 73
  74. 74. Extending the ‘Flick Right’ Definition Reducing false positives using negation ?p1 ?p2 ?p3 74
  75. 75. Extending the ‘Flick Right’ Definition Reducing false positives using negation ?p1 ?p2 ?p3 75
  76. 76. Extending the ‘Flick Right’ Definition ?p1 ?p2 ?p3 76
  77. 77. Extending the ‘Flick Right’ Definition ?p1 ?p2 ?p3(not (and (Point2D (y ?b y) (time ?b time)) … there should not be a point where the … (test (< ?b time ?p3.time)) (test (> ?b time ?p1.time)) (test (> (abs (- ?p1.y ?b y)) 245)))) 77
  78. 78. Extending the ‘Flick Right’ Definition ?p1 ?p2 ?p3(not (and (Point2D (y ?b y) (time ?b time)) … there should not be a point where the … (test (< ?b time ?p3.time)) … time lies in between ?p1 and ?p3 … (test (> ?b time ?p1.time)) (test (> (abs (- ?p1.y ?b y)) 245)))) 78
  79. 79. Extending the ‘Flick Right’ Definition y ?p1 ?p2 ?p3 y(not (and (Point2D (y ?b y) (time ?b time)) … there should not be a point where the … (test (< ?b time ?p3.time)) … time lies in between ?p1 and ?p3 … (test (> ?b time ?p1.time)) (test (> (abs (- ?p1.y ?b y)) 245)))) … for which y is greater than 245. 79
  80. 80. Evaluation22 77.50 52.10 78.75 56.5024 83.13 47.16 84.38 52.5326 90.63 42.40 91.25 46.7928 93.75 39.47 94.38 43.2630 97.50 35.37 97.50 39.2932 98.75 32.78 98.75 36.41 • 16 gestures • 1760 gesture samples • 10 subjects 80
  81. 81. Future WorkScale / rotational Visual 3D gestureinvariant spotting definitions 81
  82. 82. Summary lhoste@vub.ac.be bderooms@vub.ac.be bsigner@vub.ac.beAuto infer control points Generate Spotting Code Call classification • Extensive spotting: support for real-time continuous streams. • No gesture overshooting: support for overlapping submatches. • Comprehensible: easy to understand and to correct. • Extensible: allows for expert adjustments. 82
  83. 83. Lode Hoste, Brecht De Roomsand Beat Signer, DeclarativeGesture Spotting UsingInferred and Refined ControlPoints, Proceedings ofICPRAM 2013, InternationalConference on PatternRecognition, Barcelona, Spain,February 2013 83

×