symbolic object localization
   through active sampling of
         acceleration and sound signatures


       Paul Lukowi...
motivation




“The Phone of the Future”
The Economist,
                            2
02-08 2006
… no, seriously
    healthcare applications
      is a monitoring device on the body?
      Where is it? (pick it up befor...
derived requirements



      infrastructure-less

      cheap/simple sensors

      symbolic location interesting

      ...
approach




           A mobile phone ringing or
           vibrating sounds differently
           depending on where it...
approach
-abstraction-

mechanical stimuli
 in this case:
         – narrow frequency ‘beeps’ (high frequency)
         – ...
low frequency stimulus
-vibration-
vibration acceleration:
  coupled directly to surface
  absorption <=> resonance
  info...
High frequency stimulus
-sound beeps-
  structure specific
  closed vs. open

  material specific
  absorption
    well un...
applying the approach

 two distinct modes:
 specific location mode
       + exact location information
       - need for ...
issues to consider
  • microphone and speaker placement
     – speakers and mics are cheap

  • variations inside a symbol...
recognition method
-features used-
 From over 40 features calculated the following 10 are used:
         •   zero crossing...
recognition method
• sliding window feature extraction
   – Over 30 standard features calculated, 10 used
      • 2 freque...
fusion

     fp sound                      vib sound                vib accel


   extract features                       ...
experiments
data acquisition:
             Nokia 5500 Sport




          recognition method: batch processing


2 distinc...
scenarios                  30 samples per location
                           10 for training 20 for testing




living ro...
abstract classes
surface types:
padding
glass
iron
metal
stone
wood

compartment:
Open/closed (except metal)




 For each...
experiments
-’beeps’ used-
intensity




                           time


                                  500 1000 1500...
audio fingerprint examples


backpack                                drawer




                     intensity


         ...
vibration acceleration
                                   stereo




                                   intensity
 intensi...
fingerprint and vibration sounds




                                        carpet
desk




                             ...
frequency range power
                               and
                               sums power wavelet
               ...
results




          22
… including second best




                          23
over 35 specific location (2nd and 3rd best)




                                         24
confusion matrix (over all third best)

                                     ledges
     radiator          metal table   (...
abstract class results (2nd best)

             iron open




                                    26
… summing up
• best: audio fingerprint

• worst: vibration acceleration
   – dependence of vibration on battery

• quality...
‘Your phone is on the desk!’




                               28
opportunistic activity recognition
(publications, soon: data + code)
http://opportunistic.de/

blog about wearable / ubiqu...
Upcoming SlideShare
Loading in …5
×

symbolic object location

2,092 views

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,092
On SlideShare
0
From Embeds
0
Number of Embeds
38
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

symbolic object location

  1. 1. symbolic object localization through active sampling of acceleration and sound signatures Paul Lukowicz, Kai Kunze 1
  2. 2. motivation “The Phone of the Future” The Economist, 2 02-08 2006
  3. 3. … no, seriously healthcare applications is a monitoring device on the body? Where is it? (pick it up before you go) Supporting elderly and cognitive impaired people MonAMI 3 12 million Euro budget over 4 years
  4. 4. derived requirements infrastructure-less cheap/simple sensors symbolic location interesting the two most likely ‘hits’ help 4
  5. 5. approach A mobile phone ringing or vibrating sounds differently depending on where it is. 5
  6. 6. approach -abstraction- mechanical stimuli in this case: – narrow frequency ‘beeps’ (high frequency) – vibration (low frequency) analysis high frequency stimulus -high frequency response (over microphone) low frequency stimulus – low frequency response (over accelerometer) – high frequency response (over microphone) 6
  7. 7. low frequency stimulus -vibration- vibration acceleration: coupled directly to surface absorption <=> resonance information: fixed vs. free hard vs. elastic vibration sound: sound of device hitting surface depends not only on surface, 7 but on overall structure
  8. 8. High frequency stimulus -sound beeps- structure specific closed vs. open material specific absorption well understood in construction and music 8 Table from Olson, H.: Music, Physics and Engineering. (1967)
  9. 9. applying the approach two distinct modes: specific location mode + exact location information - need for training data abstract location class + training problem avoided - only fuzzy location information 9
  10. 10. issues to consider • microphone and speaker placement – speakers and mics are cheap • variations inside a symbolic location – though luck • number of relevant locations – Room-level location (RF) • sensor requirements – cheap sensors sufficient • complexity – procedure performed seldom 10
  11. 11. recognition method -features used- From over 40 features calculated the following 10 are used: • zero crossing rate • median • variance • 75% percentile • inter quartile range • root mean square • frequency range power • sums power wavelet determinant coefficient • number of peaks • peak height sound fingerprint vibration sound vibration acceleartion 11
  12. 12. recognition method • sliding window feature extraction – Over 30 standard features calculated, 10 used • 2 frequency features • separate classifiers for each stimulus response – C 4.5, Naïve Bayes, KNN, HMMs etc. – comparable results fp sound vib sound vib accel • fusion techniques: extract features extract features extract features sliding window sliding window sliding window – majority decision classification classification classification – lookup table (using Naïve Bayes) (using Naïve Bayes) (using Naïve Bayes) best two classifications from each lookup table (created by training data) 12 Result
  13. 13. fusion fp sound vib sound vib accel extract features extract features extract features sliding window sliding window sliding window classification classification classification (using Naïve Bayes) (using Naïve Bayes) (using Naïve Bayes) best two classifications from each lookup table (created by training data) Result 13
  14. 14. experiments data acquisition: Nokia 5500 Sport recognition method: batch processing 2 distinct experimental setups: specific location scenarios: office, living room, apartment abstract location class driven data collection: furniture store 14
  15. 15. scenarios 30 samples per location 10 for training 20 for testing living room office apartment 15 9 locations 12 locations 11 locations
  16. 16. abstract classes surface types: padding glass iron metal stone wood compartment: Open/closed (except metal) For each type and compartment: 6 different kinds of furniture 12 samples each 16 2 pieces of furniture for training, 4 for testing
  17. 17. experiments -’beeps’ used- intensity time 500 1000 1500 2000 2500 3000 3500 time frequency (Hz) Eight distinct sound beeps from 500 -3500 Hz 17
  18. 18. audio fingerprint examples backpack drawer intensity time time 18
  19. 19. vibration acceleration stereo intensity intensity time bed time using norm of the 3 axis intensity accelerometer 19 time
  20. 20. fingerprint and vibration sounds carpet desk 20
  21. 21. frequency range power and sums power wavelet determinant coefficient parameters adjusted for each frequency ‘beep’ feature vectors … rms1 frp 500 wavelet 500 rms1 frp 500 wavelet 500 21
  22. 22. results 22
  23. 23. … including second best 23
  24. 24. over 35 specific location (2nd and 3rd best) 24
  25. 25. confusion matrix (over all third best) ledges radiator metal table (solid metal, stone) 25
  26. 26. abstract class results (2nd best) iron open 26
  27. 27. … summing up • best: audio fingerprint • worst: vibration acceleration – dependence of vibration on battery • quality of accelerometer / vibration-motor / microphone • improvements: – audio fingerprint – vibration sampling (changing levels) • suitable abstractions 27 • acceptable for users?
  28. 28. ‘Your phone is on the desk!’ 28
  29. 29. opportunistic activity recognition (publications, soon: data + code) http://opportunistic.de/ blog about wearable / ubiquitous computing http://wearcomp.eu/ DEMO? 29

×