2. Where were we?● Droidcon 2011, London: Motion recognition on Android devices ● http://mylifewithandroid.blogspot.com/2011/10/my- presentation-about-motion.html● Processing only the accelerometer for motion recognition
3. AccelerationAcceleration caused by Acceleration caused by the changethe change of direction of velocity v1 dV dV v2 v2 v1 ΔV a= Δt
4. Extract motion information from accelerometer data● Accelerometer data is a vector, having 3 axes (x,y,z)● This vector has the following components: ● Gravity acceleration – Pointing toward the center of the Earth – Value of about 10 m/s2 – Thats what we measure when the accelerometer is used to calculate tilt ● Any other acceleration the device is subject to – Added to the gravity acceleration – “Disturbs” tilt measurement in gaming (swift movements cause acceleration) – hence the reason for gyroscopes – Can be used for movement detection
5. Measured acceleration
6. Absolute value● x, y, z: acceleration vector components● g – value of the gravity acceleration (can be approximated as 10) a= √ x +y +z −g 2 2 2
7. Snap – one way accelerating Movement starts: Movement ends: decelerating
8. Droidcon 2011 flashback● Conclusions: ● Power consumption is a problem ● Some neat functionality can be implemented by doing pattern recognition on the acceleration vectors absolute value ● In general case the gravity and motion acceleration components cannot be separated ● You can try to use an additional sensor like the gyro to help the separation
9. Gyroscope● Very new phenomenon as gyroscopes suitable for consumer electronic devices appeared very recently● First appearance: Wii Motion Plus accessory, 2009 June● First Android smart phone: Nexus S (end of 2010)● Pros: ● Not sensitive to gravity● Cons: ● Currently supported only by high-end Android phones ● Drift problems (more about that later)
10. Compass● Measures the device orientation wrt. the magnetic vector of the Earth ● This vector points toward the magnetic center of the Earth – It has a component that points to the magnetic North pole – thats what we use for orientation – Beware of the z component! (also called magnetic inclination). If the device is not held horizontally, the downward vector element influences the measurement● Pros: ● Can be used to deduce gravity, not sensitive to motion acceleration ● Widely available in Android devices● Cons: ● Requires calibration ● Sensitive to metal objects, magnetic fields (e.g. electric motors)
11. This time it is gyroscope only
12. Gyroscope
13. Gyroscope measurement data● Measures rotation around 3 axes● More exactly: measures rotation speed (angular velocity) around the axes Δφ v x= Δt
14. Getting the rotation angle● Get the angle difference Δ φ=v x Δ t● Get the absolute angle φ =φ+Δ φ
15. Drift
16. Noise
17. Gyro as support sensor● Because of accumulating error, gyro alone can be rarely used● But ● The accelerometer has no accumulated error but has the gravity component problem ● The gyro has accumulated error but is not sensitive to gravity● Sensor fusion: the use of multiple sensors so that they compensate each others weaknesses
18. Accelerometer-gyro fusion● The easy way ● Use the virtual sensors that calculate gravity and linear acceleration from multiple sensors● The hard way ● Process raw accelerometer and gyroscope data to yield the motion information you need
19. Virtual sensors Gravity and motion acceleration deduced from the accelerometer and the gyroscope Roll/pitch/yaw from the compass Drift-compensated gyroscope
20. Drift-compensated gyroscope
21. The hard way● Why would you go the hard way? ● Sensor fusion co-processing provided by the phone is not precise enough or can have undesirable properties (like auto-calibration in Nexus S) ● Virtual sensors are not available (is there any such case with gyro-equipped phone?) ● You would like to understand how it works and what to expect from built-in sensor fusion ● Just for the fun of it :-)
22. What we want● Remember: accelerometer measures the sum of gravity and motion acceleration● Kills two use cases: ● If you need device tilt, the motion acceleration component corrupts the measurement ● If you want motion acceleration, it is impossible to subtract the gravity acceleration in a general case● Separate gravity and motion acceleration with the help of the gyroscope
23. Idea
24. Idea in words● Pick a reliable gravity vector measurement (make sure that theres no motion then)● If you detect motion (more about later), rotate the previous gravity vector using the gyroscope data and use it as gravity vector estimation● Subtract this gravity vector estimation from the measured acceleration – this yields the motion acceleration
25. Updating the gravity vector estimation● The gravity vector estimation has to be updated time to time as rotation angle errors accumulate● If we detect an acceleration measurement where there is no motion acceleration, we can take it as new reliable gravity vector estimation● Remember slide #7: if the absolute value of the accelerometer output is close to the Earths gravity, we can assume that theres no motion → the gravity vector estimation can be updated with the current accelerometer output
26. Implementation● Example program: http://www.sfonge.com/forum/topic/example- application-accelerometergyroscope- processing-android
27. Now what?3D linear acceleration signal of a well-known motion
28. Recognizing motion● 3D linear acceleration signals are not so intuitive● Motion recognition: ● Record acceleration pattern of reference motion and compare with these references ● Convert from acceleration domain to something more intuitive like velocity – Accelerometer/gyroscope bias will become linearly growing drift after you integrate the acceleration signal!
29. Walking with swinging hand
30. Walking with steady hand
31. Cutting corners
32. Conclusions● Each sensor has strengths and weaknesses● Combine them and they compensate each other● Some sensor fusion is already built-in● If not → dont worry, come up with your own, its fun!● Motion recognition based on 3D linear acceleration signal is much more exact than doing the same from 1D signal