Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Sensor fusion between car and smartphone


Published on

Droidcon London 2013 presentation about car sensor access

Published in: Technology, Business

Sensor fusion between car and smartphone

  1. 1. Sensor fusion between car and smartphone Gabor Paller Sfonge Ltd.
  2. 2. Thanks for the memory
  3. 3. Car offers ● Power (no need to worry about battery drain) ● Fixed position (if in car kit) ● Sensors ● More sensors → more exciting applications! ● Sensor fusion: the use of multiple sensors so that they compensate each other's weaknesses
  4. 4. Relevant phone sensors ● ● ● ● GPS – if the phone is placed in the car so that it “sees” the satellites Accelerometer – shaking, vibration, violent movements like aggressive braking or turns Gyroscope – turn detection, support sensor for the accelerometer Compass (magnetometer) – direction (not sensitive to drift), magnetic signature
  5. 5. Relevant car sensors ● ● ● ● Speed – driving style, distance (if integrated by time) RPM – driving style Built-in GPS – not subject to incorrect placement within the car Large number of car diagnostic indicators – remote diagnostic → not the subject of this presentation
  6. 6. Obtaining car sensor data ● ● ● ● OBDII (like OBD-2, On-board diagnostic) Widely deployed (mandatory in the EU since 2001/2004) Developed for diagnostic, not for obtaining traffic data Nevertheless, used for a variety of applications like fleet management
  7. 7. (MirrorLink) ● ● Emerging standard for car infotainment system-phone connection Developed by Car Connectivity Consortium (CCC), ● Supported by some recent phones and cars ● Even the specification costs 150 USD ● Not used in the experiments in this presentation
  8. 8. ELM327 ● ● ● ● ● ELM327 is a popular microcontroller for OBD2 access. Developed by ELM Electronics cloned by many On the car side, it talks OBD2 to the car's on-board computer On the terminal side, it offers a friendly serial interface with AT commands reminiscent of the Hayes modem command set Actual devices often offer USB or Bluetooth interface (anything goes that has serial line support) Bluetooth is ideal for Android device connectivity ● ● Basic Bluetooth example programs like BluetoothChat can be used to communicate with the device almost without changes Example program:
  9. 9. ELM327 sequence ● ATD OK ● Set all to defaults ● ATZ ELM327 v1.4 ● Reset all ● ATE0 OK ● Switch off echo ● ATM0 OK ● Don't store discovered protocols ● ATL0 OK ● Don't send LF after CR ● ATH0 OK ● Don't show OBD headers ● ATS0 OK ● Don't insert spaces into hex digits ● ATSP0 OK ● Discover OBD protocol automatically ● Query current data 0x00 (register support) ● Query current data 0x0D (speed, 0 km/h) ● ● 0100 SEARCHING.. 4100BE3EB811 010D 410D00
  10. 10. Cutting edge technology!
  11. 11. OBDII dongle
  12. 12. Precise fixing is crucial
  13. 13. First use case ● ● Dead reckoning-type navigation when GPS is not available E.g. in tunnels, parking houses, dense urban settings (urban valleys) ...
  14. 14. Sensors used ● ● ● Base coordinate p0 comes from the GPS or from earlier measurement Absolute value of velocity vector (v) comes from the speed sensor of the car Direction of velocity vector ● Gyroscope ● Compass
  15. 15. Gyroscope recap ● ● Measures rotation around 3 axes More exactly: measures rotation speed (angular velocity) around the axes Δφ v x= Δt Δ φ =v x Δ t φ ' =φ + Δ φ
  16. 16. The problem of devices ● Newer phones are not always better ● ● Particularly not their sensors It happens that a new sensor type – like gyroscope – is introduced in good quality ● Then the cost-cutting starts in later, “more advanced” device types
  17. 17. Nexus S gyro Car making 90 degree turns, device horizontal. Red: angular velocity, blue: aggregated rotation degree
  18. 18. Galaxy Nexus gyro Car making 90 degree turns, device horizontal. Red: angular velocity, blue: aggregated rotation degree Origin of the spikes: unknown
  19. 19. Only Nexus S measurements will be used in this presentation
  20. 20. The problem of drift
  21. 21. Compass ● ● Measures the device orientation wrt. the magnetic vector of the Earth This vector points toward the magnetic center of the Earth ● ● It has a component that points to the magnetic North pole – that's what we use for orientation Beware of the z component! (also called magnetic inclination). If the device is not held horizontally, the downward vector element influences the measurement
  22. 22. Car's magnetic field Turning around, out of car Phone is inside the car, ignition on, car makes full circle
  23. 23. Find the offset ● ● ● Identify 3 points on the circle (use the gyroscope to figure out whether there was enough rotation) Calculate the center of the circle Subtract the offset
  24. 24. Move the circle to the center Circle moved to the zero point of the coordinate system (on the x-y plane) Initial circle
  25. 25. Compass vs. gyro Compass Gyro Compass is harder to calibrate and less exact but it has no drift
  26. 26. Tunnels
  27. 27. Tunnels
  28. 28. Effect of external magnetic fields Not called Chain Bridge by chance ...
  29. 29. Parking houses
  30. 30. Parking houses
  31. 31. Urban valleys
  32. 32. Urban valleys No GPS reading in the circle
  33. 33. Results ● ● ● Dead reckoning using speed (from car) and direction (from smartphone) provides reasonable results when GPS is not available Compass and gyroscope need to support each other to obtain sufficiently precise direction vector Compass is an extremely problematic sensor in car/urban environment that needs constant recalibration
  34. 34. Second use case: vibrations ● ● Goal: use the phone's accelerometer for something useful (car does not have any similar sensor) Separating motion acceleration: already discussed here: (can be used to classify driving habits) ● Vibration: ● Damaged road ● Vehicle faults (not discussed here)
  35. 35. bumpy_road
  36. 36. Raw vibration Bumpy road section
  37. 37. After high-pass filter Effects of slow acceleration changes (turns) are removed
  38. 38. Power Bumpy road section
  39. 39. Luckily we have exact position ...
  40. 40. Speed bumps
  41. 41. Drivers slow down ... Speed drops & vibration spike at the same time: speed bump Red: speed, blue: acceleration absolute value
  42. 42. Or they don't ... Not possible to detect Red: speed, blue: acceleration absolute value power
  43. 43. Results ● ● ● Localized road damages cannot always be recognized Damages that span longer distances may be detected more reliably In any case, with a large number of measurements, road damages can be mapped efficiently
  44. 44. Idea of a platform ● Available for Android applications ● Discovers car connectivity options and sensors ● ● ● Notifies applications about the connectivity status and available sensors Allows single or repeated sampling of sensor data Controlling car's systems?
  45. 45. Questions?