Learn about the new Generic Sensor API for the web coming to Chrome soon!

1. Generic Sensors
Kenneth R. Christiansen - @kennethrohde

2. Hi! I am Kenneth
@kennethrohde

4. Our team mission
Move the Web Platform forward
➫ Keep the web open and relevant
➫ Invest in the web, specification work, feature implementation
➫ Works on browsers to add features and optimize so it runs well on our hardware
➫ Ensure good collaboration with our partners
➫ Adopt the web in our strategies
➫ Identify industry trends and participate

5. Motion sensors sense speed, direction, or state of rest.
➫ Device orientation as user input or game controller
➫ VR / AR head mounted display (HMD) tracking
➫ Indoor mapping and navigation
➫ Photo and video capture stabilization
➫ Step counting and monitoring of other physical activities
...and much more!
What are sensors good for?

6. Environmental sensors monitor environmental properties around you.
➫ Night mode
➫ Light levels as user input or game controller
➫ Magnetic switch (like on first version of Google Cardboard)
➫ Altitude (via air pressure)
...and much more!
What are sensors good for?

7. ➫ Sensors exposed to the web in many different ways
○ Geolocation, Acceleration, Device Orientation
➫ Kind of high level API, hard to create derived or fusion sensors
➫ Old fashioned API (we now have ES2015+, promises etc)
➫ Data is exposed differently on different platforms like iOS vs Android
➫ Not designed with security in mind
The starting point

8. ➫ Based on DOM events, inherits DOM event model limitations
➫ Sensors not configurable, e.g. firing frequency not defined
➫ Uses high-level concepts, introduces interoperability issues
➫ No timestamps for readings, impossible to do proper sensor fusion
➫ Difficult to extend and add new sensor types
➫ Privacy & security issues and interoperability issues across browsers
➫ Specification not maintained, known issues cannot be fixed
The starting point: DeviceOrientation/Motion has many issues

9. ➫ More and more sensors in devices, from motion sensors to
environment sensors
➫ Smart devices (aka internet of things) are around us, and many edge
devices are sensors
➫ Web Bluetooth allows talking to these smart devices
➫ Node.js developers also want to talk to sensors, with projects like
Johnny Five - http://johnny-five.io/
The landscape

10. ➫ Create a new well thought out API that solves the problems today
➫ Focus on simplicity and familiarity
➫ Design for the web platform first and foremost but with other cases
like Node.js in mind
➫ Build with security in mind
Make is easy to create new sensors specs and refer to the main
spec for all the details
The idea

11. "The goal of the Generic Sensor API is to promote consistency across sensor
APIs, enable advanced use cases thanks to performant low-level APIs, and
increase the pace at which new sensors can be exposed to the Web by
simplifying the specification and implementation processes."
https://w3c.github.io/sensors/
The idea

12. Base API shown as TypeScript
Any API inherits from this
Simple API

13. Absolute and Relative Orientation Sensors just extend
Simple API

14. Sensors are either low-level, derived or fusion (used data from multiple)
Grouping
Sensor
Accelerometer Gyroscope Ambient Light OrientationSensor ...
AbsoluteOrientationSensor RelativeOrientationSensorLinearAccelerationSensor

15. Simple life cycle - cannot fingerprint or know if options can be followed
before activation
Life cycle

16. ➫ Location tracking
➫ Eavesdropping
➫ Keystroke monitoring
➫ Device fingerprinting
➫ User identifying
Security threads

17. ➫ Secure context
➫ Top-level browsing context
➫ Losing focus
➫ Visibility state
➫ Permission API
Checked by security experts from Intel, Google and the W3C TAG
Mitigation

18. The low level sensors include
➫ Accelerometer
➫ Gyroscope
➫ Magnetometer*
*Not a motion sensor as such, but often used with
Introduction to motion sensors

19. A raw accelerometer sensor measures changes in acceleration in 3 different
directions, but is affected by gravity
The Accelerometer sensor is an inertial-frame sensor
➫ When in free fall, the acceleration is 0 m/s2
in the falling direction
➫ When laying flat on a table, the acceleration in upwards direction will be equal
to the Earth gravity, i.e. g ≡ 9.8 m/s2
Accelerometer

20. Accelerometers are less useful by themselves and often take part in other fusion
sensors, but they do have some purposes like registering shakes, steps and the like
Often for such use-cases the developer is interested in the linear acceleration which is the acceleration
without the gravity, called gravity compensation; or the developer is interested in the isolated gravity, in
order to know the gravity vector, which can be useful for some kinds of sensor fusion like creating a
magnetic compass.
Accelerometer

21. High and low pass filters
For acceleration, you usually care about the big changes and want to avoid noise, like the gravity, thus a
high-pass filter can help isolate the linear acceleration and a low-pass filter can help isolate the gravity. A
low-pass filter can thus be useful for measuring a tilt.
Unfortunately, any high-pass filter or low-pass filter introduces a delay, which may or may not be acceptable.
Accelerometer and filters

22. Low pass: allows low frequencies to pass
High pass: allow high frequencies to pass
A common way to create a low-pass filter is to only use a percentage of the latest value and take the rest
from the existing value.
In a way this means that the filter remembers common values and thus smoothes out uncommon values
which most often are a result of noise. As it uses a big percentage of the existing value, this solution
introduces a delay in registering the actual events.
Accelerometer and filters

23. Example low-pass filter

24. Notice, as accelerometers report acceleration, you need to integrate to get velocity:
v = ∫a×∂t
And again to get position:
x = ∫v×∂t
An integral creates drift, and a double integral amplifies that:
So position from an accelerometer is very imprecise and not very useful.
Accelerometer values

25. Most accelerometers are MEMS (Micro Electro-Mechanical Systems)
How an accelerometer work

26. A gyroscope senses angular velocity, relative to itself, thus it measures its own
rotation, using an inertial force called the Coriolis effect.
Gyroscopes oscillate at relative high frequency in order to measure this and are
thus one of the most power hungry motion sensors.
This also means that they can easily be affected by other vibrations, like a vibration
(rumble) motor or speaker on the same device.
Gyroscope

27. When a mass is moving in a particular direction with a particular velocity and when an external angular rate
will be applied (green arrow) a force will occur (blue and red arrow), which will cause perpendicular
displacement of the mass.
So similar to the accelerometer, this displacement will cause change in capacitance which will be
measured, processed and it will correspond to a particular angular rate.
How a gyroscope work

28. In order to get rotation (angle) from a gyroscope, which senses angular velocity, you need to perform a
single integration.
f ≡ frequency
∫cos(2π×ft)) = (1/(2π×f)) × sin(2π×ft)
But be aware that integration turns noise into drift.
As we see above, the integration gets a 1/f outside, meaning that high frequency (f) noise disappears with
integration, i.e. a noise of frequency will drop by a factor of a 100, but a very low frequency will be amplified,
meaning the gyroscope will drift over time.
Gyroscope values

29. Magnetometers are magnetic field sensors, which means that without any strong
magnetic influence close by, it will sense the Earth’s magnetic field, which more or
less points in the direction of North, but not true North.
Magnetometers are very sensitive to outside influence, like anything on a table that has been slightly
magnetized, and it is even affected by other things inside a device, though the device manufacturer can
compensate for this somewhat. In practise though, these sensors work quite well for most common
use-cases.
The most common use-case for magnetometers are as part of sensor fusion, in order to generate an
Orientation Sensor which is stationary to the Earth plane, or a compass, which is basically the former with
corrections to the declination depending on geolocation position, such that it points to the true North.
Magnetometer

30. Actually almost 90% of the sensors on the market use the Hall Effect
If we bring some magnetic field near a conductive plate with current flowing through it we would disturb the
straight flow and the electrons would deflect to one side of the plate.
That means if we put a meter now between these two sides we will get some voltage which depends from
the magnetic field strength and its direction.
How a Magnetometer work

31. Each sensor has its own issues, such as noise and drift, and often need
some kind of compensation using input from a different sensor.
Put another way, one sensor might not be very precise on its own, but...
The sum of multiple sensory input can be much more stable.
High-level sensors

32. Unfortunately, sensors require power, and the more sensors and the higher the
measuring frequency, higher the power consumption (remember gyroscope)
It is always important to consider the minimum set of sensors which solves a task
satisfactory.
As many devices today can do certain kinds of sensor fusion in hardware, it most
often makes sense to use these from a power and performance point of view.
High-level sensors

33. High-level sensors
Sensor type Underlying physical sensors
Relative Orientation Sensor Accelerometer, Gyroscope, MUST NOT USE Magnetometer
Absolute Orientation Sensor Accelerometer, Magnetometer, AND (when present) Gyroscope
Geomagnetic Orientation Sensor Accelerometer, Magnetometer, MUST NOT USE Gyroscope
Gravity Sensor Accelerometer, Gyroscope
Linear Acceleration Sensor Accelerometer, AND EITHER Gyroscope OR Magnetometer
Below is a list of fusion sensors and what sensors they usually are made up of:

34. The Absolute Orientation Sensor, is one of the common use-cases of a
magnetometer
Represents an orientation stationary (fixed to the magnetic field vector and gravity
vector) to the Earth plane.
As the reference frame of an absolute orientation sensor is stationary, it is not useful as a controller for say
a driving game on a phone, as it would not allow you to move around, even slightly or slowly, without
affecting your driving direction.
Example: Absolute Orientation Sensor

35. By taking the cross product between the the magnetic field vector and gravity
vector (isolated from accelerometer), we get a vector which points East on the
ground plane, using the right hand rule.
Now if we take the cross product between the gravity vector and the newly found
East vector, the resulting vector will point in the northern direction towards the
Earth’s magnetic field.
How is an Absolute Orientation Sensor implemented

36. The illustration below represents the case where the device is at rest and y-axis points towards the North.
The reading from the Magnetometer is {x: 0, y: 11, z: -16} and Accelerometer reports {x: 0.11, y: 0.07, z: 9.81}
acceleration.
The uG is a unit vector representing the gravity
uB represents magnetic field vector
uE = uB × uG points East.
uN = uG × uE points to the northern direction.
That means an Absolute Orientation Sensor is a fusion sensor of the Magnetometer and the Accelerometer, and potentially
the Gyroscope for better isolated gravity.
How is an Absolute Orientation Sensor implemented

37. A Geomagnetic Orientation Sensor, is like a Absolute Orientation Sensor, but
doesn’t use the Gyroscope, which means it uses less power.
This also means that it is more sensitive to shakes and movement.
As the main use-case for a Geomagnetic Orientation Sensor is to create a compass, or use compass
direction within a mapping application, this is not much of a problem since people usually hold the device
steady for these use-cases.
The actual heading (N, S, E, W) can be found by adjusting the rotation vector with the local declination
compensation calculated from the current geolocation position.
Example: Geomagnetic Orientation Sensor

38. On most sensor hubs,
gravity is isolated from the accelerometer using the gyroscope,
linear acceleration is isolated by removing the gravity, from the accelerometer
This avoids the delay which low and high pass filters introduce.
Example: Gravity and Linear Acceleration

39. You can think of a relative orientation sensor as a gyroscope adjusted to the gravity
vector
One way of doing this is using a Kalman filter or complementary filter.
As a complementary filter yields quite good results and is easy to implement in
hardware, this is a common solution.
Example: Relative Orientation Sensor

40. A complementary filter can be thought of as a low-pass filter and high-pass filter in
one, complementing the gyroscope values with the accelerometer values:
θn = α × (θn-1 + ω × ∂t) + (1.0 - α) × a
With α being the weight constant, a the acceleration from accelerometer, ω the
angular velocity from gyroscope and ∂t being the time between measurements.
A common value for α is 0.98, which means that 98% of the weight lays on the
gyroscope measurements.
Complementary filter

41. Complementary
filter
implementation
When rotating you get
most from the gyro
When still you get the
existing values with no
difference in rotation,
so the gravity wins

42. From the example, we notice that the alpha represented the initial heading
orientation.
We also know that this heading might drift over time due to being based on the
gyroscope.
Fun: a device-adjusting, relative orientation sensor

43. In some situations you might want the orientation to drift towards your current
position.
This can be useful for a controller inside a virtual reality environment, where you
want a car to follow the heading of your controller, but you might move and turn
around while playing.
That would more or less work like driving a real car.
Fun: a device-adjusting, relative orientation sensor

44. Changing one line in the above accomplishes that.
This example shows how useful manual fusion can be at times.
With the above 2% of the alpha consists of the value 0. Thus, when the device is being held more or less
steady, the heading will move towards 0, meaning being adjusted to your current device position and not
positioned according to the surroundings.
Fun: a device-adjusting, relative orientation sensor

45. ➫ Many specs
○ https://w3c.github.io/sensors/ Main spec
○ https://w3c.github.io/motion-sensors/ Motion Sensor Explainer
○ https://w3c.github.io/accelerometer/ Accelerometer, Gravity, Linear Acceleration
○ https://w3c.github.io/gyroscope/ Gyroscope
○ https://w3c.github.io/magnetometer/ Magnetometer
○ https://w3c.github.io/orientation-sensor/ Relative and Absolute Orientation
○ https://w3c.github.io/ambient-light/ Ambient Light (lux)
○ https://w3c.github.io/proximity/ Proximity
What exists today?

46. Browser support?

47. ➫ Base spec is implemented in Chrome
➫ Origin Trial is ongoing for motion sensors (ie. no magnetometer)
➫ Some interest from Mozilla
➫ Google's WebVR polyfill now uses the API
➫ There is a polyfill, so you can use it today on iOS!
○ https://github.com/kenchris/sensor-polyfills
Implementation

48. 48
It can take less than
100 lines of code
to add a new
sensor type
Implementation in Chrome

49. 49
Implemented under feature flags
▪ Generic Sensor flag
– Accelerometer
– Gyroscope
– LinearAccelerationSensor
– AbsoluteOrientationSensor
– RelativeOrientationSensor
▪ Generic Sensor Extra Classes flag
– AmbientLightSensor
– Magnetometer
Implementation in Chrome

50. 50
Origin Trial started in
Chrome 63
Ends February 27, 2018
https://bit.ly/OriginTrials
Implementation in Chrome

51. 51
1. Go to https://bit.ly/OriginTrialsSignup to get a token
2. Add the token to your web page
<!-- Origin Trial Token, feature = Generic Sensors, origin = https://example.org, expires ="2018-01-18" -->
<meta http-equiv="origin-trial" data-feature="Generic Sensors" data-expires="2018-01-18" content="...">
3. Add motion-sensors.js polyfill:
4.
5.
<script type="module">
import { Gyroscope, AbsoluteOrientationSensor } from './motion-sensors.js';
</script>
Enabling Generic Sensors for an origin

52. 52
Polyfill works great on iOS and Android

54. 54
➫ Generic Sensor APIs aren’t bound to the DOM
➫ Implementable in embedded environments, like Zephyr.js
○ https://github.com/01org/zephyr.js/blob/master/docs/sensors.md
Embedded applications

55. ➫ Article on Google Developers
○ https://developers.google.com/web/updates/2017/09/sensors-for-the-web
➫ Compass
○ Shows a lot of examples of sensor fusion plus device sensors exposed over Web Bluetooth
○ Has an example of a compass implemented with correction for True North
○ https://sensor-compass.appspot.com/
➫ A multitude of mini demos
○ https://intel.github.io/generic-sensor-demos/
Demos and articles

56. Some fun stuff

57. Let's look at a video!
➫ Daydream controller exposed via Web Bluetooth
➫ Exposed as Generic Sensors
➫ Try it https://sensor-compass.appspot.com/

58. This is the
Daydream
controller

59. https://www.youtube.com/watch?v=qkvHJf8N7W0

60. What about
Smart devices?

61. That is
quite tiny

62. https://www.youtube.com/watch?v=K0efnqWDrkY

66. Thanks for listening!

67. 67
You can participate!
▪ W3C Device and Sensors Working Group
https://www.w3.org/2009/dap/
▪ Mailing List
public-device-apis@w3.org
▪ Generic Sensor issues
https://github.com/w3c/sensors/issues