The improvements in size, features and price of hardware has ushered new opportunities in creating small, smart devices (aka internet-of-things) which can be integrated in homes and industries. In these places, the devices can help automate common tasks, as well as give information about the state of things, such as temperature of a machine, air quality etc. Installing an app for one such devices might seem fine at first, but it doesn’t scale nicely to 100s of devices, or devices you seldom interact with, like in an industrial setting. Devices might even have different security restrictions, like being locked behind a physical key. So how do we communicate with these devices? The web has always been known for its low fraction and easy onboarding. No need to install any software, just type in a URL and off you go. And it has always been very secure with its sandbox system, and companies can even have URLs be restricted to certain WiFi networks (intranet). In the last couple of years, the web has taken a quantum leap in usability, with offline support, and many ways to make the experience very app-like. So the question is unavoidable, can the web be the platform to make smart devices succeed? The web lives in a sandbox, and its security model has allowed people to trust it and for it to grow enormously over time, but the world is changing around us. There is a growing need to access new hardware capabilities such as sensors or just connect to devices around us. The great news is that the web sandbox is growing with new capabilities and with new security models, allowing us to connect to devices via Bluetooth, USB or even talk NFC. There are now even ways to directly get magnetometer readings on Android devices. In this talk we will look at this new landscape and how it enables the new wave of smart devices. We will also look at how easy it is to use some of these new APIs. Come join me for a look at how the web can make your smart devices success

1. Bridging the web and the physical world
Kenneth Rohde Christiansen
Web Platform Architect

2. Hi! I am Kenneth

3. Danish - happiest people in the world
second most

7. Smart devices

8. What makes a thing "smart"?

9. Devices which are complicated or slow
to use, are they really smart?

10. ➫ Solve a real issue
➫ Be there when you need them
➫ Ease to use, with low friction

12. I don't want to download your app,
I just want to do stuff

13. Now! Now! Nowww!

14. With many devices around usapps don't scale well

15. Too many icons!
Lack of overview
Lots of updates pending for stuff I never use!

16. Too many icons!
➫ Lack of overview
➫ Updates pending for stuff I
never use!

17. Let's look at usages

18. There are different common usages
- Everyday use turn on the light!
- Occasionally change temperature of my fridge
- Once paying for parking when travelling

19. Better model?

20. We want low friction and ease of use

23. Focus on speed
HTTP 2.0 push, Service Workers, PRPL pattern, streams, scrolling, CSS Houdini,
Compositing …

24. Fast with Service Worker caching

26. The web is up to task

28. https://www.youtube.com/watch?v=Dsv5NT4PYG0

29. But that can't be web?!
No address bar - super fast - icon on homescreen

30. We call that experience for
Progressive Web Apps
https://developers.google.com/web/progressive-web-apps/

31. Great for one-time use
The low friction of the web

32. Always handy if I want it to be
Add to homescreen

33. No clutter
Looks and acts like an app

34. And safe

35. But...
Copyright Club Skirts Dinah Shore Weekend https://commons.wikimedia.org/wiki/File:White_Diamonds_Party.jpg

36. Devices live outside the
sandbox

37. And yes, the web can do that too!

38. Keep in mind
Users and companies don't care about
how stuff works, it should just work

39. Two great technologies

40. But first, it might make sense to listen
You can actually win stuff!

41. Lars and Coldfront has joined up for a quiz!
More on that later!
WebLight

42. Advantages
Widely used, Low energy, Wireless

43. Advantages
Cheap, Easy to use, Beacon functionality

44. Disadvantages
Security (just radio waves), Not powered,
Stability, Speed

45. https://www.youtube.com/watch?v=fBCPA9gIxlU

47. ➫ Based on Bluetooth 4+ (Low Energy) GATT protocol
➫ Easy to use, modern web API
➫ Uses promises and typed arrays (ie. Uint8Array etc)
➫ No Classic mode support
➫ Only Central, not Peripheral
Chrome only for now

49. Drawing by Noa Lee Shaked

50. An example
HTTPS only
User Gesture required to request device

51. https://www.youtube.com/watch?v=Fb_Pgf3rCdA

52. Requesting device
navigator.bluetooth.requestDevice({ filters: [{ services: ['battery_service'] }] })
.then(device => { /* ... */ })
.catch(error => { console.log(error); });
navigator.bluetooth.requestDevice({
filters: [{ services: [0x1234, 0x12345678, '99999999-0000-1000-8000-00805f9b34fb'] }]
})
.then(device => { /* ... */ })
.catch(error => { console.log(error); });
You may provide either the full Bluetooth UUID or a short 16- or 32-bit form, if not a standardized service
Standardized services are easy to connect to

53. Requesting device
navigator.bluetooth.requestDevice({
filters: [{ name: "Kenneth's robot" }],
optionalServices: ['battery_service']
})
.then(device => { /* ... */ })
.catch(error => { console.log(error); });
You can also search by advertized name. Services can be optional, but needs to be listed if you want to access them when
available

54. BLE Services and Characteristics
GATT (Generic Attribute profile) defines how two BLE devices
transfer data back and forth using concepts known as Services and
Characteristics
➫ Services are logical entities, and contain specific chunks of
data, known as Characteristics
➫ Characteristics encapsulate a single data point
Services and Characterics are identified by 16- or 128 bit UUID's
Characteristic
Characteristic
Characteristic
Characteristic
Characteristic
Service
Service
Profile

55. BLE Services and Characteristics
In JavaScript you can think of a Service being something like
an object - ie. a collection of properties
Characteristics are similar to properties on an object
Characteristic
Characteristic
Characteristic
Characteristic
Characteristic
Service
Service
Profile

56. Connect to a device
navigator.bluetooth.requestDevice({ filters: [{ services: ['battery_service'] }] })
.then(device => {
// Human-readable name of the device.
console.log(device.name);
// Attempts to connect to remote GATT Server.
return device.gatt.connect();
})
.then(server => { /* ... */ })
.catch(error => { console.log(error); });
When you have a device, you can connect to the GATT (Generic Attributes) server from where you can access
Characteristics

57. Deal with Characteristics
.then(service => {
// Getting Battery Level Characteristic...
return service.getCharacteristic('battery_level');
})
.then(characteristic => {
// Reading Battery Level...
return characteristic.readValue();
})
.then(value => {
console.log('Battery percentage is ' + value.getUint8(0));
})
readValue() returns a DataView (part of the Typed Array family), so it is easy to extract values from.

58. Deal with Characteristics
.then(service => service.getCharacteristic('heart_rate_control_point'))
.then(characteristic => {
// Writing 1 is the signal to reset energy expended.
const resetEnergyExpended = Uint8Array.of(1);
return characteristic.writeValue(resetEnergyExpended);
})
.then(_ => {
console.log('Energy expended has been reset.');
})
Writing is easy too!

59. Deal with Characteristics
.then(service => service.getCharacteristic('heart_rate_measurement'))
.then(characteristic => characteristic.startNotifications())
.then(characteristic => {
characteristic.addEventListener('characteristicvaluechanged', handleCharacteristicValueChanged);
console.log('Notifications have been started.');
})
.catch(error => { console.log(error); });
function handleCharacteristicValueChanged(event) {
var value = event.target.value;
console.log('Received ' + value);
// TODO: Parse Heart Rate Measurement value.
// See https://github.com/WebBluetoothCG/demos/blob/gh-pages/heart-rate-sensor/heartRateSensor.js
}
GATT notifications - subscribe to Characteristics changes

60. Bootstrapping - aka. how do people find my site from my device

61. Bluetooth devices can advertize web sites around them!
That is Eddystone

62. Shows as notifications
Click, and they open

63. Shows as notifications
Click, and they open
Can be regular sites
No additional Bluetooth
functionality needed

64. In near future
Clicking on a Physical
Web notification*, will
allow the opened page to
auto connect to device
*Eddystone URL

65. Get started today! (or tomorrow)
https://developers.google.com/web/updates/2015/07/
interact-with-ble-devices-on-the-web

66. Advantages
Widely in use, Cheap, Cabled, More Secure,
Fast, Stable, Can power devices

67. Disvantages
Cabled

68. Safe and secure
USB has no radio interference
USB connector can be locked behind a hatch

69. Recall
Users and companies don't care about
how stuff works - it should just work
Bluetooth and USB coexists just fine!

70. https://medium.com/@kennethrohde/program-a-smart-device-directly-no-install-needed-cd8b29320d76#.nla4jzls9
https://www.youtube.com/watch?v=o7wGt9RfHVA

71. Nice example showing
➫ A device capable of running JS!
➫ Bootstrapping and auto connect
➫ Device being powered over USB

72. Three different transfer types
Bulk, Interrupt, Isochronous

73. Bulk
Bulk transfers are the fastest* but have no guaranteed timing
Printers and most CDC** (e.g serial) use bulk transfers.
* Bulk transfers will use spare un-allocated bandwidth on the bus after all other transactions have been allocated.
If the bus is busy with isochronous and/or interrupt then bulk data may slowly trickle over the bus.
** Communications Device Class

74. Interrupt
Interrupt transfers have guaranteed maximum latency, or time
between transaction attempts
Mice and keyboards use interrupt transfers.

75. Isochronous
Isochronous transfers have guaranteed timing but no error
correcting
Streaming audio and video use isochronous transfers

76. Web USB specific headers
Creates further security and restrictions

77. Web USB specific headers
➫ Optional
➫ Required for bootstrapping popup
➫ Restrict which sites can access the peripheral

79. Issues
The Linux modemmanager (enabled by default on most distros) hijacks (claims for 16sec)
CDC devices unless their VID/PID are blacklisted (like the Arduino Leonardo)
Note: Only affects CDC devices

80. Issues
Windows autoloads drivers for a range of classes, e.g. HID, MSD and CDC (Win 10)
Other Web USB devices additionally requires MS OS descriptors in order to autoload (or
INF file, see below)
Earlier versions (< 10) of Windows, requires a signed "driver", which is basically a signed
INF text file, binding the VIP/PID to usbser.sys (the default USB serial driver)
More info: https://medium.com/@larsgk/web-enabling-legacy-devices-dc3ecb9400ed#.fj7bd1ba3

81. API example
let device;
navigator.usb.requestDevice({ filters: [{ vendorId: 0x2341 }] })
.then(selectedDevice => {
device = selectedDevice;
return device.open(); // Begin a session.
})
.then(() => device.selectConfiguration(1)) // Select configuration #1 for the device.
.then(() => device.claimInterface(2)) // Request exclusive control over interface #2.
.then(() => device.controlTransferOut({
requestType: 'class',
recipient: 'interface',
request: 0x22,
value: 0x01,
index: 0x02
})) // Ready to receive data
.then(() => device.transferIn(5, 64)) // Waiting for 64 bytes of data from endpoint #5.
.then(result => {
let decoder = new TextDecoder();
console.log('Received: ' + decoder.decode(result.data));
})
.catch(error => { console.log(error); });
In many ways similar in spirit to Web Bluetooth, but of course USB specific
try {
let device = await navigator.usb.requestDevice({ filters: [{ vendorId: 0x2341 }] })
await device.open(); // Begin a session.
await device.selectConfiguration(1)) // Select configuration #1 for the device.
await device.claimInterface(2)) // Request exclusive control over interface #2.
await device.controlTransferOut({
requestType: 'class',
recipient: 'interface',
request: 0x22,
value: 0x01,
index: 0x02
});
// Ready to receive data
let result = device.transferIn(5, 64)) // Waiting for 64 bytes of data from endpoint #5.
let decoder = new TextDecoder();
console.log('Received: ' + decoder.decode(result.data));
}
catch(error) {
console.log(error);
}

82. Availability
Works in Chrome today on Linux, Mac and Windows

83. Get started today!
https://developers.google.com/web/updates/2016/03/a
ccess-usb-devices-on-the-web

84. Wow!
But there is more!
Photo by Andrés Þór

85. Many companies often need to
track stuff

86. Bar codes and similar tech
➫ Quickly read and track things and products
➫ Get information about things around you
➫ Quick actions

87. Bluetooth Beacons might be the best solution

88. Or even NFC tags!

89. Or Quick Response codes and barcodes
+ they are cheap and printable

90. Barcode reader
The Shape Detection API covering barcodes, faces and more: https://wicg.github.io/shape-detection-api/
try {
let barcodeDetector = new BarcodeDetector();
// Assuming |theImage| is e.g. a <img> content, or a Blob.
let detectedCodes = await barcodeDetector.detect(theImage);
for (const barcode of detectedCodes) {
const box = barcode.boundingBox;
console.log(`Barcode ${barcode.rawValue} @(${box.x}, ${box.y}) with size ${box.width}x${box.height}`);
}
} catch(err) {
console.error(`Barcode Detection failed, boo: ${err}`);
}
Behind a flag
In Chrome

91. NFC reader/writer
The Web NFC API covers reading from and writing to NFC tags: https://w3c.github.io/web-nfc/
try {
await navigator.nfc.push({
data: [{ recordType: "url", data: "https://w3c.github.io/web-nfc/" }]
});
console.log("Message pushed.");
} catch(err) {
console.log("Push failed :-( try again.");
}
Behind a flag
In Chrome

92. NFC reader/writer
navigator.nfc.watch(message => {
if (message.data[0].recordType == 'empty') {
navigator.nfc.push({
data: [{
recordType: "json",
mediaType: "application/json",
data: { firstName: 'Kenneth' }
}]
});
} else {
console.log(`Read msg written by ${message.url}`;
processMessage(message);
}
});
Behind a flag
In Chrome
function processMessage(message) {
for (let d of message.data) {
if (d.recordType == "json") {
console.log(`Hello ${record.data.firstName}!`);
}
if (d.recordType == "opaque" && d.mediaType == 'image/png') {
const blob = new Blob(d.data, d.mediaType);
const img = document.createElement("img");
img.src = URL.createObjectURL(blob);
img.onload = _ => URL.revokeObjectURL(img.src);
document.querySelector('images').appendChild(img);
}
}
}
}

93. Pretty cool, even Sir Tim Berners-Lee is excited

94. https://my-grocery-plan.glitch.me/
https://www.youtube.com/watch?v=rkjr6tveEuw

95. But what about
Connecting to other devices
to control or receive data

96. Like Sensorial input

97. But my already has sensors!
Gimme gimme access pretty pretty please with sugar on top!

98. Could an API work for
Local and Remote sensors?
Web Platform, Node, IoT?

100. Find sensors hard?
Read my motion sensors explainer
https://w3c.github.io/motion-sensors/
Gravity Sensor
Absolute Orientation Sensor
Linear
Acceleration
Sensor
Geomagnetic Orientation Sensor
Low pass filter
Sensor fusion

101. Very easy to use API
Low-level, so you can do your own filters and sensor fusion
const accel = new Accelerometer({ frequency: 50 });
const gyro = new Gyroscope({ frequency: 50 });
gyro.onreading = _ => { … }
accel.start();
gyro.start();
Behind a flag
In Chrome

102. let timestamp = null;
gyro.onreading = _ => {
let norm = Math.sqrt(accel.x ** 2 + accel.y ** 2) + accel.z ** 2);
let xAccel = (accel.x / norm) * 90;
let yAccel = (accel.y / norm) * -90;
let zAccel = (accel.z / norm);
let xGyro = gyro.x * 180 / Math.PI;
let yGyro = gyro.y * 180 / Math.PI;
let zGyro = gyro.z * 180 / Math.PI;
let dt = timestamp ? ((gyro.timestamp - this.timestamp) / 1000) : 0;
timestamp = gyro.timestamp;
// Complementary filter
// E.g.: new angle = 98% * (current angle + gyro rotation rate) + (2% * Accelerometer angle)
this.alpha = weight * (this.alpha + zGyro * dt) + (1.0 - weight) * zAccel;
this.beta = weight * (this.beta + xGyro * dt) + (1.0 - weight) * xAccel;
this.gamma = weight * (this.gamma + yGyro * dt) + (1.0 - weight) * yAccel;
console.log(`(${this.alpha}, ${this.beta}, ${this.gamma})`);
};

103. Let's look at a video!
➫ Daydream controller exposed via Web Bluetooth
➫ Exposed as Generic Sensors
➫ Try it https://sensor-compass.appspot.com/

104. This is the
Daydream
controller

105. https://www.youtube.com/watch?v=qkvHJf8N7W0

106. What about
Smart devices?

107. That is
quite tiny

108. https://www.youtube.com/watch?v=K0efnqWDrkY

112. Summary
The web is ready for nice app like experiences
⬤ ⬤ ⬤ ⬤

113. Summary
You can effortless connect to devices over Bluetooth
and USB
Chrome for now, but expect other browser vendors to follow suit
⬤ ⬤ ⬤ ⬤

114. Summary
NFC support and barcode readers are coming too!
⬤ ⬤ ⬤ ⬤

115. Summary
Generic Sensors exposes sensors to the web
Plus the API is easy to replace with custom implementation, say, via Web Bluetooth
⬤ ⬤ ⬤ ⬤

116. The web is ready,
let's build the future

117. Follow me on Twitter, where
I will post these slides today
@kennethrohde

118. https://bit.ly/2iU4KtH
WebLight