The improvements in size, features and price of hardware has ushered new opportunities in creating small, smart devices (aka internet-of-things) which can be integrated in homes and industries. In these places, the devices can help automate common tasks, as well as give information about the state of things, such as temperature of a machine, air quality etc.
Installing an app for one such devices might seem fine at first, but it doesn’t scale nicely to 100s of devices, or devices you seldom interact with, like in an industrial setting. Devices might even have different security restrictions, like being locked behind a physical key.
So how do we communicate with these devices? The web has always been known for its low fraction and easy onboarding. No need to install any software, just type in a URL and off you go. And it has always been very secure with its sandbox system, and companies can even have URLs be restricted to certain WiFi networks (intranet).
In the last couple of years, the web has taken a quantum leap in usability, with offline support, and many ways to make the experience very app-like. So the question is unavoidable, can the web be the platform to make smart devices succeed?
The web lives in a sandbox, and its security model has allowed people to trust it and for it to grow enormously over time, but the world is changing around us. There is a growing need to access new hardware capabilities such as sensors or just connect to devices around us.
The great news is that the web sandbox is growing with new capabilities and with new security models, allowing us to connect to devices via Bluetooth, USB or even talk NFC. There are now even ways to directly get magnetometer readings on Android devices.
In this talk we will look at this new landscape and how it enables the new wave of smart devices. We will also look at how easy it is to use some of these new APIs.
Come join me for a look at how the web can make your smart devices success
18. There are different common usages
- Everyday use turn on the light!
- Occasionally change temperature of my fridge
- Once paying for parking when travelling
47. ➫ Based on Bluetooth 4+ (Low Energy) GATT protocol
➫ Easy to use, modern web API
➫ Uses promises and typed arrays (ie. Uint8Array etc)
➫ No Classic mode support
➫ Only Central, not Peripheral
Chrome only for now
52. Requesting device
navigator.bluetooth.requestDevice({ filters: [{ services: ['battery_service'] }] })
.then(device => { /* ... */ })
.catch(error => { console.log(error); });
navigator.bluetooth.requestDevice({
filters: [{ services: [0x1234, 0x12345678, '99999999-0000-1000-8000-00805f9b34fb'] }]
})
.then(device => { /* ... */ })
.catch(error => { console.log(error); });
You may provide either the full Bluetooth UUID or a short 16- or 32-bit form, if not a standardized service
Standardized services are easy to connect to
53. Requesting device
navigator.bluetooth.requestDevice({
filters: [{ name: "Kenneth's robot" }],
optionalServices: ['battery_service']
})
.then(device => { /* ... */ })
.catch(error => { console.log(error); });
You can also search by advertized name. Services can be optional, but needs to be listed if you want to access them when
available
54. BLE Services and Characteristics
GATT (Generic Attribute profile) defines how two BLE devices
transfer data back and forth using concepts known as Services and
Characteristics
➫ Services are logical entities, and contain specific chunks of
data, known as Characteristics
➫ Characteristics encapsulate a single data point
Services and Characterics are identified by 16- or 128 bit UUID's
Characteristic
Characteristic
Characteristic
Characteristic
Characteristic
Service
Service
Profile
55. BLE Services and Characteristics
In JavaScript you can think of a Service being something like
an object - ie. a collection of properties
Characteristics are similar to properties on an object
Characteristic
Characteristic
Characteristic
Characteristic
Characteristic
Service
Service
Profile
56. Connect to a device
navigator.bluetooth.requestDevice({ filters: [{ services: ['battery_service'] }] })
.then(device => {
// Human-readable name of the device.
console.log(device.name);
// Attempts to connect to remote GATT Server.
return device.gatt.connect();
})
.then(server => { /* ... */ })
.catch(error => { console.log(error); });
When you have a device, you can connect to the GATT (Generic Attributes) server from where you can access
Characteristics
57. Deal with Characteristics
.then(service => {
// Getting Battery Level Characteristic...
return service.getCharacteristic('battery_level');
})
.then(characteristic => {
// Reading Battery Level...
return characteristic.readValue();
})
.then(value => {
console.log('Battery percentage is ' + value.getUint8(0));
})
readValue() returns a DataView (part of the Typed Array family), so it is easy to extract values from.
58. Deal with Characteristics
.then(service => service.getCharacteristic('heart_rate_control_point'))
.then(characteristic => {
// Writing 1 is the signal to reset energy expended.
const resetEnergyExpended = Uint8Array.of(1);
return characteristic.writeValue(resetEnergyExpended);
})
.then(_ => {
console.log('Energy expended has been reset.');
})
Writing is easy too!
59. Deal with Characteristics
.then(service => service.getCharacteristic('heart_rate_measurement'))
.then(characteristic => characteristic.startNotifications())
.then(characteristic => {
characteristic.addEventListener('characteristicvaluechanged', handleCharacteristicValueChanged);
console.log('Notifications have been started.');
})
.catch(error => { console.log(error); });
function handleCharacteristicValueChanged(event) {
var value = event.target.value;
console.log('Received ' + value);
// TODO: Parse Heart Rate Measurement value.
// See https://github.com/WebBluetoothCG/demos/blob/gh-pages/heart-rate-sensor/heartRateSensor.js
}
GATT notifications - subscribe to Characteristics changes
73. Bulk
Bulk transfers are the fastest* but have no guaranteed timing
Printers and most CDC** (e.g serial) use bulk transfers.
* Bulk transfers will use spare un-allocated bandwidth on the bus after all other transactions have been allocated.
If the bus is busy with isochronous and/or interrupt then bulk data may slowly trickle over the bus.
** Communications Device Class
74. Interrupt
Interrupt transfers have guaranteed maximum latency, or time
between transaction attempts
Mice and keyboards use interrupt transfers.
76. Web USB specific headers
Creates further security and restrictions
77. Web USB specific headers
➫ Optional
➫ Required for bootstrapping popup
➫ Restrict which sites can access the peripheral
78.
79. Issues
The Linux modemmanager (enabled by default on most distros) hijacks (claims for 16sec)
CDC devices unless their VID/PID are blacklisted (like the Arduino Leonardo)
Note: Only affects CDC devices
80. Issues
Windows autoloads drivers for a range of classes, e.g. HID, MSD and CDC (Win 10)
Other Web USB devices additionally requires MS OS descriptors in order to autoload (or
INF file, see below)
Earlier versions (< 10) of Windows, requires a signed "driver", which is basically a signed
INF text file, binding the VIP/PID to usbser.sys (the default USB serial driver)
More info: https://medium.com/@larsgk/web-enabling-legacy-devices-dc3ecb9400ed#.fj7bd1ba3
81. API example
let device;
navigator.usb.requestDevice({ filters: [{ vendorId: 0x2341 }] })
.then(selectedDevice => {
device = selectedDevice;
return device.open(); // Begin a session.
})
.then(() => device.selectConfiguration(1)) // Select configuration #1 for the device.
.then(() => device.claimInterface(2)) // Request exclusive control over interface #2.
.then(() => device.controlTransferOut({
requestType: 'class',
recipient: 'interface',
request: 0x22,
value: 0x01,
index: 0x02
})) // Ready to receive data
.then(() => device.transferIn(5, 64)) // Waiting for 64 bytes of data from endpoint #5.
.then(result => {
let decoder = new TextDecoder();
console.log('Received: ' + decoder.decode(result.data));
})
.catch(error => { console.log(error); });
In many ways similar in spirit to Web Bluetooth, but of course USB specific
try {
let device = await navigator.usb.requestDevice({ filters: [{ vendorId: 0x2341 }] })
await device.open(); // Begin a session.
await device.selectConfiguration(1)) // Select configuration #1 for the device.
await device.claimInterface(2)) // Request exclusive control over interface #2.
await device.controlTransferOut({
requestType: 'class',
recipient: 'interface',
request: 0x22,
value: 0x01,
index: 0x02
});
// Ready to receive data
let result = device.transferIn(5, 64)) // Waiting for 64 bytes of data from endpoint #5.
let decoder = new TextDecoder();
console.log('Received: ' + decoder.decode(result.data));
}
catch(error) {
console.log(error);
}
90. Barcode reader
The Shape Detection API covering barcodes, faces and more: https://wicg.github.io/shape-detection-api/
try {
let barcodeDetector = new BarcodeDetector();
// Assuming |theImage| is e.g. a <img> content, or a Blob.
let detectedCodes = await barcodeDetector.detect(theImage);
for (const barcode of detectedCodes) {
const box = barcode.boundingBox;
console.log(`Barcode ${barcode.rawValue} @(${box.x}, ${box.y}) with size ${box.width}x${box.height}`);
}
} catch(err) {
console.error(`Barcode Detection failed, boo: ${err}`);
}
Behind a flag
In Chrome
91. NFC reader/writer
The Web NFC API covers reading from and writing to NFC tags: https://w3c.github.io/web-nfc/
try {
await navigator.nfc.push({
data: [{ recordType: "url", data: "https://w3c.github.io/web-nfc/" }]
});
console.log("Message pushed.");
} catch(err) {
console.log("Push failed :-( try again.");
}
Behind a flag
In Chrome
92. NFC reader/writer
navigator.nfc.watch(message => {
if (message.data[0].recordType == 'empty') {
navigator.nfc.push({
data: [{
recordType: "json",
mediaType: "application/json",
data: { firstName: 'Kenneth' }
}]
});
} else {
console.log(`Read msg written by ${message.url}`;
processMessage(message);
}
});
Behind a flag
In Chrome
function processMessage(message) {
for (let d of message.data) {
if (d.recordType == "json") {
console.log(`Hello ${record.data.firstName}!`);
}
if (d.recordType == "opaque" && d.mediaType == 'image/png') {
const blob = new Blob(d.data, d.mediaType);
const img = document.createElement("img");
img.src = URL.createObjectURL(blob);
img.onload = _ => URL.revokeObjectURL(img.src);
document.querySelector('images').appendChild(img);
}
}
}
}
101. Very easy to use API
Low-level, so you can do your own filters and sensor fusion
const accel = new Accelerometer({ frequency: 50 });
const gyro = new Gyroscope({ frequency: 50 });
gyro.onreading = _ => { … }
accel.start();
gyro.start();
Behind a flag
In Chrome