www.medando.de
Quantified Self
Android Apps for Self Tracking
with Wearables and Health-
Monitoring Devices
Andreas Schreiber <andreas.schreiber@medando.de>
Droidcon Berlin, 08.05.2014
Slide 2 www.medando.deDroidcon Berlin2014, 08.05.2014
My Humble Self
Scientist,
Head of department
Co-Founder, CEO
Co-Founder
Slide 3 www.medando.deDroidcon Berlin2014, 08.05.2014
My Self Tracking
• With sensors
• With smartphone apps
Source: SAT.1/Weckup, http://bit.ly/10CEfUX
Slide 4 www.medando.deDroidcon Berlin2014, 08.05.2014
Slide 5 www.medando.deDroidcon Berlin2014, 08.05.2014
What is The Quantified Self?
Self-knowledge through numbers
• Analyze trends and set goals to improve yourself
Recording of daily activities
• Fitness, sleep, location, …
• Monitoring and display of information from various devices,
services, and applications
Slide 6 www.medando.deDroidcon Berlin2014, 08.05.2014
Other Terms
• Self Tracking
• Life Hacking
• Life Logging
• …
Slide 7 www.medando.deDroidcon Berlin2014, 08.05.2014
Google Trends: “Quantified Self”
Slide 8 www.medando.deDroidcon Berlin2014, 08.05.2014
Google Trends: “Quantified Self”
Slide 9 www.medando.deDroidcon Berlin2014, 08.05.2014
Quantified Self Meetups
http://quantified-self.meetup.com
Slide 10 www.medando.deDroidcon Berlin2014, 08.05.2014
Objects of Tracking
Slide 11 www.medando.deDroidcon Berlin2014, 08.05.2014
Wearable Sensors,
Devices, and Apps
Slide 12 www.medando.deDroidcon Berlin2014, 08.05.2014
Technologies for Self-Tracking
0%
10%
20%
30%
40%
50%
60%
70%
Mobile phones
and apps
Web- and
desktop
applications
Self-tracking
hardware
Self-made
desktop tools
(spreadsheets
etc.)
Pen and paper Other
Deployed technologies for self-tracking
Source: Marcia Nißen, Quantified Self – An Exploratory Study on the Profiles and Motivations of Self-Tracking, Bachelor Thesis (2013)
Slide 13 www.medando.deDroidcon Berlin2014, 08.05.2014
Steps (Fitbit)
Slide 14 www.medando.deDroidcon Berlin2014, 08.05.2014
Weight (Withings)
Slide 15 www.medando.deDroidcon Berlin2014, 08.05.2014
Stress (W/Me)
Slide 16 www.medando.deDroidcon Berlin2014, 08.05.2014
Sleep (Sleep as Android)
Slide 17 www.medando.deDroidcon Berlin2014, 08.05.2014
Blood Pressure (BloodpressureCompanion)
Slide 18 www.medando.deDroidcon Berlin2014, 08.05.2014
Activity & Location (Moves)
Source: WDR/Servicezeit, http://bit.ly/DigitaleSelbstvermessung
Slide 19 www.medando.deDroidcon Berlin2014, 08.05.2014
Activity & Location (Moves)
Slide 20 www.medando.deDroidcon Berlin2014, 08.05.2014
Car (Dash)
Slide 21 www.medando.deDroidcon Berlin2014, 08.05.2014
Mobile Phone Usage & Well-being (Menthal)
Slide 23 www.medando.deDroidcon Berlin2014, 08.05.2014
WeightCompanion
Slide 24 www.medando.deDroidcon Berlin2014, 08.05.2014
Analysis: Blood Pressure vs. Weight
Slide 25 www.medando.deDroidcon Berlin2014, 08.05.2014
Sharing
Slide 26 www.medando.deDroidcon Berlin2014, 08.05.2014
Connecting Devices & Apps
Slide 27 www.medando.deDroidcon Berlin2014, 08.05.2014
The Internet of Things
Slide 28 www.medando.deDroidcon Berlin2014, 08.05.2014
Internet of Things
Billions of devices, sensors, and chips
• Connected physical objects (devices)
• Embedded controllers, sensors, actuators
• Connected via the internet
• Uniquely identified
• They interact
Slide 29 www.medando.deDroidcon Berlin2014, 08.05.2014
Growth
Number of devices connected to the internet grow every day
50.000.000.000 “Things” by 2020
Slide 30 www.medando.deDroidcon Berlin2014, 08.05.2014
Communication
Internet
of Things
Communication
infrastructure
Slide 31 www.medando.deDroidcon Berlin2014, 08.05.2014
MQTT
MQ Telemetry Transport
• Machine-to-machine (M2M) connectivity protocol
• Publish/subscribe messaging
• Expect unreliable networks with low bandwidth and high
latency
• Expect clients with limited processing resources
• Provides Quality of Service, if network allows
• Easy to implement
Slide 32 www.medando.deDroidcon Berlin2014, 08.05.2014
MQTT Broker
MQTT
broker
MQTT
broker
Client
Client
Client
Client
publish
subscribe
topic/subtopic
(optional)
bridge
Client
Slide 33 www.medando.deDroidcon Berlin2014, 08.05.2014
MQTT Protocol
• One-to-many message distribution over TCP/IP
• Notifies if clients disconnect abnormally
• Message format
• Fixed 2-byte header
• Variable header for some message type
• Payload (e.g., the topic or small pieces of data)
Slide 34 www.medando.deDroidcon Berlin2014, 08.05.2014
MQTT Topics
• Messages in MQTT are published on topics
• No need to configure, just publish on it
• Topics are hierarchical, with “/” as separator
my/home/temperature/kitchen
my/home/temperature/livingroom
my/server/temperature
Slide 35 www.medando.deDroidcon Berlin2014, 08.05.2014
MQTT Implementations
Server/Broker
• Mosquitto
• Eclipse Paho
• IBM Websphere MQ
• RSMB
• MQTT.js
• Apache ActiveMQ
• RabittMQ
• HiveMQ
Client Libraries for
• C/C++
• Java
• Python
• Perl
• PHP
• Ruby
• …
http://mqtt.org/wiki/software
Slide 36 www.medando.deDroidcon Berlin2014, 08.05.2014
Open Source Broker Mosquitto
Implemented in C. Fast. Small.
For testing: Publicly available Mosquitto MQTT broker
Slide 37 www.medando.deDroidcon Berlin2014, 08.05.2014
Subscribe
import import paho.mqtt.client as paho
def on_message(mosq, obj, msg):
print(msg.topic + ' ' + str(msg.payload))
mqtt_client = paho.Mosquitto()
mqtt_client.on_message = on_message
mqtt_client.connect('test.mosquitto.org')
mqtt_client.subscribe('#', 0) # all topics
return_code = 0
while return_code == 0:
return_code = mqtt_client.loop()
Slide 38 www.medando.deDroidcon Berlin2014, 08.05.2014
Publish
import paho.mqtt.client as paho
mqtt_client = paho.Mosquitto()
mqtt_client.connect('test.mosquitto.org')
mqtt_client.publish(‘droidcon/demo',
'hello world', 1)
Slide 39 www.medando.deDroidcon Berlin2014, 08.05.2014
MQTT on Android
Eclipse Paho (http://www.eclipse.org/paho)
import org.eclipse.paho.client.mqttv3.MqttClient;
import org.eclipse.paho.client.mqttv3.MqttException;
import org.eclipse.paho.client.mqttv3.MqttMessage;
. . .
client = new MqttClient("tcp://localhost:1883",
"myJavaClient");
client.connect();
MqttMessage message = new MqttMessage();
message.setPayload("Hello World".getBytes());
client.publish("droidcon/demo", message);
client.disconnect();
Slide 40 www.medando.deDroidcon Berlin2014, 08.05.2014
Android Push Notifications
Getting data from Quantified Self gadgets to Android
• The Gadget sends data to “somewhere” in the Cloud
• Withings, Fitbit, etc. provide APIs to access the data
• Register for callbacks to get notifications
• We use a Django app that registers as callback listener and
send MQTT messages on updates
• MQTT Java client on Android receives notifications
Slide 41 www.medando.deDroidcon Berlin2014, 08.05.2014
MQTT Push Notification Architecture
Django
App
MQTT
broker
Gadget
Vendor
(API)
Gadget
Android
phone
publish
notification
measure-
ments
register
phone
register callback
receives callbacks
send messages
Slide 42 www.medando.deDroidcon Berlin2014, 08.05.2014
Implementation & Deployment
• Implementation includes OAuth stuff
• Most complex part was the Java code on Android (error handling
etc.)
• Deployment on Amazon Web Services
Django/Mosquitto
EC2
instance
User/Device
registry
Apps
Withings
Elastic IP
Amazon
CloudWatch
Alarm
SNS developer@med
ando.de
Slide 43 www.medando.deDroidcon Berlin2014, 08.05.2014
Callback Implementation (Withings)
def callback(request):
""" Callback function for Withings notifications. """
. . . # request parameter handling
devices = RegisteredWithingsUser.objects.filter(user_id=user_id)
mqtt_client = MosquittoHandler(len(devices))
for device in devices:
device_id = device.device_id
mqtt_topic = 'medando/weightcompanion/weights/%s/%s' %
(user_id, device_id)
payload = simplejson.dumps({'startdate': startdate, 'enddate': enddate})
mqtt_client.publish(mqtt_topic, payload, 2, True)
mqtt_client.wait()
Slide 44 www.medando.deDroidcon Berlin2014, 08.05.2014
MQTT Messages
medando/weightcompanion/weights/1883073/34bae8cbe8dd92f3 0 {"startdate": "1371856646", "enddate": "1371856647"}
medando/weightcompanion/weights/1791607/898efc38ac5d4211 0 {"startdate": "1372742400", "enddate": "1372742401"}
medando/weightcompanion/weights/1527601/2ebcf034b8585668 0 {"startdate": "1368851117", "enddate": "1368851118"}
medando/weightcompanion/weights/16121/f2a8ca66fd067954 0 {"startdate": "1372750563", "enddate": "1372750564"}
medando/weightcompanion/weights/449599/4d701e076912648f 0 {"startdate": "1372751111", "enddate": "1372751112"}
medando/weightcompanion/weights/642578/b33356881163a389 0 {"startdate": "1370585275", "enddate": "1370585276"}
medando/weightcompanion/weights/2019258/33b1d416aeaec9ef 0 {"startdate": "1371377131", "enddate": "1371377132"}
medando/weightcompanion/weights/2019258/61bdf242b37d8a29 0 {"startdate": "1371377131", "enddate": "1371377132"}
medando/weightcompanion/weights/2019258/61bdf242b37d8a29 0
{"startdate": "1371377131", "enddate": "1371377132"}
Slide 45 www.medando.deDroidcon Berlin2014, 08.05.2014
Notification on Android
Slide 46 www.medando.deDroidcon Berlin2014, 08.05.2014
Status Page
Slide 47 www.medando.deDroidcon Berlin2014, 08.05.2014
Sending MQTT from Android
WeightCompanion
• Experimental feature
• User defines MQTT topic
• Weight as Payload (JSON)
{
"unit":"kg",
"weight":80.1
}
Slide 48 www.medando.deDroidcon Berlin2014, 08.05.2014
Slide 49 www.medando.deDroidcon Berlin2014, 08.05.2014
Conclusions
• Quantified Self community
is growing
• Many devices and apps
• Mobile! Wearable!
• Communication in the IoT
• Messaging with MQTT
• Platform independent push
notification services
Source: Kölner Stadt-Anzeiger, http://bit.ly/JsnQ3s
Slide 50 www.medando.deDroidcon Berlin2014, 08.05.2014
Questions?
Andreas.Schreiber@medando.de
@MedandoDE | @onyame

20140508 quantified self droidcon

  • 1.
    www.medando.de Quantified Self Android Appsfor Self Tracking with Wearables and Health- Monitoring Devices Andreas Schreiber <andreas.schreiber@medando.de> Droidcon Berlin, 08.05.2014
  • 2.
    Slide 2 www.medando.deDroidconBerlin2014, 08.05.2014 My Humble Self Scientist, Head of department Co-Founder, CEO Co-Founder
  • 3.
    Slide 3 www.medando.deDroidconBerlin2014, 08.05.2014 My Self Tracking • With sensors • With smartphone apps Source: SAT.1/Weckup, http://bit.ly/10CEfUX
  • 4.
    Slide 4 www.medando.deDroidconBerlin2014, 08.05.2014
  • 5.
    Slide 5 www.medando.deDroidconBerlin2014, 08.05.2014 What is The Quantified Self? Self-knowledge through numbers • Analyze trends and set goals to improve yourself Recording of daily activities • Fitness, sleep, location, … • Monitoring and display of information from various devices, services, and applications
  • 6.
    Slide 6 www.medando.deDroidconBerlin2014, 08.05.2014 Other Terms • Self Tracking • Life Hacking • Life Logging • …
  • 7.
    Slide 7 www.medando.deDroidconBerlin2014, 08.05.2014 Google Trends: “Quantified Self”
  • 8.
    Slide 8 www.medando.deDroidconBerlin2014, 08.05.2014 Google Trends: “Quantified Self”
  • 9.
    Slide 9 www.medando.deDroidconBerlin2014, 08.05.2014 Quantified Self Meetups http://quantified-self.meetup.com
  • 10.
    Slide 10 www.medando.deDroidconBerlin2014, 08.05.2014 Objects of Tracking
  • 11.
    Slide 11 www.medando.deDroidconBerlin2014, 08.05.2014 Wearable Sensors, Devices, and Apps
  • 12.
    Slide 12 www.medando.deDroidconBerlin2014, 08.05.2014 Technologies for Self-Tracking 0% 10% 20% 30% 40% 50% 60% 70% Mobile phones and apps Web- and desktop applications Self-tracking hardware Self-made desktop tools (spreadsheets etc.) Pen and paper Other Deployed technologies for self-tracking Source: Marcia Nißen, Quantified Self – An Exploratory Study on the Profiles and Motivations of Self-Tracking, Bachelor Thesis (2013)
  • 13.
    Slide 13 www.medando.deDroidconBerlin2014, 08.05.2014 Steps (Fitbit)
  • 14.
    Slide 14 www.medando.deDroidconBerlin2014, 08.05.2014 Weight (Withings)
  • 15.
    Slide 15 www.medando.deDroidconBerlin2014, 08.05.2014 Stress (W/Me)
  • 16.
    Slide 16 www.medando.deDroidconBerlin2014, 08.05.2014 Sleep (Sleep as Android)
  • 17.
    Slide 17 www.medando.deDroidconBerlin2014, 08.05.2014 Blood Pressure (BloodpressureCompanion)
  • 18.
    Slide 18 www.medando.deDroidconBerlin2014, 08.05.2014 Activity & Location (Moves) Source: WDR/Servicezeit, http://bit.ly/DigitaleSelbstvermessung
  • 19.
    Slide 19 www.medando.deDroidconBerlin2014, 08.05.2014 Activity & Location (Moves)
  • 20.
    Slide 20 www.medando.deDroidconBerlin2014, 08.05.2014 Car (Dash)
  • 21.
    Slide 21 www.medando.deDroidconBerlin2014, 08.05.2014 Mobile Phone Usage & Well-being (Menthal)
  • 22.
    Slide 23 www.medando.deDroidconBerlin2014, 08.05.2014 WeightCompanion
  • 23.
    Slide 24 www.medando.deDroidconBerlin2014, 08.05.2014 Analysis: Blood Pressure vs. Weight
  • 24.
    Slide 25 www.medando.deDroidconBerlin2014, 08.05.2014 Sharing
  • 25.
    Slide 26 www.medando.deDroidconBerlin2014, 08.05.2014 Connecting Devices & Apps
  • 26.
    Slide 27 www.medando.deDroidconBerlin2014, 08.05.2014 The Internet of Things
  • 27.
    Slide 28 www.medando.deDroidconBerlin2014, 08.05.2014 Internet of Things Billions of devices, sensors, and chips • Connected physical objects (devices) • Embedded controllers, sensors, actuators • Connected via the internet • Uniquely identified • They interact
  • 28.
    Slide 29 www.medando.deDroidconBerlin2014, 08.05.2014 Growth Number of devices connected to the internet grow every day 50.000.000.000 “Things” by 2020
  • 29.
    Slide 30 www.medando.deDroidconBerlin2014, 08.05.2014 Communication Internet of Things Communication infrastructure
  • 30.
    Slide 31 www.medando.deDroidconBerlin2014, 08.05.2014 MQTT MQ Telemetry Transport • Machine-to-machine (M2M) connectivity protocol • Publish/subscribe messaging • Expect unreliable networks with low bandwidth and high latency • Expect clients with limited processing resources • Provides Quality of Service, if network allows • Easy to implement
  • 31.
    Slide 32 www.medando.deDroidconBerlin2014, 08.05.2014 MQTT Broker MQTT broker MQTT broker Client Client Client Client publish subscribe topic/subtopic (optional) bridge Client
  • 32.
    Slide 33 www.medando.deDroidconBerlin2014, 08.05.2014 MQTT Protocol • One-to-many message distribution over TCP/IP • Notifies if clients disconnect abnormally • Message format • Fixed 2-byte header • Variable header for some message type • Payload (e.g., the topic or small pieces of data)
  • 33.
    Slide 34 www.medando.deDroidconBerlin2014, 08.05.2014 MQTT Topics • Messages in MQTT are published on topics • No need to configure, just publish on it • Topics are hierarchical, with “/” as separator my/home/temperature/kitchen my/home/temperature/livingroom my/server/temperature
  • 34.
    Slide 35 www.medando.deDroidconBerlin2014, 08.05.2014 MQTT Implementations Server/Broker • Mosquitto • Eclipse Paho • IBM Websphere MQ • RSMB • MQTT.js • Apache ActiveMQ • RabittMQ • HiveMQ Client Libraries for • C/C++ • Java • Python • Perl • PHP • Ruby • … http://mqtt.org/wiki/software
  • 35.
    Slide 36 www.medando.deDroidconBerlin2014, 08.05.2014 Open Source Broker Mosquitto Implemented in C. Fast. Small. For testing: Publicly available Mosquitto MQTT broker
  • 36.
    Slide 37 www.medando.deDroidconBerlin2014, 08.05.2014 Subscribe import import paho.mqtt.client as paho def on_message(mosq, obj, msg): print(msg.topic + ' ' + str(msg.payload)) mqtt_client = paho.Mosquitto() mqtt_client.on_message = on_message mqtt_client.connect('test.mosquitto.org') mqtt_client.subscribe('#', 0) # all topics return_code = 0 while return_code == 0: return_code = mqtt_client.loop()
  • 37.
    Slide 38 www.medando.deDroidconBerlin2014, 08.05.2014 Publish import paho.mqtt.client as paho mqtt_client = paho.Mosquitto() mqtt_client.connect('test.mosquitto.org') mqtt_client.publish(‘droidcon/demo', 'hello world', 1)
  • 38.
    Slide 39 www.medando.deDroidconBerlin2014, 08.05.2014 MQTT on Android Eclipse Paho (http://www.eclipse.org/paho) import org.eclipse.paho.client.mqttv3.MqttClient; import org.eclipse.paho.client.mqttv3.MqttException; import org.eclipse.paho.client.mqttv3.MqttMessage; . . . client = new MqttClient("tcp://localhost:1883", "myJavaClient"); client.connect(); MqttMessage message = new MqttMessage(); message.setPayload("Hello World".getBytes()); client.publish("droidcon/demo", message); client.disconnect();
  • 39.
    Slide 40 www.medando.deDroidconBerlin2014, 08.05.2014 Android Push Notifications Getting data from Quantified Self gadgets to Android • The Gadget sends data to “somewhere” in the Cloud • Withings, Fitbit, etc. provide APIs to access the data • Register for callbacks to get notifications • We use a Django app that registers as callback listener and send MQTT messages on updates • MQTT Java client on Android receives notifications
  • 40.
    Slide 41 www.medando.deDroidconBerlin2014, 08.05.2014 MQTT Push Notification Architecture Django App MQTT broker Gadget Vendor (API) Gadget Android phone publish notification measure- ments register phone register callback receives callbacks send messages
  • 41.
    Slide 42 www.medando.deDroidconBerlin2014, 08.05.2014 Implementation & Deployment • Implementation includes OAuth stuff • Most complex part was the Java code on Android (error handling etc.) • Deployment on Amazon Web Services Django/Mosquitto EC2 instance User/Device registry Apps Withings Elastic IP Amazon CloudWatch Alarm SNS developer@med ando.de
  • 42.
    Slide 43 www.medando.deDroidconBerlin2014, 08.05.2014 Callback Implementation (Withings) def callback(request): """ Callback function for Withings notifications. """ . . . # request parameter handling devices = RegisteredWithingsUser.objects.filter(user_id=user_id) mqtt_client = MosquittoHandler(len(devices)) for device in devices: device_id = device.device_id mqtt_topic = 'medando/weightcompanion/weights/%s/%s' % (user_id, device_id) payload = simplejson.dumps({'startdate': startdate, 'enddate': enddate}) mqtt_client.publish(mqtt_topic, payload, 2, True) mqtt_client.wait()
  • 43.
    Slide 44 www.medando.deDroidconBerlin2014, 08.05.2014 MQTT Messages medando/weightcompanion/weights/1883073/34bae8cbe8dd92f3 0 {"startdate": "1371856646", "enddate": "1371856647"} medando/weightcompanion/weights/1791607/898efc38ac5d4211 0 {"startdate": "1372742400", "enddate": "1372742401"} medando/weightcompanion/weights/1527601/2ebcf034b8585668 0 {"startdate": "1368851117", "enddate": "1368851118"} medando/weightcompanion/weights/16121/f2a8ca66fd067954 0 {"startdate": "1372750563", "enddate": "1372750564"} medando/weightcompanion/weights/449599/4d701e076912648f 0 {"startdate": "1372751111", "enddate": "1372751112"} medando/weightcompanion/weights/642578/b33356881163a389 0 {"startdate": "1370585275", "enddate": "1370585276"} medando/weightcompanion/weights/2019258/33b1d416aeaec9ef 0 {"startdate": "1371377131", "enddate": "1371377132"} medando/weightcompanion/weights/2019258/61bdf242b37d8a29 0 {"startdate": "1371377131", "enddate": "1371377132"} medando/weightcompanion/weights/2019258/61bdf242b37d8a29 0 {"startdate": "1371377131", "enddate": "1371377132"}
  • 44.
    Slide 45 www.medando.deDroidconBerlin2014, 08.05.2014 Notification on Android
  • 45.
    Slide 46 www.medando.deDroidconBerlin2014, 08.05.2014 Status Page
  • 46.
    Slide 47 www.medando.deDroidconBerlin2014, 08.05.2014 Sending MQTT from Android WeightCompanion • Experimental feature • User defines MQTT topic • Weight as Payload (JSON) { "unit":"kg", "weight":80.1 }
  • 47.
    Slide 48 www.medando.deDroidconBerlin2014, 08.05.2014
  • 48.
    Slide 49 www.medando.deDroidconBerlin2014, 08.05.2014 Conclusions • Quantified Self community is growing • Many devices and apps • Mobile! Wearable! • Communication in the IoT • Messaging with MQTT • Platform independent push notification services Source: Kölner Stadt-Anzeiger, http://bit.ly/JsnQ3s
  • 49.
    Slide 50 www.medando.deDroidconBerlin2014, 08.05.2014 Questions? Andreas.Schreiber@medando.de @MedandoDE | @onyame