People as
Sensors
Tom Raftery
ThingMonk,
London Dec 2013.

1
Tom Raftery
•

Lead analyst, energy and
sustainability practice,
RedMonk

•

GreenMonk.net

•

twitter.com/tomraftery

•

tom@redmonk.com

•

+34 677 695 468

•

SlideShare.net/TomRaftery

2
Mobile data

http://www.flickr.com/photos/traftery/8551389911/

3
Mobile phones with GPS, accelerometers, compass, M7, barometers, moisture, etc
Mobile data

http://www.zeit.de/datenschutz/malte-spitz-data-retention

4
Mobile phones with GPS and data retention laws
Green party sued Deutsche Telecom for data
Put it online
Source of concern
Social Media

5

Mobile is not the only data we’re broadcasting
Facebook

http://newsroom.fb.com/Key-Facts

6

1.19 billion monthly active users (MAU) as of September 30, 2013
727 million daily active users (DAU) in Sept 2013
874 million mobile MAUs as of September 30 2013
Facebook

http://techcrunch.com/2013/05/17/facebook-growth/

7
Most recent usage data is from May 2013 - so well out-of-date
4.75 billion content items shared per day (status updates + wall posts + photos + videos + comments)
4.5 billion Likes per day
Twitter

http://hide.dyndns.info/tweetcounter/index-en.cgi

8
Between 4,000-8,500 tweets per second per day
Avg over 6,000 tweets per sec, all stored in perpetuity
Tweets drop off at midnight ET, start picking up again at 6am ET!
Twitter

http://hide.dyndns.info/tweetcounter/index-en.cgi

9
Increasing tweets per sec over the last year (max value per day used)
Two peaks on the right-hand-side
Twitter

http://hide.dyndns.info/tweetcounter/index-en.cgi

10
Increasing tweets per day over the last year
Twitter

https://blog.twitter.com/2013/new-tweets-per-second-record-and-how

11

Most recent numbers from Twitter:
Twitter avg - >500m TPD
Twitter avg - 5,700 TPS
Aug 2nd had a 143k TPS record (>28x the average) - no blip to service
Data for sale…

http://online.wsj.com/news/articles/SB10001424052702304441404579118531954483974

12
Twitter in its IPO filings disclosed it is making $47.5m from selling access to its data
Google+

http://googleblog.blogspot.com.es/2013/10/google-hangouts-and-photos-save-some.html

13

Google+
540m MAU
300m active in stream
1.5bn photos per month
Sina Weibo

http://sg.finance.yahoo.com/news/sina-weibo-passes-500-million-151054944.html

14
China’s Sina Weibo is growing with 74% year on year user growth (http://www.chinadaily.com.cn/bizchina/2013-02/21/content_16243933.htm)
Has 220m ‘active users’ while Twitter has 170m ‘active users’ http://blogs.wsj.com/chinarealtime/2013/03/12/how-many-people-really-usesina-weibo/
Waze

https://www.waze.com

15
Waze had an estimated 50m users in June 2013
Waze

https://www.waze.com

16
Waze had an estimated 50m users in June 2013
Use Cases

17

Some positive use cases of the data
Crowd-Sourcing

http://csce.uark.edu/~tingxiny/courses/5013spring13/readingList/crowdsource.pdf

18

Academic study on feasibility of using Twitter to crowdsourced data
Meteorology

http://uksnowmap.com/#/

19

UK Snow Map by Ben Marsh
Tweet #uksnow, postcode and x/10 rating
Utilities

http://greenmonk.net/2012/11/01/sustainability-social-media-and-big-data/

20
GE’s Grid IQ Insight can mine social media for mentions of outages
Gives early notifications of an outage in an area
If geotagged and/or includes images/video can confirm cause of outage and speed up time to resolution
Utilities

http://greenmonk.net/2012/11/01/sustainability-social-media-and-big-data/

21

Utilities are aware of reason for outage, speeds up time to resolution (reduces need for
investigatory truck roll)
Risk Analysis

http://europeandcis.undp.org/blog/2012/11/16/social-media-and-political-risk-analysis/

22
United Nations Development Program & their Recorded Future project
Using publicly sourced data looking for signs of disruption or unrest
Risk Analysis

http://europeandcis.undp.org/blog/2012/11/16/social-media-and-political-risk-analysis/

23
The same graph turned into media sources- who is talking about Georgia in this period of time
Risk Analysis

http://europeandcis.undp.org/blog/2012/11/16/social-media-and-political-risk-analysis/

24
Mentions turned into social network analysis- who is talking to whom, who is meeting whom
“next phase will focus on conducting a regional political risk analysis and forecasting for South Eastern Europe and Central Asia”
UNDP slides courtesy of Milica Begovich (aka @elim????)
Automotive

http://www.magazine.pamplin.vt.edu/fall12/vehicledefects.html

25
Social Media monitoring tool developed by Pamplin College of Business
Initial version worked from automotive fora and blogs, now expanding to take in Twitter and Facebook
“Robust” way to discover and classify vehicle defects from social media posts across multiple automotive brands
Faster than reporting back up through the dealer chain
Finance

http://arxiv.org/PS_cache/arxiv/pdf/1010/1010.3003v1.pdf

26
Academic paper from University of Manchester and Indiana University shows that Twitter can predict the Dow Jones Industrial Average with 87.6%
accuracy
Finance

http://www.caymanatlantic.com/investment-management/4574471088

27
UK Firm Derwent Capital Markets signed an exclusive deal with the authors to create a hedge fund - became Cayman Atlantic
Law Enforcement

http://support.sas.com/resources/papers/proceedings12/309-2012.pdf

28
SAS produced an interesting white paper on this space and bought UK firm Memex - definitely chasing this space
Citing use cases like - finding individuals - analysing their social graph to find accomplices/gang structure
Also identify precursor activity to events like riots
Law Enforcement

http://www.policemag.com/blog/technology/story/2012/09/social-media-analytics-in-law-enforcement.aspx

29
“Social media is a huge network of informants—and one you don't have to pay for.”
Law Enforcement use cases (information distribution, fake profile creation, etc.)
Helps first responders gain situational awareness prior to having feet on the ground
Helps Emergency Operations Centres gain information in the event of natural disasters, for example
Sunday’s train crash in NY, for example
Law Enforcement

http://www.huffingtonpost.com/2012/09/04/web-surveillance-social-media_n_1854750.html

X
Other vendors in this space outlined in this article
3i-Mind - http://www.3i-mind.com/
HMS Technologies - http://www.hmstech.com/
Visible Technologies - http://www.visibletechnologies.com/
Attensity - http://www.attensity.com/home/
CrowdControlHQ - http://www.crowdcontrolhq.com/index.php
and
As well as Law Enforcement use cases (information distribution, fake profile creation, etc.)
Law Enforcement

http://www.lexisnexis.com/government/investigations/

30
Good infographic on Law Enforcement use of social media - based on a LexisNexis Risk Solutions survey of 1,200 law enforcement professionals
Full report is available at http://solutions.lexisnexis.com/forms/GV12LEOMPSoMeSurveyforLE9677
Law Enforcement

31
Needs to be approached sensitively - the way some of this is reported often prompts visions of ‘pre-crime’ and Minority Report
Smart Cities

https://itunes.apple.com/us/app/boston-citizens-connect/id330894558

32
Graffiti, Pothole, & LA school district apps
Healthcare

http://www.google.org/flutrends/intl/en_us/

33
Google use frequency of certain search terms as a way to estimate flu activity
Also have one for Dengue Fever
Search data is increasingly mobile
Healthcare

http://www.nature.com/nature/journal/v457/n7232/full/nature07634.html

34
Google wrote this up as an academic paper and it was published in Nature
Healthcare

http://www.ajtmh.org/content/86/1/39.abstract

35
A group led from Harvard Medical School studied viability of using social media for predicting cholera outbreak
Found that the data from Twitter closely corresponded with government data, was available up to two weeks earlier
The paper concludes informal media could be used to study the activity of other disease outbreaks around the world
Financial support was provided by Google.org
Healthcare

http://propellerhealth.com

36
Formerly asthmapolis - wireless asthma puff data - where/when
Can map where asthma outbreaks occur - people sensitive can avoid triggers
Healthcare

http://www.huffingtonpost.com/keith-runyon/louisville-chooses-asthma_b_4086297.html

37
Rolled out in conjunction with city of Louisville Ky
Residents experience as much as a 13-year gap in life expectancy depending upon where they live
Findings eagerly anticipated - only rolled out in Oct
CRM

http://www.ft.com/intl/cms/s/0/bd5a5ce2-aa57-11e1-899d-00144feabdc0.html#axzz2OAlD9lav

38
T-Mobile in US analysed its 33m customer data records, web logs, billing data and social media information to predict customer defections
It halved customer defections in 3 months
Brand Management

http://www.youtube.com/watch?v=VaJjPRwExO8

39
Nestle were Greenpeace’d because palm oil in Kit Kat came from Sinar Mas - company involved in deforestation
Brand Management

http://greenmonk.net/2010/03/19/can-corporate-social-responsibility-affect-your-companys-bottom-line/

40
In the social media storm which followed Nestlé made every mistake in the book
Nestlé received over 200,000 protest emails and their share price was negatively affected
So they decided to work with Greenpeace to fix their supply chain and
To initiate a social media strategy for the organisation
Brand Management

http://uk.reuters.com/article/2012/10/26/uk-nestle-online-water-idUKBRE89P07Q20121026

41
Set up a social media command centre, staffed by their Digital Acceleration Team
Brand Management

http://www.reputationinstitute.com/thought-leadership/global-reptrak-100

42
In 2013 Nestle entered the Reputation Institute’s Global top 10 for the first time.
Transportation

https://twitter.com/ehn/status/396307684661530624

43
Waze data now being incorporated in Google Maps
Transportation

http://www.bbc.co.uk/news/technology-22357748

44

2.5bn anonymised call records from 5m Orange phone users in Ivory Coast
Looked at patterns of people’s movements in Abidjan - capital city of Ivory Coast
Realised they could reduce travel times of ppl by 10%
Looking Ahead

http://www.flickr.com/photos/35468133931@N01/8699901706

45
Google Glass
Looking Ahead

http://www.instabeat.me

46
Instabeat gives swimmers stats in their goggles as they swim
And subsequent download
Looking ahead

http://www.fitbit.com/force

47
Fitbit force, Nike+ Fuelband, Jawbone Up
Can see a situation where sports players are broadcasting vital stats similar to F1
Conclusion
Data and data sources are increasing
exponentially - go hack that data for good.

48
Thanks!
Contact information:
Tom Raftery
Principal Analyst, Energy & Sustainability, RedMonk
Tom@redmonk.com,
GreenMonk.net,
Twitter.com/tomraftery
+34 677 695 468
No tweets were hurt in the making of this presentation
49

People as sensors - mining social media for meaningful information