People as
Sensors
Tom Raftery
ThingMonk,
London Dec 2013.

1
Tom Raftery
•

Lead analyst, energy and
sustainability practice,
RedMonk

•

GreenMonk.net

•

twitter.com/tomraftery

•

...
Mobile data

http://www.flickr.com/photos/traftery/8551389911/

3
Mobile phones with GPS, accelerometers, compass, M7, baro...
Mobile data

http://www.zeit.de/datenschutz/malte-spitz-data-retention

4
Mobile phones with GPS and data retention laws
G...
Social Media

5

Mobile is not the only data we’re broadcasting
Facebook

http://newsroom.fb.com/Key-Facts

6

1.19 billion monthly active users (MAU) as of September 30, 2013
727 millio...
Facebook

http://techcrunch.com/2013/05/17/facebook-growth/

7
Most recent usage data is from May 2013 - so well out-of-da...
Twitter

http://hide.dyndns.info/tweetcounter/index-en.cgi

8
Between 4,000-8,500 tweets per second per day
Avg over 6,000...
Twitter

http://hide.dyndns.info/tweetcounter/index-en.cgi

9
Increasing tweets per sec over the last year (max value per ...
Twitter

http://hide.dyndns.info/tweetcounter/index-en.cgi

10
Increasing tweets per day over the last year
Twitter

https://blog.twitter.com/2013/new-tweets-per-second-record-and-how

11

Most recent numbers from Twitter:
Twitter...
Data for sale…

http://online.wsj.com/news/articles/SB10001424052702304441404579118531954483974

12
Twitter in its IPO fili...
Google+

http://googleblog.blogspot.com.es/2013/10/google-hangouts-and-photos-save-some.html

13

Google+
540m MAU
300m ac...
Sina Weibo

http://sg.finance.yahoo.com/news/sina-weibo-passes-500-million-151054944.html

14
China’s Sina Weibo is growing...
Waze

https://www.waze.com

15
Waze had an estimated 50m users in June 2013
Waze

https://www.waze.com

16
Waze had an estimated 50m users in June 2013
Use Cases

17

Some positive use cases of the data
Crowd-Sourcing

http://csce.uark.edu/~tingxiny/courses/5013spring13/readingList/crowdsource.pdf

18

Academic study on fea...
Meteorology

http://uksnowmap.com/#/

19

UK Snow Map by Ben Marsh
Tweet #uksnow, postcode and x/10 rating
Utilities

http://greenmonk.net/2012/11/01/sustainability-social-media-and-big-data/

20
GE’s Grid IQ Insight can mine soc...
Utilities

http://greenmonk.net/2012/11/01/sustainability-social-media-and-big-data/

21

Utilities are aware of reason fo...
Risk Analysis

http://europeandcis.undp.org/blog/2012/11/16/social-media-and-political-risk-analysis/

22
United Nations D...
Risk Analysis

http://europeandcis.undp.org/blog/2012/11/16/social-media-and-political-risk-analysis/

23
The same graph t...
Risk Analysis

http://europeandcis.undp.org/blog/2012/11/16/social-media-and-political-risk-analysis/

24
Mentions turned ...
Automotive

http://www.magazine.pamplin.vt.edu/fall12/vehicledefects.html

25
Social Media monitoring tool developed by Pa...
Finance

http://arxiv.org/PS_cache/arxiv/pdf/1010/1010.3003v1.pdf

26
Academic paper from University of Manchester and Ind...
Finance

http://www.caymanatlantic.com/investment-management/4574471088

27
UK Firm Derwent Capital Markets signed an excl...
Law Enforcement

http://support.sas.com/resources/papers/proceedings12/309-2012.pdf

28
SAS produced an interesting white ...
Law Enforcement

http://www.policemag.com/blog/technology/story/2012/09/social-media-analytics-in-law-enforcement.aspx

29...
Law Enforcement

http://www.huffingtonpost.com/2012/09/04/web-surveillance-social-media_n_1854750.html

X
Other vendors in ...
Law Enforcement

http://www.lexisnexis.com/government/investigations/

30
Good infographic on Law Enforcement use of socia...
Law Enforcement

31
Needs to be approached sensitively - the way some of this is reported often prompts visions of ‘pre-cr...
Smart Cities

https://itunes.apple.com/us/app/boston-citizens-connect/id330894558

32
Graffiti, Pothole, & LA school distr...
Healthcare

http://www.google.org/flutrends/intl/en_us/

33
Google use frequency of certain search terms as a way to estima...
Healthcare

http://www.nature.com/nature/journal/v457/n7232/full/nature07634.html

34
Google wrote this up as an academic ...
Healthcare

http://www.ajtmh.org/content/86/1/39.abstract

35
A group led from Harvard Medical School studied viability of...
Healthcare

http://propellerhealth.com

36
Formerly asthmapolis - wireless asthma puff data - where/when
Can map where ast...
Healthcare

http://www.huffingtonpost.com/keith-runyon/louisville-chooses-asthma_b_4086297.html

37
Rolled out in conjuncti...
CRM

http://www.ft.com/intl/cms/s/0/bd5a5ce2-aa57-11e1-899d-00144feabdc0.html#axzz2OAlD9lav

38
T-Mobile in US analysed it...
Brand Management

http://www.youtube.com/watch?v=VaJjPRwExO8

39
Nestle were Greenpeace’d because palm oil in Kit Kat came...
Brand Management

http://greenmonk.net/2010/03/19/can-corporate-social-responsibility-affect-your-companys-bottom-line/

4...
Brand Management

http://uk.reuters.com/article/2012/10/26/uk-nestle-online-water-idUKBRE89P07Q20121026

41
Set up a socia...
Brand Management

http://www.reputationinstitute.com/thought-leadership/global-reptrak-100

42
In 2013 Nestle entered the ...
Transportation

https://twitter.com/ehn/status/396307684661530624

43
Waze data now being incorporated in Google Maps
Transportation

http://www.bbc.co.uk/news/technology-22357748

44

2.5bn anonymised call records from 5m Orange phone user...
Looking Ahead

http://www.flickr.com/photos/35468133931@N01/8699901706

45
Google Glass
Looking Ahead

http://www.instabeat.me

46
Instabeat gives swimmers stats in their goggles as they swim
And subsequent dow...
Looking ahead

http://www.fitbit.com/force

47
Fitbit force, Nike+ Fuelband, Jawbone Up
Can see a situation where sports pl...
Conclusion
Data and data sources are increasing
exponentially - go hack that data for good.

48
Thanks!
Contact information:
Tom Raftery
Principal Analyst, Energy & Sustainability, RedMonk
Tom@redmonk.com,
GreenMonk.ne...
Upcoming SlideShare
Loading in...5
×

People as sensors - mining social media for meaningful information

13,957

Published on

The video of this talk is available at https://www.youtube.com/watch?v=4ZdknOPY_jQ

More and more we are all broadcasting information. Geolocation data, “this x sucks” data, weather data, etc.

More and more that data is being parsed and analysed in realtime, such that we have now become sensors.

How does this work, what does this mean, and what risks/benefits will it bring?

Published in: Technology, Business
1 Comment
1 Like
Statistics
Notes
No Downloads
Views
Total Views
13,957
On Slideshare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
33
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

People as sensors - mining social media for meaningful information

  1. 1. People as Sensors Tom Raftery ThingMonk, London Dec 2013. 1
  2. 2. Tom Raftery • Lead analyst, energy and sustainability practice, RedMonk • GreenMonk.net • twitter.com/tomraftery • tom@redmonk.com • +34 677 695 468 • SlideShare.net/TomRaftery 2
  3. 3. Mobile data http://www.flickr.com/photos/traftery/8551389911/ 3 Mobile phones with GPS, accelerometers, compass, M7, barometers, moisture, etc
  4. 4. Mobile data http://www.zeit.de/datenschutz/malte-spitz-data-retention 4 Mobile phones with GPS and data retention laws Green party sued Deutsche Telecom for data Put it online Source of concern
  5. 5. Social Media 5 Mobile is not the only data we’re broadcasting
  6. 6. Facebook http://newsroom.fb.com/Key-Facts 6 1.19 billion monthly active users (MAU) as of September 30, 2013 727 million daily active users (DAU) in Sept 2013 874 million mobile MAUs as of September 30 2013
  7. 7. Facebook http://techcrunch.com/2013/05/17/facebook-growth/ 7 Most recent usage data is from May 2013 - so well out-of-date 4.75 billion content items shared per day (status updates + wall posts + photos + videos + comments) 4.5 billion Likes per day
  8. 8. Twitter http://hide.dyndns.info/tweetcounter/index-en.cgi 8 Between 4,000-8,500 tweets per second per day Avg over 6,000 tweets per sec, all stored in perpetuity Tweets drop off at midnight ET, start picking up again at 6am ET!
  9. 9. Twitter http://hide.dyndns.info/tweetcounter/index-en.cgi 9 Increasing tweets per sec over the last year (max value per day used) Two peaks on the right-hand-side
  10. 10. Twitter http://hide.dyndns.info/tweetcounter/index-en.cgi 10 Increasing tweets per day over the last year
  11. 11. Twitter https://blog.twitter.com/2013/new-tweets-per-second-record-and-how 11 Most recent numbers from Twitter: Twitter avg - >500m TPD Twitter avg - 5,700 TPS Aug 2nd had a 143k TPS record (>28x the average) - no blip to service
  12. 12. Data for sale… http://online.wsj.com/news/articles/SB10001424052702304441404579118531954483974 12 Twitter in its IPO filings disclosed it is making $47.5m from selling access to its data
  13. 13. Google+ http://googleblog.blogspot.com.es/2013/10/google-hangouts-and-photos-save-some.html 13 Google+ 540m MAU 300m active in stream 1.5bn photos per month
  14. 14. Sina Weibo http://sg.finance.yahoo.com/news/sina-weibo-passes-500-million-151054944.html 14 China’s Sina Weibo is growing with 74% year on year user growth (http://www.chinadaily.com.cn/bizchina/2013-02/21/content_16243933.htm) Has 220m ‘active users’ while Twitter has 170m ‘active users’ http://blogs.wsj.com/chinarealtime/2013/03/12/how-many-people-really-usesina-weibo/
  15. 15. Waze https://www.waze.com 15 Waze had an estimated 50m users in June 2013
  16. 16. Waze https://www.waze.com 16 Waze had an estimated 50m users in June 2013
  17. 17. Use Cases 17 Some positive use cases of the data
  18. 18. Crowd-Sourcing http://csce.uark.edu/~tingxiny/courses/5013spring13/readingList/crowdsource.pdf 18 Academic study on feasibility of using Twitter to crowdsourced data
  19. 19. Meteorology http://uksnowmap.com/#/ 19 UK Snow Map by Ben Marsh Tweet #uksnow, postcode and x/10 rating
  20. 20. Utilities http://greenmonk.net/2012/11/01/sustainability-social-media-and-big-data/ 20 GE’s Grid IQ Insight can mine social media for mentions of outages Gives early notifications of an outage in an area If geotagged and/or includes images/video can confirm cause of outage and speed up time to resolution
  21. 21. Utilities http://greenmonk.net/2012/11/01/sustainability-social-media-and-big-data/ 21 Utilities are aware of reason for outage, speeds up time to resolution (reduces need for investigatory truck roll)
  22. 22. Risk Analysis http://europeandcis.undp.org/blog/2012/11/16/social-media-and-political-risk-analysis/ 22 United Nations Development Program & their Recorded Future project Using publicly sourced data looking for signs of disruption or unrest
  23. 23. Risk Analysis http://europeandcis.undp.org/blog/2012/11/16/social-media-and-political-risk-analysis/ 23 The same graph turned into media sources- who is talking about Georgia in this period of time
  24. 24. Risk Analysis http://europeandcis.undp.org/blog/2012/11/16/social-media-and-political-risk-analysis/ 24 Mentions turned into social network analysis- who is talking to whom, who is meeting whom “next phase will focus on conducting a regional political risk analysis and forecasting for South Eastern Europe and Central Asia” UNDP slides courtesy of Milica Begovich (aka @elim????)
  25. 25. Automotive http://www.magazine.pamplin.vt.edu/fall12/vehicledefects.html 25 Social Media monitoring tool developed by Pamplin College of Business Initial version worked from automotive fora and blogs, now expanding to take in Twitter and Facebook “Robust” way to discover and classify vehicle defects from social media posts across multiple automotive brands Faster than reporting back up through the dealer chain
  26. 26. Finance http://arxiv.org/PS_cache/arxiv/pdf/1010/1010.3003v1.pdf 26 Academic paper from University of Manchester and Indiana University shows that Twitter can predict the Dow Jones Industrial Average with 87.6% accuracy
  27. 27. Finance http://www.caymanatlantic.com/investment-management/4574471088 27 UK Firm Derwent Capital Markets signed an exclusive deal with the authors to create a hedge fund - became Cayman Atlantic
  28. 28. Law Enforcement http://support.sas.com/resources/papers/proceedings12/309-2012.pdf 28 SAS produced an interesting white paper on this space and bought UK firm Memex - definitely chasing this space Citing use cases like - finding individuals - analysing their social graph to find accomplices/gang structure Also identify precursor activity to events like riots
  29. 29. Law Enforcement http://www.policemag.com/blog/technology/story/2012/09/social-media-analytics-in-law-enforcement.aspx 29 “Social media is a huge network of informants—and one you don't have to pay for.” Law Enforcement use cases (information distribution, fake profile creation, etc.) Helps first responders gain situational awareness prior to having feet on the ground Helps Emergency Operations Centres gain information in the event of natural disasters, for example Sunday’s train crash in NY, for example
  30. 30. Law Enforcement http://www.huffingtonpost.com/2012/09/04/web-surveillance-social-media_n_1854750.html X Other vendors in this space outlined in this article 3i-Mind - http://www.3i-mind.com/ HMS Technologies - http://www.hmstech.com/ Visible Technologies - http://www.visibletechnologies.com/ Attensity - http://www.attensity.com/home/ CrowdControlHQ - http://www.crowdcontrolhq.com/index.php and As well as Law Enforcement use cases (information distribution, fake profile creation, etc.)
  31. 31. Law Enforcement http://www.lexisnexis.com/government/investigations/ 30 Good infographic on Law Enforcement use of social media - based on a LexisNexis Risk Solutions survey of 1,200 law enforcement professionals Full report is available at http://solutions.lexisnexis.com/forms/GV12LEOMPSoMeSurveyforLE9677
  32. 32. Law Enforcement 31 Needs to be approached sensitively - the way some of this is reported often prompts visions of ‘pre-crime’ and Minority Report
  33. 33. Smart Cities https://itunes.apple.com/us/app/boston-citizens-connect/id330894558 32 Graffiti, Pothole, & LA school district apps
  34. 34. Healthcare http://www.google.org/flutrends/intl/en_us/ 33 Google use frequency of certain search terms as a way to estimate flu activity Also have one for Dengue Fever Search data is increasingly mobile
  35. 35. Healthcare http://www.nature.com/nature/journal/v457/n7232/full/nature07634.html 34 Google wrote this up as an academic paper and it was published in Nature
  36. 36. Healthcare http://www.ajtmh.org/content/86/1/39.abstract 35 A group led from Harvard Medical School studied viability of using social media for predicting cholera outbreak Found that the data from Twitter closely corresponded with government data, was available up to two weeks earlier The paper concludes informal media could be used to study the activity of other disease outbreaks around the world Financial support was provided by Google.org
  37. 37. Healthcare http://propellerhealth.com 36 Formerly asthmapolis - wireless asthma puff data - where/when Can map where asthma outbreaks occur - people sensitive can avoid triggers
  38. 38. Healthcare http://www.huffingtonpost.com/keith-runyon/louisville-chooses-asthma_b_4086297.html 37 Rolled out in conjunction with city of Louisville Ky Residents experience as much as a 13-year gap in life expectancy depending upon where they live Findings eagerly anticipated - only rolled out in Oct
  39. 39. CRM http://www.ft.com/intl/cms/s/0/bd5a5ce2-aa57-11e1-899d-00144feabdc0.html#axzz2OAlD9lav 38 T-Mobile in US analysed its 33m customer data records, web logs, billing data and social media information to predict customer defections It halved customer defections in 3 months
  40. 40. Brand Management http://www.youtube.com/watch?v=VaJjPRwExO8 39 Nestle were Greenpeace’d because palm oil in Kit Kat came from Sinar Mas - company involved in deforestation
  41. 41. Brand Management http://greenmonk.net/2010/03/19/can-corporate-social-responsibility-affect-your-companys-bottom-line/ 40 In the social media storm which followed Nestlé made every mistake in the book Nestlé received over 200,000 protest emails and their share price was negatively affected So they decided to work with Greenpeace to fix their supply chain and To initiate a social media strategy for the organisation
  42. 42. Brand Management http://uk.reuters.com/article/2012/10/26/uk-nestle-online-water-idUKBRE89P07Q20121026 41 Set up a social media command centre, staffed by their Digital Acceleration Team
  43. 43. Brand Management http://www.reputationinstitute.com/thought-leadership/global-reptrak-100 42 In 2013 Nestle entered the Reputation Institute’s Global top 10 for the first time.
  44. 44. Transportation https://twitter.com/ehn/status/396307684661530624 43 Waze data now being incorporated in Google Maps
  45. 45. Transportation http://www.bbc.co.uk/news/technology-22357748 44 2.5bn anonymised call records from 5m Orange phone users in Ivory Coast Looked at patterns of people’s movements in Abidjan - capital city of Ivory Coast Realised they could reduce travel times of ppl by 10%
  46. 46. Looking Ahead http://www.flickr.com/photos/35468133931@N01/8699901706 45 Google Glass
  47. 47. Looking Ahead http://www.instabeat.me 46 Instabeat gives swimmers stats in their goggles as they swim And subsequent download
  48. 48. Looking ahead http://www.fitbit.com/force 47 Fitbit force, Nike+ Fuelband, Jawbone Up Can see a situation where sports players are broadcasting vital stats similar to F1
  49. 49. Conclusion Data and data sources are increasing exponentially - go hack that data for good. 48
  50. 50. Thanks! Contact information: Tom Raftery Principal Analyst, Energy & Sustainability, RedMonk Tom@redmonk.com, GreenMonk.net, Twitter.com/tomraftery +34 677 695 468 No tweets were hurt in the making of this presentation 49
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×