Social pixels acm_mm


Published on

Using social media to understand the situations occurring in the real world.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Social pixels acm_mm

  1. 1. SOCIAL PIXELS: GENESIS & EVALUATIONVivek Singh, Mingyan Gao, and Ramesh Jain University of California, Irvine
  2. 2. Outline Concept Approach Applications Challenges
  3. 3. Motivation People are sharing massive amounts of information on the web (Twitter, Flickr, Facebook, …) How to do effective data consumption, not just data creation  Geo-spatial situation awareness  Real time updates of the world state  From data to actionable knowledge
  4. 4. Concept Understanding evolving world situations by combining spatio-temporal-thematic data coming from social media (e.g. Twitter/Flickr). „Iphone‟ social image for mainland USA. Jun 11, 2009
  5. 5. Social Pixels Traditional Pixels  Photons aggregating at locations on CCD Social Pixels  User interest aggregating at geo-locations Create social Image, social Video… Image/Media Processing operators Situation Detection operators (e.g. convolution, filtering, background subtraction)
  6. 6. Design principles Humans as sensors Social pixel approach  Visualization  Intuitive query and mental model  Common spatio-temporal data representation  Data analysis using media processing Combining media processing with declarative query algebra
  7. 7. Overall Approach1. Micro-event detection2. Spatio-temporal aggregation using social pixel approach3. Media processing engine4. Query engine
  8. 8. Micro-event detection Simple bag-of-words approach for detecting what event is the user talking about.  e.g. „Sore throat‟, „Flu‟, „H1N1‟, … Tweet: „caught sore-throat today…arrrgh !‟ Micro-event detected for user X. Spatial Temporal Thematic
  9. 9. Spatio-temporal aggregation using social pixels Higher level abstractions have trade-offs with lower level details Percolate up what is necessary for the application Can be:  Count of tweets with the term  Average green channel value of images  Mean audio energy  Average monthly income, rainfall, population etc.
  10. 10. Data Model Spatio-temporal element  stel = [s-t-coord, theme(s), value(s), pointer(s)] E-mage g = (x, {(tm, v(x))}|xϵ X = R2 , tm ϵ θ, and v(x) ϵ V = N) Temporal E-mage Set  TES= {(t1, g1), ..., (tn, gn)}, Temporal Pixel Set  TPS = {(t1, p1), ..., (tn, pn)},
  11. 11. Operations1. Selection Operation2. Arithmetic and Logical Operation3. Aggregation Operation α4. Grouping Operation5. Characterization Operation  Spatial  Temporal6. Pattern Matching Operation  Spatial  Temporal
  12. 12. 1. Selection Operation Select part of E-mage based on predicate P Input: Temporal E-mage Set TES = {(t1, g1), …, (tn , gn)} Output: Temporal E-mage Set TES‟ Spatial or Value predicate Pi on Emage  Pi(TES) = {(t1, Pi(g1)), …, (tn, Pi(gn))}, where Pi(g) = {(x, y) | y=g(x), if Pi(x,y) is true; y=0, otherwise} Boolean predicate Pt on time  Pt(TES) = {(t1‟ g1‟), …, (tm‟, gm‟)}, where P(ti‟) is true, e.g. date = „2010-03-10‟
  13. 13. Selection Examples Show last one week‟s E-mages of California for topic „Obama‟  R=cal t <= 1wk theme= Obama(TES)
  14. 14. 2. Arithmetic Operation Binary operations between two (or more) E- mage Sets (g1, g2) = g3(x, (v1(x), v2(x))), where {+, -, *, /, max, min, convolution}, g1 and g2 are the same size. Example:  TES1=Temporal E-mage Set for „Unemployment rate‟  TES2=Temporal E-mage Set for „normalized Gas prices‟  TES3= (TES1, TES2)
  15. 15. 3. Aggregation Operation α Aggregates multiple E-mages in TES based on function . (g1, g2) = g3(x, (v1(x), v2(x))), where {+, *, mean, max, min}, g1 and g2 are the same size. Example:  Show the average emage of last one week‟s emages from California for Obama.  α mean ( R=cal t <= 1wk theme= Obama(TES))
  16. 16. 4. Grouping Operation Group stels in an E-mage g based on certain function f Input: Temporal E-mage Set TES = {(t1, g1), …, (tn , gn)} Output: Temporal E-mage Set TES‟ Function f essentially splits g, into multiple sub-e- mages. f(TES) = f((t1, g1)) … f((tn,gn)), where f((ti, gi)) = {(ti , gi1‟), …, (ti , gik‟)}, and each gij‟ is a sub-E-mage of g based on f f {segmentation, clustering, blob-detection, etc.}
  17. 17. Grouping Example Identify 3 clusters for each E-mage in the TES set having last one week‟s E-mages of California.  clustering, n=3( R=cal t <= 1wk(TES))
  18. 18. 5a. Characterization Op. (Spatial) Represent each E-mage g based on a characteristic C, and store result as a stel. Input: Temporal E-mage Set TES = {(t1, g1), …, (tn, gn)} Output: Temporal Pixel Set TPS = {(t1, p1), …, (tn, pn)} C(TES) = {(t1, (g1)), …, (tn, (gn))}, where (gi) is a pixel characterizing gi C {count, max, min, sum, average, coverage, epi center, density, shape, growth_rate, periodicity }
  19. 19. Characterization Examples(Spatial) Find the epicenter of each cluster E-mage in the last one week‟s E-mages of USA from TES  epicenter ( clustering, n=3( R=USA t <= 1wk theme=Obama(TES))
  20. 20. 5b. Characterization Op.(Temporal) Characterize a temporal pixel set, which is the result of E-mage characterization Input: Temporal Pixel Set TPS = {(t1, p1), …, (tn, pn)} Output: Temporal Pixel Set TPS‟ (TPS) = {(tk , ((t1, p1), …, (tk, pk))) | k [2, n]}, where {displacement, distance, velocity, speed, accel eration, linear extrapolation, exponential growth, exponential decay, etc.}
  21. 21. Temporal CharacterizationExamples Find the velocity of epicenter of each cluster E- mage over the last one week‟s E-mages of California from TES for theme Katrina  velocity ( epicenter ( clustering, n=3( R=Cal t <= 1wk theme = Katrina (TES))))
  22. 22. 5. Pattern Matching Pattern Matching (Spatial)  Compare the similarity between each E-mage and a given pattern P  Input: Temporal E-mage Set TES = {(t1, g1), …, (tn, gn)}, and pattern P  Output: Temporal Pixel Set TPS  P(TES) = {(t1, p1), …, (tn, pn)}, where each value in pi represents the similarity between the E-mage and the given pattern  Patterns (i.e. Kernels) can be loaded from a library or be historical data samples.
  23. 23. Pattern Matching Temporal Pattern matching:  Compare the similarity of the temporal value changing with a given pattern, e.g. „increasing‟, „decreasing‟, or „Enron‟s stock in 1999‟, … Input: Temporal Pixel Set TPS = {(t1, p1), …, (tn, pn)}, and a pattern P Output: Temporal Pixel Set TPS‟ P(TPS) = {(tn , p)}, where v(x) in p is the similarity value
  24. 24. Pattern Matching Examples Compare the similarity between each E-mage in the last one week‟s E-mages of California from TES with radial decay  radial_decay( R=cal t <= 1wk theme = Obama (TES)) How close is the similarity above to pattern of “Enron‟s stock price in 1999”?  Enron‟s stock( radial_decay( R=cal t <= 1wk(TES)))
  25. 25. Situation detection operatorsS. No Operator Input Output1 Selection Temporal Temporal E-mage Set E-mage Set2 Arithmetic & K*Temporal E-mage Temporal E-mage Set Logical Set3 Aggregation α Temporal E-mage set Temporal E-mage Set4 Grouping Temporal E-mage Set Temporal E-mage Set5 Characterization : •Spatial •Temporal E-mage Set •Temporal Pixel Set •Temporal •Temporal Pixel Set •Temporal Pixel Set6 Pattern Matching •Spatial •Temporal E-mage Set •Temporal Pixel Set •Temporal •Temporal Pixel Set •Temporal Pixel Set
  26. 26. Mediaprocessing engine
  27. 27. Implementation and results Twitter feeds  Geo-coding user home location  Loops of location based queries for different terms  Over 100 million tweets using „Spritzer‟ stream (since Jun 2009), and the higher rate „Gardenhose‟ stream since Nov, 2009. Flickr feeds  API  Tags, RGB values from >800K images
  28. 28. Correlation with real worldevents
  29. 29. Applications Business decision making Political event analytics Seasonal characteristics analysis
  30. 30. Situation awareness: iPhonelaunch
  31. 31. Spatio temporal variation:Visualization
  32. 32. Business intelligence: Queries
  33. 33. iPhone theme AT&T based e-mage, retail Jun 2 to Jun 11 locations . Convolution Store + Aggregation * catchment area DifferenceAggregateinterest Combination of operators - AT&T total catchmen t area <geoname> Convolution . MAXIMA <name>College City</name> Decision <lat>39.0057303</lat> <lng>-122.0094129</lng> Best Location is at <geonameId>5338600</geonameId> * <countryCode>US</countryCode> Geocode [39, - <countryName>United States</countryName> 122] , just north of <fcl>P</fcl> Bay Area, CA <fcode>PPL</fcode> <fclName>city, village,...</fclName> <fcodeName>populated place</fcodeName> <population/> Under-served <distance>1.0332</distance> </geoname> interest areas Store catchment
  34. 34. Political event analytics:Queries
  35. 35. Snapshot
  36. 36. Flickr Social Emages Jan – Dec 2009
  37. 37. Seasonal characteristicsanalysis
  38. 38. Year average Peak of green At [35, -84], at the junction of Chattahoochee National Forest, Nantahala National Forest, Cherokee National Forest and Great Smoky Mountains National Park
  39. 39. Variations throughout the year Total Energy Jan DecFall colors of New England  [R-G] channel data 0 Jan Dec
  40. 40. Conclusions Combining spatio-temporal event data for visualization, and analytics. An e-mage representation of spatio-temporal thematic data coming in real-time. Defined operators for real-time situation analysis Applications in multiple domains
  41. 41. Challenges: Future work Defining a (visual) query language using operators Scalability  Realtime data management for all possible topics which user might be interested in Automatic tweets from sensors A reverse-911 like control/recommendation mechanism Creating an event web by connecting all event related data
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.