SlideShare a Scribd company logo
Utilizing Social Media to Understand
        the Dynamics of a City
                         Justin Cranshaw
          jcransh@cs.cmu.edu | @jcransh | justincranshaw.com


    Raz Schwartz           Jason I. Hong           Norman Sadeh
   razs@andrew.cmu.edu     jasonh@cs.cmu.edu       sadeh@cs.cmu.edu


                 School of Computer Science
                  Carnegie Mellon University

                 @livehoods | livehoods.org
Neighborhoods
Neighborhoods
•   Neighborhoods provide order to the chaos of the city

•   Help us determine: where to live / work / play

•   Provide haven / safety / territory

•   Municipal government: organizational unit in resource allocation

•   Centers of commerce and economic development. Brands.

•   A sense of cultural identity to residents
Neighborhoods
• Neighborhoods are cultural entities
• We each carry around our own biases of how we
  view the city, and its neighborhoods

• Neighborhoods often evolve fast, much faster than
  the boundaries that delineate them

• Often city boundaries can be misaligned with the
  cultural realities of a neighborhood
What comes to mind
when you picture your
  neighborhood?

  @livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
The Image of a Neighborhood
                      You’re probably not imagining this.




  @livehoods | livehoods.org                      Mobile Commerce Lab, Carnegie Mellon University
The Image of a Neighborhood




What you’re imagining most likely
looks a lot more like this.
Every citizen has had long associations with some
part of his city, and his image is soaked in
memories and meanings.

                ---Kevin Lynch, The Image of a City


             @livehoods | livehoods.org               Mobile Commerce Lab, Carnegie Mellon University
Studying Perceptions:
Cognitive Maps




  Kevin Lynch, 1960            Stanley Milgram, 1977

  @livehoods | livehoods.org      Mobile Commerce Lab, Carnegie Mellon University
Two Perspectives
   “Politically constructed”           “Socially constructed”




Neighborhoods have fixed            Neighborhoods are organic,
borders defined by the city         cultural artifacts. Borders are
government.                        blurry, imprecise, and may be
                                   different to different people.

      @livehoods | livehoods.org          Mobile Commerce Lab, Carnegie Mellon University
Collective Cognitive Maps
                                     “Socially constructed”
    Can we discover
   automated ways of
identifying the “organic”
 boundaries of the city?

 Can we extract local
cultural knowledge from
     social media?
                                 Neighborhoods are organic,
Can we build a collective        cultural artifacts. Borders are
cognitive map from data?         blurry, imprecise, and may be
                                 different to different people.

    @livehoods | livehoods.org          Mobile Commerce Lab, Carnegie Mellon University
Observing the City:
   • Answering such questions about a city historically
        requires an immense amount of field work
        (interviews, surveys, observations)

   • Such methodologies can only every offer a small
        scale, inherently partial view of the social dynamics
        of a city

William Whyte spent 1000s of
hours observing the public social
life of New York

Images from The Social Life of Small Urban
Places by William Whyte.


          @livehoods | livehoods.org         Mobile Commerce Lab, Carnegie Mellon University
Observing the City:
SmartPhones
Studying the city through direct observation historically requires an
immense amount of field work, and often yields small-scale,
partial results

We seek computational ways of
uncovering local cultural
knowledge by leverage rich new
sources of social media.

Can social media help us
uncover an aggregate
cognitive map of the city?

      @livehoods | livehoods.org               Mobile Commerce Lab, Carnegie Mellon University
Observing the City:
SmartPhones
We seek to leverage location-based mobile social
networks such as foursquare, which let users
broadcast the places
they visit to their
friends via
check-ins.




    @livehoods | livehoods.org    Mobile Commerce Lab, Carnegie Mellon University
Check-in




@livehoods | livehoods.org              Mobile Commerce Lab, Carnegie Mellon University
Hypothesis
 • The character of an urban area is defined not just by
     the the types of places found there, but also by the
     people who make the area part of their daily routine.

 • Thus we can characterize a place by observing the
     people that visit it.

 • To discover areas of unique character, we should look
     for clusters of nearby venues that are visited the same
     people
The moving elements of a city, and in particular the people and their
activities, are as important as the stationary physical parts.

                                        ---Kevin Lynch, The Image of a City
       @livehoods | livehoods.org                  Mobile Commerce Lab, Carnegie Mellon University
Clustering The City
• Clusters should be of geographically contiguous
  foursquare venues.

• Clusters should have a distinct character from one
  another, perceivable by city residents.

• Clusters should be such that venues within a cluster
  are more likely to be visited by the same users than
  venues in different clusters.




    @livehoods | livehoods.org        Mobile Commerce Lab, Carnegie Mellon University
Clustering Intuition


@livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
If you watch check-ins over
time, you’ll notice that
groups of like-minded people
tend to stay in the same
areas.



                               ☺
      ☺
                               ☺
         ☺
We can aggregate these
patterns to compute
relationships between check-
in venues.
These relationships can then
be used to identify natural
borders in the urban
landscape.
We call the discovered
clusters “Livehoods”
reflecting their dynamic
character.




                          Livehood 2

Livehood 1
Clustering
                Methodology

@livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
Social Venue Similarity
We can get a notion of how similar two           2                                         3
places are by looking at who has checked           # of u1 checkins v
                                                 6 # of u2 checkins to v                   7
into them.                                       6
                                            cv = 6            .
                                                                                           7
                                                                                           7
                                                 4            .
                                                              .                            5
                                                   # of unU checkins to v
cv is a vector where the uth component
counts the number of times user u
checks in to venue v.
                                                  Problems With This
We can think of cv as the bag of                      Approach
checkins to venue v .
                                            (1) The resulting similarity
We can then look at the cosine similarity       graph is quite sparse
between venues.                             (2) Similarity tends to be
                          ci · cj               dominated by “hub”
s(i, j) =              ||ci || ||cj ||
                                                venues


       @livehoods | livehoods.org              Mobile Commerce Lab, Carnegie Mellon University
Venue Affinity Matrix
•    In addition to capturing the social similarity between places, we
     want clusters to be geographically contiguous

•    We also need to overcome the limitations of social similarity
     (sparsity and hub biases)

We derive an affinity matrix A that blends
social affinity with spatial proximity:                              Restricting to nearest
                                                                    neighbors overcomes bias by
                                                                    “hub” venues such as airports,
A = (ai,j )i,j=1,...,nV                                             and it adds geographic
       ⇢                                                            contiguity.
          s(i, j) + ↵                 if j 2 Nm (i) or i 2 Nm (j)
ai,j =
                      0               otherwise                     ↵ is a small positive constant
                                                                    that overcomes sparsity in
Nm (i) are the m nearest geographic neighbors
                                                                    pairwise co-occurrence data.
to venue i.

         @livehoods | livehoods.org                                   Mobile Commerce Lab, Carnegie Mellon University
Graph Interpretation
Viewed as a graph, we connect each node to its m
nearest neighbors in geographic distance,
and we weight these
edges
                      Nm (i)
according
to their
social distance.

We can then use well
studied methodologies
                                       s(i, j
                                i             )+               ↵
in graph clustering to
discover the contiguous Livehoods.
                                                                                 j
     @livehoods | livehoods.org      Mobile Commerce Lab, Carnegie Mellon University
Spectral Clustering
•   Given the affinity matrix A, we            Ng, Jordon, and Weiss
    segment check-in venues using well
    studied spectral clustering            (1) D is the diagonal degree
                                               matrix
    techniques
                                           (2) L = D A
•   We use the variation of spectral       (3) Lnorm = D 1/2 LD1/2
                                           (4) Find e1 , . . . , ek , the k
    clustering introduced by Ng, Jordan,       smallest evects’s of Lnorm
    and Weiss [NIPS 2001]                  (5) E = [e1 , . . . , ek ] and let
                                               y1 , . . . , ynV be the rows
                                               -
•   We select k (number of clusters) as        of E
    is common by looking for large gaps    (6) Clustering y1 , . . . , ynV
    in consecutive eigenvalues                 with KMeans induces a
                                               clustering A1 , . . . , Ak of
    between and upper and lower                the original data.
    allowable k.

     @livehoods | livehoods.org               Mobile Commerce Lab, Carnegie Mellon University
Post Processing

• We introduce a post processing step to clean up any
  degenerate clusters

• We separate the subgraph induced by each A   into            i
  connected components, creating new clusters for
  each.

• We delete any clusters that span too large a
  geographic area (“background noise”) and reapportion
  the venues to the closest non-degenerate cluster by
  (single linkage) geographic distance

    @livehoods | livehoods.org       Mobile Commerce Lab, Carnegie Mellon University
Related Livehoods
To examine how the Livehoods are related to one another, for
each pair of Livehoods, we
compute a similarity score.

For Livehood Ai the vector cAi
is the bag of check-ins to Ai .

Each component measures
for a given user u the number of
times u has checked in to
any venue in Ai.
                 cAi · cAj
s(Ai , Aj ) =
              ||cAi || ||cAj ||
      @livehoods | livehoods.org           Mobile Commerce Lab, Carnegie Mellon University
Data


@livehoods | livehoods.org          Mobile Commerce Lab, Carnegie Mellon University
The Data
• Foursquare check-ins are by default private
• We can gather check-ins that have been shared
  publicly on Twitter.

• Combine the 11 million foursquare check-ins from the
  dataset Chen et al. dataset [ICWSM 2011] with our
  own dataset of 7 million checkins gathered between
  June and December of 2011.

• Aligned these Tweets with the underlying foursquare
  venue data (venue ID and venue category)


    @livehoods | livehoods.org      Mobile Commerce Lab, Carnegie Mellon University
livehoods.org


@livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
Currently maps of New York, San Francisco Bay, Pittsburgh, and Montreal

     @livehoods | livehoods.org                Mobile Commerce Lab, Carnegie Mellon University
@livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
@livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
@livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
Evaluation


@livehoods | livehoods.org       Mobile Commerce Lab, Carnegie Mellon University
Pittsburgh Livehoods
How do we evaluate this?
• Livehoods are different from neighborhoods, but
  how? In our evaluation, we want to characterize
  what Livehoods are.
  •     Do residents derive social meaning from the Livehoods
        mapping?

  •     Can Livehoods help elucidate the various forces that
        shape and define the city?

• Quantitative (algorithmic) evaluation methods fall far
  short of capturing such concepts.

      @livehoods | livehoods.org          Mobile Commerce Lab, Carnegie Mellon University
Field Work
Evaluation
• To see how well our algorithm performed, we
   interviewed 27 residents of Pittsburgh

• Residents recruited through a social media campaign,
   with various neighborhood groups and entities as seeds

• Semi-structured Interview protocol explored the
   relationship among Livehoods, municipal borders,
   and the participants own perceptions of the city.

• Participants must have lived in their neighborhood for at
   least 1 year


    @livehoods | livehoods.org         Mobile Commerce Lab, Carnegie Mellon University
Interview Protocol
•   Each interview lasted approximately one hour

•   Began with a discussion of their backgrounds in relation their
    neighborhood.

•   Asked them to draw the boundaries of their neighborhood over it
    (their cognitive map).

•   Is there a “shift in feel” of the neighborhood?

•   Municipal borders changing?

•   Show Livehoods clusters (ask for feedback)

•   Show related Livehoods (ask for feedback)

     @livehoods | livehoods.org                 Mobile Commerce Lab, Carnegie Mellon University
Intersection Patterns

• To guide our interview, we developed a framework to
  explore the categorize and explore the relationships
  between Livehoods and and municipal neighborhoods
  •     Split: when a neighborhood is split into several Livehoods

  •     Spilled: when a Livehoods spills over a municipal border

  •     Corresponding: when neighborhood border and Livehood
        border correspond




      @livehoods | livehoods.org               Mobile Commerce Lab, Carnegie Mellon University
Split Pattern
One municipal
neighborhood can be
split into several
Livehoods.




     @livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
Spilled Pattern
A Livehood can spill
across the boundaries
of one or more
municipal
neighborhoods.




     @livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
Corresponding Pattern
The borders
between Livehoods
and municipal
neighborhoods can
correspond with
one another.




     @livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
Combining Patterns
All these patterns
can appear in
across the
Livehoods and
municipal hoods of
a city.




     @livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
Split                            Spilled                   Corresponding
To explore split                   Spilled patterns were         For corresponding
patterns, we asked if              explored by asking if         patterns, we
there was anywhere                 there was anywhere            compared their
that has a “distinct               where the subject felt        cognitive maps of the
character or shift in              the “municipal                city with the shared
feel” within the                   boundaries were               municipal and
neighborhood.                      shifting or in flux.”          Livehoods borders.




      @livehoods | livehoods.org                            Mobile Commerce Lab, Carnegie Mellon University
Interview Results


@livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
South Side Pittsburgh




  @livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
South Side Pittsburgh




  @livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
South Side Pittsburgh




Carson Street runs along the length of
South Side, and is densely packed with
bars, restaurants, tattoo parlors, and
clothing and furniture shops. It is the most
popular destination for nightlife.

      @livehoods | livehoods.org               Mobile Commerce Lab, Carnegie Mellon University
South Side Pittsburgh




South Side Works is a recently built,
mixed-use outdoor shopping mall,
containing nationally branded apparel
stores and restaurants, upscale
condominiums, and corporate offices.

      @livehoods | livehoods.org        Mobile Commerce Lab, Carnegie Mellon University
South Side Pittsburgh




There is an small, somewhat older strip-
mall that contains the only super market
(grocery) in South Side. It also has a liquor
store, an auto-parts store, a furniture
rental store and other small chain stores.

      @livehoods | livehoods.org                Mobile Commerce Lab, Carnegie Mellon University
South Side Pittsburgh




The rest of South Side is predominantly
residential, consisting of mostly smaller row
houses.



      @livehoods | livehoods.org                Mobile Commerce Lab, Carnegie Mellon University
South Side Pittsburgh




 My Cognitive Map of South Side




    @livehoods | livehoods.org    Mobile Commerce Lab, Carnegie Mellon University
South Side Pittsburgh

                               LH6

   LH7                                                      LH9
                                     LH8


          The Livehoods of South Side
        I’ll show the evidence in support of the Livehoods
  clusters in South Side, and will describe the forces that shape
                the city that the Livehoods highlight.

  @livehoods | livehoods.org                  Mobile Commerce Lab, Carnegie Mellon University
South Side Pittsburgh

                                LH6

    LH7                                                          LH9
                                      LH8


  LH8 vs LH9
                                  “Ha! Yes! See, here is my division! Yay!
Demographic                       Thank you algorithm! ... I definitely feel
 Differences                      where the South Side Works, and all of
                                  that is, is a very different feel.”

   @livehoods | livehoods.org                      Mobile Commerce Lab, Carnegie Mellon University
South Side Pittsburgh

                                LH6

    LH7                                                           LH9
                                      LH8


  LH7 vs LH8

 Architecture &                       “from an urban standpoint it is a lot
                                      tighter on the western part once you
 Urban Design                         get west of 17th or 18th [LH7].”

   @livehoods | livehoods.org                       Mobile Commerce Lab, Carnegie Mellon University
South Side Pittsburgh

                                 LH6

    LH7                                                                   LH9
                                         LH8


 LH7 vs LH8                     “Whenever I was living down on 15th Street [LH7] I
                                had to worry about drunk people following me home,
 Safety                         but on 23rd [LH8] I need to worry about people trying
                                to mug you... so it’s different. It’s not something I had
                                anticipated, but there is a distinct difference between
                                the two areas of the South Side.”

   @livehoods | livehoods.org                               Mobile Commerce Lab, Carnegie Mellon University
South Side Pittsburgh

                                LH6

    LH7                                                               LH9
                                      LH8


     LH6
                                 “There is this interesting mix of people there I don’t
Demographic                      see walking around the neighborhood. I think they
                                 are coming to the Giant Eagle from lower income
 Differences                     neighborhoods...I always assumed they came from
                                 up the hill.”

   @livehoods | livehoods.org                           Mobile Commerce Lab, Carnegie Mellon University
South Side Pittsburgh   “I always assumed they
                        came from up the hill.”
Shadyside and East Liberty



                                     A Teaser...




  @livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
Shadyside




@livehoods | livehoods.org               Mobile Commerce Lab, Carnegie Mellon University
East Liberty




@livehoods | livehoods.org              Mobile Commerce Lab, Carnegie Mellon University
The Train Tracks




@livehoods | livehoods.org        Mobile Commerce Lab, Carnegie Mellon University
The Whole Foods




@livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
The Pedestrian Bridge




@livehoods | livehoods.org   Mobile Commerce Lab, Carnegie Mellon University
Conclusions
• Throughout our interviews we found very strong
  evidence in support of the clustering

• Interviews showed that residence found strong social
  meaning behind the Livehood clusters.

• We also found that Livehoods can help shed light on
  the various forces that shape people’s behavior in the
  city, the city including demographics, economic
  factors, cultural perceptions and architecture.



    @livehoods | livehoods.org       Mobile Commerce Lab, Carnegie Mellon University
A Final Note on Qualitative
Evaluations
In this work, we gave a qualitative evaluation of a quantitative model. This
idea is subtle and it might initially make some uncomfortable, especially
those used to conducting machine learning research. However it’s an idea
that deserve more thought and attention from the research community. Of
course, we can’t conclude that our model is the best or even better than
some known baseline, but our objective in this task is different from
standard settings. We want to uncover what insights our model tells us
about the social texture of city life. The only way to effectively explore
the relationships between our model and the social realities of citizens is by
going out and talking to them, learning about their lives and the ways that
they perceive the urban landscape. Could other models uncover these
insights? Absolutely. However, we can conclude based on a
preponderance of evidence from our interviews, that Livehoods can be
used as an effective tool for exploring the social dynamics of a city.


       @livehoods | livehoods.org                   Mobile Commerce Lab, Carnegie Mellon University
Limitations
•   Most Livehoods had real social meaning to participants, but no algorithm
    is perfect. There are certainly Livehoods that don’t make sense.

•   There are obvious biases to using foursquare data. However this a
    limitation to the data, not our methodology.

•   There is experimenter bias associated with tuning the clustering
    parameters.

•   Some populations are left out (the digital divide)

•   We don’t want to overemphasize sharp divisions between Livehoods. In
    reality neighborhoods blend into one another.

•   This is not comparative work. We’re not making the claim that ours
    model is the best model for capturing the areas of a city, only that its a
    good model.

     @livehoods | livehoods.org                      Mobile Commerce Lab, Carnegie Mellon University
Thanks!
  Please explore our maps at livehoods.org
You can also find us on on Facebook and Twitter @livehoods.

Justin Cranshaw                    Raz Schwartz         Jason I. Hong                Norman Sadeh
jcransh@cs.cmu.edu                razs@andrew.cmu.edu   jasonh@cs.cmu.edu            sadeh@cs.cmu.edu




                                   School of Computer Science
                                    Carnegie Mellon University

                                   @livehoods | livehoods.org

     @livehoods | livehoods.org                                    Mobile Commerce Lab, Carnegie Mellon University
http://distilleryimage7.s3.amazonaws.com/0a7e8f2c6ad711e180d51231380fcd7e_7.jpg                           Photo Credits
                           http://distilleryimage0.s3.amazonaws.com/b57fcd849d6b11e1b10e123138105d6b_7.jpg

                                           http://distilleryimage9.s3.amazonaws.com/94a7a046453711e19e4a12313813ffc0_7.jpg
                                                                                        http://distilleryimage3.s3.amazonaws.com/1733a942876411e1be6a12313820455d_7.jpg

                                                       http://distilleryimage1.s3.amazonaws.com/22c1c74e6af911e180d51231380fcd7e_7.jpg
                    http://instagr.am/p/LHByf3CtNl/
http://distilleryimage2.s3.amazonaws.com/70cb3dca9ebc11e18cf91231380fd29b_7.jpg
                                                    http://distilleryimage9.s3.amazonaws.com/d7b9e2249bca11e1a92a1231381b6f02_7.jpg
                                                       http://distilleryimage6.s3.amazonaws.com/4de101bc414411e1abb01231381b65e3_7.jpg
                                                                     http://distilleryimage7.s3.amazonaws.com/93bcdec8a04311e18bb812313804a181_7.jpg

         http://distilleryimage4.instagram.com/3d3ef156a9eb11e181bd12313817987b_7.jpg

                                                         http://distilleryimage0.s3.amazonaws.com/db4691625bdd11e19896123138142014_7.jpg
                                                                     http://distilleryimage0.s3.amazonaws.com/65ba3af242d811e19896123138142014_7.jpg

                                                                     http://instagr.am/p/K_ZE9roeq6/
                    http://www.flickr.com/photos/award33/4730513546          http://distilleryimage7.s3.amazonaws.com/88a6ca749b8b11e180c9123138016265_7.jpg
                                                                        http://distilleryimage8.s3.amazonaws.com/9cc5f582919211e1be6a12313820455d_7.jpg

                                                               http://distillery.s3.amazonaws.com/media/2011/10/07/1d2d0bc47db44acf9cf15017103af06a_7.jpg

                         http://distilleryimage5.s3.amazonaws.com/fe99d252a69f11e1abd612313810100a_7.jpg

                                                                          http://instagr.am/p/LN_tt3MzHl/

                                                                     http://distilleryimage8.s3.amazonaws.com/88d3de6639a111e1a87612313804ec91_7.jpg


                                        http://distilleryimage2.s3.amazonaws.com/d57f456c534011e1abb01231381b65e3_7.jpg

                                                                       http://instagr.am/p/K8O4JLoY-4/ http://distilleryimage8.s3.amazonaws.com/dfb52f1268b011e19e4a12313813ffc0_7.jpg


                                                                        http://distilleryimage1.s3.amazonaws.com/fe0235ce7a0c11e181bd12313817987b_7.jpg

                                                 http://distilleryimage0.s3.amazonaws.com/2dc3f470923911e181bd12313817987b_7.jpg

                                               http://distilleryimage0.s3.amazonaws.com/0c6a6cac9af711e19dc71231380fe523_7.jpg

                                    http://distilleryimage5.s3.amazonaws.com/fe99d252a69f11e1abd612313810100a_7.jpg
                                                                        http://distilleryimage4.s3.amazonaws.com/d3db2b5ca36a11e18bb812313804a181_7.jpg

               http://distilleryimage7.s3.amazonaws.com/ca6d892a13f011e180c9123138016265_7.jpg




                       @livehoods | livehoods.org                                                                                 Mobile Commerce Lab, Carnegie Mellon University

More Related Content

Similar to Livehoods: Understanding cities through social media

Fuzzy foss4g 2006 tim waters poster
Fuzzy foss4g 2006 tim waters posterFuzzy foss4g 2006 tim waters poster
Fuzzy foss4g 2006 tim waters poster
chippy
 
Complexity-Lecture-3.pdf
Complexity-Lecture-3.pdfComplexity-Lecture-3.pdf
Complexity-Lecture-3.pdf
NutagNegten1
 
[Digest] Eigenbehaviors- identifying structure in routine
[Digest] Eigenbehaviors- identifying structure in routine[Digest] Eigenbehaviors- identifying structure in routine
[Digest] Eigenbehaviors- identifying structure in routine
Hsing-chuan Hsieh
 
Nethood Aesop09 Liverpool
Nethood Aesop09 LiverpoolNethood Aesop09 Liverpool
Nethood Aesop09 Liverpool
nethood
 
Bridging the Gap Between Physical Location and Online Social Networks, at Ubi...
Bridging the Gap Between Physical Location and Online Social Networks, at Ubi...Bridging the Gap Between Physical Location and Online Social Networks, at Ubi...
Bridging the Gap Between Physical Location and Online Social Networks, at Ubi...
Jason Hong
 
Modeling the Social, Spatial, and Temporal dimensions of Human Mobility in a ...
Modeling the Social, Spatial, and Temporal dimensions of Human Mobility in a ...Modeling the Social, Spatial, and Temporal dimensions of Human Mobility in a ...
Modeling the Social, Spatial, and Temporal dimensions of Human Mobility in a ...
Dima Karamshuk
 
Intro to data visualization
Intro to data visualizationIntro to data visualization
Intro to data visualization
Jan Aerts
 
SSRI_pt1.ppt
SSRI_pt1.pptSSRI_pt1.ppt
SSRI_pt1.ppt
9260SahilPatil
 
4 musatov
4 musatov4 musatov
4 musatov
Yandex
 
ARGUMENT.pptx
ARGUMENT.pptxARGUMENT.pptx
ARGUMENT.pptx
SIDDHIKANKARIYA1
 
Studying perceptions of urban space and neighbourhood with moblogging
Studying perceptions of urban space and neighbourhood with mobloggingStudying perceptions of urban space and neighbourhood with moblogging
Studying perceptions of urban space and neighbourhood with moblogging
Dania Abdel-aziz
 
City Games: Up and Down and Sideways on the Ladder of Abstraction
City Games: Up and Down and Sideways on the Ladder of AbstractionCity Games: Up and Down and Sideways on the Ladder of Abstraction
City Games: Up and Down and Sideways on the Ladder of Abstraction
Sebastian Deterding
 
Mapping The Fear of Crime
Mapping The Fear of CrimeMapping The Fear of Crime
Mapping The Fear of Crime
chippy
 
Caught In A Web
Caught In A WebCaught In A Web
Caught In A Web
Roberto Ovalle
 
Hive NYC 3X3X3
Hive NYC 3X3X3Hive NYC 3X3X3
Hive NYC 3X3X3
hivelearningnyc
 
A Perspective on Graph Theory and Network Science
A Perspective on Graph Theory and Network ScienceA Perspective on Graph Theory and Network Science
A Perspective on Graph Theory and Network Science
Marko Rodriguez
 
Research Directions for Cross Reality Interfaces
Research Directions for Cross Reality InterfacesResearch Directions for Cross Reality Interfaces
Research Directions for Cross Reality Interfaces
Mark Billinghurst
 
Image of the city, kevin lynch & case study.
Image of the city, kevin lynch & case study.Image of the city, kevin lynch & case study.
Image of the city, kevin lynch & case study.
Prasenjit Karmakar
 
Mining customer location data to provide behavioral information and business ...
Mining customer location data to provide behavioral information and business ...Mining customer location data to provide behavioral information and business ...
Mining customer location data to provide behavioral information and business ...
HitReach
 
Travellersim
TravellersimTravellersim
Travellersim
DoctorG
 

Similar to Livehoods: Understanding cities through social media (20)

Fuzzy foss4g 2006 tim waters poster
Fuzzy foss4g 2006 tim waters posterFuzzy foss4g 2006 tim waters poster
Fuzzy foss4g 2006 tim waters poster
 
Complexity-Lecture-3.pdf
Complexity-Lecture-3.pdfComplexity-Lecture-3.pdf
Complexity-Lecture-3.pdf
 
[Digest] Eigenbehaviors- identifying structure in routine
[Digest] Eigenbehaviors- identifying structure in routine[Digest] Eigenbehaviors- identifying structure in routine
[Digest] Eigenbehaviors- identifying structure in routine
 
Nethood Aesop09 Liverpool
Nethood Aesop09 LiverpoolNethood Aesop09 Liverpool
Nethood Aesop09 Liverpool
 
Bridging the Gap Between Physical Location and Online Social Networks, at Ubi...
Bridging the Gap Between Physical Location and Online Social Networks, at Ubi...Bridging the Gap Between Physical Location and Online Social Networks, at Ubi...
Bridging the Gap Between Physical Location and Online Social Networks, at Ubi...
 
Modeling the Social, Spatial, and Temporal dimensions of Human Mobility in a ...
Modeling the Social, Spatial, and Temporal dimensions of Human Mobility in a ...Modeling the Social, Spatial, and Temporal dimensions of Human Mobility in a ...
Modeling the Social, Spatial, and Temporal dimensions of Human Mobility in a ...
 
Intro to data visualization
Intro to data visualizationIntro to data visualization
Intro to data visualization
 
SSRI_pt1.ppt
SSRI_pt1.pptSSRI_pt1.ppt
SSRI_pt1.ppt
 
4 musatov
4 musatov4 musatov
4 musatov
 
ARGUMENT.pptx
ARGUMENT.pptxARGUMENT.pptx
ARGUMENT.pptx
 
Studying perceptions of urban space and neighbourhood with moblogging
Studying perceptions of urban space and neighbourhood with mobloggingStudying perceptions of urban space and neighbourhood with moblogging
Studying perceptions of urban space and neighbourhood with moblogging
 
City Games: Up and Down and Sideways on the Ladder of Abstraction
City Games: Up and Down and Sideways on the Ladder of AbstractionCity Games: Up and Down and Sideways on the Ladder of Abstraction
City Games: Up and Down and Sideways on the Ladder of Abstraction
 
Mapping The Fear of Crime
Mapping The Fear of CrimeMapping The Fear of Crime
Mapping The Fear of Crime
 
Caught In A Web
Caught In A WebCaught In A Web
Caught In A Web
 
Hive NYC 3X3X3
Hive NYC 3X3X3Hive NYC 3X3X3
Hive NYC 3X3X3
 
A Perspective on Graph Theory and Network Science
A Perspective on Graph Theory and Network ScienceA Perspective on Graph Theory and Network Science
A Perspective on Graph Theory and Network Science
 
Research Directions for Cross Reality Interfaces
Research Directions for Cross Reality InterfacesResearch Directions for Cross Reality Interfaces
Research Directions for Cross Reality Interfaces
 
Image of the city, kevin lynch & case study.
Image of the city, kevin lynch & case study.Image of the city, kevin lynch & case study.
Image of the city, kevin lynch & case study.
 
Mining customer location data to provide behavioral information and business ...
Mining customer location data to provide behavioral information and business ...Mining customer location data to provide behavioral information and business ...
Mining customer location data to provide behavioral information and business ...
 
Travellersim
TravellersimTravellersim
Travellersim
 

Recently uploaded

Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024
siddu769252
 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
SynapseIndia
 
Step-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From ScratchStep-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From Scratch
softsuave
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
alexjohnson7307
 
Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
Bhajan Mehta
 
Vulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive OverviewVulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive Overview
Steven Carlson
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
shyamraj55
 
Acumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptxAcumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptx
BrainSell Technologies
 
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptxMAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
janagijoythi
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
DianaGray10
 
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and DisadvantagesBLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
SAI KAILASH R
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
bhumivarma35300
 
Camunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptxCamunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptx
ZachWylie3
 
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and CitiesThe Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
Arpan Buwa
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
shanihomely
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
FIDO Alliance
 
Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
ssuser1915fe1
 

Recently uploaded (20)

Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024
 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
 
Step-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From ScratchStep-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From Scratch
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
 
Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
 
Vulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive OverviewVulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive Overview
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
 
Acumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptxAcumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptx
 
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptxMAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
 
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and DisadvantagesBLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
 
Camunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptxCamunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptx
 
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and CitiesThe Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
 
Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
 

Livehoods: Understanding cities through social media

  • 1. Utilizing Social Media to Understand the Dynamics of a City Justin Cranshaw jcransh@cs.cmu.edu | @jcransh | justincranshaw.com Raz Schwartz Jason I. Hong Norman Sadeh razs@andrew.cmu.edu jasonh@cs.cmu.edu sadeh@cs.cmu.edu School of Computer Science Carnegie Mellon University @livehoods | livehoods.org
  • 3. Neighborhoods • Neighborhoods provide order to the chaos of the city • Help us determine: where to live / work / play • Provide haven / safety / territory • Municipal government: organizational unit in resource allocation • Centers of commerce and economic development. Brands. • A sense of cultural identity to residents
  • 4. Neighborhoods • Neighborhoods are cultural entities • We each carry around our own biases of how we view the city, and its neighborhoods • Neighborhoods often evolve fast, much faster than the boundaries that delineate them • Often city boundaries can be misaligned with the cultural realities of a neighborhood
  • 5. What comes to mind when you picture your neighborhood? @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 6. The Image of a Neighborhood You’re probably not imagining this. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 7. The Image of a Neighborhood What you’re imagining most likely looks a lot more like this. Every citizen has had long associations with some part of his city, and his image is soaked in memories and meanings. ---Kevin Lynch, The Image of a City @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 8. Studying Perceptions: Cognitive Maps Kevin Lynch, 1960 Stanley Milgram, 1977 @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 9. Two Perspectives “Politically constructed” “Socially constructed” Neighborhoods have fixed Neighborhoods are organic, borders defined by the city cultural artifacts. Borders are government. blurry, imprecise, and may be different to different people. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 10. Collective Cognitive Maps “Socially constructed” Can we discover automated ways of identifying the “organic” boundaries of the city? Can we extract local cultural knowledge from social media? Neighborhoods are organic, Can we build a collective cultural artifacts. Borders are cognitive map from data? blurry, imprecise, and may be different to different people. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 11. Observing the City: • Answering such questions about a city historically requires an immense amount of field work (interviews, surveys, observations) • Such methodologies can only every offer a small scale, inherently partial view of the social dynamics of a city William Whyte spent 1000s of hours observing the public social life of New York Images from The Social Life of Small Urban Places by William Whyte. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 12. Observing the City: SmartPhones Studying the city through direct observation historically requires an immense amount of field work, and often yields small-scale, partial results We seek computational ways of uncovering local cultural knowledge by leverage rich new sources of social media. Can social media help us uncover an aggregate cognitive map of the city? @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 13. Observing the City: SmartPhones We seek to leverage location-based mobile social networks such as foursquare, which let users broadcast the places they visit to their friends via check-ins. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 14. Check-in @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 15. Hypothesis • The character of an urban area is defined not just by the the types of places found there, but also by the people who make the area part of their daily routine. • Thus we can characterize a place by observing the people that visit it. • To discover areas of unique character, we should look for clusters of nearby venues that are visited the same people The moving elements of a city, and in particular the people and their activities, are as important as the stationary physical parts. ---Kevin Lynch, The Image of a City @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 16. Clustering The City • Clusters should be of geographically contiguous foursquare venues. • Clusters should have a distinct character from one another, perceivable by city residents. • Clusters should be such that venues within a cluster are more likely to be visited by the same users than venues in different clusters. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 17. Clustering Intuition @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 18. If you watch check-ins over time, you’ll notice that groups of like-minded people tend to stay in the same areas. ☺ ☺ ☺ ☺
  • 19. We can aggregate these patterns to compute relationships between check- in venues.
  • 20. These relationships can then be used to identify natural borders in the urban landscape.
  • 21. We call the discovered clusters “Livehoods” reflecting their dynamic character. Livehood 2 Livehood 1
  • 22. Clustering Methodology @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 23. Social Venue Similarity We can get a notion of how similar two 2 3 places are by looking at who has checked # of u1 checkins v 6 # of u2 checkins to v 7 into them. 6 cv = 6 . 7 7 4 . . 5 # of unU checkins to v cv is a vector where the uth component counts the number of times user u checks in to venue v. Problems With This We can think of cv as the bag of Approach checkins to venue v . (1) The resulting similarity We can then look at the cosine similarity graph is quite sparse between venues. (2) Similarity tends to be ci · cj dominated by “hub” s(i, j) = ||ci || ||cj || venues @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 24. Venue Affinity Matrix • In addition to capturing the social similarity between places, we want clusters to be geographically contiguous • We also need to overcome the limitations of social similarity (sparsity and hub biases) We derive an affinity matrix A that blends social affinity with spatial proximity: Restricting to nearest neighbors overcomes bias by “hub” venues such as airports, A = (ai,j )i,j=1,...,nV and it adds geographic ⇢ contiguity. s(i, j) + ↵ if j 2 Nm (i) or i 2 Nm (j) ai,j = 0 otherwise ↵ is a small positive constant that overcomes sparsity in Nm (i) are the m nearest geographic neighbors pairwise co-occurrence data. to venue i. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 25. Graph Interpretation Viewed as a graph, we connect each node to its m nearest neighbors in geographic distance, and we weight these edges Nm (i) according to their social distance. We can then use well studied methodologies s(i, j i )+ ↵ in graph clustering to discover the contiguous Livehoods. j @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 26. Spectral Clustering • Given the affinity matrix A, we Ng, Jordon, and Weiss segment check-in venues using well studied spectral clustering (1) D is the diagonal degree matrix techniques (2) L = D A • We use the variation of spectral (3) Lnorm = D 1/2 LD1/2 (4) Find e1 , . . . , ek , the k clustering introduced by Ng, Jordan, smallest evects’s of Lnorm and Weiss [NIPS 2001] (5) E = [e1 , . . . , ek ] and let y1 , . . . , ynV be the rows - • We select k (number of clusters) as of E is common by looking for large gaps (6) Clustering y1 , . . . , ynV in consecutive eigenvalues with KMeans induces a clustering A1 , . . . , Ak of between and upper and lower the original data. allowable k. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 27. Post Processing • We introduce a post processing step to clean up any degenerate clusters • We separate the subgraph induced by each A into i connected components, creating new clusters for each. • We delete any clusters that span too large a geographic area (“background noise”) and reapportion the venues to the closest non-degenerate cluster by (single linkage) geographic distance @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 28. Related Livehoods To examine how the Livehoods are related to one another, for each pair of Livehoods, we compute a similarity score. For Livehood Ai the vector cAi is the bag of check-ins to Ai . Each component measures for a given user u the number of times u has checked in to any venue in Ai. cAi · cAj s(Ai , Aj ) = ||cAi || ||cAj || @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 29. Data @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 30. The Data • Foursquare check-ins are by default private • We can gather check-ins that have been shared publicly on Twitter. • Combine the 11 million foursquare check-ins from the dataset Chen et al. dataset [ICWSM 2011] with our own dataset of 7 million checkins gathered between June and December of 2011. • Aligned these Tweets with the underlying foursquare venue data (venue ID and venue category) @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 31. livehoods.org @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 32. Currently maps of New York, San Francisco Bay, Pittsburgh, and Montreal @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 33. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 34. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 35. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 36. Evaluation @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 38. How do we evaluate this? • Livehoods are different from neighborhoods, but how? In our evaluation, we want to characterize what Livehoods are. • Do residents derive social meaning from the Livehoods mapping? • Can Livehoods help elucidate the various forces that shape and define the city? • Quantitative (algorithmic) evaluation methods fall far short of capturing such concepts. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 40. Evaluation • To see how well our algorithm performed, we interviewed 27 residents of Pittsburgh • Residents recruited through a social media campaign, with various neighborhood groups and entities as seeds • Semi-structured Interview protocol explored the relationship among Livehoods, municipal borders, and the participants own perceptions of the city. • Participants must have lived in their neighborhood for at least 1 year @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 41. Interview Protocol • Each interview lasted approximately one hour • Began with a discussion of their backgrounds in relation their neighborhood. • Asked them to draw the boundaries of their neighborhood over it (their cognitive map). • Is there a “shift in feel” of the neighborhood? • Municipal borders changing? • Show Livehoods clusters (ask for feedback) • Show related Livehoods (ask for feedback) @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 42. Intersection Patterns • To guide our interview, we developed a framework to explore the categorize and explore the relationships between Livehoods and and municipal neighborhoods • Split: when a neighborhood is split into several Livehoods • Spilled: when a Livehoods spills over a municipal border • Corresponding: when neighborhood border and Livehood border correspond @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 43. Split Pattern One municipal neighborhood can be split into several Livehoods. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 44. Spilled Pattern A Livehood can spill across the boundaries of one or more municipal neighborhoods. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 45. Corresponding Pattern The borders between Livehoods and municipal neighborhoods can correspond with one another. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 46. Combining Patterns All these patterns can appear in across the Livehoods and municipal hoods of a city. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 47. Split Spilled Corresponding To explore split Spilled patterns were For corresponding patterns, we asked if explored by asking if patterns, we there was anywhere there was anywhere compared their that has a “distinct where the subject felt cognitive maps of the character or shift in the “municipal city with the shared feel” within the boundaries were municipal and neighborhood. shifting or in flux.” Livehoods borders. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 48. Interview Results @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 49. South Side Pittsburgh @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 50. South Side Pittsburgh @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 51. South Side Pittsburgh Carson Street runs along the length of South Side, and is densely packed with bars, restaurants, tattoo parlors, and clothing and furniture shops. It is the most popular destination for nightlife. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 52. South Side Pittsburgh South Side Works is a recently built, mixed-use outdoor shopping mall, containing nationally branded apparel stores and restaurants, upscale condominiums, and corporate offices. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 53. South Side Pittsburgh There is an small, somewhat older strip- mall that contains the only super market (grocery) in South Side. It also has a liquor store, an auto-parts store, a furniture rental store and other small chain stores. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 54. South Side Pittsburgh The rest of South Side is predominantly residential, consisting of mostly smaller row houses. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 55. South Side Pittsburgh My Cognitive Map of South Side @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 56. South Side Pittsburgh LH6 LH7 LH9 LH8 The Livehoods of South Side I’ll show the evidence in support of the Livehoods clusters in South Side, and will describe the forces that shape the city that the Livehoods highlight. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 57. South Side Pittsburgh LH6 LH7 LH9 LH8 LH8 vs LH9 “Ha! Yes! See, here is my division! Yay! Demographic Thank you algorithm! ... I definitely feel Differences where the South Side Works, and all of that is, is a very different feel.” @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 58. South Side Pittsburgh LH6 LH7 LH9 LH8 LH7 vs LH8 Architecture & “from an urban standpoint it is a lot tighter on the western part once you Urban Design get west of 17th or 18th [LH7].” @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 59. South Side Pittsburgh LH6 LH7 LH9 LH8 LH7 vs LH8 “Whenever I was living down on 15th Street [LH7] I had to worry about drunk people following me home, Safety but on 23rd [LH8] I need to worry about people trying to mug you... so it’s different. It’s not something I had anticipated, but there is a distinct difference between the two areas of the South Side.” @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 60. South Side Pittsburgh LH6 LH7 LH9 LH8 LH6 “There is this interesting mix of people there I don’t Demographic see walking around the neighborhood. I think they are coming to the Giant Eagle from lower income Differences neighborhoods...I always assumed they came from up the hill.” @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 61. South Side Pittsburgh “I always assumed they came from up the hill.”
  • 62. Shadyside and East Liberty A Teaser... @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 63. Shadyside @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 64. East Liberty @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 65. The Train Tracks @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 66. The Whole Foods @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 67. The Pedestrian Bridge @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 68. Conclusions • Throughout our interviews we found very strong evidence in support of the clustering • Interviews showed that residence found strong social meaning behind the Livehood clusters. • We also found that Livehoods can help shed light on the various forces that shape people’s behavior in the city, the city including demographics, economic factors, cultural perceptions and architecture. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 69. A Final Note on Qualitative Evaluations In this work, we gave a qualitative evaluation of a quantitative model. This idea is subtle and it might initially make some uncomfortable, especially those used to conducting machine learning research. However it’s an idea that deserve more thought and attention from the research community. Of course, we can’t conclude that our model is the best or even better than some known baseline, but our objective in this task is different from standard settings. We want to uncover what insights our model tells us about the social texture of city life. The only way to effectively explore the relationships between our model and the social realities of citizens is by going out and talking to them, learning about their lives and the ways that they perceive the urban landscape. Could other models uncover these insights? Absolutely. However, we can conclude based on a preponderance of evidence from our interviews, that Livehoods can be used as an effective tool for exploring the social dynamics of a city. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 70. Limitations • Most Livehoods had real social meaning to participants, but no algorithm is perfect. There are certainly Livehoods that don’t make sense. • There are obvious biases to using foursquare data. However this a limitation to the data, not our methodology. • There is experimenter bias associated with tuning the clustering parameters. • Some populations are left out (the digital divide) • We don’t want to overemphasize sharp divisions between Livehoods. In reality neighborhoods blend into one another. • This is not comparative work. We’re not making the claim that ours model is the best model for capturing the areas of a city, only that its a good model. @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 71. Thanks! Please explore our maps at livehoods.org You can also find us on on Facebook and Twitter @livehoods. Justin Cranshaw Raz Schwartz Jason I. Hong Norman Sadeh jcransh@cs.cmu.edu razs@andrew.cmu.edu jasonh@cs.cmu.edu sadeh@cs.cmu.edu School of Computer Science Carnegie Mellon University @livehoods | livehoods.org @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University
  • 72. http://distilleryimage7.s3.amazonaws.com/0a7e8f2c6ad711e180d51231380fcd7e_7.jpg Photo Credits http://distilleryimage0.s3.amazonaws.com/b57fcd849d6b11e1b10e123138105d6b_7.jpg http://distilleryimage9.s3.amazonaws.com/94a7a046453711e19e4a12313813ffc0_7.jpg http://distilleryimage3.s3.amazonaws.com/1733a942876411e1be6a12313820455d_7.jpg http://distilleryimage1.s3.amazonaws.com/22c1c74e6af911e180d51231380fcd7e_7.jpg http://instagr.am/p/LHByf3CtNl/ http://distilleryimage2.s3.amazonaws.com/70cb3dca9ebc11e18cf91231380fd29b_7.jpg http://distilleryimage9.s3.amazonaws.com/d7b9e2249bca11e1a92a1231381b6f02_7.jpg http://distilleryimage6.s3.amazonaws.com/4de101bc414411e1abb01231381b65e3_7.jpg http://distilleryimage7.s3.amazonaws.com/93bcdec8a04311e18bb812313804a181_7.jpg http://distilleryimage4.instagram.com/3d3ef156a9eb11e181bd12313817987b_7.jpg http://distilleryimage0.s3.amazonaws.com/db4691625bdd11e19896123138142014_7.jpg http://distilleryimage0.s3.amazonaws.com/65ba3af242d811e19896123138142014_7.jpg http://instagr.am/p/K_ZE9roeq6/ http://www.flickr.com/photos/award33/4730513546 http://distilleryimage7.s3.amazonaws.com/88a6ca749b8b11e180c9123138016265_7.jpg http://distilleryimage8.s3.amazonaws.com/9cc5f582919211e1be6a12313820455d_7.jpg http://distillery.s3.amazonaws.com/media/2011/10/07/1d2d0bc47db44acf9cf15017103af06a_7.jpg http://distilleryimage5.s3.amazonaws.com/fe99d252a69f11e1abd612313810100a_7.jpg http://instagr.am/p/LN_tt3MzHl/ http://distilleryimage8.s3.amazonaws.com/88d3de6639a111e1a87612313804ec91_7.jpg http://distilleryimage2.s3.amazonaws.com/d57f456c534011e1abb01231381b65e3_7.jpg http://instagr.am/p/K8O4JLoY-4/ http://distilleryimage8.s3.amazonaws.com/dfb52f1268b011e19e4a12313813ffc0_7.jpg http://distilleryimage1.s3.amazonaws.com/fe0235ce7a0c11e181bd12313817987b_7.jpg http://distilleryimage0.s3.amazonaws.com/2dc3f470923911e181bd12313817987b_7.jpg http://distilleryimage0.s3.amazonaws.com/0c6a6cac9af711e19dc71231380fe523_7.jpg http://distilleryimage5.s3.amazonaws.com/fe99d252a69f11e1abd612313810100a_7.jpg http://distilleryimage4.s3.amazonaws.com/d3db2b5ca36a11e18bb812313804a181_7.jpg http://distilleryimage7.s3.amazonaws.com/ca6d892a13f011e180c9123138016265_7.jpg @livehoods | livehoods.org Mobile Commerce Lab, Carnegie Mellon University