Daniel




Content, Connections, and Context
Daniel Tunkelang
Principal Data Scientist at LinkedIn

      Recruiting Solutions                      1
Recommendation Products at LinkedIn
                             Similar Profiles




                                  Connections




           Network updates
                                    Events You May
                                    Be Interested In




                                 News




                                                       2
More than 50%
Recommendations drive:
> 50% of connections
            > 50% of job applications
                         > 50% of group joins




                                                3
Inputs for Recommender Systems
               Social Graph
 Content

                                 Behavior

                                     Queries
                                 Page Views
                                    Actions




           …
                                               4
Take-Aways

  Content is king.


  Connections provide social dimension.


  Context determines where and when
  a recommendation is appropriate.


                                          5
What is the goal of recommendations?


O  Growth


O  Engagement


O  Revenue



                                       6
What is the goal of recommendations?


O  Growth


O  Engagement


O  Revenue



                                       7
Users First!


          Provide relevant content


      and establish social connections


           in appropriate context.



                                         8
Content is King




                  9
Goal-Seeking is about Content




                                10
Content (Ir)relevance

§  No right answer, but many wrong answers.




  %no                         WTF!
   [Voorhees, 2004]             @k
    http://bit.ly/percentno   http://bit.ly/wtfatk



                                                     11
Example: Related Searches




                            12
Collaborative Filtering as Content Signal

§  Use temporal locality within sessions.

§  Find queries with clicks on similar results.

§  Look for query overlap.

§  Learn more at CIKM! [Reda et al, 2012]

                                                   13
Example: Jobs You Might Be Interested In




                                           14
Content Signals Dominate Social Signals
              Job                                             Corpus Stats
                                           Matching   Transition probabilities
                                                      Connectivity
                                   Binary             yrs of experience to reach title
title         industry       …
                                     Exact matches:   education needed for this title
geo           description                             …
company       functional area        geo, industry,
                                     …

          User Base                Soft                              Similarity
                                                        (candidate expertise, job description)
                                     transition
           Filtered                                                    0.56
                                     probabilities,
                                                                     Similarity
          Candidate                  similarity,       (candidate specialties, job description)
                                     …                                  0.2
                                                               Transition probability
                                   Text                   (candidate industry, job industry)
General       Current Position                                         0.43
expertise     title
specialties   summary                                               Title Similarity

education     tenure length                                             0.8
headline      industry
                                                              Similarity (headline, title)
geo           functional area
experience    …                                                         0.7
                                                                          .
                      derive
                               d
                                                                          .
                                                                          .
                                                                                             15
Summary


      Provide relevant content


    and establish social connections


          in appropriate context.



                                       16
Connections




              17
Connections are Social Dimensions




                                    18
Example: People You May Know




                               19
Relationships are Social




                           20
Shared Connections as a Signal




                                 21
The Power of Social Proof




                            22
Beyond Triadic Closure

§  Triads suggest and affect relationships.
    [Simmel, 1908], [Granovetter, 1973]

§  Triangle closing is a Big Data problem.
    [Shah, 2011]

§  Use machine learning to rank candidates.



                                               23
Summary


          Provide relevant content


   and establish social connections


          in appropriate context.



                                      24
Context




     "to every thing there is a season”
             [Ecclesiastes 3:1]

                                          25
One Platform, Many Users, Many Needs




          http://blog.lab42.com/the-linkedin-profile


                                                       26
No Weekend Meetings!


 Weekdays

                       navigational queries


                       exploratory queries
        Weekends



                                             27
Different Devices for Different Needs




                                        28
Time and Place for a Career Change




                                     29
Active vs. Passive Job Seekers




                  >10x more likely
                  to reply if active
                                 [Rodriguez et al, 2012]


                                                           30
Summary


      Provide relevant content


    and establish social connections


          in appropriate context.



                                       31
Recap

  Content is king.


  Connections provide social dimension.


  Context determines where and when
  a recommendation is appropriate.


                                          32
Thank You!
                                     175M+           2/sec
                                     62% non U.S.


                                                    25th
                               90          We’re    Most visit website worldwide
                                                    (Comscore 6-12)



                          55
                                          Hiring!   >2M
                                                    Company pages



                                                    85%
                    32

               17
           8
 2    4                                             Fortune 500 Companies use
                                                    LinkedIn to hire
2004 2005 2006 2007 2008 2009 2010 2011
          LinkedIn Members (Millions)



          Learn more at http://data.linkedin.com/
                                                                                   33

Content, Connections, and Context

  • 1.
    Daniel Content, Connections, andContext Daniel Tunkelang Principal Data Scientist at LinkedIn Recruiting Solutions 1
  • 2.
    Recommendation Products atLinkedIn Similar Profiles Connections Network updates Events You May Be Interested In News 2
  • 3.
    More than 50% Recommendationsdrive: > 50% of connections > 50% of job applications > 50% of group joins 3
  • 4.
    Inputs for RecommenderSystems Social Graph Content Behavior Queries Page Views Actions … 4
  • 5.
    Take-Aways Contentis king. Connections provide social dimension. Context determines where and when a recommendation is appropriate. 5
  • 6.
    What is thegoal of recommendations? O  Growth O  Engagement O  Revenue 6
  • 7.
    What is thegoal of recommendations? O  Growth O  Engagement O  Revenue 7
  • 8.
    Users First! Provide relevant content and establish social connections in appropriate context. 8
  • 9.
  • 10.
  • 11.
    Content (Ir)relevance §  Noright answer, but many wrong answers. %no WTF! [Voorhees, 2004] @k http://bit.ly/percentno http://bit.ly/wtfatk 11
  • 12.
  • 13.
    Collaborative Filtering asContent Signal §  Use temporal locality within sessions. §  Find queries with clicks on similar results. §  Look for query overlap. §  Learn more at CIKM! [Reda et al, 2012] 13
  • 14.
    Example: Jobs YouMight Be Interested In 14
  • 15.
    Content Signals DominateSocial Signals Job Corpus Stats Matching Transition probabilities Connectivity Binary yrs of experience to reach title title industry … Exact matches: education needed for this title geo description … company functional area geo, industry, … User Base Soft Similarity (candidate expertise, job description) transition Filtered 0.56 probabilities, Similarity Candidate similarity, (candidate specialties, job description) … 0.2 Transition probability Text (candidate industry, job industry) General Current Position 0.43 expertise title specialties summary Title Similarity education tenure length 0.8 headline industry Similarity (headline, title) geo functional area experience … 0.7 . derive d . . 15
  • 16.
    Summary Provide relevant content and establish social connections in appropriate context. 16
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
    The Power ofSocial Proof 22
  • 23.
    Beyond Triadic Closure § Triads suggest and affect relationships. [Simmel, 1908], [Granovetter, 1973] §  Triangle closing is a Big Data problem. [Shah, 2011] §  Use machine learning to rank candidates. 23
  • 24.
    Summary Provide relevant content and establish social connections in appropriate context. 24
  • 25.
    Context "to every thing there is a season” [Ecclesiastes 3:1] 25
  • 26.
    One Platform, ManyUsers, Many Needs http://blog.lab42.com/the-linkedin-profile 26
  • 27.
    No Weekend Meetings! Weekdays navigational queries exploratory queries Weekends 27
  • 28.
    Different Devices forDifferent Needs 28
  • 29.
    Time and Placefor a Career Change 29
  • 30.
    Active vs. PassiveJob Seekers >10x more likely to reply if active [Rodriguez et al, 2012] 30
  • 31.
    Summary Provide relevant content and establish social connections in appropriate context. 31
  • 32.
    Recap Contentis king. Connections provide social dimension. Context determines where and when a recommendation is appropriate. 32
  • 33.
    Thank You! 175M+ 2/sec 62% non U.S. 25th 90 We’re Most visit website worldwide (Comscore 6-12) 55 Hiring! >2M Company pages 85% 32 17 8 2 4 Fortune 500 Companies use LinkedIn to hire 2004 2005 2006 2007 2008 2009 2010 2011 LinkedIn Members (Millions) Learn more at http://data.linkedin.com/ 33