The geographical life of search

The Geographical Life of Search Ricardo Baeza-Yates(1), Christian Middleton(2), Carlos Castillo(1) (1) Yahoo! Research; Barcelona, Spain (2) Universitat Pompeu Fabra; Barcelona, Spain

Contents  Introduction  Experimental Framework  Generic Top-Level Domains (gTLD)  Country Top-Level Domains (ccTLD)  Internal Search Traffic of a Country  Traffic Among Continents and Countries  Geographic Dynamics  Conclusions

Introduction  Three-fold goals a) Understand how search engines are used from a geographic point of view b) Search process implies a social network, and geographical user behavior gives information of this network and how the Web reflects society c) Search traffic among countries is interesting for the development of distributed search engines

Introduction  Main objective “Describe how users behave, based on their location and clicked URL”  Explainfrom where a user need comes and where it is resolved  Implies traffic of information and transactions

Experimental Framework • Top-Level Domains (TLD) – Generic (gTLD) -- .org, .com, .edu, … – Country Code (ccTLD) -- .cl, .es, .fr, … • Query log – 840M clicked queries from Yahoo! log (2008) – Eliminated 0.01% of inter-country traffic – 593K unique gTLD hosts – 166K unique ccTLD hosts

Generic TLD (gTLD)  Traffic contribution to .com  Most countries have 65% traffic to .com (least is >45%)  Some countries have significant servers on .com: ◦ Canada, Spain, Great Britain

Generic TLD (gTLD) Traffic to other gTLD

Generic TLD (gTLD)  Which countries host gTLDs?  US is Top-1 on all domains  .gov & .edu are highly concentrated (use of secondary level)  .biz, .net, .mil are geographically spread

Server Location – Vanity TLDs  Vanity Score  Hostingprobability ◦ Non-uniform ◦ Median 0.66

Internal search traffic of country “Visits to pages in which both the searcher and the clicked page are in the same country.”  Ratio of internal destination:  Ratio of traffic from internal sources:

Internal search traffic of country  Sites on domain only  People only interested on relevant to people from that sites from their country. country.

Traffic to ccTLD ◦ At least 4% of outgoing traffic ◦ Omitted 60 countries with internal traffic ◦ Significance of .uk and .ru

Source & destination Entropy  Sources entropy  Destination entropy

Intra-country & Incoming Entropy  Inverse correlation (r = -0.772) “when ccTLD has narrow set of traffic sources, country is one of the most important sources”

Traffic amongst Continents  .uk visited from Europe, Oceania, and Africa  Diversified traffic in South America  .es sites highly influent in South America

Traffic Similarity  Linguistic/geograp hic characteristics: a) Indo-European b) French c) Dutch d) Swazi e) Tetum f) English

Traffic Similarity  Analyzed 24 demographic features  Laplacian Eigenmaps over country similarity graph ◦ Feature x such that minimizes:  Most relevant features: ◦ Oil consumption ◦ Latitude of center of country ◦ Human Development Index (HDI) ◦ Number of mobile phones ◦ Total area

Similarity between Continents

Geographic Dynamics  Compared with 2005 Yahoo! Log  Similar traffic to gTLDs, and location of servers hosting gTLDs.  In2005, European ccTLDs had more diversified sources, while in 2008 it was the Asian ccTLDs.  Demographic features: ◦ Latitude, HDI, unemployment rate, migration, and area of country. t This confirms our observation that countries sharing latitude or human development have similar traffic.

Conclusions  Analyzed data from large query log to describe the way the user behaves based on their location and the URL clicked.  Findings: ◦ .com has the largest share of traffic t Most hosts are located in US ◦ Inverse correlation between incoming traffic entropy and internal referral ratio. ◦ Traffic among countries is concentrated in few domains (.uk, .ru) – linguistic & cultural ◦ At a continent level, .uk is the most common domain among continents.

Conclusions  Findings: ◦ Users located in North America, Europe, Asia, and Africa visit similar set of domains. ◦ Users in South America and Oceania show different behavior. ◦ Results show that language can be more important than geography, based on search similarity. ◦ From demographics analysis, we observed that countries in similar latitudes or with similar HDI share traffic destinations.

The Geographical Life of Search Ricardo Baeza-Yates(1), Christian Middleton(2), Carlos Castillo(1) (1) Yahoo! Research; Barcelona, Spain (2) Universitat Pompeu Fabra; Barcelona, Spain

The geographical life of search

by Carlos Castillo on Oct 13, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 1

Statistics

The geographical life of search — Presentation Transcript