The Future of Search: How to Stay Relevant in Sourcing
The future of search: How to stay relevant in Sourcing Greg Lindahl CTO, Blekko October 13, 2011 - SourceCon
A li@le about me • Technologist, not a sourcer • I did get sourced once, in 1995 • I m proud to have the ugliest slides at the conference!
A li@le about you • You guys are heavy, sophisJcated users of Google & specialized engines like Topsy • You re eager to learn about new things • You quickly form opinions about what s useful
Challenges in Sourcing • The order of search results is based on incoming links & Page Rank • You guys are heavy users of advanced features such as boolean search • Social networks are walled gardens
Advanced interfaces • No one uses the advanced search interface of Google • You re lucky that it hasn t disappeared! • New Google algorithms trying to guess your intent are probably a net minus for Sourcers
Social Networks • Facebook is mostly a walled garden -‐ many users don t want their personal info to be public • Facebook mixes work and play • LinkedIn is a pure play, but younger people don t use it • Twi@er is open, but a mixture of work and play
Search implicaJons of social • General search engines won t be good enough, now or in the future • It will remain hard to try to match up candidates with several social accounts plus a web presence
Things you should know: BiGrams • Google counts: – java programmer OR developer: 80 million – java programmer OR java devloper : 23 million • Why? – Everyone indexes pairs and triples of words which are thought to be related • Names, job Jtles, common word pairs
Future search direcJons • semanJc search • 2 parts: – understanding the source documents be@er • bigrams of names just a start – understanding your query intent be@er • This will hurt advanced search and boolean queries!
Future search direcJons • real-‐Jme search: twi@er and non-‐twi@er • Non-‐twi@er real-‐Jme incorporated directly into the major search engines • Twi@er search best in specialized engines
New market entrants • blekko • yandex • duckduckgo
Algorithms + People = Better Search"ü Wikipedia model: Users idenJfy top sites for every category ü Technology: blekko uses social data plus algorithms to make more relevant, spam-‐free search results
Slashtag basics • Both algorthmic and human-‐curated • Curated slashtags developed in conjuncJon with outside partners, such as Stack Overﬂow • Type em directly into the search box: Greg Lindahl /date
Slashtags • Sort order: /date, /relevance • Narrow your search – Algorthmic: /forum /blog /gov /edu – Site: /foxnews.com – Curated list of websites: /health • Human-‐edited by groups (think dmoz or wikipedia) • Every user has their own namespace, plus there s a namespace for /blekko/ tags
Tips: • Use /ﬁndslashtags to ﬁnd tags: python /ﬁndslashags • Use /web to get rid of any unwanted autoslashtags • Watch the suggesJons for slashtag suggesJons as you type • We have some API outcalls that might save you Jme: /twi@er = Twi@er search API, /video = YouTube, /imges = bing, /bing, /google
Advanced Slashtags • Can be negated: -‐/foxnews.com • Implicit -‐/spam on every searce – and you can add any result to it with 1 click – use this to get rid of all those people ﬁnders • Can use mulJple tags to intersect /linux /blogs /date • Can include tags in other tags to OR them
Sourcer Slashtags • /people -‐-‐ algorithmic a@empt to ﬁnd resumes, bios, etc • /blogs and /forums, -‐/blogs and -‐/forums • Develop a /spam slashtag, or maybe even several of them that you manually add to various searches
A few examples • Java programers with high performance compuJng experience: java hpc /people • Followup on a candidate marcus wa@s java /blogs marcus wa@s java -‐/blogs marcus wa@s java -‐/blogs /date marcus wa@s java -‐/blogs /date=2008-‐2009 • Ok, more marcus wa@s /twi@er marcus wa@s /youtube -‐-‐ oops, basketball guy
The bad stuﬀ • blekko s crawl is fairly small, 2 billion pages today – increasing this Fall • Some of the programming slashtags aren t as good as others – this will improve over Jme
For more info • help.blekko.com – Add blekko to the list on the upper right search box • blekko toolbar
To Sum Up • Let me explain. No, there is too much. Let me sum up. • Search is evolving in good and bad ways • New tools pop up every year • I would love to hear feedback: firstname.lastname@example.org
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.