Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Yury Lifshits Yahoo! Research http://yury.name Future of Search St. Petersburg | Helsinki December 2008
Outline <ul><li>Structured Search </li></ul><ul><li>Yahoo! Work in Search  </li></ul><ul><ul><li>SearchMonkey </li></ul></...
Structured Search: work in progress
Structured Search = Bring structured data to search users M.K. Bergman. The Deep Web: Surfacing Hidden Value. 2001.
Value Proposition <ul><li>Coverage </li></ul><ul><ul><li>Real-time data </li></ul></ul><ul><ul><li>Semi-private data </li>...
User Interface: Query <ul><li>Search assist: Yahoo! </li></ul><ul><li>Selector: LinkedIn, VKontakte.ru </li></ul><ul><li>M...
User Interface: Results <ul><li>Federated page  </li></ul><ul><li>Facets </li></ul><ul><li>Search transfer / search form <...
 
 
Data Supply Chain <ul><li>Atomic fact  </li></ul><ul><li>Flight, Event, Patent </li></ul><ul><li>Data aggregator </li></ul...
Getting structured data <ul><li>Entity extraction </li></ul><ul><li>Markup </li></ul><ul><li>Feeds </li></ul><ul><li>Searc...
Give Us Your Data For …  <ul><li>Traffic via search transfer </li></ul><ul><li>Firefox search box   </li></ul><ul><li>Bett...
Yahoo! Work in Search
Slides by: Paul Tarjan, Chief Technical Monkey ( [email_address] ) Full version  http://www.slideshare.net/ptarjan/searchm...
What is SearchMonkey? an open platform for using structured data to build more useful and relevant search results Before A...
Enhanced Result: Zagat Key/Value Pairs or Abstract Links Image
Infobar: Wikipedia Preview Summary Blob
Creating an Infobar <ul><li>Infobar advantages </li></ul><ul><ul><li>Annotate someone else’s site </li></ul></ul><ul><ul><...
How to get data to SearchMonkey? <ul><li>Humans see: </li></ul><ul><li>name </li></ul><ul><li>picture of a person </li></u...
How does it work? Acme.com’s database Index RDF/Microformat Markup site owners/publishers share structured data with Yahoo...
SearchMonkey Resources <ul><li>Main: </li></ul><ul><ul><li>http:// developer.yahoo.com/searchmonkey </li></ul></ul><ul><li...
Vik Singh  (Architect) Graham Mudd  (Senior PMM)
BOSS =  B uild your  O wn  S earch  S ervice Open Yahoo’s core search features via web services to let 3rd parties revolut...
<ul><li>Unrestricted: </li></ul><ul><li>Unlimited queries </li></ul><ul><li>Blend, re-order, discard </li></ul><ul><li>Ful...
<ul><li>Barriers to entry are massive </li></ul><ul><li>$300M, top talent, a prayer to get to basic parity </li></ul><ul><...
Why Traditional Search Distribution + BOSS Distribution
Tracks API A self-service, web services model for developers and start-ups to quickly build and deploy new search experien...
http://boss.yahooapis.com/ysearch/{vert}/v1/{q} {vert} := {web, news, images, spelling} @ required appid @ optional  (Y!OS...
Python (v2.5+) library BOSS Search SDK plus … SQL for remixing  arbitrary  XML/JSON sources Loosely Functional programming...
Ported enhanced version of BMF to GAE platform http://zooie.wordpress.com/2008/08/04/yahoo-boss-google-app-engine-integrat...
http://www.4hoursearch.com http://123people.com Mashable! Contest for BOSS search engines http://mashable.com/boss/   Exam...
BOSS Custom for TechCrunch
TechCrunch Network Search <ul><li>CrunchBase + Posts + Web </li></ul><ul><li>Sort by time / relevance </li></ul><ul><li>En...
Research Agenda
Structured Search <ul><li>Analysis of search demand  </li></ul><ul><ul><li>Intent classification </li></ul></ul><ul><ul><l...
Key Scientific Challenges Draft:  http://research.yahoo.com/ksc   <ul><li>Search intent </li></ul><ul><li>Quality metrics ...
More Problems <ul><li>Discovery search </li></ul><ul><li>Web search vs. asking people </li></ul><ul><li>Event search </li>...
Thanks for your attention! <ul><li>Yury Lifshits  </li></ul><ul><li>http://yury.name   </li></ul><ul><li>[email_address]  ...
Upcoming SlideShare
Loading in …5
×

Future of Search | Yury Lifshits, Yahoo! Research

http://yury.name

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

Future of Search | Yury Lifshits, Yahoo! Research

  1. 1. Yury Lifshits Yahoo! Research http://yury.name Future of Search St. Petersburg | Helsinki December 2008
  2. 2. Outline <ul><li>Structured Search </li></ul><ul><li>Yahoo! Work in Search </li></ul><ul><ul><li>SearchMonkey </li></ul></ul><ul><ul><li>BOSS </li></ul></ul><ul><li>Research Agenda </li></ul>
  3. 3. Structured Search: work in progress
  4. 4. Structured Search = Bring structured data to search users M.K. Bergman. The Deep Web: Surfacing Hidden Value. 2001.
  5. 5. Value Proposition <ul><li>Coverage </li></ul><ul><ul><li>Real-time data </li></ul></ul><ul><ul><li>Semi-private data </li></ul></ul><ul><li>Structured queries </li></ul><ul><li>Ordering and filtering results </li></ul><ul><li>Straight-to-answers </li></ul>
  6. 6. User Interface: Query <ul><li>Search assist: Yahoo! </li></ul><ul><li>Selector: LinkedIn, VKontakte.ru </li></ul><ul><li>Multiple search buttons: Gmail </li></ul><ul><li>Search tabs: Yahoo / Google </li></ul>
  7. 7. User Interface: Results <ul><li>Federated page </li></ul><ul><li>Facets </li></ul><ul><li>Search transfer / search form </li></ul>K.P. Yee, K. Swearingen, K. Li, M. Hearst. Faceted metadata for image search and browsing. CHI 2003. Fernando Diaz. Aggregation of News Content Into Web Results. WSDM 2009. http:// glue.yahoo.com http:// au.alpha.yahoo.com
  8. 10. Data Supply Chain <ul><li>Atomic fact </li></ul><ul><li>Flight, Event, Patent </li></ul><ul><li>Data aggregator </li></ul><ul><li>US Patents, Amadeus/Sabre flights, Upcoming.com </li></ul><ul><li>Domain search </li></ul><ul><li>Expedia, Spock </li></ul><ul><li>General purpose search </li></ul><ul><li>Yahoo!, Google, Yandex, Baidu </li></ul>
  9. 11. Getting structured data <ul><li>Entity extraction </li></ul><ul><li>Markup </li></ul><ul><li>Feeds </li></ul><ul><li>Search API (OpenSearch) </li></ul><ul><li>OR </li></ul><ul><li>Do a search transfer </li></ul>
  10. 12. Give Us Your Data For … <ul><li>Traffic via search transfer </li></ul><ul><li>Firefox search box </li></ul><ul><li>Better presentation in search </li></ul><ul><li>SearchMonkey </li></ul><ul><li>Hosted search </li></ul><ul><li>BOSS Custom </li></ul><ul><li>Showing your ads </li></ul><ul><li>Yahoo Local + AT&T </li></ul>
  11. 13. Yahoo! Work in Search
  12. 14. Slides by: Paul Tarjan, Chief Technical Monkey ( [email_address] ) Full version http://www.slideshare.net/ptarjan/searchmonkey-presentation
  13. 15. What is SearchMonkey? an open platform for using structured data to build more useful and relevant search results Before After
  14. 16. Enhanced Result: Zagat Key/Value Pairs or Abstract Links Image
  15. 17. Infobar: Wikipedia Preview Summary Blob
  16. 18. Creating an Infobar <ul><li>Infobar advantages </li></ul><ul><ul><li>Annotate someone else’s site </li></ul></ul><ul><ul><li>Use links and images from other domains </li></ul></ul><ul><ul><ul><li>Mash up info from multiple sites </li></ul></ul></ul><ul><ul><ul><li>Affiliate / coupon links? Hmmm… </li></ul></ul></ul><ul><ul><li>Can act on *, all websites </li></ul></ul><ul><ul><ul><li>But these apps can be annoying if poorly designed </li></ul></ul></ul><ul><li>Key design principles </li></ul><ul><ul><li>Put something useful in the summary </li></ul></ul><ul><ul><li>Be creative with the HTML </li></ul></ul>
  17. 19. How to get data to SearchMonkey? <ul><li>Humans see: </li></ul><ul><li>name </li></ul><ul><li>picture of a person </li></ul><ul><li>current job </li></ul><ul><li>industry, … </li></ul><ul><li>Computers see: </li></ul><ul><li>an undifferentiated </li></ul><ul><li>blob of HTML </li></ul><ul><li>Can we make computers smarter? </li></ul>
  18. 20. How does it work? Acme.com’s database Index RDF/Microformat Markup site owners/publishers share structured data with Yahoo!. 1 consumers customize their search experience with Enhanced Results or Infobars 3 site owners & third-party developers build SearchMonkey apps. 2 DataRSS feed Web Services Page Extraction Acme.com’s Web Pages
  19. 21. SearchMonkey Resources <ul><li>Main: </li></ul><ul><ul><li>http:// developer.yahoo.com/searchmonkey </li></ul></ul><ul><li>Lists and forums: </li></ul><ul><ul><li>[email_address] </li></ul></ul><ul><ul><li>http:// suggestions.yahoo.com/searchmonkey </li></ul></ul>
  20. 22. Vik Singh (Architect) Graham Mudd (Senior PMM)
  21. 23. BOSS = B uild your O wn S earch S ervice Open Yahoo’s core search features via web services to let 3rd parties revolutionize Search Unrestricted What
  22. 24. <ul><li>Unrestricted: </li></ul><ul><li>Unlimited queries </li></ul><ul><li>Blend, re-order, discard </li></ul><ul><li>Full presentation control </li></ul><ul><li>Non-search apps OK </li></ul><ul><li>Monetization: Free or CPM or Ads </li></ul>What
  23. 25. <ul><li>Barriers to entry are massive </li></ul><ul><li>$300M, top talent, a prayer to get to basic parity </li></ul><ul><li>No monopoly over great ideas </li></ul><ul><li>Search anywhere </li></ul><ul><li>Improve Vertical Quality w/ Web comprehensiveness </li></ul><ul><li>Fragment the market, foster more players, choice, competition </li></ul><ul><li>Yahoo extends advertising reach, 3rd parties revenue share </li></ul>Why
  24. 26. Why Traditional Search Distribution + BOSS Distribution
  25. 27. Tracks API A self-service, web services model for developers and start-ups to quickly build and deploy new search experiences. <ul><ul><li>UIUC </li></ul></ul><ul><ul><li>CMU </li></ul></ul><ul><ul><li>Stanford </li></ul></ul><ul><ul><li>Purdue </li></ul></ul><ul><ul><li>IIT Bombay </li></ul></ul><ul><ul><li>MIT </li></ul></ul><ul><ul><li>UMass </li></ul></ul>CUSTOM Working with 3rd parties to build a more relevant, brand/site specific web search experience. This option is jointly built by Yahoo! and select partners. <ul><ul><li>ACADEMIC </li></ul></ul><ul><ul><li>Working with the following universities to allow for wide-scale research in the search field: </li></ul></ul>Interested in Custom? Email us [email_address]
  26. 28. http://boss.yahooapis.com/ysearch/{vert}/v1/{q} {vert} := {web, news, images, spelling} @ required appid @ optional (Y!OS compliant) start, count, lang, region, format, callback, sites BOSS API v1
  27. 29. Python (v2.5+) library BOSS Search SDK plus … SQL for remixing arbitrary XML/JSON sources Loosely Functional programming paradigm BOSS Mashup Framework
  28. 30. Ported enhanced version of BMF to GAE platform http://zooie.wordpress.com/2008/08/04/yahoo-boss-google-app-engine-integrated/ Easiest way to deploy a BOSS application online BMF + Google App Engine
  29. 31. http://www.4hoursearch.com http://123people.com Mashable! Contest for BOSS search engines http://mashable.com/boss/ Examples
  30. 32. BOSS Custom for TechCrunch
  31. 33. TechCrunch Network Search <ul><li>CrunchBase + Posts + Web </li></ul><ul><li>Sort by time / relevance </li></ul><ul><li>Enhanced results </li></ul><ul><li>Domain-specific facets </li></ul><ul><li>Yahoo! sponsored search </li></ul><ul><li>Real-time indexing </li></ul><ul><li>Special results </li></ul>
  32. 34. Research Agenda
  33. 35. Structured Search <ul><li>Analysis of search demand </li></ul><ul><ul><li>Intent classification </li></ul></ul><ul><ul><li>General search vs. vertical </li></ul></ul><ul><li>Incentives in data supply </li></ul><ul><li>Push & real-time indexing </li></ul><ul><li>Search user interface </li></ul><ul><ul><li>One box vs. multi-box </li></ul></ul><ul><ul><li>General vs. vertical </li></ul></ul><ul><li>Deciding search transfer </li></ul><ul><ul><li>When? </li></ul></ul><ul><ul><li>To whom? </li></ul></ul>
  34. 36. Key Scientific Challenges Draft: http://research.yahoo.com/ksc <ul><li>Search intent </li></ul><ul><li>Quality metrics </li></ul><ul><li>Web mining </li></ul><ul><li>Multilingual IR </li></ul><ul><li>Nextgen search </li></ul><ul><ul><li>Synthesized result pages </li></ul></ul><ul><li>World knowledge </li></ul>A.Z. Broder. Taxonomy of web search. SIGIR 2002.
  35. 37. More Problems <ul><li>Discovery search </li></ul><ul><li>Web search vs. asking people </li></ul><ul><li>Event search </li></ul>
  36. 38. Thanks for your attention! <ul><li>Yury Lifshits </li></ul><ul><li>http://yury.name </li></ul><ul><li>[email_address] </li></ul>

×