Future of Search | Yury Lifshits, Yahoo! Research


Published on


Published in: Technology, Design
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Future of Search | Yury Lifshits, Yahoo! Research

    1. 1. Yury Lifshits Yahoo! Research http://yury.name Future of Search St. Petersburg | Helsinki December 2008
    2. 2. Outline <ul><li>Structured Search </li></ul><ul><li>Yahoo! Work in Search </li></ul><ul><ul><li>SearchMonkey </li></ul></ul><ul><ul><li>BOSS </li></ul></ul><ul><li>Research Agenda </li></ul>
    3. 3. Structured Search: work in progress
    4. 4. Structured Search = Bring structured data to search users M.K. Bergman. The Deep Web: Surfacing Hidden Value. 2001.
    5. 5. Value Proposition <ul><li>Coverage </li></ul><ul><ul><li>Real-time data </li></ul></ul><ul><ul><li>Semi-private data </li></ul></ul><ul><li>Structured queries </li></ul><ul><li>Ordering and filtering results </li></ul><ul><li>Straight-to-answers </li></ul>
    6. 6. User Interface: Query <ul><li>Search assist: Yahoo! </li></ul><ul><li>Selector: LinkedIn, VKontakte.ru </li></ul><ul><li>Multiple search buttons: Gmail </li></ul><ul><li>Search tabs: Yahoo / Google </li></ul>
    7. 7. User Interface: Results <ul><li>Federated page </li></ul><ul><li>Facets </li></ul><ul><li>Search transfer / search form </li></ul>K.P. Yee, K. Swearingen, K. Li, M. Hearst. Faceted metadata for image search and browsing. CHI 2003. Fernando Diaz. Aggregation of News Content Into Web Results. WSDM 2009. http:// glue.yahoo.com http:// au.alpha.yahoo.com
    8. 10. Data Supply Chain <ul><li>Atomic fact </li></ul><ul><li>Flight, Event, Patent </li></ul><ul><li>Data aggregator </li></ul><ul><li>US Patents, Amadeus/Sabre flights, Upcoming.com </li></ul><ul><li>Domain search </li></ul><ul><li>Expedia, Spock </li></ul><ul><li>General purpose search </li></ul><ul><li>Yahoo!, Google, Yandex, Baidu </li></ul>
    9. 11. Getting structured data <ul><li>Entity extraction </li></ul><ul><li>Markup </li></ul><ul><li>Feeds </li></ul><ul><li>Search API (OpenSearch) </li></ul><ul><li>OR </li></ul><ul><li>Do a search transfer </li></ul>
    10. 12. Give Us Your Data For … <ul><li>Traffic via search transfer </li></ul><ul><li>Firefox search box </li></ul><ul><li>Better presentation in search </li></ul><ul><li>SearchMonkey </li></ul><ul><li>Hosted search </li></ul><ul><li>BOSS Custom </li></ul><ul><li>Showing your ads </li></ul><ul><li>Yahoo Local + AT&T </li></ul>
    11. 13. Yahoo! Work in Search
    12. 14. Slides by: Paul Tarjan, Chief Technical Monkey ( [email_address] ) Full version http://www.slideshare.net/ptarjan/searchmonkey-presentation
    13. 15. What is SearchMonkey? an open platform for using structured data to build more useful and relevant search results Before After
    14. 16. Enhanced Result: Zagat Key/Value Pairs or Abstract Links Image
    15. 17. Infobar: Wikipedia Preview Summary Blob
    16. 18. Creating an Infobar <ul><li>Infobar advantages </li></ul><ul><ul><li>Annotate someone else’s site </li></ul></ul><ul><ul><li>Use links and images from other domains </li></ul></ul><ul><ul><ul><li>Mash up info from multiple sites </li></ul></ul></ul><ul><ul><ul><li>Affiliate / coupon links? Hmmm… </li></ul></ul></ul><ul><ul><li>Can act on *, all websites </li></ul></ul><ul><ul><ul><li>But these apps can be annoying if poorly designed </li></ul></ul></ul><ul><li>Key design principles </li></ul><ul><ul><li>Put something useful in the summary </li></ul></ul><ul><ul><li>Be creative with the HTML </li></ul></ul>
    17. 19. How to get data to SearchMonkey? <ul><li>Humans see: </li></ul><ul><li>name </li></ul><ul><li>picture of a person </li></ul><ul><li>current job </li></ul><ul><li>industry, … </li></ul><ul><li>Computers see: </li></ul><ul><li>an undifferentiated </li></ul><ul><li>blob of HTML </li></ul><ul><li>Can we make computers smarter? </li></ul>
    18. 20. How does it work? Acme.com’s database Index RDF/Microformat Markup site owners/publishers share structured data with Yahoo!. 1 consumers customize their search experience with Enhanced Results or Infobars 3 site owners & third-party developers build SearchMonkey apps. 2 DataRSS feed Web Services Page Extraction Acme.com’s Web Pages
    19. 21. SearchMonkey Resources <ul><li>Main: </li></ul><ul><ul><li>http:// developer.yahoo.com/searchmonkey </li></ul></ul><ul><li>Lists and forums: </li></ul><ul><ul><li>[email_address] </li></ul></ul><ul><ul><li>http:// suggestions.yahoo.com/searchmonkey </li></ul></ul>
    20. 22. Vik Singh (Architect) Graham Mudd (Senior PMM)
    21. 23. BOSS = B uild your O wn S earch S ervice Open Yahoo’s core search features via web services to let 3rd parties revolutionize Search Unrestricted What
    22. 24. <ul><li>Unrestricted: </li></ul><ul><li>Unlimited queries </li></ul><ul><li>Blend, re-order, discard </li></ul><ul><li>Full presentation control </li></ul><ul><li>Non-search apps OK </li></ul><ul><li>Monetization: Free or CPM or Ads </li></ul>What
    23. 25. <ul><li>Barriers to entry are massive </li></ul><ul><li>$300M, top talent, a prayer to get to basic parity </li></ul><ul><li>No monopoly over great ideas </li></ul><ul><li>Search anywhere </li></ul><ul><li>Improve Vertical Quality w/ Web comprehensiveness </li></ul><ul><li>Fragment the market, foster more players, choice, competition </li></ul><ul><li>Yahoo extends advertising reach, 3rd parties revenue share </li></ul>Why
    24. 26. Why Traditional Search Distribution + BOSS Distribution
    25. 27. Tracks API A self-service, web services model for developers and start-ups to quickly build and deploy new search experiences. <ul><ul><li>UIUC </li></ul></ul><ul><ul><li>CMU </li></ul></ul><ul><ul><li>Stanford </li></ul></ul><ul><ul><li>Purdue </li></ul></ul><ul><ul><li>IIT Bombay </li></ul></ul><ul><ul><li>MIT </li></ul></ul><ul><ul><li>UMass </li></ul></ul>CUSTOM Working with 3rd parties to build a more relevant, brand/site specific web search experience. This option is jointly built by Yahoo! and select partners. <ul><ul><li>ACADEMIC </li></ul></ul><ul><ul><li>Working with the following universities to allow for wide-scale research in the search field: </li></ul></ul>Interested in Custom? Email us [email_address]
    26. 28. http://boss.yahooapis.com/ysearch/{vert}/v1/{q} {vert} := {web, news, images, spelling} @ required appid @ optional (Y!OS compliant) start, count, lang, region, format, callback, sites BOSS API v1
    27. 29. Python (v2.5+) library BOSS Search SDK plus … SQL for remixing arbitrary XML/JSON sources Loosely Functional programming paradigm BOSS Mashup Framework
    28. 30. Ported enhanced version of BMF to GAE platform http://zooie.wordpress.com/2008/08/04/yahoo-boss-google-app-engine-integrated/ Easiest way to deploy a BOSS application online BMF + Google App Engine
    29. 31. http://www.4hoursearch.com http://123people.com Mashable! Contest for BOSS search engines http://mashable.com/boss/ Examples
    30. 32. BOSS Custom for TechCrunch
    31. 33. TechCrunch Network Search <ul><li>CrunchBase + Posts + Web </li></ul><ul><li>Sort by time / relevance </li></ul><ul><li>Enhanced results </li></ul><ul><li>Domain-specific facets </li></ul><ul><li>Yahoo! sponsored search </li></ul><ul><li>Real-time indexing </li></ul><ul><li>Special results </li></ul>
    32. 34. Research Agenda
    33. 35. Structured Search <ul><li>Analysis of search demand </li></ul><ul><ul><li>Intent classification </li></ul></ul><ul><ul><li>General search vs. vertical </li></ul></ul><ul><li>Incentives in data supply </li></ul><ul><li>Push & real-time indexing </li></ul><ul><li>Search user interface </li></ul><ul><ul><li>One box vs. multi-box </li></ul></ul><ul><ul><li>General vs. vertical </li></ul></ul><ul><li>Deciding search transfer </li></ul><ul><ul><li>When? </li></ul></ul><ul><ul><li>To whom? </li></ul></ul>
    34. 36. Key Scientific Challenges Draft: http://research.yahoo.com/ksc <ul><li>Search intent </li></ul><ul><li>Quality metrics </li></ul><ul><li>Web mining </li></ul><ul><li>Multilingual IR </li></ul><ul><li>Nextgen search </li></ul><ul><ul><li>Synthesized result pages </li></ul></ul><ul><li>World knowledge </li></ul>A.Z. Broder. Taxonomy of web search. SIGIR 2002.
    35. 37. More Problems <ul><li>Discovery search </li></ul><ul><li>Web search vs. asking people </li></ul><ul><li>Event search </li></ul>
    36. 38. Thanks for your attention! <ul><li>Yury Lifshits </li></ul><ul><li>http://yury.name </li></ul><ul><li>[email_address] </li></ul>