Search

Semantic Searchmonkey

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

2 comments

Comments 1 - 2 of 2 previous next Post a comment

Post a comment
Embed Video
Edit your comment Cancel

Notes on slide 1

A SearchMonkey Enhanced result contains a great deal of structured data. It could have a picture, key/value pairs, deep links…This kind of information goes far beyond what normal search results give you – a title and an autoextracted summary. Where does this information come from?

Likewise, an Infobar has a summary (what the user sees before the pane is expanded) and a “blob”, an area of free-form HTML.

XSLT custom data services are excellent when there is no good structured data available, either because you don’t own the site in question, or because you just want to get a prototype out quickly without having to to change your site’s template markup. You can use these data services to mock up what is possible with SearchMonkey.As with the PHP, the XSLT is fairly simple. The “hard” part of writing the stylesheet is really just finding the right xpath expression for extracting the information you want. The other thing you need to do is pick a good vocabulary for describing the extracted data. For example, a description is a dc:description (Dublin Core description) and so on.If the page is not well-formed XHTML, have no fear, we tidy up the page ahead of time and run the XSLT on that. The tidying can fail, but only if the markup is really pathologically bad.As we mentioned before, XSLT custom data services are good for mocking up Enhanced Results, but they’re too slow in practice. For a production-quality app, you’ll need to use them in infobars.[Show demo]

10 Favorites

Semantic Searchmonkey - Presentation Transcript

  1. Monkey with the Semantic Web
  2. SearchMonkey Presentation by: Paul Tarjan, Chief Technical Monkey (ptarjan@yahoo-inc.com) Online at: http://www.slideshare.net/ptarjan/semantic-searchmonkey
  3. The web was / is fragmented Funny pictures Super secret military site Friend’s website University Cool event page bookmarks
  4. So we added search to find stuff Google Yahoo Super Funny secret pictures military site Friend’s University Cool website event page bookmarks
  5. But there are many similar sites Facebook Events Evite Events Upcoming Events Youtube Metacafe Vimeo Digg Reddit Technorati Let’s treat these as “views” onto “objects”
  6. Wouldn’t it be cool if you could do: •  object:video creator:”Paul Tarjan” length<=60s
  7. Wouldn’t it be cool if you could do: •  object:video creator:http://paulisageek.com/ length<=60s
  8. Wouldn’t it be cool if you could do: •  object:game name:”Desktop Tower Defense” version:1.5 publishdate:”May 2 2005”
  9. Wouldn’t it be cool if you could do: •  object:video author:”The Escapist” game:”Left 4 Dead”
  10. It gets even cooler
  11. Aggregation: •  object:review type:camera make:canon model:D40
  12. Aggregation: •  object:event date:”May 16, 2008” type:party price<$5
  13. Aggregation: •  object:photo person:“Paul Tarjan”
  14. Aggregation: •  object:photo person:http://paulisageek.com
  15. The Semantic What? •  Web pages are views of data for people to read •  Search Engines are a hack •  They treat pages as a bucket of words •  Lets turn the web into a database •  APIs are good, but there is no “web” of APIs •  If you figure out a good way of doing that, let me know 
  16. Ok, I want to do it. Now what?
  17. Recommendation: µF •  If there is a microformat for your data, use it –  hcard –  hreview –  hresume –  hcalendar –  rel-tag –  rel-licence –  xfn –  hatom –  geo
  18. µF in a nutshell •  Change your @class to something that is known •  <div> –  <span class=“name”>Paul Tarjan</span> –  <span class=‘email’>spam@paulisageek.com</span> •  </div> •  BECOMES •  <div class=“vcard”> –  <span class=“fn”>Paul Tarjan</span> –  <span class=“email”>spam@paulisageek.com</span> •  </div>
  19. Recommendation: RDFa •  If you have data that doesn’t really fit in a µF •  Examples: –  Markup APIs (YUI, javadoc, etc) –  Media (Audios, Videos, Games, Presentations) –  Job Postings
  20. RDFa in a nutshell •  Make a namespace •  Use @property, @rel and @resource •  For DATA: @property makes the node contents into the value •  For URLs: @rel makes the @resource into the value
  21. Normal HTML •  <html> … <div class=\"private”> private static String <strong>_createCookieHash </strong> (hash) …
  22. RDFa: example •  <html xmlns:yui=\"http://yuilibrary.com/rdf/ 1.0/yui.rdf#\"> … <div class=\"private” rel=\"yui:method\" resource=\"#method__createCookieHash\"> private static String <strong property=\"yui:name\"> _createCookieHash </strong> (hash) …
  23. That’s it! •  Automatically picked up by semantic parsers / crawlers •  Can build a SearchMonkey app on it •  Can make a mashup way easier than screen scraping •  Can get the data from Yahoo! BOSS
  24. What is SearchMonkey? an open platform for using structured data to build more useful and relevant search results Before After
  25. Enhanced Result: Zagat Image Links Key/Value Pairs or Abstract
  26. Infobar: Wikipedia Preview Summary Blob
  27. Part of the puzzle Semantic vocabularies Semantic markup on web pages SearchMonkey
  28. Vocabularies •  Need to speak the same language •  I like to see girls of that... caliber. •  English, French, Spanish, Esparanto? •  URLs to the rescue –  Dublin Core (http://purl.org/dc/elements/1.1/) –  Friend of a Friend (http://xmlns.com/foaf/0.1/) –  X-Friend Network (http://gmpg.org/xfn/11/) –  … (many more)
  29. Syntax •  Nouns, Verbs, and Adjectives, oh my! •  All phrases become lots of triples •  (Subject, Verb / Adj. / Prep. / etc, Object) •  Key / Value pairs ++ –  Everything is a URL or String –  Subject doesn’t have to be the document
  30. Syntax 2 •  Key / Value pair –  Title = Awesome SearchMonkey Presentation –  Homepage = http://search.yahoo.com/searchmonkey •  Triples –  (self, http://purl.org/dc#title, “Awesome SearchMonkey Presentation”) –  (self, http://vcard#url, http://search.yahoo.com/searchmonkey)
  31. Decompose to triples •  My friend “Bob” is an idiot. –  (self, http://xmlns.com/foaf/0.1/knows, genid:Ui__152310312_366) –  (genid:Ui__152310312_366, http:// www.w3.org/2001/vcard-rdf/3.0#fn, “Bob”) –  (genid:Ui__152310312_366, http:// example.org/ptarjan/isInstanceOf, http:// example.org/ptarjan/idiot) •  Unnamed nodes are O.K.
  32. Writing URLs takes a lot of work! •  xmlns:foaf=http://xmlns.com/foaf/0.1/ •  xmlns:vcard=http://www.w3.org/2001/vcard-rdf/ 3.0# •  xmlns:junk=http://example.org/ptarjan/ •  My friend “Bob” is an idiot. –  (self, foaf:knows, genid:Ui__152310312_366) –  (genid:Ui__152310312_366, vcard:fn, “Bob”) –  (genid:Ui__152310312_366, junk:isInstanceOf, junk:idiot) •  Unnamed nodes are O.K.
  33. RDFa •  <html xmlns:foaf=“http://xmlns.com/foaf/0.1” xmlns:vcard=http://www.w3.org/2001/vcard-rdf/ 3.0# xmlns:junk=http://example.org/ptarjan/> <div rel=“foaf:knows”> <span property=“vcard:fn”>Bob</span> <span rel=“junk:isInstanceOf” resource=“junk:idiot” /> </div> </html>
  34. •  </SemanticWeb> •  Questions?
  35. Innards of SearchMonkey •  You build a web-service inside our framework •  When a search page renders –  We check which SM apps are enabled –  We call them • 50ms for in-page • Long time for AJAX –  They return data in our template –  We render them (and cache)
  36. Prototyping with XSLT •  What if I don’t have structured data? –  I don’t own the site –  I do own the site, but I want to prototype first •  Build an XSLT custom data service first –  Write some XSLT to extract the data and transform it into DataRSS –  Mostly about finding the right XPath (use Firebug or XPather ) –  Quick to implement, but brittle –  Can’t do a good Enhanced Result
  37. Do it for real •  Demo
  38. Examples •  Rubic’s cube •  VTA Bus •  API Monkey •  BugMeNot •  RetailMeNot •  Amazon
  39. questions?

+ Paul TarjanPaul Tarjan, 8 months ago

custom

2413 views, 10 favs, 3 embeds more stats

Semantic Search + SeachMonkey talk given at Yahoo! more

More info about this document

© All Rights Reserved

Go to text version

  • Total Views 2413
    • 2386 on SlideShare
    • 27 from embeds
  • Comments 2
  • Favorites 10
  • Downloads 53
Most viewed embeds
  • 14 views on http://developer.yahoo.net
  • 11 views on http://www.paulscott.za.net
  • 2 views on http://paulscott.za.net

more

All embeds
  • 14 views on http://developer.yahoo.net
  • 11 views on http://www.paulscott.za.net
  • 2 views on http://paulscott.za.net

less

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

Cancel
File a copyright complaint
Having problems? Go to our helpdesk?

Categories

Tags

more
-->
Search
RSS Feed
What's new?

© 2009 SlideShare Inc. All Rights Reserved