0
Finding things on the
   web with BOSS


 Christian Heilmann | http://wait-till-i.com | http://scriptingenabled.org

     ...
What is innovation?
Innovation is improving the
  current state of affairs.
In the best of all cases this
means that it gets easier for
the person using a product
      to reach a goal.
This can be achieved by
connecting several different
 products and turning them
 into one. (pssst... Mashup).
I’ve seen this followed
cleverly on several levels
       here in India.
BOSS is your chance to make
the web an easier to navigate
           space.
You can help us turn a search
  engine into a find engine.
Web search is broken!



http://luckyrobot.com/2009/02/06/search-is-broken-%E2%80%93-really-broken/

http://www.borthwick....
Back in the
     days the
     web was
    small and
largely static
  documents.
Nowadays it
is huge and
its content
and the
publication
speed
increased a
lot.
This is one challenge of
search engines these days.
The others are old, but also
        increasing
Now, say you have
the most awesome
idea of a search
engine that works
around these
issues.
Where you will get stuck is
  the overhead of indexing,
storing and filtering the data
         of the web.
And this is where BOSS
      comes in.
It is an API interface to our
   data stores for search.
BOSS is Build Your Own
   Search Service:
 http://developer.yahoo.com/search/boss/
To use it, you need a
   Application ID:
https://developer.yahoo.com/wsregapp/
And there is full
 documentation available:
http://developer.yahoo.com/search/boss/boss_guide/
Happy Hacking!
... oh alright then ...
You can get the code
examples I will show here:
 http://isithackday.com/hacks/
    bangalore/bosscode.zip
Say you want to search the
     web for donkeys.
Because
Donkeys alright then ...
   ... oh
 Rock!
Using BOSS you can do this
 with a REST API and display
the results any way you want!
The REST API:
boss.yahooapis.com/ysearch/{type}/v1/
               {search}
The REST API:
    boss.yahooapis.com/ysearch/{type}/v1/
                   {search}

     type is what you want to
       ...
The REST API:
     boss.yahooapis.com/ysearch/{type}/v1/
                    {search}

 search is the term to look for
   ...
The REST API:
     boss.yahooapis.com/ysearch/{type}/v1/
                    {search}

            Other parameters:
appid...
Web search REST API:
 boss.yahooapis.com/ysearch/web/v1/{search}

             Extra parameters:
filter: To filter out nas...
Image search REST API:
     boss.yahooapis.com/ysearch/images/v1/
                    {search}

              Extra parame...
News search REST API:
 boss.yahooapis.com/ysearch/news/v1/{search}

            Extra parameters:
age: how old the news ar...
There are restrictions how to
     display results and
information as to what data
        comes back.
For this, read the guide!
http://developer.yahoo.com/search/boss/boss_guide/
Everybody Duck!
There will be code
The easiest way to use BOSS
    is using JavaScript.
http://boss.yahooapis.com/
 ysearch/web/v1/donkeys?
  format=json&appid={id}
{quot;ysearchresponsequot;:
{quot;responsecodequot;:quot;200quot;,quot;nextpagequot;:quot;/ysearch/web/
v1/donkeys?
format...
To use this across domains,
 simply define a callback
        parameter:
http://boss.yahooapis.com/
  ysearch/web/v1/donkeys?
format=json&callback=foundd
     onkeys&appid={id}
founddonkeys({quot;ysearchresponsequot;:
{quot;responsecodequot;:quot;200quot;,quot;nextpagequot;:quot;/ysearch/web
/v1/do...
All you then need to do is put
 this url in a script node and
    write the founddonkeys
            function:
<div id=quot;searchresultsquot;></div>
 <script type=quot;text/javascriptquot;>
   function founddonkeys(o){
     var donk...
document.getElementById('searchresults').appendChild(r
esults);
   }
 </script>
 <script type=quot;text/javascriptquot; ch...
Two problems though:
First of all - without
JavaScript there are no
        donkeys!
Secondly - you can only find
         donkeys!
The solution: Event Handling
    and dynamic script
        generation.
<p>Warning: this is terrible code, USE A LIBRARY INSTEAD!</p>
<ul id=quot;searchesquot;>
  <li><a href=quot;http://search....
<script type=quot;text/javascriptquot; charset=quot;utf-8quot;>
  function founddonkeys(o){
    var donkeys = o.ysearchres...
var APIkey = 'TX6b4XHV34EnPXW0sYEr51hP1pn5O8KAGs'+
  '.LQSXer1Z7RmmVrZouz5SvyXkWsVk-';
  var searchlinks = document.getEle...
*click*
Using the YUI library (YUI3
JavaScript and CSS grids) you
  can easily make this much
            cooler:
To make using BOSS with
JavaScript easier, I’ve written
   a BOSS wrapper called
           YBOSS:
 http://icant.co.uk/san...
<div id=quot;resultsquot;></div>
<script type=quot;text/javascriptquot; src=quot;yboss-lib.jsquot;>
</script>
<script type...
For server side code there are
 code examples available at:
 http://www.saurabhsahni.com/boss-
           examples.zip
There’s also the Python
   mashup framework that
 allows for SQL for remixing
arbitrary XML/JSON sources:
http://developer...
And there’s an easy way to
deploy BOSS apps at Google
        App Engine:
http://zooie.wordpress.com/2008/08/04/yahoo-boss...
All this
has been
around for
a while.
Here are some new things
      added lately:
People are trying to make the
 web a less messier place by
  adding semantic data to
     HTML documents.
Using
SearchMonkey
technology
BOSS now
lists this
information in
the results.
                 http://www.flickr.com/photo...
view=searchmonkey_feed


                      view=searchmonkey_rdf


   http://developer.yahoo.com/search/boss/
        ...
Using the view=keyterms
 parameter you can get
keywords associated with
       each result.
http://keywordfinder.org
In order to get longer
 descriptions of each result
    you can now use the
abstract=long parameter to
  get up to 300 cha...
Another thing we’ve done is
using the Yahoo Site Explorer
  and bundle it with BOSS.
This way you can now get
page information and page
inlink information with two
     new BOSS services.
http://boss.yahooapis.com/
ysearch/se_inlink/v1/{URL}?
       appid={APPID}
Gets you a list of pages that
linked to the UR...
http://boss.yahooapis.com/
 ysearch/se_pagedata/v1/
   {URL}?appid={APPID}
 Gets you a list of all pages
belonging to a do...
So what has been done using
          BOSS?
http://ask-boss.appspot.com/
http://ask-boss.appspot.com/
http://ask-boss.appspot.com/
http://hakia.com/
http://www.oneriot.com/
And on a more
lighter note:
The client side is where
  strange things happen.



http://isithackday.com/hacks/web-the-adventure/
The motherlode of BOSS
     implementations:
    http://mashable.com/boss/
http://delicious.com/tag/bossmashup

Add yours ...
THANKS!
Keep in touch:
Christian Heilmann
http://wait-till-i.com
http://scriptingenabled.org
http://twitter.com/codepo8
Finding things on the web with BOSS
Finding things on the web with BOSS
Finding things on the web with BOSS
Finding things on the web with BOSS
Finding things on the web with BOSS
Upcoming SlideShare
Loading in...5
×

Finding things on the web with BOSS

3,155

Published on

Introdcution to Yahoo's open search platform BOSS at Open Hack Day in Bangalore, India.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
3,155
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
69
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Finding things on the web with BOSS"

  1. 1. Finding things on the web with BOSS Christian Heilmann | http://wait-till-i.com | http://scriptingenabled.org Open Hack Day 2009 - Bangalore, India
  2. 2. What is innovation?
  3. 3. Innovation is improving the current state of affairs.
  4. 4. In the best of all cases this means that it gets easier for the person using a product to reach a goal.
  5. 5. This can be achieved by connecting several different products and turning them into one. (pssst... Mashup).
  6. 6. I’ve seen this followed cleverly on several levels here in India.
  7. 7. BOSS is your chance to make the web an easier to navigate space.
  8. 8. You can help us turn a search engine into a find engine.
  9. 9. Web search is broken! http://luckyrobot.com/2009/02/06/search-is-broken-%E2%80%93-really-broken/ http://www.borthwick.com/weblog/2009/02/05/creative-destruction-google-slayed-by-the-notificator/
  10. 10. Back in the days the web was small and largely static documents.
  11. 11. Nowadays it is huge and its content and the publication speed increased a lot.
  12. 12. This is one challenge of search engines these days.
  13. 13. The others are old, but also increasing
  14. 14. Now, say you have the most awesome idea of a search engine that works around these issues.
  15. 15. Where you will get stuck is the overhead of indexing, storing and filtering the data of the web.
  16. 16. And this is where BOSS comes in.
  17. 17. It is an API interface to our data stores for search.
  18. 18. BOSS is Build Your Own Search Service: http://developer.yahoo.com/search/boss/
  19. 19. To use it, you need a Application ID: https://developer.yahoo.com/wsregapp/
  20. 20. And there is full documentation available: http://developer.yahoo.com/search/boss/boss_guide/
  21. 21. Happy Hacking!
  22. 22. ... oh alright then ...
  23. 23. You can get the code examples I will show here: http://isithackday.com/hacks/ bangalore/bosscode.zip
  24. 24. Say you want to search the web for donkeys.
  25. 25. Because Donkeys alright then ... ... oh Rock!
  26. 26. Using BOSS you can do this with a REST API and display the results any way you want!
  27. 27. The REST API: boss.yahooapis.com/ysearch/{type}/v1/ {search}
  28. 28. The REST API: boss.yahooapis.com/ysearch/{type}/v1/ {search} type is what you want to search: web: the interwebs news: new stuff images: pictures
  29. 29. The REST API: boss.yahooapis.com/ysearch/{type}/v1/ {search} search is the term to look for (url-encoded) Put “” around terms to ensure the right order, f.e. “donkey fur” (you don’t want to see cats, do you?) Filter with a -, f.e. donkey -shrek Restrict to a site using site:, f.e. donkey site:flickr.com
  30. 30. The REST API: boss.yahooapis.com/ysearch/{type}/v1/ {search} Other parameters: appid: your app ID (needed) count: amount of results start: where to start the counting region / lang: country and language format: xml or json sites: restrict to certain sites (comma separated)
  31. 31. Web search REST API: boss.yahooapis.com/ysearch/web/v1/{search} Extra parameters: filter: To filter out nasties, use filter=-porn-hate type: to search different types. You can use html, text, pdf, xl, msword, ppt or groups like msoffice and nonhtml. You can also do a type=msoffice,-xl
  32. 32. Image search REST API: boss.yahooapis.com/ysearch/images/v1/ {search} Extra parameters: filter: no nudies dimensions: all, small, medium, large, wallpaper, widewallpaper refererurl: all images in that url url: image at that url
  33. 33. News search REST API: boss.yahooapis.com/ysearch/news/v1/{search} Extra parameters: age: how old the news are in days. Last five days would be “5d”
  34. 34. There are restrictions how to display results and information as to what data comes back.
  35. 35. For this, read the guide! http://developer.yahoo.com/search/boss/boss_guide/
  36. 36. Everybody Duck!
  37. 37. There will be code
  38. 38. The easiest way to use BOSS is using JavaScript.
  39. 39. http://boss.yahooapis.com/ ysearch/web/v1/donkeys? format=json&appid={id}
  40. 40. {quot;ysearchresponsequot;: {quot;responsecodequot;:quot;200quot;,quot;nextpagequot;:quot;/ysearch/web/ v1/donkeys? format=json&appid=[...]&start=10quot;,quot;totalhitsquot;:quot;492 215quot;,quot;deephitsquot;:quot;15700000quot;,quot;countquot;:quot;10quot;,quot;startquot;:quot;0 quot;,quot;resultset_webquot;:[{quot;abstractquot;:quot;Hyperlinked description of the domesticated mammal discussing its appearance, relationship to horses, economic <b>...</b> horses and <b>donkeys</b> were brought back <b>...</b>quot;,quot;clickurlquot;:quot;http:// lrd.yahooapis.com/ _ylc=X3oDMTU4b2NoaDR2BF9TAzIwMjMxNTI3MDIEYXBwaWQDV Fg2YjRYSFYzNEVuUFhXMHNZRXI1MWhQMXBuNU84S0FHcy5MUVN YZXIxWjdSbW1WclpvdXo1U3Z5WGtXc1ZrLQRwb3MDMARzZXJ2a WNlA1lTZWFyY2hXZWIEc2xrA3RpdGxlBHNyY3B2aWQDR3lDaEg wU081cTlmSktUNG1ndTVUUUJNdlNjaS4wa1ZUVndBQVF5Sw-- /SIG=11820sato/**http%3A//en.wikipedia.org
  41. 41. To use this across domains, simply define a callback parameter:
  42. 42. http://boss.yahooapis.com/ ysearch/web/v1/donkeys? format=json&callback=foundd onkeys&appid={id}
  43. 43. founddonkeys({quot;ysearchresponsequot;: {quot;responsecodequot;:quot;200quot;,quot;nextpagequot;:quot;/ysearch/web /v1/donkeys? format=json&callback=founddonkeys&appid=TX6b4XHV3 4EnPXW0sYEr51hP1pn5O8KAGs.LQSXer1Z7RmmVrZouz5SvyX kWsVk- &start=10quot;,quot;totalhitsquot;:quot;492215quot;,quot;deephitsquot;:quot;15700 000quot;,quot;countquot;:quot;10quot;,quot;startquot;:quot;0quot;,quot;resultset_webquot;: [{quot;abstractquot;:quot;Hyperlinked description of the domesticated mammal discussing its appearance, relationship to horses, economic <b>...</b> horses and <b>donkeys</b> were brought back <b>...</b>quot;,quot;clickurlquot;:quot;http:// lrd.yahooapis.com/ _ylc=X3oDMTU4cG05cXJwBF9TAzIwMjMxNTI3MDIEYXBwaWQD VFg2YjRYSFYzNEVuUFhXMHNZRXI1MWhQMXBuNU84S0FHcy5MU VNYZXIxWjdSbW1WclpvdXo1U3Z5WGtXc1ZrLQRwb3MDMARzZX
  44. 44. All you then need to do is put this url in a script node and write the founddonkeys function:
  45. 45. <div id=quot;searchresultsquot;></div> <script type=quot;text/javascriptquot;> function founddonkeys(o){ var donkeys = o.ysearchresponse.resultset_web; var results = document.createElement('ul'); for(var i=0,j=donkeys.length;i<j;i++){ var item = document.createElement('li'); var link = document.createElement('a'); var abstract = document.createElement('p'); link.setAttribute('href',donkeys[i].clickurl); link.innerHTML = donkeys[i].title; item.appendChild(link); abstract.innerHTML = donkeys[i].abstract; item.appendChild(abstract); results.appendChild(item); }
  46. 46. document.getElementById('searchresults').appendChild(r esults); } </script> <script type=quot;text/javascriptquot; charset=quot;utf-8quot; src=quot;http://boss.yahooapis.com/ysearch/web/v1/donkeys? format=json&callback=founddonkeys&appid=xxxquot;></script>
  47. 47. Two problems though:
  48. 48. First of all - without JavaScript there are no donkeys!
  49. 49. Secondly - you can only find donkeys!
  50. 50. The solution: Event Handling and dynamic script generation.
  51. 51. <p>Warning: this is terrible code, USE A LIBRARY INSTEAD!</p> <ul id=quot;searchesquot;> <li><a href=quot;http://search.yahoo.com/search?va=donkeysquot;> Search for Donkeys </a> </li> <li><a href=quot;http://search.yahoo.com/search?va=kittensquot;> Search for kittens </a> </li> </ul> <div id=quot;searchresultsquot;></div>
  52. 52. <script type=quot;text/javascriptquot; charset=quot;utf-8quot;> function founddonkeys(o){ var donkeys = o.ysearchresponse.resultset_web; var results = document.createElement('ul'); for(var i=0,j=donkeys.length;i<j;i++){ var item = document.createElement('li'); var link = document.createElement('a'); var abstract = document.createElement('p'); link.setAttribute('href',donkeys[i].clickurl); link.innerHTML = donkeys[i].title; item.appendChild(link); abstract.innerHTML = donkeys[i].abstract; item.appendChild(abstract); results.appendChild(item); } var resultsbox = document.getElementById('searchresults'); resultsbox.innerHTML = ''; resultsbox.appendChild(results); } var APIkey = 'TX6b4XHV34EnPXW0sYEr51hP1pn5O8KAGs'+ '.LQSXer1Z7RmmVrZouz5SvyXkWsVk-'; var searchlinks = document.getElementById('searches').getElementsByTagName('a'); for(var i=0;searchlinks[i];i++){
  53. 53. var APIkey = 'TX6b4XHV34EnPXW0sYEr51hP1pn5O8KAGs'+ '.LQSXer1Z7RmmVrZouz5SvyXkWsVk-'; var searchlinks = document.getElementById('searches'). getElementsByTagName('a'); for(var i=0;searchlinks[i];i++){ searchlinks[i].onclick = function(){ var searchterm = this.href.split('va=')[1]; var url = 'http://boss.yahooapis.com/ysearch/web/v1/' + searchterm + '?format=json&' + 'callback=founddonkeys' + '&appid=' + APIkey var s = document.createElement('script'); s.setAttribute('type','text/javascript'); s.setAttribute('src',url); document.getElementsByTagName('head')[0].appendChild(s); return false; } } </script>
  54. 54. *click*
  55. 55. Using the YUI library (YUI3 JavaScript and CSS grids) you can easily make this much cooler:
  56. 56. To make using BOSS with JavaScript easier, I’ve written a BOSS wrapper called YBOSS: http://icant.co.uk/sandbox/yboss/
  57. 57. <div id=quot;resultsquot;></div> <script type=quot;text/javascriptquot; src=quot;yboss-lib.jsquot;> </script> <script type=quot;text/javascriptquot;> YBOSS.get({ searches:'search,news', query:'obama', count:10, callback:seedpics }); function seedpics(o){ var all = '<h4>Web Sites</h4>' + o.webHTML + '<h4>News</h4>' + o.newsHTML; var out = document.getElementById('results'); out.innerHTML = all; } </script>
  58. 58. For server side code there are code examples available at: http://www.saurabhsahni.com/boss- examples.zip
  59. 59. There’s also the Python mashup framework that allows for SQL for remixing arbitrary XML/JSON sources: http://developer.yahoo.com/search/boss/mashup.html
  60. 60. And there’s an easy way to deploy BOSS apps at Google App Engine: http://zooie.wordpress.com/2008/08/04/yahoo-boss- google-app-engine-integrated/
  61. 61. All this has been around for a while.
  62. 62. Here are some new things added lately:
  63. 63. People are trying to make the web a less messier place by adding semantic data to HTML documents.
  64. 64. Using SearchMonkey technology BOSS now lists this information in the results. http://www.flickr.com/photos/glenscott/3273401181/
  65. 65. view=searchmonkey_feed view=searchmonkey_rdf http://developer.yahoo.com/search/boss/ structureddata.html
  66. 66. Using the view=keyterms parameter you can get keywords associated with each result.
  67. 67. http://keywordfinder.org
  68. 68. In order to get longer descriptions of each result you can now use the abstract=long parameter to get up to 300 characters instead of 130.
  69. 69. Another thing we’ve done is using the Yahoo Site Explorer and bundle it with BOSS.
  70. 70. This way you can now get page information and page inlink information with two new BOSS services.
  71. 71. http://boss.yahooapis.com/ ysearch/se_inlink/v1/{URL}? appid={APPID} Gets you a list of pages that linked to the URL provided.
  72. 72. http://boss.yahooapis.com/ ysearch/se_pagedata/v1/ {URL}?appid={APPID} Gets you a list of all pages belonging to a domain in the Yahoo! index.
  73. 73. So what has been done using BOSS?
  74. 74. http://ask-boss.appspot.com/
  75. 75. http://ask-boss.appspot.com/
  76. 76. http://ask-boss.appspot.com/
  77. 77. http://hakia.com/
  78. 78. http://www.oneriot.com/
  79. 79. And on a more lighter note:
  80. 80. The client side is where strange things happen. http://isithackday.com/hacks/web-the-adventure/
  81. 81. The motherlode of BOSS implementations: http://mashable.com/boss/ http://delicious.com/tag/bossmashup Add yours by tagging it with “bossmashup” on Del.icio.us!
  82. 82. THANKS! Keep in touch: Christian Heilmann http://wait-till-i.com http://scriptingenabled.org http://twitter.com/codepo8
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×