BOSS is a service from Yahoo that allows developers to build custom search applications. It provides APIs to search over web pages, images, news articles, and blogs. BOSS is free to use for non-commercial applications. Developers can get an OAuth consumer key and secret to make authenticated requests to BOSS. The document includes examples of YQL queries to search for images and news related to "The Dark Knight Rises" using the BOSS APIs.

1. BOSS around the web
Souri Datta
Structured Data Extraction Team
http://www.flickr.com/photos/sumrow/1267682594/sizes/l/

2. BOSS is
Build your own
search service
http://developer.yahoo.com/search/boss/

3. Provides APIs
To our Search
database

4. TO BUILD your own
powerful
Search applications

5. BOSS allows you to
search over
Web, images,
news & Blogs

7. BOSS Pricing

8. Free for building your hacks!!

9. BOSS uses OAuth for security
Code : https://github.com/sourind/hacku/

10. Get a FREE
consumer key and
secret
http://hackyourworld.org/hacku/

11. http://developer.yahoo.com/yql/consol
e/

13. 1. Select yql query
2. Select output format
3. Copy This url

15. Finding images of “The Dark Knight
Rises”
select * from boss.search where q="The
Dark Knight Rises" and service="images"
and ck="..." and secret="..."

16. Finding “The Dark Knight Rises” in
IMDB, movies.yahoo.com
select * from boss.search where q="The Dark
Knight Rises" and
sites="imdb.com,movies.yahoo.com" and
ck="..." and secret="..."

17. Spell Check and Correction
select * from boss.search where q="The Dirk
Knight Rises" and service="spelling" and
ck="..." and secret="..."

18. Finding news on “The Dark Knight
Rises”
select * from boss.search where q="The Dark
Knight Rises" and service="news" and
ck="..." and secret="..."

19. select * from contentanalysis.analyze where
text="Sachin Tendulkar is batting very well"

20. Content Analysis from a URL
select * from contentanalysis.analyze where
url="http://www.cnn.com/"

21. Lets See it in Action!

22. Query Cheatsheet
• Find images of “The Dark Knight Rises”
• select * from boss.search where q="The Dark Knight
Rises" and service="images" and ck="..." and secret="..."
• Find reviews of “The Dark Knight Rises”
• select * from boss.search where q="reviews intitle:The
Dark Knight Rises" and service="web" and ck="..." and
secret="…"
• Search for Avatar but not the movie:
• select * from boss.search where q="Avatar -movie" and
ck="..." and secret="... "
• Search pdfs of “The Dark Knight Rises”
• select * from boss.search where q="The Dark Knight
Rises" and type="pdf" and ck="..." and secret="..."

23. Query Cheatsheet
• Find all the news of “The Dark Knight Rises”
• select * from boss.search where q="The Dark Knight
Rises" and service="news" and ck="..." and secret="..."
• Get long abstracts in the results
• select * from boss.search where q="The Dark Knight
Rises" and abstract="long" and ck="..." and secret="…"
• Retrieve 51-100 results of the query
• select * from boss.search where q="The Dark Knight
Rises" and start=51 and ck="..." and secret="... "

24. EXAMPLES

25. duckduckgo.com

28. Why extraction is difficult?
• Internet has lot of information
• Not all can be processed by machines
– Unstructured data
– E.g. DiscountedPrice and RedcudedPrice of a product
(both mean the same)
• Ultimate aim is to publish data in structured
format
• Most simple way- xml,json

29. Web Scraping
• Demo Dapper

30. More Resources
• Yahoo! BOSS: http://developer.yahoo.com/boss
• BOSS Technical Documentation:
http://developer.yahoo.com/search/boss/boss_api
_guide/
• Content Analysis :
http://developer.yahoo.com/contentanalysis/
• Oauth sample code :
https://github.com/sourind/hacku/

31. Questions??
http://www.flickr.com/photos/reem_unique/4119729692/

32. Thanks!!
• http://slideshare.net/souridatta
• https://github.com/sourind/