HackU IIT Kgp 2013 BOSS + CA

818 views

Published on

Presentation talks about BOSS and Content Analysis along with Dapper.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
818
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Give you an overview of YQLHow can you use it for your hacksWe will see lots of examples How you can extend by creating new APIs out of webpages/data sources and for executing JS on server orQnA
  • Community ResourcesHelping you be successful in using Yahoo! Web Services isn't just about code and documentation. We want to make it easy for developers to get in touch with us and others who are using our services to answer and ask questions, toss around ideas, and give your feedback on what services we could offer next.ForumsYDN General BoardsDesign PatternsHotJobsSearchYOSAPTMusicWidgetsGeoPlanetOpenIDY! GroupsAddress BookAnswersFireEagleDel.icio.usTrafficFlickrMailSearchBOSSMapsJavaScriptSilverlightMerchant SolutionsSearch MarketingExceptional PerformanceYUIJava.NetFlashYDN EUAuthPHPPythonRubyPerlEventsWant your tech event appear in the YDN 'Upcoming Events' stream on developer.yahoo.com homepage? List your event on Upcoming and then add it to the Yahoo! Developer Network group on Upcoming. If you are setting up a tech event for developers or designers and would like some Yahoo! involvement, please contact us.ExpertsYahoo! Developer Network is on the lookout for experts to share knowledge and help grow our developer communities. If you are interested in applying to moderate a *group* or forum and become an official expert, please contact us for more information. Remember, we pay in hugs, link-love and schwag only.BlogThe YDN blog is where we talk about our various Yahoo! developer offerings and YDN events around the world. Your comments welcome. If you have a suggestion for the blog or a blog post you'd like to write about your experiences using our APIs or services, let us know. We are particularly interested in posts using Yahoo! APIs, tools, webservices and coverage of tech events. If you need inspiration or want to run some ideas by us, please contact us.YDN TheaterYDN Theater is chock-full of API deep dives, developer interviews, cutting- edge technologies, and recaps of tech events you wish you hadn't missed. If you have an idea for a story, please contact us.
  • Community ResourcesHelping you be successful in using Yahoo! Web Services isn't just about code and documentation. We want to make it easy for developers to get in touch with us and others who are using our services to answer and ask questions, toss around ideas, and give your feedback on what services we could offer next.ForumsYDN General BoardsDesign PatternsHotJobsSearchYOSAPTMusicWidgetsGeoPlanetOpenIDY! GroupsAddress BookAnswersFireEagleDel.icio.usTrafficFlickrMailSearchBOSSMapsJavaScriptSilverlightMerchant SolutionsSearch MarketingExceptional PerformanceYUIJava.NetFlashYDN EUAuthPHPPythonRubyPerlEventsWant your tech event appear in the YDN 'Upcoming Events' stream on developer.yahoo.com homepage? List your event on Upcoming and then add it to the Yahoo! Developer Network group on Upcoming. If you are setting up a tech event for developers or designers and would like some Yahoo! involvement, please contact us.ExpertsYahoo! Developer Network is on the lookout for experts to share knowledge and help grow our developer communities. If you are interested in applying to moderate a *group* or forum and become an official expert, please contact us for more information. Remember, we pay in hugs, link-love and schwag only.BlogThe YDN blog is where we talk about our various Yahoo! developer offerings and YDN events around the world. Your comments welcome. If you have a suggestion for the blog or a blog post you'd like to write about your experiences using our APIs or services, let us know. We are particularly interested in posts using Yahoo! APIs, tools, webservices and coverage of tech events. If you need inspiration or want to run some ideas by us, please contact us.YDN TheaterYDN Theater is chock-full of API deep dives, developer interviews, cutting- edge technologies, and recaps of tech events you wish you hadn't missed. If you have an idea for a story, please contact us.
  • Community ResourcesHelping you be successful in using Yahoo! Web Services isn't just about code and documentation. We want to make it easy for developers to get in touch with us and others who are using our services to answer and ask questions, toss around ideas, and give your feedback on what services we could offer next.ForumsYDN General BoardsDesign PatternsHotJobsSearchYOSAPTMusicWidgetsGeoPlanetOpenIDY! GroupsAddress BookAnswersFireEagleDel.icio.usTrafficFlickrMailSearchBOSSMapsJavaScriptSilverlightMerchant SolutionsSearch MarketingExceptional PerformanceYUIJava.NetFlashYDN EUAuthPHPPythonRubyPerlEventsWant your tech event appear in the YDN 'Upcoming Events' stream on developer.yahoo.com homepage? List your event on Upcoming and then add it to the Yahoo! Developer Network group on Upcoming. If you are setting up a tech event for developers or designers and would like some Yahoo! involvement, please contact us.ExpertsYahoo! Developer Network is on the lookout for experts to share knowledge and help grow our developer communities. If you are interested in applying to moderate a *group* or forum and become an official expert, please contact us for more information. Remember, we pay in hugs, link-love and schwag only.BlogThe YDN blog is where we talk about our various Yahoo! developer offerings and YDN events around the world. Your comments welcome. If you have a suggestion for the blog or a blog post you'd like to write about your experiences using our APIs or services, let us know. We are particularly interested in posts using Yahoo! APIs, tools, webservices and coverage of tech events. If you need inspiration or want to run some ideas by us, please contact us.YDN TheaterYDN Theater is chock-full of API deep dives, developer interviews, cutting- edge technologies, and recaps of tech events you wish you hadn't missed. If you have an idea for a story, please contact us.
  • A.K,.A a big waste of time
  • A.K,.A a big waste of time
  • A.K,.A a big waste of time
  • A.K,.A a big waste of time
  • A.K,.A a big waste of time
  • A.K,.A a big waste of time
  • A.K,.A a big waste of time
  • A.K,.A a big waste of time
  • A.K,.A a big waste of time
  • A.K,.A a big waste of time
  • A.K,.A a big waste of time
  • A.K,.A a big waste of time
  • A.K,.A a big waste of time
  • A.K,.A a big waste of time
  • Community ResourcesHelping you be successful in using Yahoo! Web Services isn't just about code and documentation. We want to make it easy for developers to get in touch with us and others who are using our services to answer and ask questions, toss around ideas, and give your feedback on what services we could offer next.ForumsYDN General BoardsDesign PatternsHotJobsSearchYOSAPTMusicWidgetsGeoPlanetOpenIDY! GroupsAddress BookAnswersFireEagleDel.icio.usTrafficFlickrMailSearchBOSSMapsJavaScriptSilverlightMerchant SolutionsSearch MarketingExceptional PerformanceYUIJava.NetFlashYDN EUAuthPHPPythonRubyPerlEventsWant your tech event appear in the YDN 'Upcoming Events' stream on developer.yahoo.com homepage? List your event on Upcoming and then add it to the Yahoo! Developer Network group on Upcoming. If you are setting up a tech event for developers or designers and would like some Yahoo! involvement, please contact us.ExpertsYahoo! Developer Network is on the lookout for experts to share knowledge and help grow our developer communities. If you are interested in applying to moderate a *group* or forum and become an official expert, please contact us for more information. Remember, we pay in hugs, link-love and schwag only.BlogThe YDN blog is where we talk about our various Yahoo! developer offerings and YDN events around the world. Your comments welcome. If you have a suggestion for the blog or a blog post you'd like to write about your experiences using our APIs or services, let us know. We are particularly interested in posts using Yahoo! APIs, tools, webservices and coverage of tech events. If you need inspiration or want to run some ideas by us, please contact us.YDN TheaterYDN Theater is chock-full of API deep dives, developer interviews, cutting- edge technologies, and recaps of tech events you wish you hadn't missed. If you have an idea for a story, please contact us.
  • A.K,.A a big waste of time
  • A.K,.A a big waste of time
  • HackU IIT Kgp 2013 BOSS + CA

    1. 1. BOSS around the web Souri Datta Structured Data Extraction Teamhttp://www.flickr.com/photos/sumrow/1267682594/sizes/l/
    2. 2. BOSS is Build your own search servicehttp://developer.yahoo.com/search/boss/
    3. 3. Provides APIsTo our Search database
    4. 4. TO BUILD your own powerfulSearch applications
    5. 5. BOSS allows you to search over Web, images, news & Blogs
    6. 6. BOSS Pricing
    7. 7. Free for building your hacks!!
    8. 8. BOSS uses OAuth for security Code : https://github.com/sourind/hacku/
    9. 9. Get a FREEconsumer key and secrethttp://hackyourworld.org/hacku/
    10. 10. http://developer.yahoo.com/yql/consol e/
    11. 11. 1. Select yql query2. Select output format 3. Copy This url
    12. 12. Finding images of “The Dark Knight Rises” select * from boss.search where q="TheDark Knight Rises" and service="images" and ck="..." and secret="..."
    13. 13. Finding “The Dark Knight Rises” in IMDB, movies.yahoo.comselect * from boss.search where q="The Dark Knight Rises" and sites="imdb.com,movies.yahoo.com" and ck="..." and secret="..."
    14. 14. Spell Check and Correctionselect * from boss.search where q="The Dirk Knight Rises" and service="spelling" and ck="..." and secret="..."
    15. 15. Finding news on “The Dark Knight Rises”select * from boss.search where q="The Dark Knight Rises" and service="news" and ck="..." and secret="..."
    16. 16. select * from contentanalysis.analyze wheretext="Sachin Tendulkar is batting very well"
    17. 17. Content Analysis from a URLselect * from contentanalysis.analyze where url="http://www.cnn.com/"
    18. 18. Lets See it in Action!
    19. 19. Query Cheatsheet• Find images of “The Dark Knight Rises”• select * from boss.search where q="The Dark Knight Rises" and service="images" and ck="..." and secret="..."• Find reviews of “The Dark Knight Rises”• select * from boss.search where q="reviews intitle:The Dark Knight Rises" and service="web" and ck="..." and secret="…"• Search for Avatar but not the movie:• select * from boss.search where q="Avatar -movie" and ck="..." and secret="... "• Search pdfs of “The Dark Knight Rises”• select * from boss.search where q="The Dark Knight Rises" and type="pdf" and ck="..." and secret="..."
    20. 20. Query Cheatsheet• Find all the news of “The Dark Knight Rises”• select * from boss.search where q="The Dark Knight Rises" and service="news" and ck="..." and secret="..."• Get long abstracts in the results• select * from boss.search where q="The Dark Knight Rises" and abstract="long" and ck="..." and secret="…"• Retrieve 51-100 results of the query• select * from boss.search where q="The Dark Knight Rises" and start=51 and ck="..." and secret="... "
    21. 21. EXAMPLES
    22. 22. duckduckgo.com
    23. 23. Why extraction is difficult?• Internet has lot of information• Not all can be processed by machines – Unstructured data – E.g. DiscountedPrice and RedcudedPrice of a product (both mean the same)• Ultimate aim is to publish data in structured format• Most simple way- xml,json
    24. 24. Web Scraping• Demo Dapper
    25. 25. More Resources• Yahoo! BOSS: http://developer.yahoo.com/boss• BOSS Technical Documentation: http://developer.yahoo.com/search/boss/boss_api _guide/• Content Analysis : http://developer.yahoo.com/contentanalysis/• Oauth sample code : https://github.com/sourind/hacku/
    26. 26. Questions?? http://www.flickr.com/photos/reem_unique/4119729692/
    27. 27. Thanks!!• http://slideshare.net/souridatta• https://github.com/sourind/

    ×