Beyond web services: supporting mashup artists at Yahoo!

9,329 views

Published on

Keynote by Chad Dickerson at Mashup Camp Dublin, November 11, 2007

Published in: Technology
  • Be the first to comment

Beyond web services: supporting mashup artists at Yahoo!

  1. 1. Beyond web services Supporting mashup artists at Yahoo! Chad Dickerson Sr. Director, Yahoo! Developer Network
  2. 2. Why I’m particularly glad to be here in Dublin
  3. 3. I love technology
  4. 4. . . . but I also love beer (search for “how to make a kegerator”)
  5. 5. Thanks for the step-by-step instructions of the kegerator. By far the best on the web. - Tim D.
  6. 6. We also really like the “camp” concept at Yahoo!
  7. 7. But mostly we like to make difficult things easy for developers
  8. 8. The web is “the most hostile programming environment ever devised” Douglas Crockford, Yahoo! (creator of JSON)
  9. 9. To make it easier for you, we provide:
  10. 10. Key building blocks for mashups
  11. 11. Pipes: a tool to mashup the web
  12. 12. Practical lessons/tools for scale
  13. 13. Access to experts
  14. 14. Key building blocks for mashups
  15. 15. developer.yahoo.com • User interface – Yahoo! User Interface Libraries (YUI) – ASTRA (ActionScript Toolkit for Rich Applications) – Yahoo! Design Patterns • Web services • Lots of RSS feeds – http://developer.yahoo.com/rss/
  16. 16. Web services del.icio.us plus 20+ more
  17. 17. Yahoo! User Interface Libraries (YUI) Industrial strength cross-browser JavaScript and CSS Generous BSD license http://developer.yahoo.com/yui/
  18. 18. The Yahoo! User Interface Library Tree Control AutoComplete Slider Calendar TabView Menu Control Control Logger Control DHTML Windowing Animation Drag & Drop Connection Event Utility Manager (Ajax) Dom Collection CSS Reset, Fonts, Grids
  19. 19. ASTRA: Flash components Tree Menu TabBar Chart AutoComplete
  20. 20. All under a liberal BSD License
  21. 21. Yahoo! Design Patterns a pattern describes an optimal solution to a common problem within a specific context Yahoo’s design patterns emerge from a rigorous design process http://developer.yahoo.com/ypatterns/
  22. 22. Example: Writing a review
  23. 23. Writing a review (cont.)
  24. 24. Yahoo! Design Patterns •Search •Interact – Search pagination – Invitation – Transition (animate, •Navigate brighten, collapse, – Breadcrumbs dim, etc) – Links – Tabs •Give Feedback •Read – Ratings and reviews – Page Grids •Customize – Pagination – Drag and drop •Select •Many more. . . . – Auto Complete – Calendar Picker http://developer.yahoo.com/ypatterns/
  25. 25. Pipes: a tool to mashup the web
  26. 26. Apt near Park How do you find an apartment near a park?
  27. 27. Apt near Park Craigslist apartment listings • For each apartment: • Click on map link • Check distance to a park on the map Tedious
  28. 28. Apt near Park Data is available • Craigslist apartment RSS feed • Yahoo! Local API to find Parks Can do it in about 50 lines of Perl code #!/usr/bin/perl -w use strict; use LWP::Simple; use XML::Simple; ...
  29. 29. Apt near Park Basically combine feeds + web services Yet another custom mashup • HousingMaps, ChicagoCrime, ... Would be nice if there was an easier way...
  30. 30. Pipes A free online service that lets you remix data and create mashups using a visual editor Craigslist Yahoo! Local
  31. 31. Demo Apartment near Something
  32. 32. Unix Pipes grep -iv yahoo.com squid.log | sort | uniq -c | sort -n > top_sources.txt Unix Pipes for the Web Build useful applications from simple primitives
  33. 33. Any Input Craigslist Yahoo! Google Ebay Your data here!
  34. 34. Any Output RSS Readers Badges HTML Your app here!
  35. 35. Any Process Yahoo! Local Fetch Your Web Service Here! Sort
  36. 36. Open Craigslist RSS Readers Yahoo! Yahoo! Local Fetch Badges Google Your Web HTML Service Here! Ebay Your app Sort here! Your data here!
  37. 37. Why use Pipes? Middleware for mashing data without having your own web server Rapid prototyping Quick to write pipes - no coding or plugins required Focus on what you’re trying to build without solving the trivial Faster network access to many APIs
  38. 38. Pipes lets you... Grab web data sources • RSS, JSON, XML, RDF, ICAL, CSV Manipulate the data • Mash it up with other data sources and data services View the results Use the Pipe data however you want
  39. 39. Pipes on the iPhone http://iphone.pipes.yahoo.com
  40. 40. Practical tools/lessons in scale
  41. 41. Numbers from Yahoo! Unique users per month: 500+ million 8 percent of all page views on the Internet globally are on yahoo.com Pipes numbers: • 6 million pipes / day • 170,000 unique sources of data
  42. 42. We live and breathe scale. Every second. Every day.
  43. 43. You can use some of the same tools and methods we use to achieve scale http://developer.yahoo.com/performance/
  44. 44. The Performance Golden Rule 80-90% of the end-user response time is spent on the front-end. Start there. • Greater potential for improvement • Simpler • Proven to work http://developer.yahoo.net/blog/archives/2007/03/high_performanc.html
  45. 45. 14 Rules for High Performance Web Sites 1. Make fewer HTTP requests 2. Use a Content Delivery Network 3. Add an Expires header 4. Gzip components 5. Put CSS at the top 6. Move JS to the bottom 7. Avoid CSS expressions 8. Make JS and CSS external 9. Reduce DNS lookups 10. Minify JS 11. Avoid redirects 12. Remove duplicate scripts 13. Turn off ETags 14. Make AJAX cacheable and small More at http://developer.yahoo.com/performance/
  46. 46. YSlow performance lint tool grades web pages for each rule Firefox extension Yahoo! internal tool http://developer.yahoo.com/yslow/
  47. 47. Hosted YUI All the benefits of YUI described earlier Hosted via our CDN, for free http://developer.yahoo.com/yui/articles/hosting/
  48. 48. Hadoop: the open source grid • Doug Cutting (Lucene, Nutch) founded the Hadoop project and joined Y! in January 2006 • Y! Hadoop team started March 2006 • Y! continues to make substantial contributions to the project • http://lucene.apache.org/hadoop/ • Example usage outside Y! – NY Times wanted to make 1851-1922 articles available for free as PDF – Needed to assemble a grid – 11 million articles processed in under 24 hours using 100 Amazon EC2 instances – http://open.blogs.nytimes.com/2007/11/01/self-service- prorated-super-computing-fun/
  49. 49. Access to experts
  50. 50. Just a couple of Yahoos Douglas Crockford: – “Yoda of lambda programming and JavaScript” according to Brendan Eich (Inventor of JavaScript) Rasmus Lerdorf – Father of PHP
  51. 51. Books by Yahoo! authors I want to give a shout-out to Yahoo! here for sharing their knowledge, earned through hard experience running one of the world's largest web sites. -Tim O’Reilly http://radar.oreilly.com/archives/2007/10/high_performance_websites.html
  52. 52. Just launched: YDN Theater http://developer.yahoo.com/blogs/theater/
  53. 53. (want to chat on camera after my talk?)
  54. 54. We provide lots of ingredients …but the rest is up to you!
  55. 55. Some inspiration. . .
  56. 56. Yahoo! Mail Web Service: Locamigos.com http://www.locamigos.com/
  57. 57. Locamigos in action
  58. 58. delimport http://www.ianhenderson.org/delimport.html
  59. 59. AIR Search http://kentbrewster.com/air-search/
  60. 60. Hardware hacks Blogging in motion • Purse • Pedometer • Cell phone • Flickr API •http://www.blogginginmotion.com/ ybox “Simple, friendly, text-based widget/channels give quick snapshots of personalized info downloaded live from the net.” http://www.uncommonprojects.com/ybox/
  61. 61. [your mashup here]
  62. 62. Everything you need web services & data from Yahoo! + Pipes (data and services from the web) + YUI (interface code) + Y! design patterns + scaling know-how + expert knowledge -------------------- = beautiful, fast, awesome mashups
  63. 63. Now it’s your turn. . .
  64. 64. Good luck! Contact: chadd@yahoo-inc.com Yahoo! Developer Network: http://developer.yahoo.com/ Slides: http://slideshare.net/chaddickerson/
  65. 65. Photo credits • Crane – http://www.flickr.com/photos/david_wilmot/28202190/ • Pipes – http://www.flickr.com/photos/monoglot/275958740/ • Speedometer – http://www.flickr.com/photos/maxim303/41509420/ • Apartments near park – http://www.flickr.com/photos/paytonc/67399288/ • Doug Crockford – http://www.flickr.com/photos/livenature/412141721/ • Rasmus Lerdorf – http://www.flickr.com/photos/chrys/56838713/ (All Creative Commons)

×