Beyond web services: supporting mashup artists at Yahoo!

Slide 1: Beyond web services Supporting mashup artists at Yahoo! Chad Dickerson Sr. Director, Yahoo! Developer Network

Slide 2: Why I’m particularly glad to be here in Dublin

Slide 3: I love technology

Slide 4: . . . but I also love beer (search for “how to make a kegerator”)

Slide 6: Thanks for the step-by-step instructions of the kegerator. By far the best on the web. - Tim D.

Slide 8: We also really like the “camp” concept at Yahoo!

Slide 10: But mostly we like to make difficult things easy for developers

Slide 11: The web is “the most hostile programming environment ever devised” Douglas Crockford, Yahoo! (creator of JSON)

Slide 12: To make it easier for you, we provide:

Slide 13: Key building blocks for mashups

Slide 14: Pipes: a tool to mashup the web

Slide 15: Practical lessons/tools for scale

Slide 16: Access to experts

Slide 17: Key building blocks for mashups

Slide 18: developer.yahoo.com • User interface – Yahoo! User Interface Libraries (YUI) – ASTRA (ActionScript Toolkit for Rich Applications) – Yahoo! Design Patterns • Web services • Lots of RSS feeds – http://developer.yahoo.com/rss/

Slide 19: Web services del.icio.us plus 20+ more

Slide 20: Yahoo! User Interface Libraries (YUI) Industrial strength cross-browser JavaScript and CSS Generous BSD license http://developer.yahoo.com/yui/

Slide 21: The Yahoo! User Interface Library Tree Control AutoComplete Slider Calendar TabView Menu Control Control Logger Control DHTML Windowing Animation Drag & Drop Connection Event Utility Manager (Ajax) Dom Collection CSS Reset, Fonts, Grids

Slide 22: ASTRA: Flash components Tree Menu TabBar Chart AutoComplete

Slide 23: All under a liberal BSD License

Slide 24: Yahoo! Design Patterns a pattern describes an optimal solution to a common problem within a specific context Yahoo’s design patterns emerge from a rigorous design process http://developer.yahoo.com/ypatterns/

Slide 25: Example: Writing a review

Slide 26: Writing a review (cont.)

Slide 27: Yahoo! Design Patterns •Search •Interact – Search pagination – Invitation – Transition (animate, •Navigate brighten, collapse, – Breadcrumbs dim, etc) – Links – Tabs •Give Feedback •Read – Ratings and reviews – Page Grids •Customize – Pagination – Drag and drop •Select •Many more. . . . – Auto Complete – Calendar Picker http://developer.yahoo.com/ypatterns/

Slide 28: Pipes: a tool to mashup the web

Slide 29: Apt near Park How do you find an apartment near a park?

Slide 30: Apt near Park Craigslist apartment listings • For each apartment: • Click on map link • Check distance to a park on the map Tedious

Slide 31: Apt near Park Data is available • Craigslist apartment RSS feed • Yahoo! Local API to find Parks Can do it in about 50 lines of Perl code #!/usr/bin/perl -w use strict; use LWP::Simple; use XML::Simple; ...

Slide 32: Apt near Park Basically combine feeds + web services Yet another custom mashup • HousingMaps, ChicagoCrime, ... Would be nice if there was an easier way...

Slide 33: Pipes A free online service that lets you remix data and create mashups using a visual editor Craigslist Yahoo! Local

Slide 34: Demo Apartment near Something

Slide 36: Unix Pipes grep -iv yahoo.com squid.log | sort | uniq -c | sort -n > top_sources.txt Unix Pipes for the Web Build useful applications from simple primitives

Slide 37: Any Input Craigslist Yahoo! Google Ebay Your data here!

Slide 38: Any Output RSS Readers Badges HTML Your app here!

Slide 39: Any Process Yahoo! Local Fetch Your Web Service Here! Sort

Slide 40: Open Craigslist RSS Readers Yahoo! Yahoo! Local Fetch Badges Google Your Web HTML Service Here! Ebay Your app Sort here! Your data here!

Slide 41: Why use Pipes? Middleware for mashing data without having your own web server Rapid prototyping Quick to write pipes - no coding or plugins required Focus on what you’re trying to build without solving the trivial Faster network access to many APIs

Slide 42: Pipes lets you... Grab web data sources • RSS, JSON, XML, RDF, ICAL, CSV Manipulate the data • Mash it up with other data sources and data services View the results Use the Pipe data however you want

Slide 43: Pipes on the iPhone http://iphone.pipes.yahoo.com

Slide 44: Practical tools/lessons in scale

Slide 45: Numbers from Yahoo! Unique users per month: 500+ million 8 percent of all page views on the Internet globally are on yahoo.com Pipes numbers: • 6 million pipes / day • 170,000 unique sources of data

Slide 46: We live and breathe scale. Every second. Every day.

Slide 47: You can use some of the same tools and methods we use to achieve scale http://developer.yahoo.com/performance/

Slide 48: The Performance Golden Rule 80-90% of the end-user response time is spent on the front-end. Start there. • Greater potential for improvement • Simpler • Proven to work http://developer.yahoo.net/blog/archives/2007/03/high_performanc.html

Slide 49: 14 Rules for High Performance Web Sites 1. Make fewer HTTP requests 2. Use a Content Delivery Network 3. Add an Expires header 4. Gzip components 5. Put CSS at the top 6. Move JS to the bottom 7. Avoid CSS expressions 8. Make JS and CSS external 9. Reduce DNS lookups 10. Minify JS 11. Avoid redirects 12. Remove duplicate scripts 13. Turn off ETags 14. Make AJAX cacheable and small More at http://developer.yahoo.com/performance/

Slide 50: YSlow performance lint tool grades web pages for each rule Firefox extension Yahoo! internal tool http://developer.yahoo.com/yslow/

Slide 52: Hosted YUI All the benefits of YUI described earlier Hosted via our CDN, for free http://developer.yahoo.com/yui/articles/hosting/

Slide 53: Hadoop: the open source grid • Doug Cutting (Lucene, Nutch) founded the Hadoop project and joined Y! in January 2006 • Y! Hadoop team started March 2006 • Y! continues to make substantial contributions to the project • http://lucene.apache.org/hadoop/ • Example usage outside Y! – NY Times wanted to make 1851-1922 articles available for free as PDF – Needed to assemble a grid – 11 million articles processed in under 24 hours using 100 Amazon EC2 instances – http://open.blogs.nytimes.com/2007/11/01/self-service- prorated-super-computing-fun/

Slide 54: Access to experts

Slide 55: Just a couple of Yahoos Douglas Crockford: – “Yoda of lambda programming and JavaScript” according to Brendan Eich (Inventor of JavaScript) Rasmus Lerdorf – Father of PHP

Slide 56: Books by Yahoo! authors I want to give a shout-out to Yahoo! here for sharing their knowledge, earned through hard experience running one of the world's largest web sites. -Tim O’Reilly http://radar.oreilly.com/archives/2007/10/high_performance_websites.html

Slide 57: Just launched: YDN Theater http://developer.yahoo.com/blogs/theater/

Slide 58: (want to chat on camera after my talk?)

Slide 59: We provide lots of ingredients …but the rest is up to you!

Slide 60: Some inspiration. . .

Slide 61: Yahoo! Mail Web Service: Locamigos.com http://www.locamigos.com/

Slide 62: Locamigos in action

Slide 63: delimport http://www.ianhenderson.org/delimport.html

Slide 64: AIR Search http://kentbrewster.com/air-search/

Slide 65: Hardware hacks Blogging in motion • Purse • Pedometer • Cell phone • Flickr API •http://www.blogginginmotion.com/ ybox “Simple, friendly, text-based widget/channels give quick snapshots of personalized info downloaded live from the net.” http://www.uncommonprojects.com/ybox/

Slide 66: [your mashup here]

Slide 67: Everything you need web services & data from Yahoo! + Pipes (data and services from the web) + YUI (interface code) + Y! design patterns + scaling know-how + expert knowledge -------------------- = beautiful, fast, awesome mashups

Slide 68: Now it’s your turn. . .

Slide 69: Good luck! Contact: chadd@yahoo-inc.com Yahoo! Developer Network: http://developer.yahoo.com/ Slides: http://slideshare.net/chaddickerson/

Slide 70: Photo credits • Crane – http://www.flickr.com/photos/david_wilmot/28202190/ • Pipes – http://www.flickr.com/photos/monoglot/275958740/ • Speedometer – http://www.flickr.com/photos/maxim303/41509420/ • Apartments near park – http://www.flickr.com/photos/paytonc/67399288/ • Doug Crockford – http://www.flickr.com/photos/livenature/412141721/ • Rasmus Lerdorf – http://www.flickr.com/photos/chrys/56838713/ (All Creative Commons)