Beyond web services
Supporting mashup artists at Yahoo!
Chad Dickerson
Sr. Director, Yahoo! Developer Network

Why I’m particularly glad to be
here in Dublin

I love technology

. . . but I also love beer
(search for “how to make a
kegerator”)

Thanks for the step-by-step instructions of
the kegerator. By far the best on the web.
- Tim D.

We also really like the “camp”
concept at Yahoo!

But mostly we like to make
difficult things easy for
developers

The web is “the most hostile
programming environment
ever devised”
Douglas Crockford, Yahoo!
(creator of JSON)

To make it easier for you,
we provide:

Key building blocks for mashups

Pipes: a tool to mashup the web

Practical lessons/tools for scale

Access to experts

Key building blocks for mashups

developer.yahoo.com
• User interface
– Yahoo! User Interface Libraries (YUI)
– ASTRA (ActionScript Toolkit for Rich
Applications)
– Yahoo! Design Patterns
• Web services
• Lots of RSS feeds
– http://developer.yahoo.com/rss/

Web services
del.icio.us
plus 20+ more

Yahoo! User Interface Libraries (YUI)
Industrial strength
cross-browser
JavaScript and CSS
Generous BSD
license
http://developer.yahoo.com/yui/

The Yahoo! User Interface Library
Tree Control AutoComplete Slider
Calendar
TabView Menu Control
Control
Logger Control DHTML Windowing
Animation Drag & Drop
Connection
Event Utility
Manager (Ajax)
Dom Collection
CSS Reset, Fonts, Grids

ASTRA: Flash components
Tree Menu TabBar
Chart
AutoComplete

All under a liberal
BSD License

Yahoo! Design Patterns
a pattern describes an optimal solution to a
common problem within a specific context
Yahoo’s design patterns emerge from a rigorous
design process
http://developer.yahoo.com/ypatterns/

Example: Writing a review

Writing a review (cont.)

Yahoo! Design Patterns
•Search •Interact
– Search pagination – Invitation
– Transition (animate,
•Navigate
brighten, collapse,
– Breadcrumbs
dim, etc)
– Links
– Tabs
•Give Feedback
•Read – Ratings and reviews
– Page Grids
•Customize
– Pagination
– Drag and drop
•Select
•Many more. . . .
– Auto Complete
– Calendar Picker
http://developer.yahoo.com/ypatterns/

Pipes: a tool to mashup the web

Apt near Park
How do you find
an apartment
near a park?

Apt near Park
Craigslist apartment listings
• For each apartment:
• Click on map link
• Check distance to a park on the map
Tedious

Apt near Park
Data is available
• Craigslist apartment RSS feed
• Yahoo! Local API to find Parks
Can do it in about 50 lines of Perl code
#!/usr/bin/perl -w
use strict;
use LWP::Simple;
use XML::Simple;
...

Apt near Park
Basically combine feeds + web
services
Yet another custom mashup
• HousingMaps, ChicagoCrime, ...
Would be nice if there was an
easier way...

Pipes
A free online service that lets you remix
data and create mashups using a visual
editor
Craigslist
Yahoo! Local

Demo
Apartment near Something

Unix Pipes
grep -iv yahoo.com squid.log |
sort | uniq -c | sort -n >
top_sources.txt
Unix Pipes for the Web
Build useful applications from simple
primitives

Any Input
Craigslist
Yahoo!
Google
Ebay
Your data
here!

Any Output
RSS Readers
Badges
HTML
Your app
here!

Any Process
Yahoo! Local
Fetch
Your Web Service
Here!
Sort

Open
Craigslist
RSS Readers
Yahoo! Yahoo! Local
Fetch
Badges
Google
Your Web
HTML
Service Here!
Ebay
Your app
Sort
here!
Your data
here!

Why use Pipes?
Middleware for mashing data without
having your own web server
Rapid prototyping
Quick to write pipes - no coding or
plugins required
Focus on what you’re trying to build
without solving the trivial
Faster network access to many APIs

Pipes lets you...
Grab web data sources
• RSS, JSON, XML, RDF, ICAL, CSV
Manipulate the data
• Mash it up with other data sources
and data services
View the results
Use the Pipe data however you want

Pipes on the
iPhone
http://iphone.pipes.yahoo.com

Practical tools/lessons in scale

Numbers from Yahoo!
Unique users per month: 500+ million
8 percent of all page views on the Internet
globally are on yahoo.com
Pipes numbers:
• 6 million pipes / day
• 170,000 unique sources of data

We live and breathe scale.
Every second.
Every day.

You can use some of the same
tools and methods we use to
achieve scale
http://developer.yahoo.com/performance/

The Performance Golden Rule
80-90% of the end-user response time is spent
on the front-end. Start there.
• Greater potential for improvement
• Simpler
• Proven to work
http://developer.yahoo.net/blog/archives/2007/03/high_performanc.html

14 Rules for High Performance Web Sites
1. Make fewer HTTP requests
2. Use a Content Delivery Network
3. Add an Expires header
4. Gzip components
5. Put CSS at the top
6. Move JS to the bottom
7. Avoid CSS expressions
8. Make JS and CSS external
9. Reduce DNS lookups
10. Minify JS
11. Avoid redirects
12. Remove duplicate scripts
13. Turn off ETags
14. Make AJAX cacheable and small
More at http://developer.yahoo.com/performance/

YSlow
performance lint tool
grades web pages for each rule
Firefox extension
Yahoo! internal tool
http://developer.yahoo.com/yslow/

Hosted YUI
All the benefits of YUI described earlier
Hosted via our CDN, for free
http://developer.yahoo.com/yui/articles/hosting/

Hadoop: the open source grid
• Doug Cutting (Lucene, Nutch) founded the
Hadoop project and joined Y! in January
2006
• Y! Hadoop team started March 2006
• Y! continues to make substantial
contributions to the project
• http://lucene.apache.org/hadoop/
• Example usage outside Y!
– NY Times wanted to make 1851-1922
articles available for free as PDF
– Needed to assemble a grid
– 11 million articles processed in under 24
hours using 100 Amazon EC2 instances
– http://open.blogs.nytimes.com/2007/11/01/self-service-
prorated-super-computing-fun/

Access to experts

Just a couple of Yahoos
Douglas Crockford:
– “Yoda of lambda programming and
JavaScript” according to Brendan Eich
(Inventor of JavaScript)
Rasmus Lerdorf
– Father of PHP

Books by Yahoo! authors
I want to give a shout-out to Yahoo! here for sharing their knowledge, earned
through hard experience running one of the world's largest web sites.
-Tim O’Reilly
http://radar.oreilly.com/archives/2007/10/high_performance_websites.html

Just launched: YDN Theater
http://developer.yahoo.com/blogs/theater/

(want to chat on camera after
my talk?)

We provide lots of ingredients
…but the rest is up to you!

Some inspiration. . .

Yahoo! Mail Web Service: Locamigos.com
http://www.locamigos.com/

Locamigos in action

delimport
http://www.ianhenderson.org/delimport.html

AIR Search
http://kentbrewster.com/air-search/

Hardware hacks
Blogging in motion
• Purse
• Pedometer
• Cell phone
• Flickr API
•http://www.blogginginmotion.com/
ybox
“Simple, friendly, text-based
widget/channels give quick
snapshots of personalized info
downloaded live from the net.”
http://www.uncommonprojects.com/ybox/

[your mashup here]

Everything you need
web services & data from Yahoo!
+ Pipes (data and services from the web)
+ YUI (interface code)
+ Y! design patterns
+ scaling know-how
+ expert knowledge
--------------------
= beautiful, fast, awesome mashups

Now it’s your turn. . .

Good luck!
Contact:
chadd@yahoo-inc.com
Yahoo! Developer Network:
http://developer.yahoo.com/
Slides:
http://slideshare.net/chaddickerson/

Photo credits
• Crane
– http://www.flickr.com/photos/david_wilmot/28202190/
• Pipes
– http://www.flickr.com/photos/monoglot/275958740/
• Speedometer
– http://www.flickr.com/photos/maxim303/41509420/
• Apartments near park
– http://www.flickr.com/photos/paytonc/67399288/
• Doug Crockford
– http://www.flickr.com/photos/livenature/412141721/
• Rasmus Lerdorf
– http://www.flickr.com/photos/chrys/56838713/
(All Creative Commons)