Yahoo! Pipes Munging, Mixing and Mashing A beginner’s guide to using Yahoo! Pipes by Neil Crosby .

Munging, Mixing and Mashing

A beginner’s guide to using Yahoo! Pipes

by Neil Crosby .

I <3 Pipes Easy to use. Allows output in multiple formats. Reduces the code I need to write.

Easy to use.

Allows output in multiple formats.

Reduces the code I need to write.

“But Pipes isn’t easy” It is, once you’re past the initial learning curve. But that initial curve puts lots of people off.

It is, once you’re past the initial learning curve.

But that initial curve puts lots of people off.

Pipes powers my stuff NeilCrosby.com Following Twitter Conversations. Making Fever º more useful to me.

NeilCrosby.com

Following Twitter Conversations.

Making Fever º more useful to me.

NeilCrosby.com

What to talk about? First, a simple teaser pipe. Next, munging data together. Then, filtering data out. Finally, making new data.

First, a simple teaser pipe.

Next, munging data together.

Then, filtering data out.

Finally, making new data.

1. A simple teaser pipe Grab an RSS feed. Truncate it. Output it.

Grab an RSS feed.

Truncate it.

Output it.

The Pipe

2. Munging data together

The Problem You have a lot of feeds. You only want one feed. You want them in chronological order. You don’t want too many items.

You have a lot of feeds.

You only want one feed.

You want them in chronological order.

You don’t want too many items.

The Solution Load the feeds using Pipes. Sort the feeds. Truncate the final feed. Job done.

Load the feeds using Pipes.

Sort the feeds.

Truncate the final feed.

Job done.

The Pipe screenshot

screenshot

3. Filtering Data Out

The problem Upcoming provides a list of all events you’ve ever been interested in. You just want the ones you’ve been to. You only want the last few. What to do?

Upcoming provides a list of all events you’ve ever been interested in.

You just want the ones you’ve been to.

You only want the last few.

What to do?

The solution Load Upcoming data as XML. Filter to only keep “attended” events. Turn Upcoming data into an RSS feed.

Load Upcoming data as XML.

Filter to only keep “attended” events.

Turn Upcoming data into an RSS feed.

The Pipe

4. Creating new data Twitter conversations.

Twitter conversations.

The Problem People ask questions on twitter that I want to hear the answer to. The lovely answers are hidden from me.

People ask questions on twitter that I want to hear the answer to.

The lovely answers are hidden from me.

The solution Use twitter search to surface replies to the original question. Create a feed from this data.

Use twitter search to surface replies to the original question.

Create a feed from this data.

The Pipe It’s a big’un. (and this doesn’t cover all of it)

It’s a big’un.

(and this doesn’t cover all of it)

And there’s more! Use YQL . Use your pipes inside your pipes . Post data to webservices .

Use YQL .

Use your pipes inside your pipes .

Post data to webservices .

Linkies @NeilCrosby everywhere. http://neilcrosby.com http://pipes.yahoo.com/neilcrosby http://www.slideshare.net/neilcrosby/pipes

@NeilCrosby everywhere.

http://neilcrosby.com

http://pipes.yahoo.com/neilcrosby

http://www.slideshare.net/neilcrosby/pipes