Using Yahoo Pipes

1,497 views
1,383 views

Published on

Slides from hands-on 'Bootcamp Sampler' session from THATCamp Canberra 2011. The aim of the session was to get data into web pages from data sources such as Trove's People and Organisations search using Yahoo Pipes

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,497
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Using Yahoo Pipes

  1. 1. Using Yahoo Pipes to access data from web pages<br />THATCamp Canberra 2011<br />
  2. 2. Why Yahoo Pipes?<br />Browsers impose same-domain restriction on requesting XML/HTML/CSV content to include in web pages<br />We can use server side technologies to retrieve data from external sources, but what about when we don’t have access to those technologies? (e.g. embedding content in a blog post) <br />Yahoo Pipes convert data sources to JSON (JavaScript Object Notation) which can be included via JavaScript on web pages without restrictions <br />http://pipes.yahoo.com<br />
  3. 3. Example mashup<br />
  4. 4.
  5. 5. convictbook data sources<br />Trove People & Organisations<br />SLQ convict data<br />Web page : hosted within a Content Management System <br />(HTML + CSS <br />+ JavaScript + jQuery)<br />CSV<br />SRU Query Result (XML)<br />Google Refine<br />(Data clean up)<br />Yahoo Pipes<br />CSV<br />JSON<br />Visualisation links<br />JSON<br />Yahoo Pipes<br />Google <br />FusionTables<br />Query (XML)<br />CSV<br />Picture Australia data<br />
  6. 6. Pipe : Trove People by name<br />http://pipes.yahoo.com/pipes/pipe.info?_id=eed133e99c38a68d20d279e46cee66e6<br />
  7. 7. Yahoo Pipes<br />
  8. 8.
  9. 9. JSON<br />{ "count" : 1,<br /> "value" : {<br /> "title" : "ConvictIDGet",<br /> "description" : "Pipes Output",<br /> "link" : "http://pipes.yahoo.com/pipes/pipe.info?_id=c8b70fd9d9f6edcfbb7c0e351992e717",<br /> "pubDate" : "Thu, 06 Oct 2011 22:49:50 -0700",<br /> "generator" : "http://pipes.yahoo.com/pipes/",<br /> "callback" : "",<br /> "items" : [{<br /> "convictid" : "200",<br /> "name" : "Frederick Beadle",<br /> "one_of" : "one of 296 convicts transported on the Pyrenees",<br /> "convicted" : "Central Criminal Court",<br /> "term_duration" : "10 years",<br /> "conviction_date" : "",<br /> "notes" : "Ticket of Leave.",<br /> "vessel" : "Pyrenees",<br /> "departure_date" : "31 January 1853",<br /> "destination" : "Western Australia",<br /> "microfilmref" : "Australian Joint Copying Project. Microfilm Roll 93, Class and Piece Number HO11/18, Page Number 3 (3)",<br /> "surname" : "Beadle",<br /> "given_names" : "Frederick",<br /> "y:row" : "2",<br /> "description" : null,<br /> "title" : null<br /> }]<br />}}<br />Tip: Use http://jsonlint.com/ to <br />validate and format JSON<br />
  10. 10. JSON for Trove People query<br />{<br /> "count" : 1,<br /> "value" : {<br /> "title" : "TrovePeople",<br /> "description" : "Pipes Output",<br /> "link" : "http://pipes.yahoo.com/pipes/pipe.info?_id=c3412dfcaf40a54bdcd0273b293174e5",<br /> "pubDate" : "Thu, 06 Oct 2011 22:50:47 -0700",<br /> "generator" : "http://pipes.yahoo.com/pipes/",<br /> "callback" : "",<br /> "items" : [{<br /> "xmlns" : "http://www.loc.gov/zing/srw/",<br /> "version" : "1.1",<br /> "numberOfRecords" : "265",<br /> "resultSetId" : "kc2umf",<br /> "resultSetIdleTime" : "300",<br /> "records”:<br /> {"record" : [{<br />"recordSchema" : "http://www.w3.org/2005/Atom",<br />"recordPacking" : "xml",<br />"recordData" : {<br />"entry" : {<br />"id" : "http://nla.gov.au/nla.party-529154",<br />"link" : [{<br />"href" : "http://nla.gov.au/nla.party-529154”<br />}, {<br />"href" : "http://nla.gov.au/anbd.aut-an35513577",<br />"rel" : "related”<br />}, {<br />"href" : "http://www.adb.online.anu.edu.au/biogs/A100015b.htm",<br /> "rel" : "related”}],<br />"author" : {<br />"name" : "National Library of Australia Party Infrastructure”},<br />"published" : "2008-12-10T15:20:01Z",<br />"title" : "Lawson, H. S. W.",<br />"content" : {<br />"type" : "text/plain",<br />"content" : "Alternate name/s: Lawson, Harry Sutherland Wightman; Lawson, Harry Sutherland Wightman, Sir”<br />}}},<br /> "recordPosition" : "1”<br />},<br /> …<br />}]}}<br />
  11. 11. Including JSON data using jQuery<br />Live example: http://jsfiddle.net/AnnaGerber/cbJrc/<br /><html><head><title>Demo</title></head><br /><script type=“text/javascript” src=“jQuery.js”></script><br /><script type="text/javascript"><br /> function findPeople(pName) {<br />    varurl = "http://pipes.yahoo.com/pipes/pipe.run?_id=eed133e99c38a68d20d279e46cee66e6&_render=json&name=" + pName;<br />    $.getJSON( url, function( data ) {<br />        varoutputHTML = "”;<br />        $.each(data.value.items[0].records.record, function( index,item ) {<br />             var record=item.recordData.entry;<br />           outputHTML += "<li><a target='_blank' href='”  + record.link[0].href  +"'>" <br />              + record.title   + "</a></li>";<br />        });<br />        $('#output').html(outputHTML); <br />    });<br />});<br />$(document).ready(function(){<br />findPeople(‘Lawson’);<br /> });<br /></script></head><body><br /><h1>People</h1><br /><ul id=“output"></ul><br /></body></html><br />Tip: use browser inspector +<br />console.log() to inspect data<br />

×