Combining data with Google Refine

Get yourself ready • Google ‘Google Refine download’ http://code.google.com/p/googlerefine/wiki/Downloads • Download and install Google Refine • Open it up - it should open in a browser at http://127.0.0.1:3333/Saturday, 15 October 2011

Google Refine Combining data OnlineJournalismBlog.com Twitter.com/PaulBradshawSaturday, 15 October 2011

In a nutshell... cell.cross("GPdata2008", "Practice Code").cells["Total Listsize"].value[0] • Using GREL to combine datasets • Using APIs to grab geographical data • Using Reconcile services to grab company dataSaturday, 15 October 2011

GREL Google Refine Expression LanguageSaturday, 15 October 2011

cell.cross("GPdata2008", "Practice Code").cells ["Total Listsize"].value[0]Saturday, 15 October 2011

Using APIs Getting contextual data .Saturday, 15 October 2011

What’s an API again? Ask it a question, it gives you an answer: “For each of these codes, give me the region.” “For each of these names, tell me their political party”Saturday, 15 October 2011

Useful APIs Geo: UK Postcodes, Google Maps Social: Twitter, Facebook, Flickr Politics: They Work For You, Data.gov.uk News: Guardian, NYT, USA Today, NPR Health, business, etc. Search for specific onesSaturday, 15 October 2011

API keys Sometimes needed - apply through the site Use it in the request as a passwordSaturday, 15 October 2011

API limits Can prevent you getting data for all your records. Try multiple APIs or split your data into multiple sheets - or buy a licenceSaturday, 15 October 2011

Saturday, 15 October 2011

Get data from an API www.chasedavis.com/refine.htmlSaturday, 15 October 2011

Reconciling An easier way to get dataSaturday, 15 October 2011

OpenCorporates.com http://vimeo.com/17924204Saturday, 15 October 2011

Walkthrough: Reconciliation with Open Corporates • Click on arrow at top of column • Select Reconcile > Start Reconciling... • Click on Add Standard Service... • http://opencorporates.com/reconcile • And start...Saturday, 15 October 2011

Walkthrough: Reconciliation with Open Corporates • Click ‘Search for Match’ and select • Click double tick icon to bulk reconcile • Reconcile > Action > Match each cell to its best candidateSaturday, 15 October 2011

FreebaseSaturday, 15 October 2011

Freebase and namespacesSaturday, 15 October 2011

Search for matchSaturday, 15 October 2011

Walkthrough: Using Google Refine and APIsSaturday, 15 October 2011

Saturday, 15 October 2011

Escaping values for URLs "http://maps.googleapis.com/ maps/api/geocode/json? sensor=false&address=" + escape(value, "url")Saturday, 15 October 2011

JSON explained {category : value} {category {nested category : value {nested category 2 : value } {category 2 : value}Saturday, 15 October 2011

JSON explained {name : citytown} {geo {latitude : 42 {longitude : 2 } {administrative : citytown council}Saturday, 15 October 2011

Walkthrough: Using Google Refine to pull out data > Create new column based on this one... GREL: value.parseJson().item1.part2[1]Saturday, 15 October 2011

Links Delicious.com/paulb/kiev11 Delicious.com/paulb/googlerefine OnlineJournalismBlog.com/tag/ google-refineSaturday, 15 October 2011

Combining data with Google Refine

by Paul Bradshaw on Oct 15, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 11

Statistics

Combining data with Google Refine — Presentation Transcript

http://paper.li	9
http://www.twylah.com	2