Jiscad viz

Visualising Activity Data
Tony Hirst
Dept of Communication and Systems,
The Open University
Scattered puzzle pieces next to solved fragment by HoriaVarlan

Today’s link shortener is bit.ly
Read: [ jlKwGq ]
as: http://bit.ly/jlKwGq

Visual Analysisvs.Presentation Graphics

This is NOT a presentation about:
data discovery
data preparation
data cleansing

BUT…

ScraperWiki
[ aGhJtK ]

Search and replace…
…add regular expressions and you have search and replace “on steroids”

Google Refine
[ aq1jUE ]
Example: walkthrough (@jenit) [ awGQPT ]
Example: merging two tables by column [ pWK3C0 ]

DataWrangler
[ gmE3yz ]

Data has shape and structure

Hierarchical Data

Many Eyes
[ qY5786 ]
Treemaps

plot srcfile using ($1):(column(focusCar) -$2) with lines title "VET", srcfileusing ($1):(column(focusCar) -$3) with lines title "WEB", srcfileusing ($1):(column(focusCar) -$4) with lines title "HAM", srcfileusing ($1):(column(focusCar) -$5) with lines title "BUT", srcfileusing ($1):(column(focusCar) -$6) with lines title "ALO", srcfileusing ($1):(column(focusCar) -$7) with lines title "MAS", srcfileusing ($1):(column(focusCar) -$8) with lines title "SCH", srcfileusing ($1):(column(focusCar) -$9) with lines title "ROS", …

Or heatmaps in R:
[ qXmPgs ]

Text processing with Unix tools[ m5tz63 ] [ lOVySX ]
Count number of lines in a file: wc-l L2sample.csv
View first few lines in a file: head L2sample.csv or head -n 4 L2sample.csv
View last few lines in a file: tail L2sample.csv or tail -n 15 L2sample.csv
Sample contiguous rows from start or end of file:
head -n 1 L2sample.csv > headers.csv
tail -n 20 L2sample.csv > subSample.csv
cat headers.csvsubSample.csv > subSampleWithHeaders.csv
Sample contiguous rows from middle of file:
head -n 15 L2sample.csv | tail -n 6 > middleSample.csv
Split large file into smaller files:
split -l 15 L2sample.csv subSamples
Search for lines containing a term:
grepmendeley L2sample.csv
grepEBSCO L2sample.csv > rowsContainingEBSCO.csv

More text processing tricks
Extract columns:
cut -f 3 L2sample.csv
cut -f 1,2,14,17 L2sample.csv > columnSample.csv
Sort data in a column:
cut -f 40 L2sample.csv | sort
Identify distinct entries in a column:
cut -f 40 L2sample.csv | sort | uniq
Count how many times each distinct term appears in a column:
cut -f 40 L2sample.csv | sort | uniq –c
Sort can also sort by column (-k), reverse order (-r):
cut -f 40 L2_2011-04.csv | sort | uniq -c | sort -k 1 -r > uniqueSID.csv

[ dAdIo3 ]

Time series data

aka “seasonal subseries”
[ j3HODr ]

matplotlib
Trends
[ qSIcrV ]
#time series data in d
#first difference
fd=np.diff(d)
Autocorrelation

Graphs and Networks

Graphviz
digraph test {
CSV [shape=box]
KML [shape=box]
JSON [shape=box]
XML [shape=box]
RDF [shape=box]
HTML [shape=box]
GoogleSpreadsheet[shape=Msquare]
RDFTripleStore [shape=Msquare]
"[SPARQL]" [shape=diamond]
"[YQL]" [shape=diamond]
"[GoogleVizDataAPI]" [shape=diamond]
"<GoogleGadgets>" [shape=doubleoctagon]
"<GoogleVizDataCharts>" [shape=doubleoctagon]
"<GoogleMaps>" [shape=doubleoctagon]
"<GoogleEarth>" [shape=doubleoctagon]
"<JQueryCharts_etc>" [shape=doubleoctagon]
"[SPARQL]"->RDF;
"[SPARQL]"->XML;
"[SPARQL]"->CSV;
"[SPARQL]"->JSON;
JSON-> "<JQueryCharts_etc>";
CSV->"{GoogleRefine}"
CSV->ScraperWiki
JSON->ScraperWiki
"[YQL]"->ScraperWiki
ScraperWiki->CSV
HTML->ScraperWiki
HTML->"[YQL]"
"[SPARQL]"->"[YQL]"
"{GoogleRefine}"->CSV [style=dashed]
CSV->"<Gephi>" [style=dashed]
"<Gephi>"->CSV [style=dashed]
RDF->"[YQL]”
}

[ nKoB4b]

Statistical Graphs

Graphics Libraries

Protovis

Processing

I hope that’s beenouseful.info….?

http://blog.activitydata.org	87
http://twitter.com	1
http://webcache.googleusercontent.com	1

Jiscad viz

by Tony Hirst

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 89

Statistics

Jiscad viz Presentation Transcript