0
Visualising Activity Data<br />Tony Hirst<br />Dept of Communication and Systems,<br />The Open University<br />Scattered ...
Today’s link shortener is bit.ly<br />Read:		 [ jlKwGq ]<br />as:		 http://bit.ly/jlKwGq<br />
Visual Analysisvs.Presentation Graphics<br />
This is NOT a presentation about:<br /><ul><li> data discovery
 data preparation
 data cleansing</li></li></ul><li>BUT…<br />
ScraperWiki<br />[ aGhJtK ]<br />
Search and replace…<br />…add regular expressions and you have search and replace “on steroids”<br />
Google Refine<br />[ aq1jUE ]<br />Example: walkthrough (@jenit) [ awGQPT ]<br />Example: merging two tables by column [ p...
DataWrangler<br />[ gmE3yz ]<br />
Data has shape and structure<br />
Hierarchical Data<br />
Many Eyes<br />[ qY5786 ]<br />Treemaps<br />
plot srcfile using ($1):(column(focusCar) -$2) with lines title "VET", srcfileusing ($1):(column(focusCar) -$3) with lines...
Or heatmaps in R:<br />[ qXmPgs ]<br />
Text processing with Unix tools[ m5tz63 ] [ lOVySX ]<br />Count number of lines in a file: wc-l L2sample.csv<br />View fir...
More text processing tricks<br />Extract columns:<br />cut -f 3 L2sample.csv<br />	cut -f 1,2,14,17 L2sample.csv > columnS...
[ dAdIo3 ]<br />
Time series data<br />
aka “seasonal subseries”<br />[ j3HODr ]<br />
matplotlib<br />Trends<br />[ qSIcrV ]<br />#time series data in d<br />#first difference<br />fd=np.diff(d)<br />Autocorr...
Graphs and Networks<br />
Graphviz<br />digraph test {<br />CSV [shape=box]<br />KML [shape=box]<br />JSON [shape=box]<br />XML [shape=box]<br />RDF...
Gephi<br />
[ nKoB4b] <br />
[ nKoB4b] <br />
Statistical Graphs<br />
R<br />
Graphics Libraries<br />
Protovis<br />
Processing<br />
Upcoming SlideShare
Loading in...5
×

Jiscad viz

1,010

Published on

Published in: Education, Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,010
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Change the basis… eg in OU, might consider different presentations (“years”) of the same course (“month”).
  • Transcript of "Jiscad viz"

    1. 1. Visualising Activity Data<br />Tony Hirst<br />Dept of Communication and Systems,<br />The Open University<br />Scattered puzzle pieces next to solved fragment by HoriaVarlan<br />
    2. 2. Today’s link shortener is bit.ly<br />Read: [ jlKwGq ]<br />as: http://bit.ly/jlKwGq<br />
    3. 3. Visual Analysisvs.Presentation Graphics<br />
    4. 4.
    5. 5. This is NOT a presentation about:<br /><ul><li> data discovery
    6. 6. data preparation
    7. 7. data cleansing</li></li></ul><li>BUT…<br />
    8. 8. ScraperWiki<br />[ aGhJtK ]<br />
    9. 9. Search and replace…<br />…add regular expressions and you have search and replace “on steroids”<br />
    10. 10. Google Refine<br />[ aq1jUE ]<br />Example: walkthrough (@jenit) [ awGQPT ]<br />Example: merging two tables by column [ pWK3C0 ]<br />
    11. 11. DataWrangler<br />[ gmE3yz ]<br />
    12. 12. Data has shape and structure<br />
    13. 13. Hierarchical Data<br />
    14. 14. Many Eyes<br />[ qY5786 ]<br />Treemaps<br />
    15. 15.
    16. 16.
    17. 17.
    18. 18.
    19. 19. plot srcfile using ($1):(column(focusCar) -$2) with lines title "VET", srcfileusing ($1):(column(focusCar) -$3) with lines title "WEB", srcfileusing ($1):(column(focusCar) -$4) with lines title "HAM", srcfileusing ($1):(column(focusCar) -$5) with lines title "BUT", srcfileusing ($1):(column(focusCar) -$6) with lines title "ALO", srcfileusing ($1):(column(focusCar) -$7) with lines title "MAS", srcfileusing ($1):(column(focusCar) -$8) with lines title "SCH", srcfileusing ($1):(column(focusCar) -$9) with lines title "ROS", …<br />
    20. 20. Or heatmaps in R:<br />[ qXmPgs ]<br />
    21. 21. Text processing with Unix tools[ m5tz63 ] [ lOVySX ]<br />Count number of lines in a file: wc-l L2sample.csv<br />View first few lines in a file: head L2sample.csv or head -n 4 L2sample.csv <br />View last few lines in a file: tail L2sample.csv or tail -n 15 L2sample.csv<br />Sample contiguous rows from start or end of file:<br />head -n 1 L2sample.csv > headers.csv<br /> tail -n 20 L2sample.csv > subSample.csv<br /> cat headers.csvsubSample.csv > subSampleWithHeaders.csv<br />Sample contiguous rows from middle of file:<br />head -n 15 L2sample.csv | tail -n 6 > middleSample.csv<br />Split large file into smaller files:<br />split -l 15 L2sample.csv subSamples<br />Search for lines containing a term:<br />grepmendeley L2sample.csv<br />grepEBSCO L2sample.csv > rowsContainingEBSCO.csv<br />
    22. 22. More text processing tricks<br />Extract columns:<br />cut -f 3 L2sample.csv<br /> cut -f 1,2,14,17 L2sample.csv > columnSample.csv<br />Sort data in a column:<br /> cut -f 40 L2sample.csv | sort<br />Identify distinct entries in a column:<br /> cut -f 40 L2sample.csv | sort | uniq<br />Count how many times each distinct term appears in a column:<br /> cut -f 40 L2sample.csv | sort | uniq –c<br />Sort can also sort by column (-k), reverse order (-r):<br />cut -f 40 L2_2011-04.csv | sort | uniq -c | sort -k 1 -r > uniqueSID.csv<br />
    23. 23. [ dAdIo3 ]<br />
    24. 24.
    25. 25. Time series data<br />
    26. 26. aka “seasonal subseries”<br />[ j3HODr ]<br />
    27. 27.
    28. 28. matplotlib<br />Trends<br />[ qSIcrV ]<br />#time series data in d<br />#first difference<br />fd=np.diff(d)<br />Autocorrelation<br />
    29. 29. Graphs and Networks<br />
    30. 30. Graphviz<br />digraph test {<br />CSV [shape=box]<br />KML [shape=box]<br />JSON [shape=box]<br />XML [shape=box]<br />RDF [shape=box]<br />HTML [shape=box]<br />GoogleSpreadsheet[shape=Msquare]<br />RDFTripleStore [shape=Msquare]<br />"[SPARQL]" [shape=diamond]<br />"[YQL]" [shape=diamond]<br />"[GoogleVizDataAPI]" [shape=diamond]<br />"<GoogleGadgets>" [shape=doubleoctagon]<br />"<GoogleVizDataCharts>" [shape=doubleoctagon]<br />"<GoogleMaps>" [shape=doubleoctagon]<br />"<GoogleEarth>" [shape=doubleoctagon]<br />"<JQueryCharts_etc>" [shape=doubleoctagon]<br />"[SPARQL]"->RDF;<br />"[SPARQL]"->XML;<br />"[SPARQL]"->CSV;<br />"[SPARQL]"->JSON;<br />JSON-> "<JQueryCharts_etc>";<br />CSV->"{GoogleRefine}"<br />CSV->ScraperWiki<br />JSON->ScraperWiki<br />"[YQL]"->ScraperWiki<br />ScraperWiki->CSV<br />HTML->ScraperWiki<br />HTML->"[YQL]"<br />"[SPARQL]"->"[YQL]"<br />"{GoogleRefine}"->CSV [style=dashed]<br />CSV->"<Gephi>" [style=dashed]<br />"<Gephi>"->CSV [style=dashed]<br />RDF->"[YQL]”<br />}<br />
    31. 31. Gephi<br />
    32. 32. [ nKoB4b] <br />
    33. 33. [ nKoB4b] <br />
    34. 34.
    35. 35.
    36. 36.
    37. 37.
    38. 38. Statistical Graphs<br />
    39. 39. R<br />
    40. 40.
    41. 41. Graphics Libraries<br />
    42. 42. Protovis<br />
    43. 43. Processing<br />
    44. 44.
    45. 45. I hope that’s beenouseful.info….?<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×