UKSG - Just Do IT Yourself


Published on

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Tony HirstTwitter:@psychemediaBlog: http://blog.ouseful.infoPresentation prepared for: UKSG, April 2011Whenever you need one, there’s never a developer to hand. But what’s stopping you from doing the IT thing yourself? In this session we’ll explore some of the ways in whichnon-developers can ‘mash-up’ their own web applications,from rich interactive visualisations to powerful, realtime current awareness monitoring systems. Sounds too much like computer code? Far from it – in many cases, all you need to be able to do is cut and paste URLs from one place to another. Suggested hashtag: #uksg11
  • …but in reality, it’s the council’s. At the end of the day, a significant proportion of the data produced by councils is data that the council has to use, or report on, as part of its daily business.So to know whether or not you’re publishing your data appropriately, you should try to use it.When council workers need this piece of that piece of public data, they should be getting it from the datastore. (You may also need a private datastore for sensitive or personal information, but if it has a similar structure and provides a similar user interface or API to the public datastore, you’ll minimise training costs and make it easier to migrate the data between the two services If it gets opened up in the future.)
  • KML – the Keyhole markup Language.
  • KML is a (relatively) simple XML representation, owned by Google but accepted as a standard, that is used to syndicate geographical data across many public web applications, such as Google Maps, Google Earth, Bing Maps, OpenStreetMap, (**check those) and so on. Which is to say: if geographically related data is published as KML, there are plenty of applications out there that can put it on an interactive map for you.
  • For many years, council websites have included certain sorts of information on the site in the form of maps. Traditionally, these have been via PDF documents, or other image formats, although an increasing number have started to use interactive maps, often in the form of Google maps or, on occasion, something rather clunkier; (now that the Ordnance Survey is getting a little bit friendlier towards web developers, we’ll start to see a few OS based maps?)But for al its value as information, publishing the locations of car parks, schools or council boundaries as images, or even as interactive maps, it doesn’t really count as data. Applying the litmus test of “could I build my own a version of that” to a council’s “Find a Library” service, if you have to scrape the data by scrabbling around the backend of the council’s whizzy interactive map looking for the building names and geographical coordinates, they haven’t really published the data…Which is why it’s so refreshing to see sites like the Lichfield District Council site publishing URLs that point to KML files containing some of that council’s geographically related data…
  • …which means I can do things like this: create my own view over ward boundaries in Google Maps, simply by copying the URL of the appropriate KML file, pasting it into the search box on Google maps, and hitting return…
  • UK city population search onwikipedia
  • You’ve got a set of data you’d like to be able to search through, but you don’t have access to a database. What do you do?You have control of your website pages and want to track what visitors from a particular referrer are doing on site. What do you do?
  • You’ve got a set of data you’d like to be able to search through, but you don’t have access to a database. What do you do?You have control of your website pages and want to track what visitors from a particular referrer are doing on site. What do you do?
  • Google Fusion Tables represent the next step on from spreadsheets in certain respects. Allowing data import from a Google docs account, or via file upload (no import of CSV data from an arbitrary URL as yet), Google Fusion Tables will create combined datasets from two or more datasets sharing similar elements in a common column, So for example, two separate datasets each containing a column containing local authority identifiers can have their data joined together around common local authority IDs. (In traditional database speak, this is akin to a JOIN. **check**)
  • Google Fusion Tables also allows the “typing” of data contained within columns. One very useful type is a location type. By declaring location type columns, Google Fusion Tables will try to geoode your data fro you, based on that location information. So what does that mean?
  • It means you can plot the data on a map – without having to do very much hard work yourself at all!And as well as the map view over the data, you can also get access to a KML view, another simple format that can be used to help data – particularly geographical and spatially related data – flow. So let’s see how…
  • Thirdly, there is a visualisation library that provides a wide range of components that can be used to render data provided in the format returned by the data source.
  • UKSG - Just Do IT Yourself

    1. 1. Just Doing IT Yourself:simple recipes for the rest of us<br />tricks<br />Tony Hirst<br />Department of Communication and Systems<br />The Open University<br /><br />@psychemedia<br />
    2. 2. From shambrarian to undeveloper…<br />
    3. 3. If You're Not Confused<br />b-tal<br />So IT and the developers do what exactly?<br />
    4. 4. “Your” computer<br />Can you install your own applications?<br />Collaboration<br />Can you work on shared files with others inside and outside your institution?<br />Data wrangling<br />Can you get the data you need in the form you want it?<br />
    5. 5. Search<br />Does the search engine on your public websites work?<br />Visualisation<br />Would you like to be able to create your own interactive visualisations?<br />Programming<br />Do you need to do the things (you think) developers do?<br />
    6. 6. Trick 1<br />
    7. 7. JUSt ask<br />
    8. 8.<br />
    9. 9.
    10. 10.<br />
    11. 11.<br />OSQA is written in Python and powered by the Django application framework.<br />It is free software licensed under the GPL<br />
    12. 12. Welcometo the institutional undeveloper’sclub… :-)<br />
    13. 13. So what’s the FIRST RULE?<br />
    14. 14. (All the answers are bl**din’ obvious)<br />
    15. 15. The second rule of institutional undeveloper’sclub…<br />
    16. 16. Every trick has its place<br />WHAT’s the context????<br />
    17. 17. {<br />discoverymanagementmanipulationpresentation<br />informationcontext<br />
    18. 18. {<br />individual/n-cast<br />synch/asynch<br />1-way/2-way<br />public or private<br />communicationcontext<br />
    19. 19. {<br />“yours” or “theirs”<br />online or offline<br />X-top or mobile<br />technologycontext<br />
    20. 20. The second rule of institutional undeveloper’sclub…<br />Other<br />
    21. 21. No trick has its place<br />Appropri-8 technology<br />
    22. 22. (ghoti and chips)<br />
    23. 23. Trick 2<br />
    24. 24. Write your diagrams<br />
    25. 25.
    26. 26. digraph test {<br />CSV [shape=box]<br />KML [shape=box]<br />JSON [shape=box]<br />XML [shape=box]<br />RDF [shape=box]<br />HTML [shape=box]<br />GoogleSpreadsheet [shape=Msquare]<br />RDFTripleStore [shape=Msquare]<br />"[SPARQL]" [shape=diamond]<br />"[YQL]" [shape=diamond]<br />"[GoogleVizDataAPI]" [shape=diamond]<br />"<GoogleGadgets>" [shape=doubleoctagon]<br />"<GoogleVizDataCharts>" [shape=doubleoctagon]<br />"<GoogleMaps>" [shape=doubleoctagon]<br />"<GoogleEarth>" [shape=doubleoctagon]<br />"<JQueryCharts_etc>" [shape=doubleoctagon]<br />…<br />…<br />"[SPARQL]"->RDF;<br />"[SPARQL]"->XML;<br />"[SPARQL]"->CSV;<br />"[SPARQL]"->JSON;<br />JSON-> "<JQueryCharts_etc>";<br />CSV->"{GoogleRefine}"<br />CSV->ScraperWiki<br />JSON->ScraperWiki<br />"[YQL]"->ScraperWiki<br />ScraperWiki->CSV<br />HTML->ScraperWiki<br />HTML->"[YQL]"<br />"[SPARQL]"->"[YQL]"<br />"{GoogleRefine}"->CSV [style=dashed]<br />CSV->"<Gephi>" [style=dashed]<br />"<Gephi>"->CSV [style=dashed]<br />RDF->"[YQL]”<br />}<br />
    27. 27. Graphvizand Gephi<br />
    28. 28. Gephi<br />
    29. 29.
    30. 30. The third rule of institutional undeveloper’sclub…<br />
    31. 31. Common representations are your friend<br />
    32. 32. Trick 3<br />
    33. 33. Learn which apps take in which formats<br />
    34. 34.
    35. 35. Learn which apps publish which formats<br />
    36. 36. IBM’sMany Eyes<br />
    37. 37.
    38. 38.
    39. 39. Let your content flow<br />
    40. 40.
    41. 41.<br />
    42. 42. JISCPress<br />digress.itWordpress theme<br />
    43. 43.
    44. 44. Feed me…<br /><ul><li> RSS
    45. 45. KML
    46. 46. CSV
    47. 47. etc</li></li></ul><li>The fourth rule of institutional undeveloper’sclub…<br />
    48. 48. Eat your own dog food<br />dog food by notto86<br />
    49. 49. Trick 4<br />
    50. 50. URLs can act as glue<br />
    51. 51.
    52. 52. KML<br />
    53. 53.
    54. 54.
    55. 55.
    56. 56.
    57. 57. Wikipedia<br />Google<br />Spreadsheet<br />=importHTML<br />Yahoo! Pipe<br />Import CSV<br />HTML<br />CSV<br />KML<br />Google Map<br />Embedded object<br /><embed><br />
    58. 58.
    59. 59.
    60. 60.<br />
    61. 61. Trick 5<br />
    62. 62. URLs<br />
    63. 63. Learn to read – and hack –URLs<br />
    64. 64. Look for your search term in the URL of search results pages…<br />
    65. 65. Try adding.xml, .jsonor .rdfto the end of a/programmepage on<br />
    66. 66. Trick 6<br />
    67. 67. Learn some geek power words<br />
    68. 68. JSON is the new XML<br />[insider knowledge]<br />
    69. 69. JSON-P is magic<br />Try ?callback=myfunc<br />
    70. 70. JSON-P lets you pull the feed into any web page<br />
    71. 71. 3rd party server<br />Local server<br />Browser<br />
    72. 72. Follow the JSON…<br />
    73. 73.<br /><br />
    74. 74. The fifth rule of institutional undeveloper’sclub…<br />
    75. 75. Be lazy…<br />
    76. 76. Trick 7<br />
    77. 77. Let tools do the work for you…<br />
    78. 78. Data<br />Wrangling<br />
    79. 79.
    80. 80.
    81. 81.
    82. 82. Google Refine<br />
    83. 83. Stanford Data Wrangler<br />
    84. 84. Paul Bradshaw’sOnlineJournalismBlog<br />
    85. 85. Trick 8<br />
    86. 86. viaRule 3 and Rule 5<br />
    87. 87. Cut and paste can get you a long way<br />
    88. 88. Developers use 3rd party libraries<br />[insider knowledge]<br />
    89. 89. Hack the examples<br />
    90. 90.
    91. 91.<br />
    92. 92. Trick 9<br />
    93. 93. Curate things yourself<br />
    94. 94. Feed aggregation<br />
    95. 95.
    96. 96.
    97. 97.
    98. 98.
    99. 99. Google Custom Search Engine<br />
    100. 100. Trick 10<br />
    101. 101. Feed content can be shaped<br />
    102. 102. Search/Aggregation<br />filter<br />
    103. 103.
    104. 104. Trick 11<br />
    105. 105. Sharing is easy<br />
    106. 106. TwitterGoogle docsGist/githubopenetherpaddropboxdeliciousannotate it<br />
    107. 107.
    108. 108. Trick 12<br />
    109. 109. Google Apps is a mashup environment<br />
    110. 110. Google spreadsheet<br />Google Apps script<br />Google Visualisation API<br />Google Spreadsheets as a mashup environment<br />
    111. 111. Martin Hawksey's<br />JISC RSC MASHe blog<br /><br /><br />
    112. 112. Collect/backup tweets in a Google Spreadsheet [Twitteralytics v2]<br />Linking a Google Form with data from the responses in the Spreadsheet<br />[Event/Resource Booking]<br />gEVS – An idea for a Google Form/Visualization mashup for electronic voting<br />The best Google Spreadsheet Event Manager (w/h Sites, Contact, Calendar integration) ever, ever, ever<br />Convert time stamped data to timed-text (XML) subtitle format using Google Spreadsheet Script<br />
    113. 113. Trick 13<br />
    114. 114. Your first trick…<br />
    115. 115. (Probably no time for)QUESTIONS…?<br /><br />@psychemedia<br />