Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Experiences as a producer, consumer,
and observer of open data
Dr. Peter Mooney
Senior Researcher, Department Computer Sci...
Where am I?
Top Down: EPA delivering environmental data
Middle: EPA-funded research projects
Bottom Up: Citizen-science, V...
5 Star Open Data
Where organisations
should, at least, be
http://5stardata.info/

Where many
organisations are

Where the ...
Mark Johnstone’s Open Data Cake

http://www.impactandlearning.org/2013/08/open-data-and-increasing-impact-of.html
Making and baking the cake
- you can introduce flair but you must stick to the recipe and the
recipe that works for others...
The icing and the presentation
SAFER (http://erc.epa.ie/safer)

Development of web-services to
meet new European Directive...
Eating the cake
Fingal Apps Competition
Using Open Data in
preference to all others

Open Data and open
standards in any t...
Thinking up
cake recipes
10 years data management experience with
EPA
Working with data in almost all the key thematic
are...
Cake Taster:
Is the cake safe to eat?
Several years research work
into the quality and usability of
Volunteered Geographic...
Sharing the cake ….
SAFER and EPA Research
Data
Many mainstream EPA
reporting obligations (Air
Quality, Drinking Water,
Gr...
Open Data EPA Ireland

http://www.flickr.com/photos/aenimation/8997912525/sizes/m/
EPA’s Research Data Archive 2005
Providing ‘open-data’ PDF reports:
knowledge distribution - but not actionable
http://www.theguardian.com/global-developme...
EPA Research Programme has been operating an open
access approach over the last 5 years

All EPA funded research projects ...
EPA’s Research Data Archive 2013
SAFER’s open-access approach has been
EPA Research Data Archive 2013
very successful

EPA Research Data Archive 2006
EPA: Drinking Water Monitoring Results
and Water Supply Details for Ireland
http://erc.epa.ie/safer/resourcelisting.jsp?oI...
EPA: Complete Archive of Air Quality
Monitoring Data for Ireland
http://erc.epa.ie/safer/resourcelisting.jsp?oID=10136&use...
EPA: Geochemistry Data for the Historic Mine Sites
Project - Inventory and Risk Classification
http://erc.epa.ie/safer/res...
SAFER providing Web display of information and
equivalent machine readable Web Services
For consumption as Open Data
Infor...
Air Quality Index For Health - multiple representations
VGI Open Data
http://www.flickr.com/photos/treaclepondphotos/10799092965/sizes/m/
Volunteered Geographic Information
(Goodchild, 2007)
The ‘spatial’ case of User-Generated Content (UGC) Passive and Active...
VGI/UGC with embedded spatial
attributes (Tweetmap)

Todd Mostak, (MIT), the
Harvard Center for
Geographic Analysis
(CGA) ...
OpenStreetMap - one of the best
known VGI projects

Gl
Global Node Density Map (2013) http:
//tyrasd.github.io/osm-node-de...
OSM continues to grow at a
considerable rate

http://osmstats.altogetherlost.com/index.php

October 8th 2013
1,391,737 (re...
http://tools.geofabrik.de/mc/
New Forest, Hampshire, England
http://resultmaps.neis-one.org/osm-typhoon-haiyan-2013/#8/11.238/125.005

Nov 10th - Nov 13th 2013
Number of OSM Contribut...
Mapping agencies and municipalities can
be contributors to OSM and VGI
http://engagingcities.com/article/new-york-city-and...
EuroSDR + AGILE
“Crowdsourcing in National Mapping”
5 Funded Projects (Phase 2 begins Q1 2014)
National Mapping Agency Dri...
COST ACTION TD1202
“Mapping and the Citizen Sensor”

Accurate and timely maps are a
fundamental resource but their
product...
OSM Network Evolution - following
well known network infrastructures

http://www.theatlanticcities.com/commute/2013/03/map...
Collaborative networks develop
amongst contributors building OSM

Berlin: All interactions (co-edits) - all
contributors t...
Using Spatial Simulation Models (Cellular Automata
Markov Chains) to predict future OSM contribution
activities
Contributi...
VGI in Africa: Who is performing the majority of the work
in VGI (OSM for example)? - Study of Edit Histories
South Africa...
Conclusions and Summary

http://www.flickr.com/photos/troudd/10797161793/sizes/m/
Delivering data and services: we have
always had telephones to worry about

http://en.wikipedia.org/wiki/File:US_Robotics_...
Deliver [open data] products which suit the requirements and skills of the
majority of your customers or stakeholders

Not...
OpenStreetMap provides access to it’s database using three methods - different
skill levels for different end applications...
What ROI are EPA getting for this
investment in an open data approach?
AQ PM10 Data

Drinking Water 2010
553 Downloads (No...
Which is more important/difficult?
The icing or the cake?
● Open Data isn’t the
problem (icing)
● The biggest obstacles ar...
For organisations Open Data is a
little like daily exercise
Open Data must become
part of organisation/office
routine
Orga...
OPEN Data
OPEN Access
OPEN Information
http://www.flickr.com/photos/sigurtor/10805521514/sizes/z/
Would Open Data work in a GitHub style of
environment?
Would Open Data work best in server-based
cataloging applications?

You will find support for a
range of standards.
Metada...
Open Data as Linked Data?
INSPIRE Air Quality Data: Example
What about BIG datasets?
Web Services (Download, View, Convert, ETL)
Application Programming Interfaces (API) - JSON,
KML,...
The answer is probably a combination
of all of these approaches

http://www.flickr.com/photos/jleighb/142336560/
http://ww...
Take home messages
Baking the Open Data cake can be difficult
- but your consumers and stakeholders
will tell you how it t...
The End: Thanks for listening!
Email: peter.mooney@nuim.ie and p.
mooney@epa.ie
Web: www.peterm7.com/mobile/me

http://www...
Experiences as a producer, consumer and observer of open data
Upcoming SlideShare
Loading in …5
×

Experiences as a producer, consumer and observer of open data

2,577 views

Published on

Peter Mooney, is an Environmental Protection Agency (EPA) funded Research Fellow at the Department of Computer Science, NUI Maynooth. He has been working with the EPA on making environmental data publicly accessibly for the last ten years.

Presentation was part of The 1st Seminar of the ERC Funded Programmable City Project based at NIRSA, NUI Maynooth, Republic of Ireland.

Published in: Technology, Education
  • Be the first to comment

Experiences as a producer, consumer and observer of open data

  1. 1. Experiences as a producer, consumer, and observer of open data Dr. Peter Mooney Senior Researcher, Department Computer Science, NUI Maynooth Data Manager and IT Specialist, STRIVE Research, EPA Ireland peterm7.com/mobile/me
  2. 2. Where am I? Top Down: EPA delivering environmental data Middle: EPA-funded research projects Bottom Up: Citizen-science, Volunteered Geographic Information Meeting Place: Where Top Down, Middle, and Bottom Up meet...
  3. 3. 5 Star Open Data Where organisations should, at least, be http://5stardata.info/ Where many organisations are Where the future of Open Data on the Web will be
  4. 4. Mark Johnstone’s Open Data Cake http://www.impactandlearning.org/2013/08/open-data-and-increasing-impact-of.html
  5. 5. Making and baking the cake - you can introduce flair but you must stick to the recipe and the recipe that works for others who are making the cake or giving you ingredients Working with different ingredients (licenses, copyright, databases, systems etc) http://www.flickr.com/photos/jillclardy/8374937036/sizes/m/
  6. 6. The icing and the presentation SAFER (http://erc.epa.ie/safer) Development of web-services to meet new European Directive Air Quality Data Exchange Regulations http://www.flickr.com/photos/nmoya/9962621026/sizes/c/
  7. 7. Eating the cake Fingal Apps Competition Using Open Data in preference to all others Open Data and open standards in any teaching I do. Conducting research into and using open data http://www.flickr.com/photos/revjim/3143266465/sizes/z/
  8. 8. Thinking up cake recipes 10 years data management experience with EPA Working with data in almost all the key thematic areas How to best deliver Open Data and Services to the public and other stakeholders Ensuring everyone's requirements are specified correctly http://www.flickr.com/photos/joannatwinn/2083379033/sizes/m/
  9. 9. Cake Taster: Is the cake safe to eat? Several years research work into the quality and usability of Volunteered Geographic Information (VGI) and Open Data 10 Journal papers on the topic and subject area http://www.steamykitchen.com/2583-wedding-cake-judging-adventures.html
  10. 10. Sharing the cake …. SAFER and EPA Research Data Many mainstream EPA reporting obligations (Air Quality, Drinking Water, Greenhouse Gas Inventories) http://www.dallasartsrevue.com/art-crit/Here_Lately/PerfArt/Performance_Art.html
  11. 11. Open Data EPA Ireland http://www.flickr.com/photos/aenimation/8997912525/sizes/m/
  12. 12. EPA’s Research Data Archive 2005
  13. 13. Providing ‘open-data’ PDF reports: knowledge distribution - but not actionable http://www.theguardian.com/global-development-professionals-network/2013/oct/21/development-open-data-action “like funding James Cameron to make Avatar, and then releasing it in a black and white flip book. We are missing all the good stuff”.
  14. 14. EPA Research Programme has been operating an open access approach over the last 5 years All EPA funded research projects must provide “significant outputs” (datasets, info resources, etc) for public access via SAFER Crucially, this couples the final reports/papers with the actual data/information used to generate the findings/recommendations etc
  15. 15. EPA’s Research Data Archive 2013
  16. 16. SAFER’s open-access approach has been EPA Research Data Archive 2013 very successful EPA Research Data Archive 2006
  17. 17. EPA: Drinking Water Monitoring Results and Water Supply Details for Ireland http://erc.epa.ie/safer/resourcelisting.jsp?oID=10206&username=EPA%20Drinking%20Water Joined SAFER in 2011 12 Resources on SAFER Over 380 CSV and Excel files Years 2000 - 2011 (2012 to appear soon) Over 1,400 downloads Success Story! Distribution of the Drinking Water files via CD is now supplemented with 24-7365 access to the archive of data online on SAFER
  18. 18. EPA: Complete Archive of Air Quality Monitoring Data for Ireland http://erc.epa.ie/safer/resourcelisting.jsp?oID=10136&username=EPA%20Air%20Quality Joined SAFER in 2010 16 Resources on SAFER Over 800 CSV and Excel files Years ~1998 - 2012 (2013 to appear Q1 2014) Over 4,500 downloads Success Story! Very significant person-hours saved as public/consultants/researchers etc can access the entire archive openly 24-7365
  19. 19. EPA: Geochemistry Data for the Historic Mine Sites Project - Inventory and Risk Classification http://erc.epa.ie/safer/resourcelisting.jsp?oID=10170&username=EPA%20Historical%20Mines Joined SAFER in 2011 4 Resources on SAFER Analysis Data (4 files) and Reports available Over 190 downloads Success Story! Collaboration with Geological Survey of Ireland - SAFER chosen as the outlet for the end-products from this project
  20. 20. SAFER providing Web display of information and equivalent machine readable Web Services For consumption as Open Data Information by other web-services and mobile apps For Web-page display
  21. 21. Air Quality Index For Health - multiple representations
  22. 22. VGI Open Data http://www.flickr.com/photos/treaclepondphotos/10799092965/sizes/m/
  23. 23. Volunteered Geographic Information (Goodchild, 2007) The ‘spatial’ case of User-Generated Content (UGC) Passive and Active data streams People driven - bottom up infrastructure - linked to ubiquitious Internet + smart-technologies and the evolution of sociotechnology online Immense historical growth - potentially unbounded future growth - yielding Spatial Big Data
  24. 24. VGI/UGC with embedded spatial attributes (Tweetmap) Todd Mostak, (MIT), the Harvard Center for Geographic Analysis (CGA) http://worldmap. harvard.edu/tweetmap/.
  25. 25. OpenStreetMap - one of the best known VGI projects Gl Global Node Density Map (2013) http: //tyrasd.github.io/osm-node-density/
  26. 26. OSM continues to grow at a considerable rate http://osmstats.altogetherlost.com/index.php October 8th 2013 1,391,737 (registered contributors) ~ 2,700 contributors active per day 2.3 billion nodes 200 million polygons 2.1 million relations
  27. 27. http://tools.geofabrik.de/mc/
  28. 28. New Forest, Hampshire, England
  29. 29. http://resultmaps.neis-one.org/osm-typhoon-haiyan-2013/#8/11.238/125.005 Nov 10th - Nov 13th 2013 Number of OSM Contributors: 608 Number of Map Changes: 1314912
  30. 30. Mapping agencies and municipalities can be contributors to OSM and VGI http://engagingcities.com/article/new-york-city-and-openstreetmap-collaborating-through-open-data#.Uk2ZBu2dkzk.twitter OSM - the conduit for allowing governments connected with open data communities and for citizens to interact with their government Citizens - generating and maintaining and updating VGI become a “collaborative intelligence” or “update intelligence”
  31. 31. EuroSDR + AGILE “Crowdsourcing in National Mapping” 5 Funded Projects (Phase 2 begins Q1 2014) National Mapping Agency Driven Development of test-bed/incubator ideas on how crowdsourcing of geospatial data could be used by National Mapping Agencies: Examples: Gamification for Update of Spanish National Gazetteer, Conflation of official spatial data in Germany with OSM, place name extraction from Flickr for IGN France Gazetteer, ‘off-the-beatentrack’ tourist sites in Lithuania (National SDI)
  32. 32. COST ACTION TD1202 “Mapping and the Citizen Sensor” Accurate and timely maps are a fundamental resource but their production in a changing world is a major scientific and practical grand challenge. This EU funded COST Action seeks to rise to the challenges and enhance the role of citizen sensors in mapping activity. COST Action TD1202 is an EU funded interdisciplinary networking activity that involves ~27 countries
  33. 33. OSM Network Evolution - following well known network infrastructures http://www.theatlanticcities.com/commute/2013/03/mapping-growth-openstreetmap/4982/ Corcoran, Mooney, Bertolotto "Analysing the growth of OpenStreetMap networks" Spatial Statistics Vol 3, 21-32. 2013
  34. 34. Collaborative networks develop amongst contributors building OSM Berlin: All interactions (co-edits) - all contributors to OSM in Berlin Berlin: All interactions (co-edits) - highest ranked contributors (by Eigenvector Centrality measures) Mooney and Corcoran (2013) “Interaction and co-editing patterns between contributors to OpenStreetMap” To Appear Transactions in GIS
  35. 35. Using Spatial Simulation Models (Cellular Automata Markov Chains) to predict future OSM contribution activities Contribution of Nodes over time... Karlsruhe Stuttgart 2014 Jokar, Mooney, Helbeich (2013) in review IJGIS “Spatio-temproal analysis of patterns of contribution in OSM - a case study” Development of a Contribution Index (CI) to indicate if cells are being edited and maintained
  36. 36. VGI in Africa: Who is performing the majority of the work in VGI (OSM for example)? - Study of Edit Histories South Africa: Cape Town (15 exclusively local from 41) Kenya: Nairobi (17 from 30) DR Congo: Kinshasa (0 from 10) Rep. Sudan: Khartoum (2 from 14) Burkina Faso: Ouagadougou (0 from 14) Cameroon: Yaoundé (1 from 7) Tanzania: Dar es Salaam (12 from 25) High-frequency contributors studied “Exclusively Local” - Contributors whose mapping activities are contained exclusively in a given city or region.
  37. 37. Conclusions and Summary http://www.flickr.com/photos/troudd/10797161793/sizes/m/
  38. 38. Delivering data and services: we have always had telephones to worry about http://en.wikipedia.org/wiki/File:US_Robotics_56K_Modem_Front.JPG http://cdn3.pcadvisor.co.uk/cmsdata/features/3405369/latest_samsung_galaxy_smartphone.jpg
  39. 39. Deliver [open data] products which suit the requirements and skills of the majority of your customers or stakeholders Note: No pizzas or fajitas were made or consumed during the making of this slide
  40. 40. OpenStreetMap provides access to it’s database using three methods - different skill levels for different end applications - tools and apps are built by the community Data access API Access to the raw data in various formats Map Rendering and Map Services It’s the OSM community which have build GUI apps for OSM data upload and editing, mobile apps, rendering engines, etc.
  41. 41. What ROI are EPA getting for this investment in an open data approach? AQ PM10 Data Drinking Water 2010 553 Downloads (Nov 1st 2013) 553 x 5 mins per request = 2765 minutes 2765 / 60 = 46 hours 2138 downloads (Nov 1st 2013) 2138 X 6 mins per request = 12828 minutes 12828 / 60 = 213 hours 8 hour working day 8 hour working day TOTAL 26.5 days TOTAL 5.75 Days DW: http://erc.epa.ie/safer/iso19115/displayISO19115.jsp?isoID=268 PM10: http://erc.epa.ie/safer/iso19115/displayISO19115.jsp?isoID=69
  42. 42. Which is more important/difficult? The icing or the cake? ● Open Data isn’t the problem (icing) ● The biggest obstacles are: Information management, governance policies, data structures, ROI, resource management, sharing culture, etc (cake)
  43. 43. For organisations Open Data is a little like daily exercise Open Data must become part of organisation/office routine Organisational support (technical and people) A clear set of targets Clear and unambiguous set of reasons for undertaking open data http://www.flickr.com/photos/tulanesally/5860177655/sizes/n/
  44. 44. OPEN Data OPEN Access OPEN Information http://www.flickr.com/photos/sigurtor/10805521514/sizes/z/
  45. 45. Would Open Data work in a GitHub style of environment?
  46. 46. Would Open Data work best in server-based cataloging applications? You will find support for a range of standards. Metadata standards (ISO19115/ISO19119/ISO1 9110 following ISO19139, FGDC and Dublin Core), Catalog interfaces (OGCCSW2.0.2 ISO profile client and server, OAI-PMH client and server, GeoRSS server, GEO OpenSearch server, WebDAV harvesting, GeoNetwork to GeoNetwork harvesting support) and Map Services interfaces (OGC-WMS, WFS, WCS, KML and others) through
  47. 47. Open Data as Linked Data?
  48. 48. INSPIRE Air Quality Data: Example
  49. 49. What about BIG datasets? Web Services (Download, View, Convert, ETL) Application Programming Interfaces (API) - JSON, KML, etc Pre-cooked, dynamically updated, download ‘packages’? IT Infrastructure Considerations http://www.flickr.com/photos/chipkrieger/10781652035/sizes/z/
  50. 50. The answer is probably a combination of all of these approaches http://www.flickr.com/photos/jleighb/142336560/ http://www.flickr.com/photos/16850197@N04/6796313167/sizes/n/ http://www.flickr.com/photos/strawberrylanecakecompany/5847996107/ http://www.flickr.com/photos/happy_homebaking/459555139/sizes/m/ http://en.wikipedia.org/wiki/File:Whitweddingcake.jpg
  51. 51. Take home messages Baking the Open Data cake can be difficult - but your consumers and stakeholders will tell you how it tastes. If you haven’t started - get baking! For organisations - Open Data does have many costs. But there is substantial ROI when Open Data + Services are rolled out and adopted. VGI and Citizen Science/Participation: Not to be underestimated. Moving very quickly and gathering momentum. These communities are capable of baking their own open data cakes! Open Data - The Future is NOW! INSPIRE, Big Data, Linked Data, Urban Sensing, Web 3.0, etc…
  52. 52. The End: Thanks for listening! Email: peter.mooney@nuim.ie and p. mooney@epa.ie Web: www.peterm7.com/mobile/me http://www.flickr.com/photos/phils-pixels/10728144746/sizes/z/

×