Open Data Tutorial at ICEGOV

3,186 views

Published on

Tutorial on Open Data by Jim Hendler and Jeanne Holm (contributions by Hadley Beeman) at the ICEGOV 2012 conference.

  • Be the first to comment

Open Data Tutorial at ICEGOV

  1. 1. ICEGOV Open Data Tutorial Jim Hendler, @jahendler Jeanne Holm, @JeanneHolm 22 October 2012Co-author: Hadley Beeman, @HadleyBeeman
  2. 2. Introductions!• Please introduce yourself – Name – Organization – Three (3) words that explain either why you are here or what you hope to learn 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 2
  3. 3. Understanding theFoundations of Open Data• Why do countries and people share data?• What will citizens, businesses, scientists, and journalists do with the data?• How can we manage it? 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 3
  4. 4. Why Countries Share Data• Meet regulatory compliance• Provide transparency into government operations 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 4
  5. 5. Why Countries Share Data• Anticipate economic development• Initiate innovation 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 5
  6. 6. 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 6
  7. 7. Why People Want Open Data Swati Ramanathan 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 7
  8. 8. Real Outcomes = Better Lives• In health care – Data empowers communities to make changes that improve the quality of life of citizens • In California, ReLeaf plants trees in areas identified as danger areas for asthma sufferers – Companies use government data to innovate and create high-value jobs – Civic Commons has a great collection of good open use cases: http://civiccommons.org/ 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 8
  9. 9. Energy Drives Innovation• Communities like Energy.Data.gov connect innovators, indus try, academia, an d government at federal, state, an d local levels 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 9
  10. 10. Challenges Spark Ideas• Energy.Data.gov connects works with challenges across the nation to integrate federal data and bring government personnel to code- a-thons 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 10
  11. 11. Data Drives Decisions• Apps transform data in understandable ways to help people make decisions 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 11
  12. 12. Changing EconomicEquations Study from Malaysian government: http://www.transknowformance.com/article.cfm?id=53 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 12
  13. 13. Why People Want Open Data 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 13
  14. 14. What Makes Data Open• Open Format – The US Government through the Open Government Directive defines an open format as “one that is platform independent, machine readable, and made available to the public without restrictions that would impede the re-use of that information.” • http://www.whitehouse.gov/omb/assets/memoranda_ 2010/m10-06.pdf 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 14
  15. 15. What Makes Data Open• Example Open Formats – PDF for documents (but not data) – CSV for data – Web standards for publishing, sharing or linking • HTML, XML, RDF – Web standards for syndication • RSS, Atom, JSON 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 15
  16. 16. What Makes Data Open• Metadata – The information about the data being shared • Who produced it • Where • When • Use restrictions • Etc. – Use standards such as ADMS or Dublin Core 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 16
  17. 17. Dataset extension to Schema.org(pending): Google, MS (Bing), Yahoo!• Improve SEO• Improve international search and federation• Unique opportunity for public/private partnershipExpress your support at:http://blog.schema.org/2012/07/describing-datasets-with-schemaorg.html 9 July 2012 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL ICEGOV Open Data Tutorial 17
  18. 18. What Topics of Data ArePublished• Analytics based on over 1,000,000 datasets from around the world can be seen at – http://logd.tw.rpi.edu/iogds_data_analytics• The examples that follow are from that page 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 18
  19. 19. Countries Sharing Data Important note: quantity is not really the most important issue 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 19
  20. 20. Countries Sharing Data Important note: quantity is not really the most important issue 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 20
  21. 21. Example: U.S. Data.gov 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 21
  22. 22. Example: UK Data.gov.uk 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 22
  23. 23. Example: Spain 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 23
  24. 24. Topics (Across All Catalogs) 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 24
  25. 25. Topics (Across All Catalogs) 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 25
  26. 26. Data “Mashups” of Many Kinds More than 50 at http://logd.tw.rpi.edu 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 26
  27. 27. Making DataOpen, Accessible, andDiscoverable• Architecture for systems and technology• Processes for publishing data• Policies for ensuring data is open, accessible, and obtainable 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 27
  28. 28. Creating an Open DataArchitecture• Key components – Workflow for release approval (often overlooked) – Dataset storage • Can be centralized or via linking – Data cataloging • Metadata critical to a good open data site – Data API • Can be via download or via access • Technical issues with syndication, usage rules, etc. 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 28
  29. 29. Processes• Publication (and cleaning)• Data reuse and integration• Community input 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 29
  30. 30. Policies Become Essential• Policies help drive the ecosystem and “motivate” departments to continue to share data openly• Build the policies based around issues that are universal• Licensing, provenance http://creativecommons.org/licenses/ Open data on food, security, transp ortation, and transparency 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 30
  31. 31. Semantic Web and Linked Data County Council Royal Mail Ordnance Survey 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 31
  32. 32. Linking Data Via CommonNaming (Usually URLs) 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 32
  33. 33. Example: Agency Names 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 33
  34. 34. Can Be Lots of Things 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 34
  35. 35. “Linking” Data Government data is currently over half the cloud in size (~17B triples), 10s of thousands of links to other data (within and without) 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL http://linkeddata.org/ Tutorial9 July 2012 ICEGOV Open Data 35
  36. 36. 5 Star Data 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 36
  37. 37. Creating the Open DataCommunity 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 37
  38. 38. Creating Community• Communities are public-facing spaces that present data, information, and subject matter knowledge about a single topic from many organizations in one place – The topics for communities can be chosen based on priorities from the public, departments based on their mission, or issues of national importance 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 38
  39. 39. Community Vision• These questions help to guide early discussions 1. Vision: What will the community connection and collaboration look like in the future? 2. Leaders: Who will help to lead the community? 3. Participants: Who will participate? 4. Outcome: What are the expected outcomes, metrics, and measurements that will show success? How will this community work to improve the lives of citizens? 5. Functionality: What types of activities will be conducted on the site (forums, blogs, wikis, ranking, rating, challenges, or apps)? 6. Content: What content should be displayed 7. Interactivity: What ways will the community interact with the leaders, with each other, and with the public? 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 39
  40. 40. Open Communities Community ✓ Developers Open Data ✓ Semantic Web ✓ Health ✓ Law ✓ Energy ✓ Education ✓ Ocean ✓ Safety ✓ Manufacturing ✓ Business ✓ Ethics ✓ Smart Disclosure ✓ Sustainable Supply Chain ✓ Cities ✓ + many more… 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 40
  41. 41. Supporting Global Events Japanese tsunami, earthquake, and radiation monitoring Restore the Gulf: Deepwater Horizon Response 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 41
  42. 42. Health.Data.gov Champion: Todd Park U.S. Chief Technology Officer Apps Forums BlogsChallenges 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL 9 July 2012 ICEGOV Open Data Tutorial 42
  43. 43. Publicizing Data to Innovators• Challenges and code‐a-thons (health2challenge.org)• Many innovator “meetups” and conferences• Annual health data-paloozas• Over 139 applications• 50 new businesses• Thousands of lives improved each day• 1700 attendees at the Health Data Palooza in 2012 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL 9 July 2012 ICEGOV Open Data Tutorial 43
  44. 44. Creating Apps That Improve Lives:Asthmapolis 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 44
  45. 45. Creating Apps That SaveLives: iTriage and HospitalCompare 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 45
  46. 46. Additional Topics• Licensing, provenance, languages• Metadata design (international)• Trust – government data is controversial, who controls it?• Scaling – over 1M datasets and growing fast – How to search, store, link, translate, and archive• Versioning and updating• Visualization beyond the single dataset• Boundaries of open data 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 46
  47. 47. Summary• More and more government agencies throughout the world are sharing “raw data” with their citizens – Enhance transparency – Increase Innovation – “Crowd source” government services• Open data in open formats allows governments, agencies, and third parties to develop analyses, information graphics, and other ways to share information• Development of correct processes and policies are an important aspect of Open Government Data sharing – Need to support, not squelch, information sharing – Need to find appropriate balance of data release with privacy, security, and other citizen/government mandates• Community mechanisms are an important aspect of an open data ecosystem 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 47
  48. 48. Questions 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 48
  49. 49. Summary and Next Steps• Join a community – W3C eGovernment Interest Group • http://www.w3.org/egov/wiki/Main_Page – Open Data Innovation Network on LinkedIn • http://bit.ly/ODNetwork 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 ICEGOV Open Data Tutorial 49

×