IOGDC Open Data Tutorial

6,214 views

Published on

An open data tutorial by Hadley Beeman, Jim Hendler, and Jeanne Holm given at the International Open Government Data Conference 2012.

Published in: Technology, Education
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,214
On SlideShare
0
From Embeds
0
Number of Embeds
337
Actions
Shares
0
Downloads
124
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide
  • A parent has a child who is illAsk questions online at HealthTapFind a hospital and compare (Hospital Compare)That doctor recommends GPS-powered inhaler (Asthmapolis)Monitor asthma levels at school through Public School RecordsKnow in advance the best places to play, how to get to school, and how to plan your dayThe data delivered through the 172 agencies participating in Data.gov eases the burden on families in caring for a sick childMore importantly, the data as it’s aggregated empower communities to make changes that improve the qualiy of life of citizens(ReLeaf plants trees in areas identified by Together We Breathe as danger areas for asthma sufferersCities see hot spots that trigger asthma problems for their citizens
  • Uses asthma patients aggregated GPS notations to create hot spots in communities where there are asthma issuesChanges individual behaviorFrom 65% daily incidence to 25% daily incidence of inhaler uses over a six month study
  • IOGDC Open Data Tutorial

    1. 2012 INTERNATIONALOPEN GOVERNMENT DATACONFERENCE Open Government Data Tutorial Hadley Beeman Jim Hendler Jeanne Holm Presenter Contact Information Organized by the World Bank and Data.gov
    2. Introductions!• Please introduce yourself – Name – Organization – Three (3) words that explain either why you are here or what you hope to learn 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 2
    3. Understanding theFoundations of Open Data• Why do countries and people share data?• What will citizens, businesses, scientists, and journalists do with the data?• How can we manage it? 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 3
    4. Why Countries Share Data• Meet regulatory compliance• Provide transparency into government operations 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 4
    5. Why Countries Share Data• Anticipate economic development• Initiate innovation 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 5
    6. 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 6
    7. Why People Want Open Data Swati Ramanathan 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 7
    8. Real Outcomes = Better Lives• Open data eases the burden on families in caring for a sick child or seeking medical information• More importantly, the data as it’s aggregated empowers communities to make changes that improve the quality of life of citizens – In California, ReLeaf plants trees in areas identified by Together We Breathe as danger areas for asthma sufferers – Cities see hot spots that trigger asthma problems for their citizens• Each company in Health 2.0 used government data to innovate and create high-value jobs in America• Civic Commons has a great collection of good open use cases: http://civiccommons.org/ 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 8
    9. Energy Drives Innovation• Energy.Data.gov connects innovators, industry, academia, and government at federal, state, and local levels 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL 9 July 2012 IOGDC Open Data Tutorial 9
    10. Challenges Spark Ideas• Energy.Data.gov connects works with challenges across the nation to integrate federal data and bring government personnel to code-a-thons 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 10
    11. Data Drives Decisions• Apps transform data in understandable ways to help people make decisions 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 11
    12. Changing EconomicEquations Study from Malaysian government: http://www.transknowformance.com/article.cfm?id=53 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 12
    13. Why People Want OpenData 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 13
    14. What Makes Data Open• Open Format – The US Government through the Open Government Directive (http://www.whitehouse.gov/omb/assets/memor anda_2010/m10-06.pdf ) defines an open format as “one that is platform independent, machine readable, and made available to the public without restrictions that would impede the re-use of that information.” 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 14
    15. What Makes Data Open• Example Open Formats: – PDF for documents (but not data) – CSV for data – Web standards for publishing, sharing or linking • HTML, XML, RDF – Web standards for syndication • RSS, Atom, JSON 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 15
    16. What Makes Data Open• Metadata – The information about the data being shared • Who produced it • Where • When • Use restrictions • Etc. – Use standards such as ADMS or Dublin Core – New (Search-oriented) Embedded dataset metadata coming 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 16
    17. What Topics of Data ArePublished• Analytics based on over 1,000,000 datasets from around the world can be seen at – http://logd.tw.rpi.edu/iogds_data_analytics• The examples that follow are from that page 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 17
    18. Countries Sharing Data Important note: quantity is not really the most important issue 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 18
    19. Countries Sharing Data Important note: quantity is not really the most important issue 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 19
    20. Example: US Data.gov 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 20
    21. Example: UK Data.gov.uk 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 21
    22. Example: Spain 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 22
    23. Topics (Across All Catalogs) 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 23
    24. Topics (Across All Catalogs) 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 24
    25. What Topics of Data ArePublished (UK) 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 25
    26. Data “Mashups” of ManyKinds More than 50 of these at http://logd.tw.rpi.edu 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 26
    27. Making Data Open,Accessible, and Discoverable• Architecture for systems and technology• Processes for publishing data• Policies for ensuring data is open, accessible, and obtainable 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 27
    28. Creating an Open DataArchitecture• Key components – Workflow for release approval (often overlooked) – Dataset storage • Can be centralized or via linking – Data Cataloging • Metadata critical to a good open data site – Data API • Can be via download or via access • Technical issues with syndication, usage rules, etc. 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 28
    29. Processes• Publication (and cleaning)• Data reuse and integration• Community input 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 29
    30. Policies Become Essential• Policies are needed to help drive the ecosystem and “motivate” departments to continue to share data openly• Build the policies based around issues that are universal• Licensing, provenance: http://creativecommons.org/licenses/ Open data on food, security, transportation, and transparency 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 30
    31. 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 31
    32. Generations Share Differently• 1930-50’s era generation – Focus on society – Friendships are forged through adversity• 1960-70’s era generation – Focus on community – Friendships forged through identification with a cause• 1980-90’s era generation – Focus on the individual – Friendships forged through individual goal accomplishment• 2000’s era generation – Focus on common interests – Friendships are created or thrive virtually… 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 32
    33. Creating Change is a SocialConstruct• The social aspects of an organization that help to facilitate, create, and ensure that the goals of the organization are met – Social networks – Norms and values – Social trust – Networks and communities – Physical spaces – Storytelling – Social capital consists of the stock of active connections among people; the trust, mutual understanding, and shared values and behaviors that bind the members of human networks and communities and make cooperative action possible • Robert Putnam, Harvard University 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 33
    34. Trust and Reciprocity • Trust can be built on • Personal experience • “I know you” • Shared experience • “We both worked on the same project” • Transfer of trust • “We know the same person who trusts us” • Shared values • “We agree to operate by the same rules” 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 34
    35. Semantic Web and LinkedData (UK) Royal Mail County Council9 July 2012 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL IOGDC Open Data Tutorial 35 Ordnance Survey
    36. Linking Data Via CommonNaming (Usually URLs) 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 36
    37. Example: Agency Names 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 37
    38. Can Be Lots of Things 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 38
    39. “Linking” Data goesbeyond govt Government Data is currently over ½ the cloud in size (~17B triples), 10s of thousands of links to other data (within and without) 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL http://linkeddata.org/ Tutorial9 July 2012 IOGDC Open Data 39
    40. 5 Star Data 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 40
    41. Creating an Open DataEcosystem 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 41
    42. Sustaining Data Publishing• A steward (someone tied to the business process) – Defines the resource – Purpose, audience, content, context, policies, operations, metrics, and access method – Manages the resource – Measures performance – Identifies needs and opportunities – Applies and enforces policies – Oversees daily operations – An open data project then is an enabler to creating the environment in which the processes can occur 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 42
    43. Embedding Publishing• There are three ways to look at creating a sustainable open data initiative – Step 1: Process: Oriented on the way in which people do their day-to-day work in the organization (the how and why) – Step 2: Services: Focused on who will help people share their knowledge and who will maintain tools and processes (the who) – Step 3: Systems: Are the IT infrastructure and tools necessary to deliver the processes and services efficiently and effectively to the end users (the what and where) 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 43
    44. Creating Community• Communities are public-facing spaces that present data, information, and subject matter knowledge about a single topic from many organizations in one place – The topics for communities can be chosen based on priorities from the public, departments based on their mission, or issues of national importance 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 44
    45. Community Vision• These questions help to guide early discussions – Vision: What will the community connection and collaboration look like in the future? – Leaders: Who will help to lead the community? – Participants: Who will participate? – Outcome: What are the expected outcomes, metrics, and measurements that will show success? How will this community work to improve the lives of citizens? – Functionality: What types of activities will be conducted on the site (forums, blogs, wikis, ranking, rating, challenges, or apps)? – Content: What content should be displayed – Interactivity: What ways will the community interact with the leaders, with each other, and with the public? 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 45
    46. Open Communities Community ✓ Developers Open Data ✓ Semantic Web ✓ Health ✓ Law ✓ Energy ✓ Education ✓ Ocean ✓ Safety ✓ Manufacturing ✓ Business ✓ Ethics ✓ Consumer Research and Development Cities + many more… 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 46
    47. Supporting Global Events Japanese tsunami, earthquake, and radiation monitoring Restore the Gulf: Deepwater Horizon Response 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 47
    48. Health.Data.gov Champion: Todd Park Chief Technology Officer US Dept. of Health and Human Services Apps Forums BlogsChallenges 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL 9 July 2012 IOGDC Open Data Tutorial 48
    49. Publicizing Data to Innovators• Challenges and code‐a-thons (health2challenge.org)• Many innovator “meetups” and conferences• Annual health data-paloozas• Over 139 applications• 50 new businesses• Thousands of lives improved each day• 1700 attendees at the Health Data Palooza in 2012 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL 9 July 2012 IOGDC Open Data Tutorial 49
    50. Creating Apps That Improve Lives:Asthmapolis 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 50
    51. Creating Apps That SaveLives: iTriage and HospitalCompare 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 51
    52. Use Case: Economic• Healthtap – Provides free, reliable, and independent health information through the use of open data and by building the world’s largest medical expert network tailored for young mothers and their children – Won 2010 Health 2.0 challenge – Success invigorated expansion and new hiring at HealthTap – Reaches to Data.gov datasets (along with others) 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 52
    53. Ensuring Use and EconomicBenefits  European Union market: EUR 28 billion in 2008 (Vickery)  £6 billion in the UK (Newbury, Bently and Pollock 2008)  The Open Data Institute 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 53
    54. Used in Collaborative Planning andGovernance Processes (from CTG SunyA) STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 Describe Identify and Identify and Rate Identify Summarize the Review Open Initiative Prioritize the Public Value Stakeholder Public Value Government Public Stakeholders Interests Assessments Value Portfolio Initiative Description: Identify a full list Identify the type For each Establish a Review public value of initiative and level of stakeholder and summary assessments Title stakeholders Public Value the each change assessment for across all initiatives Purpose initiative is mechanism the initiative for to inform decision Supporting Prioritize by expected to describe how each public value making. strategic goals rating each create for each of the expected type across all Program or policy stakeholder as the primary value will be the primary Does our open area A, B, or C stakeholders produced stakeholders government Tactics portfolio, taken as a Public Value is Change All summary whole, optimize our expressed in terms mechanisms include assessments are resources and of: impacts on: populated into capabilities while Economic value Efficiency the Portfolio meeting our Social Value Effectiveness Review mission and Political Value Intrinsic Workbook delivering Strategic Value Enhancement (Step 6) maximum public Quality of Life Transparency value to all Value Collaboration stakeholders? Ideological Value Participation Stewardship Value Initiative One-Five Workbooks Portfolio Review Workbook 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL
    55. What Can You Do WithOpen Data?• What are the uses of open data you have seen in your city, country, or organization?• Write down on the post-it notes one or more of the following: – What has been done with open data – What could be done with open data – What are the challenges you are facing – What have been your successes 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 55
    56. Additional Topics• Licensing, provenance, languages• Metadata design (international)• Trust – government data is controversial, who controls it?• Scaling – over 1M datasets and growing fast – How to search, store, link, translate, and archive• Versioning and updating• Visualization beyond the single dataset• Boundaries of open data 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 56
    57. Questions 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 57
    58. Summary and Next Steps• Join a community – W3C eGovernment Interest Group • http://www.w3.org/egov/wiki/Main_Page – Open Data Innovation Network on LinkedIn • http://bit.ly/ODNetwork 2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL9 July 2012 IOGDC Open Data Tutorial 58

    ×