Downloaded 10 times
Uploaded byLuigi Montanez
Civic Hacking @ MongoNYC
The document discusses civic hacking and how open source and open data can lead to better government. It explains that high quality raw data should be available in JSON, XML or CSV formats and through RESTful APIs. It also notes that MongoDB is used to store results from data scraping and provides RESTful JSON APIs. An example is given of querying a Drumbone API to return specific fields from a legislator record including earmark amounts and numbers. The assistant is thanked for its work and directed to Sunlight Labs for more information.
Civic Hacking @ MongoNYC
- 1. Civic Hacking Luigi Montanez luigi@sunlightfoundation.com
- 3.
- 4. Open Source +Open Data = Better Government Question? @LuigiMontanez
- 5. High Quality RawData Question? @LuigiMontanez
- 6. High Quality RawData ✴ First: Raw data in JSON, XML, or CSV Question? @LuigiMontanez
- 7. High Quality RawData ✴ First: Raw data in JSON, XML, or CSV ✴ Second: RESTful APIs in JSON or XML Question? @LuigiMontanez
- 8. High Quality RawData ✴ First: Raw data in JSON, XML, or CSV ✴ Second: RESTful APIs in JSON or XML ✴ Third: Nothing really... Question? @LuigiMontanez
- 16. How We UseMongoDB Question? @LuigiMontanez
- 17. How We UseMongoDB ✴ RESTful JSON APIs Question? @LuigiMontanez
- 18. How We UseMongoDB ✴ RESTful JSON APIs ✴ Results from Scraping Question? @LuigiMontanez
- 21.
- 22. {legislator: { in_office: true, title: "Rep", nickname: "", district: "9", bioguide_id: "L000551", govtrack_id: "400237", phone: "202-225-2661", website: "http://lee.house.gov/index.html", twitter_id: "", last_name: "Lee", name_suffix: "", last_updated: "2010/04/13 00:00:14 +0000", party: "D", chamber: "house", state: "CA", youtube_url: "http://www.youtube.com/RepLee", first_name: "Barbara", gender: "F", congress_office: "2444 Rayburn House Office Building", earmarks: { average_number: 20, total_amount: 10000000, average_amount: 22994535, total_number: 28, last_updated: "2010-03-18", fiscal_year: 2010, } ... }
- 23. ?sections=last_name,first_name,state,earmarks {legislator: { last_name: "Lee", state: "CA", first_name: "Barbara", earmarks: { average_number: 20, total_amount: 10000000, average_amount: 22994535, total_number: 28, last_updated: "2010-03-18", fiscal_year: 2010, } }
- 24. ?sections=last_name,first_name,state,earmarks.total_amount,earmarks.total_number {legislator: { last_name: "Lee", state: "CA", first_name: "Barbara", earmarks: { total_amount: 10000000, total_number: 28 } }
- 25.
- 26. Thanks! sunlightlabs.com @LuigiMontanez Question? @LuigiMontanez
Editor's Notes
- #3 I work for the Sunlight Labs, a group of developers and designers who use technology to transform government. We focus on transparency -- the idea that government can be more effective and more accountable when it makes data about its process, operations, and influences freely available to the public. Our work revolves around pushing government to open up its data, and then creating incredibly useful applications on top of that government data. We're creating an open source community around these ideas. So far, we have over a thousand members -- people committed to the idea that freeing data and writing code is the best way to make government better. I'm one of sixteen paid staff of the Sunlight Labs, and we're all employed by the Sunlight Foundation, based in Washington, D.C.
- #4 We want government to make more data about itself public. How it spends money. How it collects money. Where campaign contributions come from and how campaign money is spent. We want government agencies and departments who generate statistical economic data and scientific data for their own use to make that data public for all of us to use. That means well-structured, well-documented data published online and in real-time. That's what we want out of government transparency. Sometimes the phrase government data scares people, because it might conjure up thoughts of personally identifiable data, or classified national security data. That's not what we're trying to open up. We're talking about data that can be made public with compromising anyone's safety.
- #5 Something important to note is that we're not asking government to build some great, ground-breaking app. We're not asking them to invent the next Twitter, or have the creative energy of a startup. We're just asking them to publish government data, data that has been paid for by tax dollars, in an open, well-documented format. They open up the data, then we build the apps around that data.
- #6 So we believe that government's first and primary goal should be to publish its data in well-documented, widely-accepted formats: XML, JSON, CSV. Second, government should build APIs to allow us to get to that data programmatically, without having to keep up to date on downloaded data files. That's really all we want from government: open data sets and open APIs. When government starts building things with that data, they start to editorialize it, and enforce their opinions. We all know that even statistics can be molded to support pretty much any argument. But raw data is as close to an objective truth as we can get.
- #7 So we believe that government's first and primary goal should be to publish its data in well-documented, widely-accepted formats: XML, JSON, CSV. Second, government should build APIs to allow us to get to that data programmatically, without having to keep up to date on downloaded data files. That's really all we want from government: open data sets and open APIs. When government starts building things with that data, they start to editorialize it, and enforce their opinions. We all know that even statistics can be molded to support pretty much any argument. But raw data is as close to an objective truth as we can get.
- #8 So we believe that government's first and primary goal should be to publish its data in well-documented, widely-accepted formats: XML, JSON, CSV. Second, government should build APIs to allow us to get to that data programmatically, without having to keep up to date on downloaded data files. That's really all we want from government: open data sets and open APIs. When government starts building things with that data, they start to editorialize it, and enforce their opinions. We all know that even statistics can be molded to support pretty much any argument. But raw data is as close to an objective truth as we can get.
- #9 Fly On Time looks at the flight records compiled by the FAA, the Federal Aviation Administration, and tells you the likelihood of your particular flight being delayed, and some historic trends for your route and for your airport.
- #10 Here’s an example.
- #11 The National Geological Survey keeps track of earthquakes around the world, and Quakespotter is a 3D modeling of the globe with recent earthquakes. The data is very recent, sometimes only minutes old. The circles represent earthquakes and their magnitude is reflected by their radius. This is a desktop app that runs on Ruby Processing, and it won the contest’s best visualization prize.
- #12 Late last year, New York City held the Big Apps competition. Earlier in the fall, they released about a hundred data sets in what they called the DataMine, and put up $20,000 in prize money for developers. Here are some winners.
- #13 WayFinder is an augmented reality app for Android. It uses geolocation, the phone’s compass, and MTA transit data to point the user to the direction of the nearest subway station.
- #14 TaxiHack uses the city's taxi driver database to allow riders to tweet and email notes about their rides. Feedback positive and negative.
- #15 Big Apple Ed uses performance data about the NYC school system to help parents find the best school for their children. Yes, this one is a Rails app!
- #16 sinatra_resource
- #17 sinatra_resource
- #19 rich document