Making Open 
Data Useful 
BY 2014 CODE FOR AMERICA FELLOWS 
Danny Whalen // @invisiblefunnel 
Erik Schwartz // @eeeschwartz 
Livien Yin // @livienyin
Here’s what we’re talking about today! 
● What is Citygram? 
● How does it work? 
● Why was it built this way? 
● How can you get involved? 
● How did this work in Lexington?
This is about designing 
for the community.
Get ya awesome on 
Citygram is a notifications platform for 
subscribing to your city. It uses the power of 
open data to inform you of the things you care 
about, where you care about them.
3 GOALS FOR CITYGRAM 
01 // increase citizen engagement 
by providing automated, meaningful info about what the city is doing 
02 // be proactive, not reactive 
reduce the burden on staff to respond to repetitive requests 
03 // demonstrate the capabilities of 
city open data portals 
build trust with citizens through openness and accessibility
UNDERSTANDING THE CONSTRAINTS 
USE CASE // civic apps are not the intended 
use case for open gov data 
SPEED // gov data systems are not designed 
for speed 
CADENCE // data reflects the processes 
through which it was created
Screen Shot 
The Open Data Site
FOUR KEY GUIDING PRINCIPLES 
1. User Experience Design: start with user needs to transform 
data in a way that is accessible to citizens. 
2. Agile Development: a process of iterative, incremental 
software development that values constant learning and 
building. 
3. Flexibility: be able to hook into machine-readable data 
sources in common formats. 
4. Generalizable: across cities, for the greatest possible 
impact.
BACK-END ARCHITECTURE SKETCH
HOW IT WORKS
How can you integrate your 
open data with Citygram? 
citygram.org
HOW DOES A DATASET BECOME A CITYGRAM TOPIC? 
CRITERIA WE’RE LOOKING FOR: 
#1 meets citizen needs 
is there interest? does the city receive a lot of requests about this? 
#2 consistently updated 
anywhere from every three minutes to once a month. 
#3 accessible via API 
any standard, machine-readable format. JSON / CSV preferred. 
#4 has both time + geo component 
notifications are sent in real time if the event location is in the 
citizen’s area of interest.
DATA TRANSFORMATION // extract, transform, load to geoJSON 
{ 
"Y_COORD": "535574", 
"PROJECTNAME": "Auto Sales", 
"X_COORD": "1499638", 
"RECORDOPENDATE": "2014-02-07T00:00:00", 
"TASK": "Info Received", 
"COORDINATORNAME": "Alice Christenbury", 
"PROJECTDESCRIPTION": "Construct retail auto sales facility on vacant parcel. Grading permit, tree 
review, storm water detention, and PCCO review is required.", 
"PROJECTTYPE": "Commercial (Regular 15 business day Review)", 
"URL": "https://aca.accela.com/charlotte/Cap/CapDetail.aspx%63Module= 
LandDevelopment&TabName=LandDevelopment&capID1=14LDC&capID2=00000&capID3=00012", 
"OWNERNAME": "FRANCIS BROTHERS INVESTMENT GROUP LLC", 
"RECORDSTATUS": "Approved", 
"OWNERADDRESS": "6831 PINE LAKE LN", 
"PROJECTNUMBER": "LDC-2014-00012", 
"PARCELNUMBER": "11116306", 
"PROJECTMODULE": "LandDevelopment", 
"ADDRESS": "9907 ALBEMARLE RD, CHARLOTTE, NC 28227", 
"RECORDSTATUSDATE": "2014-08-07T00:00:00", 
"CONTACTADDRESS": "", 
"_id": 8032, 
"OWNERPHONE": "7045023809" 
}
CURATION // what elements are important to the citizen? 
{ 
"Y_COORD": "535574", 
"PROJECTNAME": "Auto Sales", 
"X_COORD": "1499638", 
"RECORDOPENDATE": "2014-02-07T00:00:00", 
"TASK": "Info Received", 
"COORDINATORNAME": "Alice Christenbury", 
"PROJECTDESCRIPTION": "Construct retail auto sales facility on vacant parcel. Grading permit, tree 
review, storm water detention, and PCCO review is required.", 
"PROJECTTYPE": "Commercial (Regular 15 business day Review)", 
"URL": "https://aca.accela.com/charlotte/Cap/CapDetail.aspx%63Module= 
LandDevelopment&TabName=LandDevelopment&capID1=14LDC&capID2=00000&capID3=00012", 
"OWNERNAME": "FRANCIS BROTHERS INVESTMENT GROUP LLC", 
"RECORDSTATUS": "Approved", 
"OWNERADDRESS": "6831 PINE LAKE LN", 
"PROJECTNUMBER": "LDC-2014-00012", 
"PARCELNUMBER": "11116306", 
"PROJECTMODULE": "LandDevelopment", 
"ADDRESS": "9907 ALBEMARLE RD, CHARLOTTE, NC 28227", 
"RECORDSTATUSDATE": "2014-08-07T00:00:00", 
"CONTACTADDRESS": "", 
"_id": 8032, 
"OWNERPHONE": "7045023809" 
}
TRANSLATION // turn machine-speak into human speak 
ELEMENTS 
> project name 
> project description 
> URL 
> address 
TEMPLATE 
Hi! A new land development 
permit has been submitted near 
you at [ADDRESS]. The project is 
called [PROJECTNAME] and 
proposes to [PROJECT 
DESCRIPTION]. Find out more at 
[URL].
➜
Citygram as 
a platform
SpyGlass: 
API adapters 
AKA 
INTRODUCING... 
“web content transformation proxy” (lolz)
Lexington, KY 
Citygram Integration: a case study
MOST BASIC REQUIREMENTS 
Create a geoJSON file for each dataset
WHERE WE WANT TO END UP 
Perfectly flowing data 
(go back in time for a great 
session at 9am today!)
GET LEAN 
We inched our way towards automation 
Kept system continually integrated
SHORT VERSION 
Pentaho Data Integration to update 
the data portal 
We schedule it 2x’s per hour
SIDE NOTE // geocoder to find address and parcel info
You can use many 
other geocoders
OPEN DATA PORTALS 
We’re still evaluating best approach
Extremely flexible: Beauty of Citygram 
Save a geoJSON file to a webserver 
Citygram picks it up!
MORE FLEXIBILITY // more better 
Use whatever automation/ETL 
processes at your disposal
VERY IMPORTANT 
Get to know and love each dataset. 
Evaluate the possible messages for the available information.
Let’s make this meaningful! 
Woo
Try it out!
thank you. 
questions, comments welcome 
citygram@codeforamerica.org 
#citygram 
@invisiblefunnel // @eeeschwartz // @livienyin

Making Open Data Useful: Citygram

  • 1.
    Making Open DataUseful BY 2014 CODE FOR AMERICA FELLOWS Danny Whalen // @invisiblefunnel Erik Schwartz // @eeeschwartz Livien Yin // @livienyin
  • 2.
    Here’s what we’retalking about today! ● What is Citygram? ● How does it work? ● Why was it built this way? ● How can you get involved? ● How did this work in Lexington?
  • 3.
    This is aboutdesigning for the community.
  • 4.
    Get ya awesomeon Citygram is a notifications platform for subscribing to your city. It uses the power of open data to inform you of the things you care about, where you care about them.
  • 5.
    3 GOALS FORCITYGRAM 01 // increase citizen engagement by providing automated, meaningful info about what the city is doing 02 // be proactive, not reactive reduce the burden on staff to respond to repetitive requests 03 // demonstrate the capabilities of city open data portals build trust with citizens through openness and accessibility
  • 6.
    UNDERSTANDING THE CONSTRAINTS USE CASE // civic apps are not the intended use case for open gov data SPEED // gov data systems are not designed for speed CADENCE // data reflects the processes through which it was created
  • 8.
    Screen Shot TheOpen Data Site
  • 9.
    FOUR KEY GUIDINGPRINCIPLES 1. User Experience Design: start with user needs to transform data in a way that is accessible to citizens. 2. Agile Development: a process of iterative, incremental software development that values constant learning and building. 3. Flexibility: be able to hook into machine-readable data sources in common formats. 4. Generalizable: across cities, for the greatest possible impact.
  • 10.
  • 11.
  • 12.
    How can youintegrate your open data with Citygram? citygram.org
  • 13.
    HOW DOES ADATASET BECOME A CITYGRAM TOPIC? CRITERIA WE’RE LOOKING FOR: #1 meets citizen needs is there interest? does the city receive a lot of requests about this? #2 consistently updated anywhere from every three minutes to once a month. #3 accessible via API any standard, machine-readable format. JSON / CSV preferred. #4 has both time + geo component notifications are sent in real time if the event location is in the citizen’s area of interest.
  • 16.
    DATA TRANSFORMATION //extract, transform, load to geoJSON { "Y_COORD": "535574", "PROJECTNAME": "Auto Sales", "X_COORD": "1499638", "RECORDOPENDATE": "2014-02-07T00:00:00", "TASK": "Info Received", "COORDINATORNAME": "Alice Christenbury", "PROJECTDESCRIPTION": "Construct retail auto sales facility on vacant parcel. Grading permit, tree review, storm water detention, and PCCO review is required.", "PROJECTTYPE": "Commercial (Regular 15 business day Review)", "URL": "https://aca.accela.com/charlotte/Cap/CapDetail.aspx%63Module= LandDevelopment&TabName=LandDevelopment&capID1=14LDC&capID2=00000&capID3=00012", "OWNERNAME": "FRANCIS BROTHERS INVESTMENT GROUP LLC", "RECORDSTATUS": "Approved", "OWNERADDRESS": "6831 PINE LAKE LN", "PROJECTNUMBER": "LDC-2014-00012", "PARCELNUMBER": "11116306", "PROJECTMODULE": "LandDevelopment", "ADDRESS": "9907 ALBEMARLE RD, CHARLOTTE, NC 28227", "RECORDSTATUSDATE": "2014-08-07T00:00:00", "CONTACTADDRESS": "", "_id": 8032, "OWNERPHONE": "7045023809" }
  • 17.
    CURATION // whatelements are important to the citizen? { "Y_COORD": "535574", "PROJECTNAME": "Auto Sales", "X_COORD": "1499638", "RECORDOPENDATE": "2014-02-07T00:00:00", "TASK": "Info Received", "COORDINATORNAME": "Alice Christenbury", "PROJECTDESCRIPTION": "Construct retail auto sales facility on vacant parcel. Grading permit, tree review, storm water detention, and PCCO review is required.", "PROJECTTYPE": "Commercial (Regular 15 business day Review)", "URL": "https://aca.accela.com/charlotte/Cap/CapDetail.aspx%63Module= LandDevelopment&TabName=LandDevelopment&capID1=14LDC&capID2=00000&capID3=00012", "OWNERNAME": "FRANCIS BROTHERS INVESTMENT GROUP LLC", "RECORDSTATUS": "Approved", "OWNERADDRESS": "6831 PINE LAKE LN", "PROJECTNUMBER": "LDC-2014-00012", "PARCELNUMBER": "11116306", "PROJECTMODULE": "LandDevelopment", "ADDRESS": "9907 ALBEMARLE RD, CHARLOTTE, NC 28227", "RECORDSTATUSDATE": "2014-08-07T00:00:00", "CONTACTADDRESS": "", "_id": 8032, "OWNERPHONE": "7045023809" }
  • 18.
    TRANSLATION // turnmachine-speak into human speak ELEMENTS > project name > project description > URL > address TEMPLATE Hi! A new land development permit has been submitted near you at [ADDRESS]. The project is called [PROJECTNAME] and proposes to [PROJECT DESCRIPTION]. Find out more at [URL].
  • 19.
  • 20.
    Citygram as aplatform
  • 21.
    SpyGlass: API adapters AKA INTRODUCING... “web content transformation proxy” (lolz)
  • 22.
    Lexington, KY CitygramIntegration: a case study
  • 23.
    MOST BASIC REQUIREMENTS Create a geoJSON file for each dataset
  • 24.
    WHERE WE WANTTO END UP Perfectly flowing data (go back in time for a great session at 9am today!)
  • 25.
    GET LEAN Weinched our way towards automation Kept system continually integrated
  • 26.
    SHORT VERSION PentahoData Integration to update the data portal We schedule it 2x’s per hour
  • 27.
    SIDE NOTE //geocoder to find address and parcel info
  • 28.
    You can usemany other geocoders
  • 29.
    OPEN DATA PORTALS We’re still evaluating best approach
  • 30.
    Extremely flexible: Beautyof Citygram Save a geoJSON file to a webserver Citygram picks it up!
  • 31.
    MORE FLEXIBILITY //more better Use whatever automation/ETL processes at your disposal
  • 32.
    VERY IMPORTANT Getto know and love each dataset. Evaluate the possible messages for the available information.
  • 33.
    Let’s make thismeaningful! Woo
  • 34.
  • 35.
    thank you. questions,comments welcome citygram@codeforamerica.org #citygram @invisiblefunnel // @eeeschwartz // @livienyin