Cultivating Open Geo-Data in the Real World

Cultivating Open Geo-Data in the Real-World: A Guided Tour of Three Portland-based Projects Matt Blair OSCON, Portland, OR July 29, 2011

Who’s working with Open Data?

What’s the Scope?• National• State/Province/Canton/Prefecture• City• Neighborhood

What’s beenchallenging?

What’s been easy?

Outline• Common project goals• The three projects, and their challenges• Solutions• Summary/Q&A

Project Goals

Free iOS AppsFor Education and Easy Participation

Open Datasets

Open Datasets• real-time geo-queryable• living/dynamic/well-maintained• available to other projects/platforms• relevant beyond lifecycle of initial apps

Broader GoalsTo Promote:• Learning and exploring• Appreciation of place (Portland)• Participation in community projects• Walking and health

The Projects

Poetry Posts

What’s a Poetry Box?

“A poetry post...is a wooden pole, usually,mounted on private property, so that itfaces pedestrians. On top of the pole is abox, with a glass or clear face and a lid.Inside the box is a sheet of paper containinga poem...” – Laura O. Foster

http://poetrybox.info

Challenges

Where are they?

Where are they? No comprehensive list.

Wait – what are they called?

A Poetry Box

Poetry Posts

A Poetry Pole

A Poetry “Tree”

Divergent Requests

PDX Trees

(a side project)

Civic AppsLots of great ideas – most without data

Civic Apps Data

What are Heritage Trees?"...a list of trees that – because of their age,size, type, historical association orhorticultural value – are of specialimportance to the City. Upon recommendation ofthe Urban Forestry Commission, the City Council maydesignate a tree as a Heritage Tree provided the treeshealth, aerial space, and open ground area for the rootsystem have been certiﬁed as sufﬁcient."

283 Heritage Trees

Challenges

Thin Metadata

A Verbose Specimen{ "address": "2403 WI/ SW JEFFERSON ST", "circumfere": "12.300000000000001", "common_nam": "Deodar cedar", "diameter": "47", "geometry": { "coordinates": [ -122.70463884770101, 45.521710633334202 ], "type": "Point" }, "gid": "103", "height": "73", "notes": "between SW Marconi Ave and SW Tichner Dr", "objectid": "103", "owner": "Right Of Way", "scientific": "Cedrus deodara", "spread": "73", "stateid": "1N1E32 100", "status": "Heritage", "treeid": "113", "year": "1996"}

Visual Appeal?

PDFs and Photos?

“...like PDX Trees, but with Public Art...

Public Art PDX

Rewind to May 2010...

Before PDX Trees...

Before Poetry Posts...

Green Line (2009)

Green Line Art on the Web

Green Line Art Podcast

Permission for re-use?

Fast Forward toNovember 2010

No longer an outsider?

Challenges

Multiple Organizations

Multiple Data Formats

Will they share data?

Do they have data?!

Inaccurate Geo-coding

Intellectual Property

Refactored List of Challenges• Building Data • Creating Visual Appeal• Lack of Consensus • Data Accuracy• Multiple Sources • IP Uncertainty• Multiple Data Formats • Teams and Roles

Solutions

Building Datasets

Data IdeasThe Open Data Universe

DataIdeas Work here. Option #1

Option #2:Assemble your own data

If it’s already digitized:• Screen-scrape• Repurpose feeds or reporting systems• Google Reﬁne• Convince governments/stakeholders to release it

If it’s not digitized?

Web Forms?

Most data arrives like this:

Tough to Automate

Poetry Posts Dozens and dozens of emailsin and outside of a Google Group

Don’t underestimate data collection.

Lack of Consensus

What’s this called?

“You should add social features...”

“You should add social features...” Comments, Photo Uploads, Likes, Friends, Flags

“You know, make it like Facebook!”

Scope Creep

Who’s the client?

Who decides? How?

Build the leastfunctional version

Rapid Feedback Cycles

Just Pick Something.

Just Pick Something. (Benevolent Dictator For This Release?)

Multiple Sources

• Who gathered it?• To what end?• Are they maintaining it?• Do they want to share?

PDX TreesOne list, managed by one department

Public Art Data in Portland

Data Sources (Nov 2010) • PDX API: 279 works of art • Civic Apps CSV ﬁle: 366 works of art • BTS: 300ish works of art (no direct access) • RACC.org: 1800+ works or art

PDXAPI RACC.org Civic Apps BTS?How do they ﬁt together?

RACC.org Civic Apps PDXAPINested subsets?

But they didn’t match...

RACC.org Permanent Collection

Text = Available Data

Text = Available Data As of November 2010: ~ 370 works of art

RACC.org

RACC.org (City of Portland)(Multnomah County)

Not quite...

(Caveat: Not To Scale) Accuracy Not Guaranteed

RACC.org Public Art

RACC.org Murals Public Art

RACC.org Murals TriMet Public Art

RACC.org Murals TriMet Convention Center? Public Art

RACC.org Murals TriMet Metro? Convention Center? Public Art

RACC.org Murals Port of TriMetPortland Metro Convention Center Public Art

Fountains RACC.org Murals Port of TriMetPortland Metro Convention Center Public Art

Fountains RACC.org MuralsParks& Rec Port of TriMetPortland Metro Convention Center Public Art

Fountains RACC.org MuralsParks& Rec Port of TriMetPortland Beaverton? Metro Convention Center Public Art

Fountains RACC.org MuralsParks& Rec Port of TriMetPortland Beaverton? Hillsboro? Metro Convention Center Public Art

Fountains RACC.org MuralsParks& Rec Port of TriMetPortland Beaverton? Hillsboro? Metro Clark Convention County? Center Public Art

Fountains RACC.org MuralsParks& Rec Port of TriMetPortland Performance Beaverton? Hillsboro? Metro Clark Convention County? Center Public Art

Fountains RACC.org MuralsParks& Rec Port of TriMetPortland Performance Temporary Installations Beaverton? Hillsboro? Metro Clark Convention County? Center Public Art

Fountains Street RACC.org Murals Art?Parks& Rec Port of TriMetPortland Performance Temporary Installations Beaverton? Hillsboro? Metro Clark Convention County? Center Public Art

(aka Grafﬁti)

“Of course not!”

“But what if it’s Banksy?”

Fountains Street RACC.org Murals Art?Parks& Rec Port of TriMetPortland Performance Temporary Installations Beaverton? Hillsboro? Metro Clark Convention County? Center Community? Public Art

Community Collection

Where’s Paul?

photo by Cacophony (via Wikipedia)

Intersection Repairphoto by City Repair (via Flickr)

Julian Voss- Andrae’s Alpha Helix(at the Linus Pauling House) photo via julianvossandrae.com

Fountains Street RACC.org Murals Art?Parks& Rec Port of TriMetPortland Performance Temporary Installations Beaverton? Hillsboro? Metro Clark Convention County? Center Community? Public Art

Fountains Street RACC.org Murals Art?Parks& Rec Available Port of Data TriMetPortland Performance Temporary Installations Beaverton? Hillsboro? Metro Clark Convention County? Center Community? Public Art

A Community-wide DatabaseBuild an inter-agency superset of public art

Yes, it’s complicated.

Yes, it’s complicated. Yet Portland has some of the best public art data in the country.

Data Source Tips• Start small• Find allies and implement an example• Don’t wait for the perfect dataset• Plan for chaos• But be ambitious in the long-term!

Multiple Data Formats

Each source has its own:• metadata• schema• volatility• level of accuracy and currency• messes to clean up

A dynamic mix of...• noisy data• shifting standards• unexpected restrictions• adapting to community requests• addressing data provider concerns• ambiguities of project ownership

Less about architecture than improvisation...

Metadata Flexibility Let early adopters set the standards

Document Databases

CouchDB

Eventual Schema

“There is alwaysschema somewhere.”

“There is alwaysschema somewhere.” If it’s not in your software, you’re forcing your audience to build it in their heads.

Delay Schema Decisions Push them up to presentation/client layer

But what about MVC?

But what about MVC? My model is in my view?!

More nimble thanrelational database Especially during development

Creating Visual Appeal

Heritage Trees{ "address": "2403 WI/ SW JEFFERSON ST", "circumfere": "12.300000000000001", "common_nam": "Deodar cedar", "diameter": "47", "geometry": { "coordinates": [ -122.70463884770101, 45.521710633334202 ], "type": "Point" }, "gid": "103", "height": "73", "notes": "between SW Marconi Ave and SW Tichner Dr", "objectid": "103", "owner": "Right Of Way", "scientific": "Cedrus deodara", "spread": "73", "stateid": "1N1E32 100", "status": "Heritage", "treeid": "113", "year": "1996"}

Public Art{ "docs": [{ "addrCity": "", "addrState": " ", "addrStreet": "", "addrZip": "", "artists": "Dan Corson", "date": "2009", "dateModified": "2011-04-18 00:00:00", "description": "Mercurial Sky is an ever-changing array of light played on LEDtubes integrated into the Director Park Canopy. The digital video only emits from thelighted bars, and provides a sense of movement through an abstract tapestry of lightand color. If you stand farther away, or look in nearby reflections, the images arecompressed and give a clearer view of the video. nn"I filmed images and patterns ofnatural phenomena like waves, clouds, fire, earthworms, and jellyfish to bring themovement and randomness of nature into this mostly hardscaped park."", "detailPageURL": "http://racc.org/public-art/search/?recid=2909.101", "dimensions": "duration: 1:23:10", "discipline": "video", "fundingSource": "Percent for Art - City of Portland", "thumbnailURL": "http://data.racc.org/pa_inventory/1844/1844thumb.jpg", "location": "Director Park Canopy", "mappableDiscipline": "other", "medium": "Digital video on DVD", "recordID": "2909", "title": "Mercurial Sky", "dataSource": "RACC", "collection": "None", "photoCredit": "RACC", "artCopyright": "TBD", "locationVerified": "YES", "geometry": { "coordinates": [ -122.681124, 45.518759 ], "type": "Point" }}

PDX Trees{ "address": "2403 WI/ SW JEFFERSON ST", "circumfere": "12.300000000000001", "common_nam": "Deodar cedar", "diameter": "47", "geometry": { + ? "coordinates": [ -122.70463884770101, 45.521710633334202 ], "type": "Point" }, "gid": "103", "height": "73", "notes": "between Marconi Ave and SW Tichner Dr", "objectid": "103", "owner": "Right Of Way", "scientific": "Cedrus deodara", "spread": "73", "stateid": "1N1E32 100", "status": "Heritage", "treeid": "113", "year": "1996"}

PDX Trees{ "address": "2403 WI/ SW JEFFERSON ST", "circumfere": "12.300000000000001", "common_nam": "Deodar cedar", "diameter": "47", "geometry": { + "coordinates": [ -122.70463884770101, 45.521710633334202 ], "type": "Point" }, "gid": "103", "height": "73", "notes": "between Marconi...", "objectid": "103", "owner": "Right Of Way", "scientific": "Cedrus deodara", "spread": "73", "stateid": "1N1E32 100", "status": "Heritage", "treeid": "113", "year": "1996"}

PDX Trees{ "address": "2403 WI/ SW JEFFERSON ST", "circumfere": "12.300000000000001", "common_nam": "Deodar cedar", "diameter": "47", "geometry": { + ?? "coordinates": [ -122.70463884770101, 45.521710633334202 ], "type": "Point" }, "gid": "103", "height": "73", "notes": "between Marconi Ave and SW Tichner Dr", "objectid": "103", "owner": "Right Of Way", "scientific": "Cedrus deodara", "spread": "73", "stateid": "1N1E32 100", "status": "Heritage", "treeid": "113", "year": "1996"}

Poetry Posts1991 SW Mill St Terrace

PDX Trees{ "address": "2403 WI/ SW JEFFERSON ST", "circumfere": "12.300000000000001", "common_nam": "Deodar cedar", "diameter": "47", "geometry": { + "coordinates": [ -122.70463884770101, 45.521710633334202 ], "type": "Point" }, "gid": "103", "height": "73", "notes": "between Marconi...", "objectid": "103", "owner": "Right Of Way", "scientific": "Cedrus deodara", "spread": "73", "stateid": "1N1E32 100", "status": "Heritage", "treeid": "113", "year": "1996"}

Sunny October Day?

Sunny October Day? No one will believe it.

More Realistic

In Rarer Weather, too...

But there are 283 Trees!

But there are 283 Trees! I need help.

Create acrowd-sourced,longitudinal,season-sortablecollection oftree photos...

Will anyone send photos?

by Brad B

by kateinoregon

“KeepPortlandGreen!”by Dan Flynn

(and I still add some...)

350+ Photos Sent In

Submitted underCreative Commons

Photos (and data)available for re-use

Data Accuracy

What’s Missingor Incorrect?

What’s Missingor Incorrect? Errors and omissions arean opportunity for participation.

> 1 mile

Artifact of Geo-Coding Address != Location

CapacityWho has the time to manually verify all these locations?

Crowd-Correction

IP Uncertainty

I thought this was “open” data?

Restrictions inTerms of Use

Linked Media

Public Art{ "docs": [{ "addrCity": "", "addrState": " ", "addrStreet": "", "addrZip": "", "artists": "Dan Corson", "date": "2009", "dateModified": "2011-04-18 00:00:00", "description": "Mercurial Sky is an ever-changing array of light played on LEDtubes integrated into the Director Park Canopy. The digital video only emits from thelighted bars, and provides a sense of movement through an abstract tapestry of lightand color. If you stand farther away, or look in nearby reflections, the images arecompressed and give a clearer view of the video. nn"I filmed images and patterns ofnatural phenomena like waves, clouds, fire, earthworms, and jellyfish to bring themovement and randomness of nature into this mostly hardscaped park."", "detailPageURL": "http://racc.org/public-art/search/?recid=2909.101", "dimensions": "duration: 1:23:10", "discipline": "video", "fundingSource": "Percent for Art - City of Portland", "thumbnailURL": "http://data.racc.org/pa_inventory/1844/1844thumb.jpg", "location": "Director Park Canopy", "mappableDiscipline": "other", "medium": "Digital video on DVD", "recordID": "2909", "title": "Mercurial Sky", "dataSource": "RACC", "collection": "None", "photoCredit": "RACC", "artCopyright": "TBD", "locationVerified": "YES", "geometry": { "coordinates": [ -122.681124, 45.518759 ], "type": "Point" }}

Linked Media Who took that photo? Who owns it? Who can license it?

Teams and Roles

Who do you need?

Standard Roles• Project Management • Software Engineering• Graphic Design • Testing• Information Architecture • Maintenance• Metadata Design • Coordination• Data Entry/Import • Marketing• UI Design • Communication

Roles That Deserve Special Attention

Data Collection

Fountains Street RACC.org Murals Art?Parks& Rec Available Port of Data TriMetPortland Performance Temporary Installations Beaverton? Hillsboro? Metro Clark Convention County? Center Community? Public Art

Data Scrubbing Probably Need Tech Skills

Research & Veriﬁcation Not necessarily a technical person

CurationWhat’s in, what’s out and why

Authority(not required, but it helps)

Advocacy Why are we doing this?Why should you release this data?

PDX Trees

PDX Trees‘Mostly’ a solo project

There are no‘solo’ projects.

PDX Trees Collaborators • Urban Forestry Commission • City Council • City of Portland’s Bureau of Technology Services • Max Ogden’s de-shapeﬁle-iﬁcation • Everyone who has contributed photos

Public Art PDX 1.0Essential Collaborators• Regional Arts & Culture Council (RACC)• The Ofﬁce of Mayor Sam Adams• City of Portland’s Bureau of Technology Services• City Attorney (and pro bono attorneys)• Former Mayor Bud Clark! (see video)

Public Art PDX 1.0Collaborators Provided:• Graphic Design • Communication• Metadata Design • Data Collection• Data Entry/Import • Data Scrubbing• Testing • Authority• Marketing • Advocacy

Beyond 1.x: Community CollectionDifferent Phases, Different Roles, New Participants

Expanding Roles for theCommunity Collection• Metadata Design • Curation• Data Entry/Import • Photography• Research and Veriﬁcation • Communication

Two More Things

Data Has Stories

“While the map makes class and racedifferences all the more evident, its great tolearn about the few murals where I live andI look forward to using this app on a walkingtour downtown soon.” – Marshal Kirkpatrick, Read Write Web

Incomplete Data

No Recent PublicBuilding Projects

Less Economic Activity

Voice andRepresentation

Share Open Data:Interpretations Emerge

A Tool for Advocacy?

The Paces of Change

– From How Buildings Learn: What Happens After Theyre Built by Stewart Brand, p13 Scan source: CC by-nc-sa via http://www.ﬂickr.com/photos/andrewcoulterenright/15663020/

The Pace of Each Layer http://www.ecobuildtrends.com/2011/02/designing-in-layers.html

"A design imperative emerges: An adaptive building hasto allow slippage between the differently-paced systems ofSite, Structure, Skin, Services, Space plan, and Stuff.Otherwise the slow systems block the flow of thequick ones, and the quick ones tear up the slow oneswith their constant change. Embedding the systemstogether may look efficient at first, but over time it isthe opposite and destructive as well."– From How Buildings Learn: What Happens After Theyre Built by Stewart Brand, p20

Websites =Presentation

Apps = Presentation

“Can I get that icon in cornﬂower blue?”

Presentation Layers are Ephemeral

Presentation Layers are Ephemeral They come and go, like fashion

This year’s attire

What costumes willyour data wear in ﬁve years’ time?

Who knows.

Data is the Substrate

Data is theSite and Structure

Design for Different Paces

A particular appis a short-term goal

Like Furniture

A Shearing LayersModel of Open Data?

Hmm...

Hmm...(Need More Input)

De-couple Data and Presentation

A Design Philosophy:Data is the generative source of the entire ecosystem.

Project Websites• http://poetrybox.info• http://pdxtrees.org• http://publicartpdx.com

Thank You• http://mattblair.net• Email: elsewisemedia@gmail.com• Blog: http://elsewisestrategic.com• github.com/mattblair

http://www.oscon.com	42
http://lanyrd.com	5
https://twitter.com	3

Cultivating Open Geo-Data in the Real World

by OSCON Byrum

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 50

Statistics

Cultivating Open Geo-Data in the Real World Presentation Transcript