State of the Map 2015

1. OpenStreetMap + Government Data of Licenses, Feedback Loops, and Analysis Andrew Turner CTO, Esri R&D DC @ajturner andrew@esri.com
2. OpenStreetMap is a Global Open Data project
3. OpenStreetMap is about feedback loops
4. Data Lifecycles
5. Government Data Credit: Tiago Fioreze
6. Building the Public Square flickr: badwsky flickr: familymwr
7. Esri Community
8. Cities at Macro http://www.urbanobservatory.org/
9. Global Open Data
10. http://opendata.arcgis.com
11. Can I use this data?
12. Treat licenses like data. Structured, Queryable, Usable
13. http://spdx.org/ { "license" : { “id": "CC0-1.0", "name": "Creative Commons Zero v1.0 Universal", "link": "https://spdx.org/licenses/CC0-1.0.html", "capabilities": { "commercial": true, "attribution": false, "modifications": true, "sharealike": false } }} /datasets/:id/license.json license ID from
14. License Commercial Use Share-Alike Custom
15. gh://jonahadkins/chesapeake-OSM-imports
16. What does that enable? OpenStreetMap in ArcGIS
17. Tunis, Tunisia 36k OpenStreetMap Commercial Tripoli, Libya 288k Bujumbura,Burundi 18k Juba, Sudan 72k
18. Road Centerlines Land Use Polygons Administrative Boundary Tree Points Railways Building Footprints Walls & Fences Landscaped Polygons Trails and paths Parking Lots Waterlines and Water bodies Points of Interest
19. Giving Back: ArcGIS Editor for OpenStreetMap http://wiki.openstreetmap.org/wiki/ArcGIS_Editor_for_OSM
20. gh://esri/arcgis-osm-editor
21. gh://esri/arcgis-osm-editor
22. http://maps.who.int/networkanalysis/
23. ata:text/x-url,http://koop.dc.esri.com/osm/points/state/New%20York/county/New%2 gh://koopjs/koop-osm
24. gh://koopjs/koop-osm
25. flickr: Scooter Lowrimore Have Fun Mapping!
26. Thank you for all you do. Andrew Turner andrew@esri.com http://heorb.it/sotmus2015

Editor's Notes

At Esri, we believe that geography is at the heart of a more resilient and sustainable future. Governments, industry leaders, academics, and nongovernmental organizations (NGOs) trust us to connect them with the analytic knowledge they need to make these critical decisions that shape the planet.
I can’t underscore the impact that OpenStreetMap has had in merely validating the theory of crowd-sourcing. Years ago when I would meet with various government agencies they would scoff at the idea that ‘anyone’ could contribute meaningful data.

Now, that conversation is flipped. They are eager to collaborate.
OpenStreetMap has also demonstrated how the community is more than merely the collection of features in a database.
OSM is the ‘Netflix of Open Data’. - Entirely API accessible, ad-hoc and emergent community practices.
Data is never at rest. It is constantly in motion and part of and permeates a larger ecosystem. However we can only visually observe a small portion of this cycle - the part which OSM has already done very well at.
A majority of this data are in places and forms that we can’t see. What’s in the ocean? (of data). How does it enter this broader ecosystem? see where I’m going with this analogy?
I can’t underscore the impact that OpenStreetMap has had in merely validating the theory of crowd-sourcing. Years ago when I would meet with various government agencies they would scoff at the idea that ‘anyone’ could contribute meaningful data.

Now, that conversation is flipped. They are eager to collaborate.
Government is in the business of building shared public infrastructure that encourages communities and businesses to convene, develop and thrive. Typically that means parks, roads, police, etc. Increasingly this also means digital and technical infrastructure that is available for public utilization.

"Those of us who believe in the progressive power of accurate data believe that we can use information to strengthen our connections to one another. Through those connections we have the potential to change the course of a city's history, a state's history, a country's history — perhaps even our planet's history — parcel-by-parcel, neighborhood-by-neighborhood, and most importantly, neighbor-by-neighbor. Today violent crime is lower, public schools are stronger and the Chesapeake Bay is healthier than at any time in recent memory. These accomplishments were not achieved by accident or in a vacuum. They have been achieved through the deliberate actions of dedicated individuals, non-profit leaders, conscientious businesses and elected officials. Together, through the connections made when ideas and information are shared, we will continue to move Baltimore and Maryland forward.” - Citistat, 10 years (2000-2010)
If there is one dept in gov’t that has decades of experience managing big, structured data through web services for the public - it’s the GIS department. It’s a community of millions of map-geeks, passionate about using geography to make their communities safer, environment + science better, and provide better services.
Cities are eager for their data to be available so it can be analyzed, compared and give them insights into how to improve living and economic conditions as their populations swell.
23,000 governments globally use ArcGIS to manage their data. We wanted to make it extremely simple and cost-free for them to share their data openly. So we launched “ArcGIS Open Data” - a “button” that any ArcGIS customer - regardless if they are the United Nations, or a county of 100 people, to publish their data via an open API that is web discoverable.

In under year - nearly 2,000 government organizations have created 1,000 public open data sites with more than 26,000 datasets. These include roads, parcels, addresses, crimes, trees, rivers, oyster sanctuaries, health facilities, demographics, historic markers, and so much more.
Organizations can create, customize and publish their own open data sites in their local languages and with their own data.
We also federate the data catalog to a central search repository at http://opendata.arcgis.com - where anyone can search across the entire global open datasets. For example - search for Addresses http://opendata.arcgis.com/datasets?q=addresses or Parcels http://opendata.arcgis.com/datasets?q=parcels

All data are coming directly from the government services via APIs.
So you can start searching and downloading all the address data from around the world - but what can you do with it?
As the OSM community is very aware, technical capability does not mean legal ability.
Each dataset has a license - but what’s there varies. Ideally it’s something like Creative-Commons-Zero or Public Domain
but too often it’s a copy & paste legalese with a lot of details. The essence may still mean “Use as you want, but don’t blame us if it’s wrong and mention where it came from”.
Even better - essentially unspecified. That’s what you get when you have a free text field for what essentially should be a Natural Language Parser.
Beyond just natural language legal - what if we could encode licenses with their capabilities?
Can I use this in OSM? Can I use it in my Project?
No one reads the metadata - so make it part of the user experience in how they search and discover data.
ArcGIS Open Data is introducing “Structured Licenses” - where data providers can choose from a list of well-known open licenses. This should help people new to the open licensing world make good initial decisions. It also encodes the capabilities within each license.

And if they choose a custom license - we want to still allow them to specify what is allowed or disallowed by their custom legal terminology.
We likely will use the SPDX license markup as a baseline. This is a standard that is also used by NPM (node package manager) for library licenses.
The Software Package Data Exchange® (SPDX®) specification is a standard format for communicating the components, licenses and copyrights associated with a software package.

It will require some extensions - which yay JSON and extra keys - though we also want to suggest these back to the core standard.
So in the end - all this Legal + JSON so that a user can search for “Show me data that I can use Commercially” or “Show me data that I can use in OSM”.
What we want to enable is make it easy for the community to leverage this global ocean of data that governments are making publicly available and integrate that with OpenStreetMap.
Jonah Adkins - a particularly astute engineer - has developed scripts which pull data from ArcGIS Open Data and convert them to OpenStreetMap format for editing and contribution.
Now Chesapeake has the building addresses that would have been expensive or taken a long time to gather manually. And ideally this can be kept up to date from the government data sources that are being published.
He’s done it for a number of cities.
Better data in OSM has multiple purposes. We also want to use it in our numerous information products.
We evaluated a number of cities globally and determined where our own data (often commercial) sources had far less quality or completeness compared with OSM. So we developed mechanisms to use OSM data to generate basemaps in areas where it could be our source of data.
And we’re doing this for increasing areas of our map and features.
So how do we round-trip this ecosystem? We’re helping to evaporate data from the ocean into the clouds to feed the streams. How does OSM and GIS more directly connect?
We develop and support an open-source editing interface integrated into ArcGIS Desktop that allows users to sync OSM data, batch-upload from their existing datasets, etc. Available on Github: https://github.com/Esri/arcgis-osm-editor - we’d love to work with others on improving the editor to support newer OSM API methods as well as improve the UX.
We developed a up-to-date extract of OSM over three most Ebola-affected countries and updated the tiles daily. This was used for Offline data gathering with Collector (offline tiles through ArcGIS Server).
http://maps.who.int/networkanalysis/
Only if this is appealing... It is OK to demo, but the data is older. In theory this could be a dataset we kept up to date with OSM edits. The key thing we provided here that OSRM did not was the drive-time service areas (and API behind it) as opposed to point-to-point routes with OSRM
Another project we use with OSM is Koop. Koop is an open-source project that makes it easy to make different Web APIs interoperable. Developed as a set of Node.js modules, developers can quickly make custom API available in other standard formats.

O.SKoop.M is a version of Koop which provides for REST API and GeoServices interface to OpenStreetMap Data. http://koopjs.github.io/o.skoop.m
We can convert OSM data into GeoJSON.
or load OSM layers into ArcGIS Online and Desktop to do, say, heatmap of pubs in NYC from OpenStreetMap.
Have fun mapping.

State of the Map 2015

You are reading a preview.

Check these out next

State of the Map 2015

More Related Content

Similar to State of the Map 2015 (20)

More from Andrew Turner (20)

Recently uploaded (20)

State of the Map 2015

Editor's Notes

State of the Map 2015

You are reading a preview.

State of the Map 2015

Recommended

More Related Content

Similar to State of the Map 2015 (20)

More from Andrew Turner (20)

Recently uploaded (20)

State of the Map 2015

Editor's Notes