Open Data and Beyond. An Open Data Story from Nanaimo


Published on

In November, 2011, Jason Birch presented this to over eighty municipal delegates to the MISA Prairies Open Data Workshop.

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • I am fortunate to work as part of a small but passionate group of analysts responsible for corporate application sustainment, business analysis, development, database administration, reporting and other duties as required Today, I’m going to talk about what triggered Nanaimo’s movement towards open data, the approach that we have taken, and what kind of results we have seen. As well, I’ll give some insights into our way forward.
  • The problem was that we were failing to meet the expectations of our citizens and businesses by communicating information in ways they could most effectively use it.Basically, frustrations were rising because it was more difficult to get information from Government than it should have been.It is important to note that our audience has a wide range of expectations and technological capacity… and that both change over time.The thing is…
  • … this isn’t a new problem for government.I don’t know if anyone recognizes this, it’s from a British sitcom called “Yes, Minister” that first aired in 1980. And its first episode: “Open Government” Even then, Open Government was a familiar enough term that the audience understood it… and were likely a bit jaded. The cynical treatment of open government being squashed by an entrenched civil service is a bit painful…
  • Take a few seconds to read these quotes.Sounds familiar, right? But…This article was written about Nanaimo’s first foray into online services that my director started work on in 1994. I’d encourage you to read it, it’s full of nuggets like banks of 14.4 baud modems and internet access for as little as $1/hour.I had always though that Nanaimo’s commitment to innovation was “just” part of the corporate culture, but reading this article makes me wonder if it is partially due to the ghost of open government past.
  • So, from that point back in the Palaeolithic web, we focussed on delivering web applications that met the needs of specific user types. Some successes included bid opportunity alerts, garbage calendar lookup, job applications and online mapping, but there were many others.
  • The problem was that while we were fixated on building more and more applications on our website we missed a fundamental change in the way that our citizens interact with technology.Businesses were becomingdata driven, using technology to help them with things as varied as business siting decisions and new property development. Time and again we heard anecdotal evidence of local businesses recreating information for internal purposes because our applications did not meet their specific needs.At the same time, online information access had changed radically, primarily because of Google.
  • Advances in search engine technology meant that citizens had become used to getting the answer to their query immediately, rather than searching for the authority’s website and braving the different navigation schemes on each site.Unfortunately…
  • … when trying to find government information they felt like they were back in the stone-ages. And despite our best efforts, we were again faced with the recurring theme that government wasn’t open. We were not doing enough to meet our citizens needs for information.
  • Having identified areas where we were falling down, we decided that we had to focus on matching the information delivery methods to the people who needed it.
  • We had to find ways of providing raw data to the technical businesses and general users who could best make use of these advanced formats and even APIs.Now, this wasn’t a new concept. The provincial and federal government had been making some of their data available for free via sites like GeoGRATIS and various FTP servers for almost a decade. Adam Chadwick at the City of Kamloops made the case for free data publishing in 2001 and quietly put it into production in 2002. In general though, freely downloadable data was a pretty foreign concept to local governments.
  • Like many of our new web-facing projects, Nanaimo went about open data publishing in a series of small steps, trying to figure out what worked and what didn’t. While Engineering started publishing raw GIS data for download, IT was working on building data feeds and services into new applications. In late 2008, Washington DC launched an open data catalogue, and this was followed by similar efforts. We saw the value in providing links to all data sources in one location, and decided to put together something quick on our site to support the same thing.The first round of our data catalogue that we launched in 2009 pointing to existing open datasets was a single unpaged list of items, took a few days to build (with an admin interface for adding new items) and didn’t have many additional features.
  • In 2010,we rewrote our data catalogue by forking OGDI (a Microsoft cloud-based open data tool) and created a version of our data catalogue with far more functionality, and which anyone running IIS and SQL Server could use.It includes items such as data browsing, links to external datasets CSV download, charts, and developer samples.
  • Apart from the open data catalog, we realized that we had to stop hiding database contents behind search forms and instead ensure that all records were easily indexed by search engines so that the public could easily find what they wanted.
  • As part of our web mapping rewrite, we made sure that all of the property reports were crawlable by Google starting from a plain HTML street index, and built sitemaps to submit as well. Initially the results were pretty spotty, but many properties were being returned by Google searches.For more on this concept, see my blog post: tool I am using to publish these reports is called GeoREST, and allows HTML (and JSON, XML, KML, etc) to be published from spatial datasets with no programming:
  • We went forward with similar initiatives for our business license database with similar results, and also extended it to the public art inventory.
  • Measurement is an important tool for us. It allows us to validate the approaches we are using, determining where we may need to change the way we are doing things, or in some cases just confirming that we are on the right path.
  • The data catalogue has proven to be reasonably popular considering that the audience for raw data is a limited one. Even though GIS data is one of the easier initial wins because of limited privacy concerns, this graph indicates that there may be more bang for your buck going with data that is more easily worked with by the general public.I would be really interested to see breakdowns from more populous cities to see whether this bears out. I suspect that business and finance data may be more interesting to people in general.
  • In the years leading up to 2010, the engineering department provided between 20 and 30 CDs of GIS information. Making the data available freely online more than at least tripled the number of times raw data is accessed in traditional formats, but what’s interesting is that a primarily visualization format (KML) totally cleans up over the GIS formats. Again, something that is accessible to non-specialists has a higher level of popularity.Apart from sheer numbers, a benefit of our open data catalogue is that it is open source. In fact, North Cowichan is already collaborating on the code and using this themselves, and we are discussing the possibility of an island-wide municipal open data catalogue with other IT departments on Vancouver Island.You can access this open source application, Open Data Publisher, if you’d like to use it yourself; we’re happy to help with configuration questions. And if there is additional functionality you’d like to add, we’re happy to add more people to the coding effort!
  • As far as web results go, fully half of all visits to our domain are direct hits to specific property report pages. This equals 5000 views in just ONE MONTH, four times the number of views that that any single dataset in our open data catalogue got in an entire year.
  • The cool thing is that the majority of these come from search terms that are not repeated in a given month. Looking at some of these “long tail” searches, it is clear that citizens are expecting Google to be able to answer questions that the government is authoritative for. If we can answer these questions in a single click, it’s a win for the citizen AND for the city. It will improve the citizen’s perception of how open we are, and reduce the number of calls and/or FOI requests we have to respond to.
  • It also has a rather nice side-effect for users of our on-site search. Information that previously would only have shown up if they had located the property search application is now inline with the other results in our Google Site Search implementation. Wouldn’t it be cool if staff could just perform a single search and it would display relevant records across all databases in the organization? There would obviously be some privacy concerns here, so this could only be done in-house using a system that honoured ACLs, but it sure is tempting. I may have to look into whether there is a valid business case for SharePoint data connectors or a Google Search Appliance…
  • In summary…
  • While I sincerely believe that we need to continue working on data downloads and exposing records to the web to meet highly technical and casual search access to information, I would also argue that we’re also about to get the carpet pulled out from under us again.Think about how Facebook works. You log onto the website and everything you see is about you. Your friends, your photos, the brands you have identified with. You may need to initially select what shows up, but once it’s there, it’s there every time you log in. Now think about how hard it is for a citizen to access all of the services they care about on your website.For instance, on Nanaimo’s website, users can sign up for emergency alerts, development permit tracking, bid opportunities, and many other services, but each time they come back the have to visit each of these applications individually. Once again, the way we provide information to our citizens and businesses is failing to measure up to current best practices.We need to provide a better experience the users who are still in the middle ground between having casual search answer all of their requirements but who don’t have the expertise to analyze the data themselves. We continue having to build _some_ applications, and we need to allow citizens to access the information they provide far more easily.
  • Last month, Nanaimo launched a new tool on its website that will make those applications we still need to build for the general public more useful and easier to find. Users can log in with any of a number of mechanisms (or they can still set up a local account). Once there, they will eventually be able to see information from all of the City of Naniamo online services they sign up for. We’ve just started out, so only a few services are currently covered, but we will soon be migrating the other personalized services on our site to this new system and adding more functionality.
  • If you have any questions, please feel free to contact me!
  • Open Data and Beyond. An Open Data Story from Nanaimo

    1. 1. An Open Data Story Or: Open Data and Beyond Jason Birch
    2. 2. Problem
    3. 3.,_Prime_Minister_episodes
    4. 4. “a new administration rode to power in the Vancouver Island city of Nanaimo on the familiar election promise of open government”“it was decided that the larger community should be invited to participate and tell us what information theyd like to make available.”
    5. 5. Approach
    6. 6. In the 21st century, data is at the heart ofeconomic activity; it is what drivesinnovation, efficiencies and productivity. - David Eaves
    7. 7. Findablilty precedes usability. In thealphabet and on the Web. You cant usewhat you cant find. - Peter Morville
    8. 8. Results
    9. 9. woo hoo!
    10. 10. What’s Next?
    11. 11. Thank You!