Open Data
                                 Nat Torkington
                              ALGIM Web Symposium
                                   May 2010




Going to start by looking at some thought-provoking projects, and then I’ll talk about the
thoughts they provoke.
Wellibus
Routes
Select
$$$?



You might ask how much this cost MetLink. iPhone developers are hot property, the skills
aren’t common, apps have to be reviewed by Apple, ....
Nick Parfene
It cost nothing. This guy wrote it. He’s a Wellingtonian, his girlfriend had been nagging him
to give her the bus schedule on her mobile device.
Public Transit Data



He did it using the Metlink Google Transit data feed. Metlink offer the data to Google so they
can appear in Google’s maps. Nick piggybacked off that.
Actively used



He has over 500 users every day.
Projects
California had a $26B budget gap in 2009, cities lost redevelopment money, Santa Cruz had
$9.2M deficit. Started a website to engage public. Wasn’t a blog, used a company
“Uservoice” whose tools promoted productive discussion and minimised unproductive.
Published 10 years of budgets. Had accountants, business people, citizens making
suggestions. They were able to make thoughtful informed cuts and optimisations.
Boston non-profit in partnership with the city, running social networks for neighbourhoods.
Started with one neighbourhood, expanded to 18.
Connect to neighbours, crime reports, common interests, chat, volunteerism, events.
A legion of systems to report problems (out lights, downed lines, flooded streets, buggered
roads, etc.).
It’s Buggered Mate. See Click Fix. FixMyStreet. Hundreds of these.
District of Columbia released a lot of data, then ran a contest. $2.3M of software creation for
$50k project (grand prize won $10k). Winner was an iPhone and Facebook app combo for
doing 311 “fix my street” style requests. Many cities following this course.
Cities are releasing data, and lots of it, in reusable formats. They hope to make businesses
more efficient, citizens better directed, and services cheaper.
I know what you’re
                    thinking


You’re thinking about your own web sites.
Local Govt Web Sites
You’re delivering information in PDF files, your web site holds PDF forms that must be mailed in, even the digital databases you maintain don’t have online
interfaces.
And I know why.
I’m talking science fiction dreams, which would take megabucks to realise,
But this is what they give you. Every year you look at your budgets and
it’s not good.
Chillax



Don’t panic, as Douglas Adams would have said. I’m not asking the impossible.
As Krishna was told by Arjuna, "a man must go forth from where he stands. He cannot jump to the Absolute, he must evolve toward it"
Let’s boil it down.
Gov 2.0



Gov 2.0, “e-democracy”, or whatever the hell it’s being called today.
Last 20 years of computers
It’s just looking at the amazing successes of the last 20 years of computing
We can do better
and applying them to government.
What’s worked?
Build open, extensible
                              systems


IBM PC took off because everyone could build compatible hardware
Web took off because anyone could use code and build their own website, and they interoperated

APIs and open data
Design for cooperation



Small systems loosely joined
DNS is federated, not centralised
Learn from your users



Google maps
(used by 45% of all online mashups)
hired the first guy to make a mashup

fedspending.org used to be run by OMBwatch
Lower the barriers to
                          experimentation



Failure should be an option.
Edison: “I didnʼt fail ten thousand times. I successfully eliminated, ten thousand times, materials and combinations that did not work.”

Much innovation comes from a single engineer within an entity like the New York Times, putting archives up on an inexpensive, rented server from Amazon. The
low cost of failure made it easier to experiment.
Build a culture of
                                  measurement



Amazon driven by numbers
Need good metrics!

You are what you measure. Canʼt change what you donʼt measure.
Build a community



You want this to succeed.
You also want to get credit.
(“Why do we need NOAA when we have weather.com?”)
Invisible Walls
                        (aka Planned
                        Interactions)


Wikipedia succeeds because of what you don’t see
Build simple systems
                           and let them evolve


Twitterʼs original design doc was 1/2 a page of paper, and there are now 11,000 applications built on top of it (written by third parties).
The hourglass model: run on many systems, support many applications, but connected by a common protocol.
“Complex systems built from scratch never work. You need to build a simple system and let it grow… Complex problems paradoxically require simple answers.”
Open Data



The easiest step for governments to take is opening data. This means
1. Machine readable
2. Reusable
(“Creative Commons”)
3. Self-service
4. Open format
0. Useful!



Temptation is always to release the easy stuff. Work backwards from the issue to figure out
what data would be most useful in the hands of citizens?
Open Data:
                         Necessary
                          Sufficient


why are we doing this?
Action



                                         Insight



                                          Data




Get the data in front of the right people.
Over the fence isn’t enough. It’s been tried, and then everyone scrambles to find users.
Community



You need to build a community of users.
fortunately you’re good at community. you already have one.
Don’t cargo-cult



Don’t use technology because it’s trendy, it ticks a box, or someone else used it.
Know what you’re hoping to accomplish, use the right technology with the right social
measures around it. What made Wikipedia great? Same invisible walls in the app that Santa
Cruz used.
Contests



Contests draw your community out of the woodwork. What if you don’t have a lot of
developers in your community?
Do the work to make
             useful data available


Contests often used to try to excite people about irrelevant data. Release data people are
passionate about. This probably means a fight.
Standards let you
                 piggyback on the
                world’s developers


Google Transit feed
Open311 project
It doesn’t end with data
Vision



So, here’s my vision that I’d like you to share.
Integrated workflow



Every process in your council is online, and has a web/mobile/smartphone/Babelfish
mindreader interface. Citizens can interact with the council conveniently, and efficiently.
“Build web services, not
    web sites.” —Rod Drury


this leads to Rod Drury’s suggestion
Takeaways
1) PDF isn’t enough
2) Web maps aren’t
     enough
3) Release open data
4) Connect with
citizens over data
5) Cultivate local
   developers
6) Built web services
    not web APIs
7) Use standards
Redux
           • PDF is not enough
           • Web maps are not enough
           • Release open data
           • Connect with citizens over the data
           • Cultivate local developers
           • Build web services not APIs
           • Use standards
With these tips, I hope you’ll go back to your councils and be thinking of the intelligent life
outside the org chart--who can you bring in to build a community of data users and app
developers so you can connect data to insight to action. If you have questions or need help,
Thank You
                          nathan@torkington.com
                        http://nathan.torkington.com
                           http://twitter.com/gnat




here’s my contact information. I’ll take questions if we have time.
2010 ALGIM Gov 2.0

2010 ALGIM Gov 2.0

  • 1.
    Open Data Nat Torkington ALGIM Web Symposium May 2010 Going to start by looking at some thought-provoking projects, and then I’ll talk about the thoughts they provoke.
  • 2.
  • 3.
  • 4.
  • 5.
    $$$? You might askhow much this cost MetLink. iPhone developers are hot property, the skills aren’t common, apps have to be reviewed by Apple, ....
  • 6.
    Nick Parfene It costnothing. This guy wrote it. He’s a Wellingtonian, his girlfriend had been nagging him to give her the bus schedule on her mobile device.
  • 7.
    Public Transit Data Hedid it using the Metlink Google Transit data feed. Metlink offer the data to Google so they can appear in Google’s maps. Nick piggybacked off that.
  • 8.
    Actively used He hasover 500 users every day.
  • 9.
  • 10.
    California had a$26B budget gap in 2009, cities lost redevelopment money, Santa Cruz had $9.2M deficit. Started a website to engage public. Wasn’t a blog, used a company “Uservoice” whose tools promoted productive discussion and minimised unproductive. Published 10 years of budgets. Had accountants, business people, citizens making suggestions. They were able to make thoughtful informed cuts and optimisations.
  • 11.
    Boston non-profit inpartnership with the city, running social networks for neighbourhoods. Started with one neighbourhood, expanded to 18. Connect to neighbours, crime reports, common interests, chat, volunteerism, events.
  • 12.
    A legion ofsystems to report problems (out lights, downed lines, flooded streets, buggered roads, etc.). It’s Buggered Mate. See Click Fix. FixMyStreet. Hundreds of these.
  • 13.
    District of Columbiareleased a lot of data, then ran a contest. $2.3M of software creation for $50k project (grand prize won $10k). Winner was an iPhone and Facebook app combo for doing 311 “fix my street” style requests. Many cities following this course.
  • 14.
    Cities are releasingdata, and lots of it, in reusable formats. They hope to make businesses more efficient, citizens better directed, and services cheaper.
  • 15.
    I know whatyou’re thinking You’re thinking about your own web sites.
  • 16.
    Local Govt WebSites You’re delivering information in PDF files, your web site holds PDF forms that must be mailed in, even the digital databases you maintain don’t have online interfaces. And I know why.
  • 17.
    I’m talking sciencefiction dreams, which would take megabucks to realise,
  • 18.
    But this iswhat they give you. Every year you look at your budgets and
  • 19.
  • 20.
    Chillax Don’t panic, asDouglas Adams would have said. I’m not asking the impossible. As Krishna was told by Arjuna, "a man must go forth from where he stands. He cannot jump to the Absolute, he must evolve toward it" Let’s boil it down.
  • 21.
    Gov 2.0 Gov 2.0,“e-democracy”, or whatever the hell it’s being called today.
  • 22.
    Last 20 yearsof computers It’s just looking at the amazing successes of the last 20 years of computing
  • 23.
    We can dobetter and applying them to government.
  • 24.
  • 25.
    Build open, extensible systems IBM PC took off because everyone could build compatible hardware Web took off because anyone could use code and build their own website, and they interoperated APIs and open data
  • 26.
    Design for cooperation Smallsystems loosely joined DNS is federated, not centralised
  • 27.
    Learn from yourusers Google maps (used by 45% of all online mashups) hired the first guy to make a mashup fedspending.org used to be run by OMBwatch
  • 28.
    Lower the barriersto experimentation Failure should be an option. Edison: “I didnʼt fail ten thousand times. I successfully eliminated, ten thousand times, materials and combinations that did not work.” Much innovation comes from a single engineer within an entity like the New York Times, putting archives up on an inexpensive, rented server from Amazon. The low cost of failure made it easier to experiment.
  • 29.
    Build a cultureof measurement Amazon driven by numbers Need good metrics! You are what you measure. Canʼt change what you donʼt measure.
  • 30.
    Build a community Youwant this to succeed. You also want to get credit. (“Why do we need NOAA when we have weather.com?”)
  • 31.
    Invisible Walls (aka Planned Interactions) Wikipedia succeeds because of what you don’t see
  • 32.
    Build simple systems and let them evolve Twitterʼs original design doc was 1/2 a page of paper, and there are now 11,000 applications built on top of it (written by third parties). The hourglass model: run on many systems, support many applications, but connected by a common protocol. “Complex systems built from scratch never work. You need to build a simple system and let it grow… Complex problems paradoxically require simple answers.”
  • 33.
    Open Data The easieststep for governments to take is opening data. This means
  • 34.
  • 35.
  • 36.
  • 37.
  • 38.
  • 39.
    0. Useful! Temptation isalways to release the easy stuff. Work backwards from the issue to figure out what data would be most useful in the hands of citizens?
  • 40.
    Open Data: Necessary Sufficient why are we doing this?
  • 41.
    Action Insight Data Get the data in front of the right people. Over the fence isn’t enough. It’s been tried, and then everyone scrambles to find users.
  • 42.
    Community You need tobuild a community of users. fortunately you’re good at community. you already have one.
  • 43.
    Don’t cargo-cult Don’t usetechnology because it’s trendy, it ticks a box, or someone else used it. Know what you’re hoping to accomplish, use the right technology with the right social measures around it. What made Wikipedia great? Same invisible walls in the app that Santa Cruz used.
  • 44.
    Contests Contests draw yourcommunity out of the woodwork. What if you don’t have a lot of developers in your community?
  • 45.
    Do the workto make useful data available Contests often used to try to excite people about irrelevant data. Release data people are passionate about. This probably means a fight.
  • 46.
    Standards let you piggyback on the world’s developers Google Transit feed Open311 project
  • 47.
  • 48.
    Vision So, here’s myvision that I’d like you to share.
  • 49.
    Integrated workflow Every processin your council is online, and has a web/mobile/smartphone/Babelfish mindreader interface. Citizens can interact with the council conveniently, and efficiently.
  • 50.
    “Build web services,not web sites.” —Rod Drury this leads to Rod Drury’s suggestion
  • 51.
  • 52.
  • 53.
    2) Web mapsaren’t enough
  • 54.
  • 55.
  • 56.
  • 57.
    6) Built webservices not web APIs
  • 58.
  • 59.
    Redux • PDF is not enough • Web maps are not enough • Release open data • Connect with citizens over the data • Cultivate local developers • Build web services not APIs • Use standards With these tips, I hope you’ll go back to your councils and be thinking of the intelligent life outside the org chart--who can you bring in to build a community of data users and app developers so you can connect data to insight to action. If you have questions or need help,
  • 60.
    Thank You nathan@torkington.com http://nathan.torkington.com http://twitter.com/gnat here’s my contact information. I’ll take questions if we have time.