Open Data: From the Information Age to the Action Age (PDF with notes)


Published on

This is the presentation I made at the UK Department for International Aid/Omidyar Network OpenUp! conference in London on November 13, 2012. I talk about open government not as a platform for transparency or citizen engagement, but for a developer ecosystem building useful services. A video of this talk is available at

Published in: Technology

Open Data: From the Information Age to the Action Age (PDF with notes)

  1. Open Data: From the Information Age to the Action Age Tim O’Reilly O’Reilly Media OpenUp November 13, 2012Thursday, November 15, 12
  2. § something about transparency and gov dataThursday, November 15, 12When people talk about open government data, they often start by thinking about government transparency. Sites likemysociety’s TheyWorkForYou, or OpenCongress in the US, are indeed important.
  3. Thursday, November 15, 12In the US, this strain of the open government movement is perhaps best exemplified by the Sunlight Foundation.
  4. Thursday, November 15, 12Another mysociety project, fixmystreet, was among the first to highlight the possibility of opening up a two way channel, usingopen data to create a context for better citizen engagement.
  5. Thursday, November 15, 12Startups like SeeClickFix have sprung up to help citizens give feedback to civil servants about what needs doing.
  6. Thursday, November 15, 12What’s encouraging that there are now emerging standards like 311 that allow applications and the back-end systems thatgovernments use to manage their workflow to speak the same language. This is the beginning of a new way of thinking abouthow government data and the private sector can interoperate, creating new interfaces for government services.
  7. Thursday, November 15, 12In the US, Code for America is a leader in providing new kinds of engagement, working with local governments to build simple,beautiful and easy-to-use interfaces to government services and challenging government to reinvent the way it engages withcitizens. Jen Pahlka talked a bit about their work earlier this morning.
  8. Thursday, November 15, 12The UK government digital service has done a fantastic job of thinking about how to put citizens at the center of governmentrather than at the periphery. Their Design Principles are a manifesto for open government everywhere.
  9. Thursday, November 15, 12The Government Digital Service Design Principles are probably the most important design document since the Macintosh HumanInterface Guidelines of the 1980s. They, and the GDS Digital Strategy, should be the template for government services aroundthe world. I particularly like that they’ve embraced the notion of APIs, which allow third parties to build new services. But Iwould go further, and argue that APIs should be open to third parties by default, and closed only when necessary.
  10. § google home page / information ageThursday, November 15, 12But even the thinking behind GDS doesn’t give the full range of impact of open government data.I’ve always found myself wondering why people aren’t more aware of how government data powers non-governmental servicesthat citizens take for granted, many of them never taking the time to think how much government investment went into buildingthe infrastructure that makes it possible for the private sector to offer services like weather predictions
  11. GPS: A 21st century platform launched in 1973Thursday, November 15, 12Now consider Global positioning satellites. Here government investment in a hard, long term project, is paying off in uncountednew private sector developments.A huge project with uncertain return, started in 1973 and now showing enormous fruit in the 21st century, with huge value addfrom the commercial sector. Everything from maps and directions on your phone tofuture self-driving cars spring from this platform investment, and the key policy decision to open the data andmake it available for commercial use. No one dreamed of the unexpected applications that became possible by opening up thisdata. That’s why we need open web services by default.
  12. Thursday, November 15, 12Mapping services have taken data developed at great expense by government and turned it into hugely powerful experiences forcitizens. Here are my public transit directions from my hotel to this venue today, courtesy of Google, but also of centuries ofgovernment investment in mapping services. Not to mention that Google Transit directions got its start with the government ofPortland, Oregon, who in classic internet style, proposed a data specification for transit timetables that could be consumed bythird party applications.
  13. § Health datapaloozaThursday, November 15, 12This same kind of “developer outreach” has characterized Todd Park’s work on open data at HHS. Rather than just openingthe data, he proactively sought out partners who could consume government data. For example, HHS had a huge database ofhospital and nursing home quality and user satisfaction metrics, which now helps to power Bing’s hospital search, and not justthe HHS Hospital Compare application.The HHS open data initiative now features a thriving developer conference,hundreds of apps, and numerous funded startups.
  14. Thursday, November 15, 12The Obama administration is now trying to replicate this playbook in other areas like energy
  15. Thursday, November 15, 12and Education.
  16. What happens when you throw open the doors to partners More than 50,000 iPhone applications in less than a year! Now at 688,000Thursday, November 15, 12Apple showed us the power of this kind of entrepreneurial explosion when they turned the smartphone into a platform with theintroduction of the iPhone app store. We went from phones that had twenty or thirty apps cooked up in a back room deal to aplatform that allows anyone to come up with new features.This is the kind of platform I’d like to see for government.
  17. Government as a platform means an end to the design of only complete, closed “applications.” Instead the government should provide fundamental services on which we, the people, (also known as “the market”) build applications.Thursday, November 15, 12That’s why...
  18. Thursday, November 15, 12At Code for America, we’ve capitalized on this idea by launching a startup accelerator, finding and nurturing creative startupswho are re-using or opening up government data, creating new interfaces to government services, or building tools that willhelp governments to engage with their community.
  19. Thursday, November 15, 12Since I know this conference is focused on International Development, I want to mention one of these startups in particular.Captricity grew out of a project in Tanzania and Uganda to digitize medical records. It is a now a powerful tool combiningmachine learning and crowdsourcing to convert any kind of paper document into structured data.
  20. Thursday, November 15, 12While we’re on the subject of Africa, I want to remind all of you that technology for international development isn’t all digital.One of the most remarkable frontiers of technology today is in something we call the Maker Movement, which involveseverything from sensors and robots to advanced 3D printing, digital design and manufacturing, but also includes old fashionedmechanical engineering and ingenuity. At O’Reilly, we run an event in California that draws over 120,000 people. But one of ourmost exciting Maker Faires happens in Nigeria.
  21. Thursday, November 15, 12You can get a great sense of how “makers” are solving everyday problems with local ingenuity on the Afrigadget blog. Many ofthe people featured there also appear at Maker Faire.Sulaiman Famro is a cheerful, 65 year old engineer, and a master of branding. He built the prototype “Farmking” three years agoand claims he can save the country $1 billion a year, just in savings on starch importation.The Farmking is a one-stop processing plant for cluster and farm-site processing of root crops and grains. It has a dieselpowered engine that allows for remote processing, with power out connections for lighting so that it can work all night, ifneeded.On one end you have 3 devices, for chipping, grating and milling. In the middle is the power plant, and in the rear is a large steeldrum that can hold 50kgs of milled cassava, that uses a spin filter to process up to 2.5 tons of milled cassava into starch.It’s used for processing of cassava, soya beans, maize, sweet potatoes, yam and many other roots and grains.
  22. Thursday, November 15, 12In an odd way, there’s also a connection between the maker movement and hot silicon valley startups like AirBnB. Both aredeeply rooted in a trend towards decentralized activities and peer-to-peer markets. With maker marketplaces like Etsy, peoplearound the world can sell their goods to each other. With sharing marketplaces like airBnb, people can share their homes.Government should support these peer economies, and not get in the way with too much regulation. At the OpeningGovernment meeting the other day, we had a fascinating conversation about the interplay between regulation and reputation,and how reputation systems can replace some of what we now do with centralized regulation.
  23. Thursday, November 15, 12I also want to give a shout out to efforts like the UN Global Pulse, as well as Ushahidi, the Global Virus Forecasting Initiative, andother groups that are using new approaches to big data and predictive analytics to literally build a new early warning system forthe planet.Data is the currency of most modern apps, and particularly important is data that helps you to “close the loop.”
  24. “What I learned from Google is to only invest in things that close the loop.” - Chris SaccaThursday, November 15, 12Efforts like this are completely aligned with the “big data” focus of Silicon Valley companies like Google.Investor Chris Sacca, who used to run special projects for Google, once remarked “What I learned...”
  25. “Perl should make easy things easy, and hard things possible.” - Larry WallThursday, November 15, 12I want to end with a few further thoughts about the role of government in sparking innovation in the private sector. Larry Wall,the creator of the Perl programming language, once said...
  26. “Government should make easy things easy, and hard things possible.” -Tim O’ReillyThursday, November 15, 12Since he’s an open source developer, I’m going to take his maxim, and as we say in the open source world, “fork it” so it nowreads... I was with the UK Government Digital Service the other day, and we did a Q&A with reporters. Someone from the WallStreet Journal asked if the new approach was just about saving money. We answered that it wasn’t at all, that it was aboutcreating new kinds of engagement between government and citizens. Saving money is a byproduct. But there’s another way inwhich the cost savings of open data and government doing less by enabling the rest of us to do more is really important.
  27. Critical Issues Facing the World • Natural Disasters • Jobs • Energy • Food • Water • Creating a vibrant, just, and bountiful economy for all the world’s citizensThursday, November 15, 12 I suspect that in the next few decades we’ll have many, many hard problems to solve - wicked problems even. One of the great urgencies of the open government movement is to make easy things easy so that we can make hard things possible. The private sector can do a lot, but there are some problems of a scale that we need to tackle them all together.
  28. “The legitimate object of government is to do for the people what needs to be done, but which they cannot, by individual effort, do at all, or do so well, for themselves.” -Abraham Lincoln, July 1, 1854Thursday, November 15, 12 Or as Abraham Lincoln put it...