The Closed World Of Company Data

7,329 views

Published on

This presentation was first given at the World Bank, April 25, 2012. A version was also given at Transparency Camp 2012. The World Bank presentation was also webcast and a recording is available at: A recording of the webcast of the World Bank presentation is at http://bit.ly/ocdw

Published in: News & Politics, Business

The Closed World Of Company Data

  1. The closed world ofcompany data And why we need to open it up
  2. We live in a corporate world ...but much of it is invisible to us (citizens, journalists, NGOs, regulators, governments, SMEs, etc...)
  3. Companies nolonger look like this
  4. Companies nolonger look like this
  5. Companies nolonger look like this nor even like this
  6. Butlikethis
  7. Butlikethis or like this
  8. Butlikethis or like this
  9. And they are more complexby the day
  10. And they are more complexby the dayGrowing in scale – not 10s of legal entities but 1000s
  11. And they are more complexby the dayGrowing in scale – not 10s of legal entities but 1000sGrowing in speed – we are seeing the beginnings ofhigh-frequency company formation
  12. And they are more complexby the dayGrowing in scale – not 10s of legal entities but 1000sGrowing in speed – we are seeing the beginnings ofhigh-frequency company formationGrowing in opacity – use of secrecy jurisdictions andoff-register entities to provide firewalls to tax, regulation,information
  13. And they are more complexby the dayGrowing in scale – not 10s of legal entities but 1000sGrowing in speed – we are seeing the beginnings ofhigh-frequency company formationGrowing in opacity – use of secrecy jurisdictions andoff-register entities to provide firewalls to tax, regulation,informationGrowing in complexity – not a hierarchy but a complex,sometimes even circular network of entities
  14. Getting the data matters In the 21st century, data is power We’ve always been governed by data, now our lives are data Huge asymmetry of access to public data Sold and collected to enrich global proprietary databases, denied to citizens
  15. Information is the currencyof democracy Thomas Jefferson
  16. ATA is the currencyInformation Dof democracy Thomas Jefferson
  17. This mattersNo understanding = no control
  18. This mattersNo understanding = no control Leads to systematic problems – Lehman’s, pollution exporting, market failures, etc
  19. This mattersNo understanding = no control Leads to systematic problems – Lehman’s, pollution exporting, market failures, etc Reduces accountability and corporate governance. Enables and encourages companies to behave like bad corporate citizens
  20. This mattersNo understanding = no control Leads to systematic problems – Lehman’s, pollution exporting, market failures, etc Reduces accountability and corporate governance. Enables and encourages companies to behave like bad corporate citizens Enables of money laundering, organised crime and corruption (see World Bank Puppet Masters report)
  21. This mattersNo understanding = no control Leads to systematic problems – Lehman’s, pollution exporting, market failures, etc Reduces accountability and corporate governance. Enables and encourages companies to behave like bad corporate citizens Enables of money laundering, organised crime and corruption (see World Bank Puppet Masters report) Remember, companies are artificial entities given legal personality by the state for the good of society
  22. ...as recognised by the OpenGovernment Partnership5 Grand Goals1.Improving Public Services2.Increasing Public Integrity3.More Effectively Managing Public Resources4.Creating Safer Communities5.Increasing Corporate Accountability 
  23. So how do the OGPcountries score for access to company data?
  24. So how do the OGPcountries score for access FA to company IL! data?
  25. 4 key measures
  26. 4 key measuresBasic search: can you search the company registerfreely, without charge and without registrationLicence: Is there a licence that allows open reuse ofthe informationData: Is the information available as open data as adata dump or an APIDepth: Is there sufficient information to get a truepicture of the company and those who control it –directors, significant shareholdings, and statutory filings
  27. The results were not good...
  28. The results were not good...
  29. For US we took a straight average of thestate registers (possibly overstates access)
  30. On corporate confidentiality& competitive advantage
  31. On corporate confidentiality& competitive advantage No good reason why a corporate hierarchy should not be public
  32. On corporate confidentiality& competitive advantage No good reason why a corporate hierarchy should not be public Competitive advantage should be about new products and services, innovation, risking capital, not devising complex corporate networks that encourage companies to evade regulation, tax, scrutiny
  33. On corporate confidentiality& competitive advantage No good reason why a corporate hierarchy should not be public Competitive advantage should be about new products and services, innovation, risking capital, not devising complex corporate networks that encourage companies to evade regulation, tax, scrutiny Disproportionally benefits big incumbents, thus stifling competition and innovation
  34. On corporate confidentiality& competitive advantage No good reason why a corporate hierarchy should not be public Competitive advantage should be about new products and services, innovation, risking capital, not devising complex corporate networks that encourage companies to evade regulation, tax, scrutiny Disproportionally benefits big incumbents, thus stifling competition and innovation Disadvantages those companies that want to be good corporate citizens, forcing a race to the bottom
  35. What is OpenCorporates?A simple (huge)goal: build anopenly licenseddatabase withan entry (andURI) for everycorporate legalentity in theworld
  36. What is OpenCorporates?A simple (huge)goal: build anopenly licenseddatabase withan entry (andURI) for every dict ionscorporate legal n 52 juris atesentity in the anies i 22 US stworld n co mp clud ing 0m illio In w ov er 4No
  37. 5 core uses
  38. 1. An open identifying system
  39. 1. An open identifying system URIs can be used as common identifiers among a variety of organisations Can be used without reference to OpenCorporates Because they map to the id issued by the company register the corresponding entry in the registry (and associated info) can be found, and vice versa Fits the new W3c/EU Business Vocabulary Can even by used for companies in jurisdiction we haven’t yet imported
  40. 2. The simple search
  41. 2. The simple searchNot to be underestimated
  42. 2. The simple searchNot to be underestimated
  43. 2. The simple searchNot to be underestimatedMassively reduces friction(how long will it take youto find and searchmultiple jurisdictions)
  44. 2. The simple searchNot to be underestimatedMassively reduces friction(how long will it take youto find and searchmultiple jurisdictions)
  45. 2. The simple searchNot to be underestimatedMassively reduces friction(how long will it take youto find and searchmultiple jurisdictions)Allows what if questions
  46. 2. The simple searchNot to be underestimatedMassively reduces friction(how long will it take youto find and searchmultiple jurisdictions)Allows what if questionsPotentially generatesstories in its own right
  47. 2. The simple searchNot to be underestimatedMassively reduces friction(how long will it take youto find and searchmultiple jurisdictions)Allows what if questionsPotentially generatesstories in its own right
  48. 3. Source for additional info
  49. 3. Source for additional info Addresses, filings, status, websites...
  50. 3. Source for additional info Addresses, filings, status, websites...
  51. 3. Source for additional info Addresses, filings, status, websites... Intl trademarks, UK govt spending, official notices, health & safety violations...
  52. 3. Source for additional info Addresses, filings, status, websites... Intl trademarks, UK govt spending, official notices, health & safety violations...
  53. 3. Source for additional info Addresses, filings, status, websites... Intl trademarks, UK govt spending, official notices, health & safety violations... Other IDs: SEC, CAGE, etc – allows reverse mapping queries, e.g. show me legal entity mapped to a CIK code
  54. 4. Reconciliation(matching names to legalClean up messycompany names(& prev names)to legal entity,and from thereto other dataGoogle Refinereconciliationservice (specificto jurisdiction)
  55. 5. The platform API: allows all information to be retrieved as data, even searches Users can now add data too Coming soon: the option to match data to companies
  56. 5. The platform API: allows all information to be retrieved as data, even searches Users can now add data too Coming soon: the option to match data to companies
  57. 5. The platform API: allows all information to be retrieved as data, even searches Users can now add data too Coming soon: the option to match data to companies
  58. 5. The platform API: allows all information to be retrieved as data, even searches Users can now add data too Coming soon: the option to match data to companies
  59. 5. The platform API: allows all information to be retrieved as data, even searches Users can now add data too Coming soon: the option to match data to companies
  60. How have we done it?
  61. How have we done it?Co-operation – we get data direct from some companyregisters (UK, NZ, a few US), and are working withinternational institutions (EC, W3c, Financial StabilityBoard, etc) to improve visibility and reuse of company info
  62. How have we done it?Co-operation – we get data direct from some companyregisters (UK, NZ, a few US), and are working withinternational institutions (EC, W3c, Financial StabilityBoard, etc) to improve visibility and reuse of company infoCommunity – a lot of the data has been contributed bythe open data community (thanks, ScraperWiki)
  63. How have we done it?Co-operation – we get data direct from some companyregisters (UK, NZ, a few US), and are working withinternational institutions (EC, W3c, Financial StabilityBoard, etc) to improve visibility and reuse of company infoCommunity – a lot of the data has been contributed bythe open data community (thanks, ScraperWiki)Cool open-source software (100% open sourceplatform/tools)
  64. How have we done it?Co-operation – we get data direct from some companyregisters (UK, NZ, a few US), and are working withinternational institutions (EC, W3c, Financial StabilityBoard, etc) to improve visibility and reuse of company infoCommunity – a lot of the data has been contributed bythe open data community (thanks, ScraperWiki)Cool open-source software (100% open sourceplatform/tools)Colossal scraping (100,000s of pages/API calls per day)
  65. Problems (& solutions)
  66. Problems (& solutions) Company registers consider themselves businesses, not public registers – sometimes block access
  67. Problems (& solutions) Company registers consider themselves businesses, not public registers – sometimes block access Slow, poorly designed company register websites (and sometimes they don’t even exist – and not just in developing countries)
  68. Problems (& solutions) Company registers consider themselves businesses, not public registers – sometimes block access Slow, poorly designed company register websites (and sometimes they don’t even exist – and not just in developing countries) Understanding global data
  69. Problems (& solutions) Company registers consider themselves businesses, not public registers – sometimes block access Slow, poorly designed company register websites (and sometimes they don’t even exist – and not just in developing countries) Understanding global data International/national jurisdictions
  70. Problems (& solutions) Company registers consider themselves businesses, not public registers – sometimes block access Slow, poorly designed company register websites (and sometimes they don’t even exist – and not just in developing countries) Understanding global data International/national jurisdictions Big-data problems – ETL, scaling, etc
  71. Problems (& solutions) Company registers consider themselves businesses, not public registers – sometimes block access Slow, poorly designed company register websites h elp to (and ow ant in developing countries) le op wh sometimes they don’t even exist – and not just pe ing global dataF nd iUnderstanding International/national jurisdictions Big-data problems – ETL, scaling, etc
  72. What next?
  73. What next?Recently started adding company directors and officers
  74. What next?Recently started adding company directors and officersMore public data – political donations, lobbyists, otherID systems
  75. What next?Recently started adding company directors and officersMore public data – political donations, lobbyists, otherID systemsRelationships between corporate entities
  76. What next?Recently started adding company directors and officersMore public data – political donations, lobbyists, otherID systemsRelationships between corporate entitiesMore options for community to add/curate data

×