1. Big data analytics can improve government efficiency and effectiveness across areas like crime prediction and prevention, healthcare, and social programs if privacy issues are addressed. 2. Open data refers to structured data that is publicly accessible online, and governments are increasingly embracing open data to increase transparency and engage citizens. 3. However, privacy remains a major challenge, as seen with Canada's Bill C-51, which aims to increase information sharing between agencies but has alarmed privacy advocates due to its broad scope and lack of oversight. Addressing these issues is key to realizing big data's benefits.

1. +
Big Data & Government
Harnessing the Power of
Open Data
Sylvia Ogweng

2. +
Summary
1. Overview of Benefits
2. Benefits by Sector
3. Privacy Matters
4. Open Data
5. Big Data and Government in Action
6. Future of Big Data & Government

3. +
Big Data & Government
n Big data analytics can improve efficiency and effectiveness
across the broad range of government responsibilities, by
improving existing processes and operations and enabling
completely new ones.
n Information that is gathered from the public using public funds
should remain publicly accessible, regardless of government
decisions to delegate its management.
n Slow process. Governments are moving past fears of releasing
data. Not asking WHY they should publish, instead WHY NOT?
n Many are still waiting for a solid business case to introduce open
data. By nature, open data does not guarantee anything specific.

4. +
Overview of Benefits
n Increased transparency
n Engagement with citizens
n Enhancement of services to citizens by improving coordination among
agencies, and between internal and external stakeholders.
n Economic rewards: encourages civic-minded developers to build products
and businesses around the data – pumping sought-after jobs and dollars into
the economy.
n Fosters collaboration
n Improved measurement of policies
n Increased government efficiency
n Deeper analytical insights

5. +
Shortcomings
n Highly imperfect reports – most government economic indicators
are based on surveys
n Cost
n Access Limitations
n Anonymization and deidentification
n Granularity

6. +
Benefits by Sector
Major Sectors Include:
n Crime prediction and prevention
n Threat prediction and prevention
n Social program fraud
n Tax compliance
n Policy development
n Traffic solutions
n Healthcare

7. +
Crime Prediction &
Prevention
Anne Milgram: Former attorney general of
the state of New Jersey and then became an
attorney at the United States Department of
Justice
n “I wanted to understand who we were
arresting, who we were charging, and who
we were putting in our nation's jails and
prisons. I also wanted to understand if we
were making decisions in a way that made
us safer. And I couldn't get this information
out”

8. +
Crime Prediction &
Prevention
n Decisions based on experience and instinct across entire Justice System
n Introduced smart statistics and data into police work
n Reduced murders by 41%
n All crimes by 26%
n Changed focus to violent crimes/gangs/political corruption
• 75 B per year on state and local
corrections costs.
• 2.3 million people in jails and prisons
• 2/3 waiting for a trial
• 7 out of 10 ppl will be rearrested

9. +
Crime Prediction &
Prevention
n Team: data scientists, statisticians, researches
n Built a predictive risk assessment tool
n 1.5M cases
n 900 risk factors
n 9 most predictive of risk
n Scalable/transferable/universal
n Predict:
n Will they commit a new crime
n Will they commit an act of violence
n Will they come back to court

10. +
Crime Prediction &
Prevention
n Deeper understanding of persons of interest
n Crime and incident patterns
n Location-based threats
n Improve case clearance rates
n Stronger inter-and intra-agency collaboration
n Faster, better informed response

11. +
Threat Prediction &
Prevention
n More reliable understanding of a target or area of interest
n Finding relevant dots and connecting them
n Helping analysts to focus on what matter, understand what they
don’t know and use their time more effectively
n Improve collaboration and ensure faster response.

12. +
Threat Prediction &
Prevention
n Past doesn’t always dictate the future
n Insufficient data to predict a threat (difficult to train the model)
n Noise
n Legal / moral implications

13. +
Threat Prediction &
Prevention

14. +
Threat Prediction &
Prevention

15. +
Threat Prediction &
Prevention

16. +
Social Program Fraud,Waste &
Errors
• Reduce social benefit overpayments
• Proactively detected & deterred fraud and abuse,
• Reduced analysis time and improved efficiency,
• Improved program integrity
• Preservation of limited budgets for eligible citizens.

17. +
Healthcare
n Gain a more timely, more complete understanding of the
patient’s current situation so that they can prescribe the
appropriate and most effective treatments.
n Increased utilization of smart devices
n Creative use of image and audio technology to predict changes
in health
n Model patient treatment effectiveness, predictive patient re-
admission analytics, and the creation of patient health scores
n Complex event processing to monitor, analyze, and flag
potential health issues on a daily or on-demand basis

18. +
Other Sectors
n Tax Compliance
n Policy development
n Traffic solutions

19. +
Privacy Matters
n Privacy: a major hurdle in the way the government collects data
n Major issues:
n Anonymization
n Profiling
n Unethical actions
n Discrimination
n Best Practices:
n Become highly competent in procuring and managing cloud services.
n Make better use of converged data with converged storage.
n Properly sanitize data
n Be transparent and open. Encourage and invite a process for consumers to
access, review and correct information that has been collected about
them, much like credit reporting agencies do.

20. +
Privacy Matters: Bill c-51
About Bill c-51:
n Harper government’s proposed Anti-Terrorism Act
n “An Act to enact the Security of Canada Information Sharing Act and the
Secure Air Travel Act, to amend the Criminal Code, the Canadian
Security Intelligence Service Act and the Immigration and Refugee
Protection Act and to make related and consequential amendments to
other Acts”
n A new information sharing law aiming to relax constraints on the flow of
information between government agencies about activities that
undermine the security of Canada.

21. +
Privacy Matters: Bill c-51
More About Bill c-51:
n Addresses terror suspect detaining, no fly list, terrorism “peace
bonds,” under which police will be allowed to limit the liberty of
someone if they have reason to believe he or she might be about to
commit a terrorism-related crime.
n The government promises its conduct will be subject to review by
the Privacy Commissioner. In 2014, the Privacy Commissioner
issued a report that recognized that the Privacy Act had not been
substantially amended since the 1980s and that the Commissioner
required more powers to ensure integrated review of secret
information.Those amendments are not included in Bill C-51.

22. +
Privacy Matters: Bill c-51
Some responses to the bill:
n “Bill C-51 Moves Us One Step Closer to the End of Privacy”
n “Without Big Changes, Bill c-51 Means Big Data”
n “Bill c-51 – The Good,The Bad and the Truly Ugly”

23. +
Privacy Matters: Bill c-51
Why is Bill c-51 alarming?
n The scale of information-sharing between government departments and
agencies proposed in this bill is unprecedented.
n Bill C-51 opens the door to collecting, analyzing and potentially keeping
forever the personal information of all Canadians in order to find the virtual
needle in the haystack.
n The legislation would allow for Canadians’ personal information to be
shared if it is deemed “relevant” to the detection of new security threats.
n In this way, the bill would provide 17 federal government agencies with
almost limitless powers to monitor and profile ordinary Canadians, with a
view to identifying security threats among them.
n Very broad definitions

24. +
Privacy Matters: Bill c-51
Why is Bill c-51 alarming?
n People may not even know that secret information has been shared.
n Privacy Commission Daniel Therrien says:The new powers of the 17
federal agencies to exchange personal and confidential information
in the name of the war on terror are “excessive” and could affect
ordinary Canadians.

25. +
Privacy Matters: Bill c-51
How to Improve Bill c-51?
n Clear and reasonable standards need to be established as to what
personal information may be collected, shared, used and kept.
n Ensure appropriate oversight and review.
n Subject the 17 agencies to independent review.
n Judicial recourse should be available to those who believe personal
information has been collected, used, disclosed or retained
improperly.

26. +
Open Data
n Open data: structured data that is machine-readable, freely shared,
used and built without restrictions
n For governments, simply making data available to the public isn’t
enough to make that data useful. Open data, requires a specific
approach based on:
n the agency or organization releasing it,
n the kind of data being released, and most importantly,
n targeted audience
n How does it differ from Big Data? Focus on the idea that public
information should be accessible to the public online

27. +
Open Data
Crucial factors of open data (Canadian Gov’t):
n Availability and Access: data must be available as a whole at no more
than a reasonable reproduction cost. Data must be available in a
convenient and modifiable form.
n Re-use and Redistribution: data must be provided under terms that
permit re-use and redistribution including the intermixing with other
datasets.
n Universal participation: everyone must be able to use, re-use and
redistribute. No discrimination against fields of endeavor or against
persons or groups. (ex:‘non-commercial’ restrictions that would prevent
‘commercial’ use, or restrictions of certain purposes are not allowed)
Essentially, data must be:
n Technically open: data is easily accessible to its intended audience
n Legally open: data allows for universal participation

28. +
Driving Support for Open Data
n Difficult to get support from politicians and policy-makers to care
about open data.
n Drive support and growth for open data by:
n Avoiding the technical jargon
n Proving open data cuts work instead of creating it
n Following city procurement plans so pitches don’t duplicate
efforts
n Hackathons – Canadian Open Data Experience, Govcode.org
n Explain benefits, what it achieves:
n Jobs
n Transparency
n Open government
n Citizen engagement
n Data-driven decisions
n Informed public

29. +
Open Data: File Formats
Best file formats:
n JSON
n XML
n RDF (not widespread but now a trend among open government
initiatives)
n HTML (if data is limited scope)
Cumbersome File Formats:
n Spreadsheets – macros & formulas
n CSV – useless without documentation since it can be almost
impossible to guess the significance of the different columns
n Text documents – lack of support to the structure, difficult to
enter data automatically
n Plain text
n Scanned Image (for information that wasn’t born electronically)

30. +
Open Data Policy Guidelines
Determine:
n What data should be public?
n How to make data public?
n How to implement a data policy?
n 31-step guide created by The Sunlight Foundation
n The Sunlight Foundation is a nonpartisan non-profit that advocates
for open government globally and uses technology to make
government more accountable to all. Based in Washington, DC.

31. +
What Data Should Be Public?
1. Proactively release government information online
2. Reference and build on existing public accountability and access
policies
3. Build on the values, goals and mission of the community and
government
4. Create a public, comprehensive list of all information holdings
5. Specify methods of determining the prioritization of data release
6. Stipulate that provisions apply to contractors or quasi-governmental
agencies
7. Appropriately safeguard sensitive information

32. +
How To Make Data Public?
8. Mandate data formats for maximal technical access.
9. Provide comprehensive and appropriate formats for varied uses
10. Remove restrictions for accessing information
11. Mandate data be explicitly license-free
12. Charge data-creating agencies with recommending an appropriate
citation form
13. Require publishing metadata
14. Require publishing data creation processes
15. Mandate the use of unique identifiers
16. Require code sharing or publishing open source

33. +
How To Make Data Public?
17. Require digitization and distribution of archival materials
18. Create a central location devoted to data publication and
policy
19. Publish bulk data
20. Create public APIs for accessing information
21. Optimize methods of data collection
22. Mandate ongoing data publication and updates
23. Create permanent, lasting access to data

34. +
How To Implement A Data
Policy?
24. Create or appoint oversight authority
25. Create guidance or other binding regulations for
implementation
26. Incorporate public perspectives into policy implementation
27. Set appropriately ambitious timelines for implementation
28. Create processes to ensure data quality
29. Ensure sufficient funding for implementation
30. Create or explore potential partnership
31. Mandate future review for potential changes to this policy

35. +
Open Data In Action
n Canada
n United States
n Toronto
n NewYork City
n London
n Developing Nations

36. +
Open Data In Action
Canada United States
• Open.canada.ca/data
• 244,093 datasets
• Canada’s Open Data Principles:
• Completeness
• Primacy
• Timeliness
• Ease o Physical and Electronic
Access
• Machine Readability
• Non-discrimination
• Use of commonly owned standards
• Licencing
• Permanence
• Usage costs
• Data.gov | @usdatagov
• 132 712 datasets | 39 states and 46
localities provide data sets to the
repository
• U.S. Open Data Action Plan: On May 9,
the White House released plans to fulfill
an executive order that asks federal
agencies to make their data open and
machine- readable.
• Objectives:
• Publish open data in a discoverable,
machine-readable, useful way
• Collaborate with public and civil society
organizations to prioritize open data sets for
release
• Support innovators and improve open data
based on feedback
• Continue to release and enhance high-
priority data sets.

37. +
Open Data in Action – Toronto
n Open Data Toronto:“Building a City that thinks like the web - open data
leads to transparent government: open, accessible and accountable.”
n http://www1.toronto.ca/wps/portal/contentonly?
n 184 data sets | @open_to | #dataeh
n JSON, XML, Google spreadsheet, XLS, ESRI Shapefile, CSV
n Who is using the data? Mobile App - Dinesafe Toronto, Rocket Man,.Web
Apps - HomeZilla,Walkonomics.com,Vote.ca visualizations
n Data catalogue covering 15 categories: Business, city government,
community services, culture and tourism, development and infrastructure,
environment, finance, garbage and recycling, health, locations and mapping,
parks and recreation, permits and licenses, public safety, transportation,
water.

38. +
Open Data in Action – Toronto
n Operating divisions and program areas within the City manage their
own data sets and update them as required to meet business needs.
n Developers do not need to ask for permission to create applications,
however, they are required to follow the City's terms of agreement
n Open data license: must acknowledge the source.The license does not
grant you any right to use:
n Personal Information
n Information not accessible under the Ontario Municipal Freedom of
Information and Protection of Privacy Act or the Ontario Personal Health
Information Protection Act, 2004
n Third party rights the Information Provider is not authorized to licence
n The names, crests, logos, or other official symbols of the Information Provider
n Information subject to other intellectual property rights, including patents,
trademarks and official marks.

39. +
Open Data in Action –
Toronto
n Toronto has partnered with Ottawa, Edmonton and Vancouver to
improve Open Data standards and practices.
n The Province of Ontario and Toronto are working with other Open
Data municipalities in a group called the Public Sector Open Data to
develop common processes and formats
n Open dialogue with those who are interested in Open Data and
Open Government - devs, startups, advocates, visualizers, students,
researchers and more.
n Note-worthy: after Mayor John Tory launched his push to tow
illegally parked cars, he learned there was no easy way to assess
the impact of the blitz.

40. +
Toronto: Open Data & Traffic
Problem: Toronto is ranked 9th worst traffic in the world (according to
GPS maker TomTom)
n “The first thing is simply understanding what’s occurring out there,
and right now, we don’t have a good handle on that…We don’t
understand, what are the worst times of day and are there potential
strategies out there where we can do things differently to perhaps
spread that demand over certain hours? Those sorts of things that I
think, right now, we can speculate [about], but we don’t have hard
data.” – Transportation Services general manager Steve Buckley

41. +
Toronto: Open Data & Traffic
Solution:
n “The availability of travel data has improved dramatically over the past few years
and is at a point where the city can – and should – be using it to better understand
travel patterns, evaluate the city’s investments and monitor performance.With this
information, we can get Toronto moving smarter […] This will be a game changer
and will establish Toronto as a leader in running a truly smart city.” – John Tory,
April 2015
n First few months: developing strategy for data use
n September 2015: Hackathon
n Impact of solutions expected to be visible next year
n The team will be building on some of the current big data work of
Transportation Services, including:
n Partnering with McMaster University to analyze historical travel data on
city expressways and streets

42. +
Toronto: Open Data & Traffic
Solution:
n Working with the Toronto Transit Commission to closely analyze surface
transit data to identify operational improvements to further improve
streetcar service
n Releasing a report from the Cycling Unit of Transportation Services
evaluating cycling travel patterns based on data collected from its
cycling tracking app — showing the impacts of Cycle Tracks bike lane
program.
n Developing a Big Travel Data strategy for Transportation Services to
determine ways to make this type of information available, and
n Vetting products and services that might be useful in assisting the city in
better decision-making and investments.
n Looking to solutions in other cities: Amsterdam, Barcelona

43. +
Open Data In Action – NYC
n https://nycopendata.socrata.com/
n 1200+ datasets | @BetaNYC
n Business, city government, education, environment, health,
housing and development, public safety, recreation, social
services, transportation and NYC Big Apps.
n All NewYork City data to be open by 2018.
n Data portal is more than just a data portal. Includes data
visualization and anecdotes from users of the data published on
the site.

44. +
Open Data In Action –
London
n Data.london.gov.uk/ | @LDN_Data | 586 datasets
Problem: London uses big data to fight gangs/crime - when criminals are likely to strike -
with police funding being cut and limited resources:
Solution:
n Analysing five years' worth of data
n Monitoring gang members across 32 boroughs, then compared to criminal acts
conducted in the fifth year to see whether the software was accurate.
n Generate accurate prediction of when / if a criminal will re-offend.
n Evaluate aspects of an individual’s record: Geography past offenses, associations,
social media activity.
n Privacy issues
n Other cases: London Underground - 3.5 million daily riders
n Improve efficiencies in maintenance, decrease travel times, etc.
n Industrial Internet to monitor signal disruptions, track conditions, even climate inside
the cars.

45. +
Open Data In Action –
Developing Nations
United Nations Global Pulse; labs in NewYork City, Jakarta &
Kampala
n Innovation initiative of the United Nations Secretary General to
utilise big data.
n Vision & mission: a future in which big data is harnessed safely
and responsibly as a public good. Aims to accelerate:
n Discovery
n Development
n Scaled adoption of big data for sustained development and humanitarian action
n Feedback on implemented policy measures

46. +
Open Data In Action –
Developing Nations
n Analysis performed on growing number of data generated by
mobile device activity: communication, buy/sell, money
transfers, searches, social networks. Resulting in:
n Insights on human well-being
n Real-time trends on behavior
n Perceptions related to sustainable development issues

47. +
Open Data In Action –
Developing Nations
n Aids the U.N. in responding appropriately and faster to emerging
crises and vulnerabilities.
n Partner with with experts from UN agencies, governments,
academia, and the private sector to research, develop, and
mainstream approaches for applying real-time digital data

48. +
Open Data In Action –
Developing Nations
n Projects include:
n Analysing seasonal mobility patterns using mobile phones
n Analysing social media conversations to understand public
perceptions of sanitation
n Analysing large-scale News media content for early warning of
conflict
n Using twitter to measure global engagement on climate change
n Using twitter data to analyze public sentiment on fuel subsidy policy
reform in El Salvador
n Mining Indonesian tweets to understand food price crises
n Using mobile phone activity for disaster management

49. +
Open Data In Action –
Developing Nations
n Using mobile phone data and airtime credit purchase to
estimate food security:
n PARTNERS: UN World Food Programme (WFP),
Université Catholique de Louvain (Belgium), Real
Impact Analytics, Pulse Lab NewYork
n Assessed the potential use of mobile phone data as a
proxy for food security and poverty indicators.
n Data extracted from airtime credit purchases/“top-
ups” and mobile phone activity in an East African
country was compared to a nationwide household
survey conducted by WFP at the same time.

50. +
Open Data In Action –
Developing Nations
n High correlations between “top-ups” and survey
results referring to consumption of several food
items, such as vitamin-rich vegetables, meat or
cereals.
n “tops-ups” acted as a proxy indicator for food
spending in market-dependent households
n Provided valuable real-time information on the levels
of several indicators related to food security, which
could be integrated with early warning and
monitoring systems, filling data gaps between
survey intervals, and in situations where timely data
is not possible or accessible.

51. +
Future of Big Data and
Government
n Human + Machine symbiosis is most optimal outcome
n Right type of cooperation