Slide presentations delivered during morning sessions of Day One of the California Statewide Health and Human Services Open DataFest - March 14 - 15, 2016, Sacramento, CA

2. Join the conversation now: #HHSDataFest
Welcome, Purpose, and Agenda Overview
Michael Wilkening, Undersecretary, California Health and Human Services Agency
Andy Krackov, Associate Director for External Engagement, The California
HealthCare Foundation
Daniel Stein, President, Stewards of Change Institute

7. Open DataFest Goals
• Share progress, celebrate accomplishments, make new connections
• Connect Social Determinants of Health and Wellness with Let’s Get Healthy California
and Open Data
• Showcase innovative local, state, and national programs
• Explore the intersection between academic research and open data
• Bring together various agencies’ involved with open data to learn from one another
and avoid building more silo
• Prepare recommendations to help shape California’s ongoing open data vision and
operations
• Introduce concepts, tools and methods to advance information sharing and
interoperability

8. Where Did We Start? Where Are We Today?
Data prison Lots of data are now ‘free’
No statewide HHS Portal New CHHSA Portal with 164 data
sets from all 12 departments
No regional / community
based events
Multiple efforts in counties and
cities across the state
No state-wide convenings HHS Open DataFest III
No data innovation focus New innovation office
Lagging the US Leading the US

11. Monday’s Open DataFest Agenda
Introductions
Overview of Open Data
Interactive: “Questions of Consequence”
Break
Exploring the Intersection between Open Data and Academic Research Data
Lunch (Affinity Table Discussion)
Social Determinants - “Let’s Get Healthy California” - Innovation Challenge Winners
Interactive: The Data Hunt
Break
Employing Cognitive Learning to Assist Decision Making and Program Effectiveness
Achieving Community Health and Creating Person-Centered Services in San Diego
Review and Wrap
Reception

13. Instructions: Questions of Consequence
Purpose:
To develop a rich understanding of what the people you interview think and feel about
the issues raised by your question.
Method:
1. Choose a partner and introduce yourselves. (OK if you have a trio)
2. You and your partner each have a unique question (#1 or #2).
3. One of you will interview the other for 8 minutes, and then switch roles.
4. After both interviews are finished, table participants will discuss both questions.
5. Whole room debrief.

14. Join the conversation now: #HHSDataFest
California’s HHS Open Data Overview, Updates
and Trends
Kevin Merritt, Founder & CEO, Socrata
Stuart Drown, Deputy Secretary for Innovation and Accountability, Government
Operations Agency
Michael Wilkening, Undersecretary, California Health and Human Services Agency

15. • A PRIMER ON OPEN DATA
Kevin Merritt
CEO
Socrata
15

16. 16

17. Who Uses Open Data?
17
• What are the major stakeholder groups interested in government data?
Ordinary but Interested
Citizens
Researchers, Scientists, Analysts,
Economists & Journalists
Developers &
Entrepreneurs
Government Employees!

18. Data Fuels the 21st Century Digital Government
18
• By becoming data-driven, governments transform themselves
Data-Driven Government
Improve
Transparency &
Accountability
Build & Promote
Public Trust
Citizen Engagement &
Empowerment
Improve
Operational
Efficiency
Support Fact-Based
Decision Making
Promote Economic
Development

19. 19
• “The key to progress for digital government
maturity is a singular focus on the exclusive use of
data in designing and delivering government
policies and services.”
Gartner, 2015

20. 20What Does Open Data Look Like?

21. 21Data Can Build Trust & Foster Community

22. Open Data Solves Real & Poignant Problems 22

23. Shedding Light on Pharmaceutical Influence 23

24. Bring Data to Where Consumers Already Are 24

25. Let Others Extend the Work of Your Data 25
(Raw Data & APIs) Ascel Bio Built the App

26. Open Data Pilot
Open Data Fest III
March 14, 2016
Sacramento Convention Center
26

27. • Pilot Goals
1. Create an efficient, consolidated, and reliable infrastructure for
data.ca.gov
2. Build capacity in state workforce
3. Make use of CalCloud to reduce cost and reach out to local
governments
• Objectives
1. Create greater efficiencies for program operations.
2. Encourage inter-agency data-sharing; Reduce duplication of
information.
3. Enable data-based decision-making.
4. Better serve Californians. Provide data more quickly and in a more
user-friendly format.
5. Stimulate innovation.
27
Pilot Goals & Objectives

28. Surveyed 88 entities
756 “public” data sets reported
91 different formats
28
Data Inventory- May 2014

29. 29
Sustainability Innovation Challenge - DGS Headquarters
October 24-25, 2015; Go to greengov.data.ca.gov

30. • 11 data sets published
• 14 open source apps
• Introduced government to new population –potential hires
• Collaboration under tight deadline built mutual respect/brand
• Learned to flex new muscles = morale boost
• Implementing winning apps required learning new skills
• Open source Innovation “sandbox”
30
Innovation Challenge Outcomes

31. 31
Open Data Pilot Timeline

32. • Pilot
• Platform
• Standards + Governance
• Federation
• Automation
• Collaboration
• Within government
• Civic engagement
• Targeted deeper access
• Cultural Change
• Siloed to Open
• Deeper analysis
• Data-driven decision-making
32
More than a platform

33. Join the conversation now: #HHSDataFest
Interactive Activity: How Open Data Can Be
Used to Advance Data-Literacy and Decision
Making
Facilitators: Stewards of Change Institute Team

34. Join the conversation now: #HHSDataFest
Exploring the Intersection between Academic Research and Open Data
to Improve Outcomes and Measure Performance; Lessons Learned from
Stanford University Event
Mark Cullen, MD, Director, Stanford University Center for Population Health Sciences
Panelists:
Phil Leaf, Ph.D., Professor, Johns Hopkins Bloomberg School of Public Health, School of Medicine, School of Education, and School of Arts and
Sciences
Aenor Sawyer, Associate Director, Strategic Relations, Center for Digital Health Innovation UCSF; Director UCSF Skeletal Health Service
Anne Neville, Director, California Research Bureau, California State
Erika Martin, Ph.D., MPH, Senior Fellow and Director of Health Policy Studies, Rockefeller Institute of Government and Associate Professor,
Department of Public Administration & Policy, State University of New York
Responder:
Regan Foust, Director, Strategic Partnerships, Research Scientist, Children’s Data Network

35. Open Data Are Valuable
Jonathan Gross, BCHD
• Open data are sometimes mistaken as not being
valuable
• However, the data are made public because they are
valuable!
• Federal, state, and local governments are making
more data available than ever before to help solve a
wide array of problems.

36. Baltimore: 19-Year Life Expectancy Gap
www.baltimorecity.gov/healthmaps Data at BNIA: http://bniajfi.org/

37. Health Insurance Estimates
• Tan and red census
tracks are areas with
low health insurance
rates
• The markers are
hospitals and FQHCs
Data source: Census/American Community Survey (ACS)
and Maryland State Data Center: http://www.mdp.state.md.us/msdc/S7_ACS.shtml
www.baltimorecity.gov/healthmaps

38. Hospital’s Proximity to Firearm Violence

39. Medicare: Open Opioid Prescription Data
Provider-level data are also available
https://www.cms.gov/Research-Statistics-Data-and-
Systems/Statistics-Trends-and-Reports/Medicare-Provider-Charge-
Data/OpioidMap.html

40. Be Open Data Savvy!
• Start early
• Be BOLD!
• Open data is valuable
• Search, search, search!
• Find a comparison group/area
• Use simple tables and colorful graphs
• Work in teams
• Don’t be afraid to start over

41. Health Dept. Map Gallery
• Baltimore City
• Social determinants of health
• Health insurance at census tract level
• Homicide epidemic – spatial patterns
• Shooting incident animation
 www.baltimorecity.gov/healthmaps
• More to come…
• At least one map per division
• Asset mapping
• Mid/Long-term possibilities…
• Health atlas
• Crowdsourced, volunteered geographic
information, or social media

42. Youth Health Index across 55 Baltimore City Community
Statistical Areas

43. ACADEMIC
RESEARCH
OPEN DATA
MARCH14,2016

44. The Case: The National
Broadband Map (2009-2015)
(You are correct. This is not health-related)

45. • First national + open dataset of broadband availability
• Really granular – census block or road segment
• Big: 25M new records every six months
• New opportunity to study broadband access in the US
• Stakeholders: policymakers, advocates, researchers,
public/private sector analysts, journalists, consumers

46. Stakeholder Outreach: Researchers
Successful event at National Press Club one month
after launch of public data
Seven research groups: Georgia Tech/U. Toronto/ Northwestern, U.
Texas, Joint Center, Murray State, Harvard, Drexel/Mich State Phoenix Center
Federal Register notice, advanced access to data for
institutions with research plans
Government agency (NTIA) has existing
relationship with researchers

47. What Happened Next?
• Dedicated space for researchers on project wiki
• Outcome: Not used
• Created a Researcher Working Group
• Outcome: Initial interest, not sustained
• Considered grants to fund independent research
• Outcome: Not created

48. The Plan Didn’t Work
What Happened?

49. Coordinating Research Activities
• Limited staff
• No initial external funding for research on this topic
• Limited funding - would research grants provide most
usefulness?

50. Research-Related Activities
• Regularly answered questions from researchers about
metadata, data collection methods
• Reviewers for academic papers

51. What’s Helpful for Gov’T?
What we hope you’ll say when you call

52. Improving Communication
• If the data is problematic, let’s talk
• We appreciate a heads up, if possible
• Specific feedback leads to specific results
Contact Info:
@CAStateLibrary
@anneneville

53. Integrating Public Health Researchers into
the Open Data Ecosystem
Erika Martin, PhD MPH
Rockefeller Institute of Government & University at Albany
California Health and Human Services Open DataFest III
Sacramento, CA
March 14, 2016
This work was supported by the Robert Wood Johnson Foundation’s Public Health Services and
Systems Research Program (grant ID #71597)

54. Project overview
 Limited guidance on tailoring open data to different users
 Open data are only valuable when used
 How can we improve the quality and usability of data for
public health research and practice?
 Systematic review of health data offerings in three open data
portals (HealthData.gov, Health Data NY, NYC OpenData)
 Key informant interviews with practitioners publishing open health
data to understand challenges and opportunities
 Pilot open data linkage project to assess the feasibility of using
open data for academic research

55. Data characteristics
(e.g. missing data, timeframe,
data collection procedures,
database design, data
elements, population)
Data user
characteristics
(e.g. intended use, expertise,
skills, tasks performed)
Platform promotion
and user training
(e.g. value propositions,
financial resources, political
support, information
technology, regulations and
data stewardship, legal
interpretation of
confidentiality protections)
Rockefeller Institute of Government 55
 Intrinsic data quality
(e.g. accuracy, reputation,
confidentiality, reliability,
validity, objectivity)
 Contextual data quality
(e.g. appropriate amount,
completeness, concise
representation, ease of
manipulation, relevance)
 Platform usability
(e.g. accessibility, functionality,
learnability, representational
consistency, visibility)
 Metadata quality
(e.g. accuracy, completeness,
consistency, interpretability,
provenance)
 Short-term impacts
(e.g. availability of health
information, data-driven
population health planning
and monitoring, mHealth
development, consumer
empowerment, research
grants and studies)
 Long-term impacts
(e.g. improved population and
patient health, enhanced
decision-making, higher
quality/value medical and
public health services)
CHARACTERISTICS OF
DATA USE
DATA QUALITY AND
USABILITY
DIRECT AND INDIRECT
HEALTH IMPACTS

56. Systematic review of open data offerings
(HealthData.gov, Health Data NY, NYC OpenData)
 Most data offerings not designed for health research
 Only one-quarter of open data offerings are structured datasets
 Most offerings do not contain demographic variables
 Variation in quality and usability across platforms
 Health Data NY scored highest on intrinsic data quality, contextual data quality,
and adherence to Dublin Core metadata standards
 Gaps in meeting “open data” deployment criteria
 All offerings met basic “web availability” open data standards
 Fewer met higher standards of being hyperlinked to other data to provide
context
 Platforms enable users to discover and access data in novel
ways, with areas for improvement
 Technical problems limit functionality, low web visibility, HealthData.gov is
primarily a search engine

57. Key informant interviews
(Policymakers and practitioners in New York and federal agencies)
 Wide range of perceived benefits
 Internal benefits: improved data/metadata quality, more efficient public
health operations (e.g. data silos, FOIA requests)
 External benefits: health literacy, data-driven improvements in healthcare
delivery and built environment, community empowerment, improved data
quality, timeliness, and usefulness
 New users bring innovative ideas
 Numerous challenges to releasing data
 Critical challenges: resources, cultural resistance, legal and regulatory issues,
and data/metadata quality
 Other challenges: technical issues with legacy systems and data platforms,
knowledge gaps, addressing needs of diverse end-users
 General optimism that open data movement will continue
 Yet success depends on sustained leadership, resources, cultural changes,
promoting the use of data, and establishing governance

58. Pilot data linkage project
(Mapping childhood obesity to the built environment in New York)
 Many datasets readily available for public health research
 Can use data creatively to evaluate multiple dimensions of the built
environment (e.g. using restaurant inspections data for fast food availability)
 Can synthesize data from different domains (health, agriculture, education)
 Challenges consistent with findings from other study phases
 Lack of standard definitions for data elements severely constrains
interoperability and ability to merge by geographic identifier
 Incomplete metadata, e.g. missing codebooks
 Data quality, e.g. incomplete addresses, inconsistent location descriptions
 Data timeliness
 High level of geographic aggregation limits value
 Some data not easily discoverable (or available) in open data platforms
 Data not yet 5-star, e.g. downloadable in multiple non-proprietary formats and
with links to provide context
 Limited usability, e.g. advanced statistical skills required to prepare data

59. Questions?
 Contact:
emartin@albany.edu
 For additional project information:
www.publichealthsystems.org/erika-martin-phd-mph-0
 For materials from fall 2013 workshop on open health data
in New York and links to open data resources:
www.rockinst.org/ohdoo

60. Bibliography
 Martin EG, Helbig N, Shah NR. Liberating data to transform healthcare:
New York’s open data experience. JAMA 2014; 311(24): 2481-2481.
 Martin EG, Helbig N, Birkhead GS. Opening health data: what do
researchers want? Early experiences with New York’s open health data
portal. J Public Health Manag Pract 2015; 21(5): e1-7.
 Martin EG, Law J, Ran W, Helbig N, Birkhead GS. Evaluating the quality
and usability of open data for public health research: a systematic
review of data offerings on three open data platforms. J Public Health
Manag Pract. [In press; online ahead of print]
 Martin EG, Begany GM. Opening government health data to the public:
benefits, challenges, and lessons learned from early innovators. [Under
review]
 Dwicaksono A, Brisette I, Birkhead GS, Bozlak CT, Martin EG. Evaluating
the contribution of the built environment to obesity among New York
State students. [Working paper.]