The document outlines frameworks for improving data access and usability within health systems, focusing on user interaction, information architecture, and data management. It discusses various approaches to structuring data presentation, including top-down and bottom-up methodologies, to facilitate better user experience and engagement. Additionally, it emphasizes the importance of publishing metadata and entity relationship diagrams to enhance data understanding and usability for different user communities.

1. The National Committee on
Vital and Health Statistics
Data Access and Use
Joshua Rosenthal, PhD
Systems & Practices

2. 2
SYSTEMS & PRACTICES
Structure
Ecosystem Interaction
How does someone find a site / delivery mechanism
or information within a site / delivery mechanism
via another channel?
UI / UX
[User Interaction / User Experience]
How does someone use and experience
a site / delivery mechanism?
Information Architecture
How does someone find information within
a site / delivery mechanism?
Data
How useful and usable is the data within
a site / delivery system?

3. 3
SYSTEMS & PRACTICES
Structure
Ecosystem Interaction
How does someone find a site / delivery mechanism
or information within a site / delivery mechanism
via another channel?
UI / UX
[User Interaction / User Experience]
How does someone use and experience
a site / delivery mechanism?
Information Architecture
How does someone find information within
a site / delivery mechanism?
Data
How useful and usable is the data within
a site / delivery system?

4. 4
SYSTEMS & PRACTICES
Structure
Data
How useful and usable is the data within
a site / delivery system?
Does the data have, and successfully meet demand,
in general and from different user communities?
Two Types of Approaches (Not Mutually Exclusive):
Top Down
Bottom Up

5. 5
SYSTEMS & PRACTICES
Structure
Data
How useful and usable is the data within
a site / delivery system?
Does the data have, and successfully meet demand,
in general and from different user communities?
Top Down Approaches
Sample – Check List from “Experts”
Is the Data Usable?
Is the Data Useful?
Is the Data Timely?
Etc.
Implement
1 - HHS Data Checklist for Producers (Damon Initiative)
2 – Publish Metadata definitions and taxonomy / ERD
(Entity Relationship Diagrams – See Appendix)
3 – Data producer Best Practice Sheet
(e.g. fill in the blank / Mad Libs – See Appendix)
4 – Review / Add / Enforce individually via
requirements of vendors via budget/delivery cycles

6. 6
SYSTEMS & PRACTICES
Structure
Ecosystem Interaction
How does someone find a site / delivery mechanism
or information within a site / delivery mechanism
via another channel?
UI / UX
[User Interaction / User Experience]
How does someone use and experience
a site / delivery mechanism?
Information Architecture
How does someone find information within
a site / delivery mechanism?
Data
How useful and usable is the data within
a site / delivery system?

7. 7
SYSTEMS & PRACTICES
Structure
Information Architecture
How does someone find information within
a site / delivery mechanism?
Two Types of IA Approaches (Not Mutually Exclusive):
Top Down
Bottom Up
Can someone find information navigating in different
ways, and is there navigation informed by / are they able
to judge the usefulness and utility of the information?

8. 8
SYSTEMS & PRACTICES
Structure
Information Architecture
How does someone find information within
a site / delivery mechanism?
Two Types of IA Approaches (Not Mutually Exclusive):
Top Down Approaches:
Sample Mechanisms (See Appendix for sample schema)
Site Map (structured around IA Taxonomy)
[PRO TIP / V2: this should intersect with metadata taxonomy]
Source vs./and Topics – Parallel navigation (supports different users / use cases)
Tags – Taxonomy Tags - Expert assigned / objective [e.g. file format; publisher]
Navigation/Exploration: Search, filter, sort, Bread Crumbs to taxonomy/site map
Implement
Data.gov has most of this (needs ERDs, bread crumbs beyond publishing org); expand and extend
Where not be possible in specific sites via security / privacy, consider in meta sites (data.gov, etc.)
Review / Add / Enforce individually via requirements of vendors via budget/delivery cycles –
(copy specs from data.gov)

9. 9
SYSTEMS & PRACTICES
Structure
Information Architecture
How does someone find information within
a site / delivery mechanism?
Two Types of IA Approaches (Not Mutually Exclusive):
Bottom Up Approaches:
Sample Mechanisms (See Appendix)
Display overall / unidentified counts of page views / data downloaded
Unidentified users to rate via overall stars; then sub rating (e.g. usefulness, utility)
Opt in user identification of user category / interest -
(not identification of person but of interest/level)
Counts of views/downloads and ratings (overall and sub) by user type
Tags – Folksonomy Tags – Open to at large input, captures alternative navigation
and providers learning of use and type to producers
PRO TIP: Learning Center / Community taxonomy intersects with user type & tags
Implement
This is new and different, but well trodden in both tech (GIT HUB) and DTC (Amazon).
Expand data.gov; spin off alternative modules/site wrapper via innovation project/challenges

10. 10
SYSTEMS & PRACTICES
Structure
Ecosystem Interaction
How does someone find a site / delivery mechanism
or information within a site / delivery mechanism
via another channel?
UI / UX
[User Interaction / User Experience]
How does someone use and experience
a site / delivery mechanism?
Information Architecture
How does someone find information within
a site / delivery mechanism?
Data
How useful and usable is the data within
a site / delivery system?

11. 11
SYSTEMS & PRACTICES
Structure
UI / UX
[User Interaction / User Experience]
How does someone use and experience
a site / delivery mechanism?
User Interaction / User Experience
Two Types of Approaches (Not Mutually Exclusive):
Score Carding (See Appendix for example)
Work from checklist, score instance and offer improvements
Emulate Model (See Appendix for example
Work from example of what already works within a similar context and modify
UI / UX
[User Interaction / User Experience]
How does someone use and experience
a site / delivery mechanism?
Implement
Copy data.gov as model for individual HHS sites/instances; modify based on constraints of individual
instance; expand, extend and improve based on users, goals and priorities of instance.
Review / Add / Enforce individually via requirements of vendors via budget/delivery cycles

12. 12
SYSTEMS & PRACTICES
Structure
Ecosystem Interaction
How does someone find a site / delivery mechanism
or information within a site / delivery mechanism
via another channel?
UI / UX
[User Interaction / User Experience]
How does someone use and experience
a site / delivery mechanism?
Information Architecture
How does someone find information within
a site / delivery mechanism?
Data
How useful and usable is the data within
a site / delivery system?

13. 13
SYSTEMS & PRACTICES
Structure
Ecosystem Interaction
How does someone find a site / delivery mechanism
or information within a site / delivery mechanism
via another channel?
Multiple Ways: Push vs. Pull (see appendix for ecosytem)
Pull: Asking Users to Find & User a Destination Site
Free standing websites (CDC, etc.); Data Enclave
Push: Push Data to Channels where User Already Are
Data Browsers (Google public data); Repositories (GIT)
Meta Sites: External sites for navigation linking to mulitple
websites (data.gov)
Distribution Mechanisms: External sites using (Pro Publica
Data Store /Yelp; USNEWS/RowdMap Best Doctors/Hospitals)

14. 14
SYSTEMS & PRACTICES
Structure
Ecosystem
Barriers to access
(cost, skill, awareness)
Enclaves
Destination Sites
Meta Sites /
Repositories
Data Browsers
(See Appendix)
Delivery Mechanisms
(See Appendix)
Ecosystem Interaction
How does someone find a site / delivery mechanism
or information within a site / delivery mechanism
via another channel?

15. 15
SYSTEMS & PRACTICES
Structure
Ecosystem Sample Interaction
Meta site
Destination
site
Destination
site
Destination
site
Destination
site
Destination
site
Destination
site
Destination
site
User aggregation
IA & UI/UX
Implement IA & UI/UX
to match
Site by site as
individual budget /
contract cycles allow
Sample approach: Use Data.gov as model
for destination sites; roll on destination site
UI/UX as budget contracts allow
Destination site build / contract / budget cycle
Ecosystem Interaction
How does someone find a site / delivery mechanism
or information within a site / delivery mechanism
via another channel?

16. 16
SYSTEMS & PRACTICES
Appendix
Taxonomy and Metadata
Entity Relationships Diagrams (ERD)

17. 17
SYSTEMS & PRACTICES
Taxonomy and Metadata Entity Relationships Diagrams
Publish Entity Relationship Diagrams
An ERD is basically the blue print for a
data set.
This is true for all types of data, from large
sets in databases to small sets in simple
files (non-beneficiary files).
Data producers should publish their ERDs,
a common practice outside health care.
This benefits users who otherwise have to
create their own models, a notable
barrier to use and usability.

18. 18
SYSTEMS & PRACTICES
Taxonomy and Metadata Entity Relationships Diagrams
http://www.medicare.gov/download/downloaddb.asp
Example taxonomy (non-beneficiary / STAR example)

19. 19
SYSTEMS & PRACTICES
Taxonomy and Metadata Entity Relationships Diagrams
What is this? How can I use it to answer business/performance questions?
Sample record from the downloaded file

20. 20
SYSTEMS & PRACTICES
Taxonomy and Metadata Entity Relationships Diagrams
In order to get insight, I need the data in a meaningful business structure

21. 21
SYSTEMS & PRACTICES
Taxonomy and Metadata Entity Relationships Diagrams
Taxonomy defines business entities and the relationships among them

22. 22
SYSTEMS & PRACTICES
Taxonomy and Metadata Entity Relationships Diagrams
Taxonomy defines attributes for business entities

23. 23
SYSTEMS & PRACTICES
Taxonomy and Metadata Entity Relationships Diagrams
Taxonomy
(business relationship of data elements)
CMS
Data Products/Tools
Research
Data Products/Tools
Commercial
Data Products/Tools
Simple System
data extract / cloud
Learning Center
user/data interaction
Databases
Beneficiary
Files
Non-beneficiary

24. 24
SYSTEMS & PRACTICES
Taxonomy and Metadata Entity Relationships Diagrams
Want to build something cool here
But has to do all the taxonomy work from scratch, each time,
sifting through files, metadata, to attempt to recreate relationships.

25. 25
SYSTEMS & PRACTICES
Appendix
Data Producer /
Best Practice Worksheet

26. 26
SYSTEMS & PRACTICES
Data Producer Best Practice Worksheet
Here is a blurb to fill out for a data producer to fill
out / contemplate before pushing out.
My data set is _________________
It can be used to answer the questions about: _______,
_______ and ________.
The planned release cycle is annual/quarterly/ad hoc.
(Or, there is no further planned release due to budget
visibility, but it is a priority set).
Etc.
This is very important for
the marketplace and
researchers – if you are
going to invest in making
something specific data,
you need visibility of the
data going forward
This is
important to
have the
producer
consider the
demand and
track its use

27. 27
SYSTEMS & PRACTICES
Appendix
Top Down Information Architecture
(IA) Schema

28. 28
SYSTEMS & PRACTICES
IA: Top Down – Schema
Sample Schema (Content of Tags)
File format
JSON
XML
PDF
Etc.
Geographic grain
Country
State
County
PUMA
HRR
HSA
Zip
Neighborhood Block
Etc.
Dates
[Range]
Dates Frequency
Topic
Agency
Producer
Etc.
Etc.

29. 29
SYSTEMS & PRACTICES
IA: Top Down – Schema
Sample Schema (Content of Tags)
Data
Usability / Usefulness
Site
Usability / Usefulness
Schema as Tags
Tags from the data become the
way that you navigate the site

30. 30
SYSTEMS & PRACTICES
Appendix
Bottom Up Information Architecture
(IA) Approaches - Various

31. 31
SYSTEMS & PRACTICES
IA: Bottom Up – Approaches
Folksonomy – Users Create & Add Tags (no schema)
Add a Tag to
This Data!
Search by User-
Generated Tags!

32. 32
SYSTEMS & PRACTICES
IA: Bottom Up – Approaches
Users (Unidentified) Rating Data
Rate this data
set!
(different than
# of views)

33. 33
SYSTEMS & PRACTICES
IA: Bottom Up – Approaches
Users Buckets
I am a:
Researcher
Government person
Community dude
Entrepreneur, baby!
Here’s the top sources for
people like you
You may like: JSON
Here’s your learning
community

34. 34
SYSTEMS & PRACTICES
Taxonomy across Data & Metadata
Taxonomy ensures users both find the data and find it meaningful
Data
Usability / Usefulness
Site
Usability / Usefulness
Taxonomy / Folksonomy
Tags from the data become the
way that you navigate the site

35. 35
SYSTEMS & PRACTICES
Appendix
Top Down User Interaction / User Experience
(UI/UX) – Score Carding

36. 36
SYSTEMS & PRACTICES
UI/UX Score Carding
Simple Version
Score cards for site evaluation
Is it clear / clean & readable?
Does it use icons / pictures to
convey information?
Can you search, sort and filter?
Does it look like a 1987 MySpace
Page blew up on your screen?
Formal Version
Is a Website Reasonable to Use?
Evaluate against Criteria

37. 37
SYSTEMS & PRACTICES
Appendix
User Interaction / User Experience
(UI/UX): Emulate a Model

38. 38
SYSTEMS & PRACTICES
UI/UX Emulate a Model
Is a Website Reasonable to Use?
Find a good site in a similar space and modify
Clear, clean & readable
with pictures / icons
Search, sort & filter
capabilities
Highlights and popularity
pushed to top

39. 39
SYSTEMS & PRACTICES
UI/UX Emulate a Model
Taxonomy Number of Views
Taxonomy displayed
Is a Website Reasonable to Use?
Find a good site in a similar space and modify

40. 40
SYSTEMS & PRACTICES
Appendix
Data Browsers

41. 41
SYSTEMS & PRACTICES
Data Browsers as Distribution
HHS/CMS/Etc release data - files or data platform
‘Users’ form communities and can:
1) Use data or portal directly
- need specialized expertise/access
(data/stat)
2) Build Apps
- need to be able to develop
(tech skills)
3) Use a data browser

42. 42
SYSTEMS & PRACTICES
Data Browsers as Distribution
Quickly check size / traffic /
ranking and inbound
sources of different sources
-
(Free/cheap/no integration
required/anyone can do it)

43. 43
SYSTEMS & PRACTICES
Data Browsers as Distribution
1) Data.gov
- Everything (not just health)
~3k sites linking in
2) Google
- (data explorer is a fraction, but big #)
~5MM sites linking in
3) Big Tech Site on data explorer contest
- ReadWriteWeb (contest with Tableau Public)
~40k sites linking in
Distribution is really important –
the secondary tier of access / users
dwarfs the primary point of access

44. 44
SYSTEMS & PRACTICES
Data Browsers as Distribution
Sample: Google Public Data Explorer
Data Browsers allow anyone to
interact with the data, explore
and analyze data without having
to touch the data / code / have
special tools
(IE you do it directly in a web site)

45. 45
SYSTEMS & PRACTICES
Data Browsers as Distribution
Sample: Google Public Data Explorer
Data Browsers
navigate via a
taxonomy
(meta data), not
what a top down
standard has
stated, but what’s
implicit / latent /
already in the data
(via scraping)

46. 46
SYSTEMS & PRACTICES
Data Browsers as Distribution
Sample: Tableau Public Using HHS data (500k users)

47. 47
SYSTEMS & PRACTICES
Data Browsers as Distribution
Sample: Public Using HHS data (without touching data)
Health Topics / HHS Data Is Hot!

48. 48
SYSTEMS & PRACTICES
Appendix
Distribution Mechanisms

49. 49
SYSTEMS & PRACTICES
Distribution Mechanisms
HHS Data Taking on Secondary & Tertiary Distribution
Many, Many, Many Consumers/Users Using HHS Here
ProPublica repackaging CMS data,
making more useful, selling
ProPublica repacking CMS data,
distributing embedded in Yelp

50. 50
SYSTEMS & PRACTICES
Distribution Mechanisms
HHS Data Taking on Secondary & Tertiary Distribution
Many, Many, Many Consumers/Users Using HHS Here
RowdMap (start up) repackaging HHS &
Dartmouth data, selling to US Market
(75MM patients, 42 States cf. CMMI
presentation) and partnering with USNEWS
RowdMap HHS data, partnering with
USNEWS to help consumers