Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Data All the Way Down
1. Data All the Way Down
Jeni Tennison
@JeniT
http://www.jenitennison.com/blog/
2. Data All the Way Down
• challenges of complex open data
• layered approach to data publishing
• essential steps
• benefits
3. Complex Datasets
• too much for a single spreadsheet
• need to navigate
• browse through data
• look at slices of larger dataset
• view summary statistics
• need to explain
• definitions of terms, provisos & disclaimers
4. User Challenge
• complex data sets have range of users
• different hardware / platforms
• different tasks / goals
• different ability / understanding
• no one interface satisfies everyone
• data owners cannot satisfy everyone
• create ecosystem around open data
6. Visualisations
• approachable for real people
• necessary for stakeholder buy-in
• beauty is in what's left out
• advertisement or taster of rich datasets
• often not possible in official data
• leaves questions unanswered
• what if we looked at the data in a different way?
7. Raw Data
• importable into own data store
• often only interested in particular slice
• data set may be massive / changing
• run whatever analysis you want
• requires at least some programming skills
• analysis might not be appropriate for the data
• documentation probably lacking
8. bridging the gap layered data access
Photo by Nikita Kravchuk http://www.flickr.com/photos/mi55er/3845619153/
9. Layered Architecture
• user interface
• navigation and global understanding
• API
• curated, targeted, programmable access
• query
• free-form programmable access
• raw data
18. Key Techniques
• resource-driven design (good URIs)
• every page built based on API calls
• explicit links to API access
• for bonus points, link to your transformation code
• consistent terminology
• clear mapping from UI to API
• caching & access control at each level
19. Benefits
• fork at any point
• don't like the visualisation / API? create your own!
• everyone is human
• reusers gain understanding from user interface
• visualisation benefits the stack
• API oriented towards achieving a goal
• visual validation of data improves quality