This document discusses uncertainties in big data. It presents several case studies where deep learning and multi-view learning techniques have been applied to predict things like traffic incident duration, school truancy factors, and event detection in video. However, it notes that all models have uncertainties and discusses the need to better quantify objectives, propagate uncertainties, evaluate robustness, and communicate uncertainties to users. The document advocates for a multidisciplinary approach to tackling uncertainties in big data predictions and outcomes.

1. DATAPIJPLIJNEN
GARBAGE-IN-GARBAGE-OUT OF EEN WASSTRAAT?
BIG DATA EXPO, 21 SEPTEMBER 2017
Freek Bomhof

2. CAN WE KNOW THE TRUTH?
Uncertainties in Big Data

3. CAN WE KNOW THE TRUTH?
Uncertainties in Big Data

4. CAN WE KNOW THE TRUTH?
Uncertainties in Big Data

5. TRAVEL TIME: WHAT IS THE BEST DECISION?
Uncertainties in Big Data

6. Uncertainties in Big Data
PREDICTION OF INCIDENT DURATION
Background: Travel time prediction is best done using classical
(statistical) methods. Incidents have significant influence on travel
time but are hard to predict
Aim: use deep learning technology to predict the incident duration
Approach: the developed ‘fingerprint’ method is combined with
Deep Learning
Input: Loop data from Dutch highway network (several years)
Result: Incident durations can be predicted with remarkable
accuracy.
Incident: starts when RWS
closes lane; ends when
average speed is >70%

7. Uncertainties in Big Data
EVIDENCE-BASED YOUTH POLICY
Multi-View Learning (MVL) is a Machine Learning framework that is
expected to be very well suited for creating predictive models.
Unlike standard statistical approaches, algorithms that are formulated within
this framework allow mining large amounts of heterogeneous data from
multiple sources, dealing with noisy and high dimensional tasks,
incorporating partially labelled data (e.g. semi-supervised or transductive
learning setting where annotated data is limited), and are based on
theoretically justified assumptions and error bounds.
Aim: to apply the Multiview Learning network approach to identify main factors
for truancy at schools. This could be the basis for evidence-based policy
Results
Tested on a database with 12 000 data subjects
Results provide some preliminary relations
Applicable to many other aspects (health: obesity; social: drugs abuse)
outcome
Feature selection to predict the effect
of an intervention
Time-resolved MV clustering to track the effect of
interventions over time

8. Uncertainties in Big Data
Source: Nature, Advances in nowcasting influenza-like
illness rates using search query logs (2015)

9. Uncertainties in Big Data
Source: tylervigen.com

10. All models are wrong
But some of them are useful

11. WHY THESE TOPICS? WHY NOW?
Our algorithms become smarter every day
Cross-domain & multistakeholder data exchange
Complexity is growing
Yet we expect the user to trust the outcomes
Uncertainties in Big Data
XAI

12. TNO RESEARCH AGENDA
Uncertainties in Big Data

13. DIVING INTO UNCERTAINTY: CASES
Adaptive Cruise Control
Factors that cause diabetes
Citizens applying for support
Find failing sensors in dikes
Evaluating cross-media effectiveness
ETA prediction container ships
Resource allocation in HD cameras
Hybrid Energy Grids
Intel from aerial observation
Machine 2 machine grids
Naval Mine Detection
Long-term effects in prematurely borns
Uncertainties in Big Data
Better decisions for risk-based Infrastructure
Assets
Youth Policy: school truancy
Detecting events in video
Smart batteries in Smart energy grids
Analysing football games for training
Stability of underground pipelines
Scenario detection for self-driving cars
Assessing safety of proteins in drugs
Predict duration of traffic incidents
Assess the size of internet-related business
Predict the need for municipal social support

14. Uncertainties in Big Data
the analysis framework is correct and complete
Source: sdxcentral.com

15. Uncertainties in Big Data
a complete and clear picture of uncertainty is useful
Source: flickr, ELKayPics (CC)

16. Uncertainties in Big Data
a higher accuracy would be valuable
Source: freegreatpicture.com

17. Uncertainties in Big Data
tackling uncertainties is
multidisciplinary
Source: Wikimedia commons

18. Uncertainties in Big Data
PREDICTION OF INCIDENT DURATION
Background: Travel time prediction is best done using classical
(statistical) methods. Incidents have significant influence on travel
time but are hard to predict
Aim: use deep learning technology to predict the incident duration
Approach: the developed ‘fingerprint’ method is combined with
Deep Learning
Input: Loop data from Dutch highway network (several years)
Result: Incident durations can be predicted with remarkable
accuracy.
Incident: starts when RWS
closes lane; ends when
average speed is >70%

19. A RESEARCH AGENDA
Data quality and representiveness
Including semantic uncertainty
Quantifying objectives
Uncertainty propagation
Model choice
Including robustness
Communicating uncertainty
Uncertainties in Big Data

20. OM OVER NA TE DENKEN… OOK VOOR DATA?
Uncertainties in Big Data

21. THANK YOU FOR YOUR
ATTENTION
Take a look:
TIME.TNO.NL