Models: pets and herds". Carlos J. Gil Bellosta

1. Models: pets and herds
Carlos J. Gil Bellosta
cgb@datanalytics.com
September 2014
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 1 / 21

2. This is a pet...
Source: http://jessfalcone.wordpress.com
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 2 / 21

3. ... and this is a herd
Source: http://bonfirehealth.com/negative-influences-comparisons-social-cues-herd/
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 3 / 21

4. Some people treat computers as pets...
Source: aliexpress.com
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 4 / 21

5. ... an others like herds
Source: Failure Trends in a Large Disk Drive Population, Pinheiro et al.
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 5 / 21

6. This is a statistical model treated as a pet
Source: http://www.ats.ucla.edu/stat/stata/seminars/interaction_sem/interaction_sem.htm
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 6 / 21

7. Pets are very demanding and require...
1 variable selection,
2 checks for outilers,
3 assessment of the goodness of

8. t,
4

9. nding con

10. dence intervals,
5 calculating p-values,
6 interpretating the results,
7 discuss the generalization,
8 ...
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 7 / 21

11. Models... as herds?
Source: http://www.gotmedieval.com
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 8 / 21

12. Model construction: population
Source: http://timyeo.wordpress.com/
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 9 / 21

13. Model construction: data enrichment (aka left join)
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 10 / 21

14. But subject data is often messy...
Source: http://arquitectolegista.com.ar/
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 11 / 21

15. ... and contains temporal data...
Source: http://thirdorderscientist.org/
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 12 / 21

16. ... that is dicult to

17. t in a box (table)
Source: http://cutestcatpics.com/cat-trying-to-fit-into-tiny-box/
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 13 / 21

18. We have a whole dataset per subject!
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 14 / 21

19. ... and a model per subject?
Source: http://www.unc.edu/
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 15 / 21

20. (Most) models are sophisticated summaries of data
Source: http://xkcd.r-forge.r-project.org/
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 16 / 21

21. Do you seek ? Build a model per stock!
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 17 / 21

22. Fitting a million models in the nineties was all of an
achievement (for some)
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 18 / 21

23. Beyond recency and frequency: self exciting processes
Source: Bursting transition in a linear self-exciting point process, Onaga, T. et al
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 19 / 21

24. One logistic regression per Gmail user...
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 20 / 21

25. Challenges: statistical, computational,... and more!
This approach faces many challenges:
1 Computational: how do you

26. t so many models? (but Spark rocks!)
2 Statistical: how do you...
1 perform variable selection?
2 evaluate the

27. t?
3 deal with outliers?
4 ...
3 And

28. nally, how do you sell these approaches to business people
(ex-Google)?
Carlos J. Gil Bellosta { datanalytics Models: pets and herds September 2014 21 / 21