Slides for my talk at City Unrulyversity on 18.03.15 in London. Discuss the term Data Science, touch upon the origins and the data scientist types. A longer discussion on the Data Science process and challenges analysts face. And here is the abstract of the talk: Data Science ... the term is everywhere now, on the news, recruitment sites, technology boards. "Data scientist" is even named to be sexiest job title of the century. But what is it, really? Is it just a hype or a term that will be with us for some time? This session will investigate where the term is originating from and how it relates to decades of research in established fields such as statistics, data mining, visualisation and machine learning. We will investigate how the field is evolving with the emergence of large, heterogeneous data resources. We will discuss the objectives, tools and challenges of data science as a practice, and look at examples from research and industrial applications.

1. Data science:
Origins, methods,
challenges & the future?
CAGATAYTURKAY,
giCentre, City University London
City Unrulyversity, 18 March 2015

2. Who?
• Lecturer in Applied Data Science, City Univ. London
• @ giCentre
• PhD @VisGroup at Univ. of Bergen, Norway

3. Research on …
Methods for
InteractiveVisual
Data Analysis

4. WHAT IS
DATA SCIENCE?

5. Somes example first – Google Flu Trends
http://www.google.org/flutrends/
Ginsberg, Jeremy, et al. "Detecting influenza epidemics using search engine query data." Nature 457.7232 (2008): 1012-1014.
Video: http://goo.gl/4ysAmw
Google says ..
.. relationship between how
many people search for flu-
related topics and how many
people actually have flu
symptoms. … to estimate
how much flu is circulating ….

6. The Shortest Path to Happiness:
Recommending Beautiful, Quiet,
and Happy Routes in the City
Quercia, Daniele, Rossano Schifanella, and Luca Maria Aiello. ACM
conference on Hypertext and social media, 2014.
http://urbangems.org/

7. Data science is a systematic study
of generalizable extraction of
knowledge from data
From “Data Science and Prediction” byVasant Dhar, Communications of the ACM, 2013

8. Data science is a process starting with
formulating a question that can be answered with
data, and iteratively collecting, cleaning,
analysing and modelling the data, while
communicating the answers to the relevant
audience along this iteration.

9. On the origins ..
Term coined by William S. Cleveland in 2001[*]
…knowledge among computer scientists about how to think
of and approach the analysis of data is limited, just as the
knowledge of computing environments by statisticians is limited.
A merger of the knowledge bases would produce a powerful
force for innovation.
[*] Cleveland, William S. "Data science: an action plan for expanding the technical areas of the field of statistics.“, (2001)

10. By Capgemini Consulting, http://www.slideshare.net/capgemini/impact-of-big-data-on-analytics

11. Data scientist ?
• Sexiest job of the 21st century, according to Harvard Business Review, 2012
• A data scientist is many people in one, someone:
–who understands domain (industry/academia) needs and
terminology
–who is able to mash-up several analytical tools
–who is able to design and implement solutions to extract
knowledge from the data
–who can communicate findings

12. On data analysts – analyst types

13. On data analysts – skills vs. types

14. DATA SCIENCE PROCESS

15. DS Process
• Understand domain needs & formulate questions
• Collect & make data available
• Get the data ready for analysis
• Exploratively (and visually) analyse the data
• Model the phenomena (if needed)
• Evaluate findings
• Communicate findings
• ITERATE (from any stage to any other stage)!

16. DS Process – Data collection, efficient storing & access

17. DS Process – Getting data ready
Many names:
Data wrangling,
data munging,
data cleaning,
data massaging,
data scrubbing,
pre-processing, ….

18. I spend more than half of my time integrating,
cleansing and transforming data without doing any actual analysis.
Most of the time I’m lucky if I get to do any analysis. Most of
the time once you transform the data you just do an average... the
insights can be scarily obvious. It’s fun when you get to do something
somewhat analytical.
Kandel, Sean, et al. "Enterprise data analysis and visualization:An interview study.", IEEETVCG (2012)

19. DS Process – (Exploratively) Analysing data
Goal: Generate / confirm ideas, findings (hypotheses)
• Analysis tasks:
–Finding anomalies
–Finding relations
–Finding groups
–Summarizing information
–Making predictions
–Understanding uncertainties
–Evaluate hypotheses

20. DS Process – Build models
• Statistical models
–For summarising, representing data
–For predictions, estimations
• Machine learning methods, e.g., neural networks
–Predictive tasks
–Classification tasks
–Unsupervised / supervised models

21. Visualise & Communicate along the process!

22. ITERATE !

23. (SOME) CHALLENGES

24. HETEROGENOUS DATA

25. +
Text
Material by Tamara Munzner, http://www.cs.ubc.ca/~tmm/talks.html#minicourse14

26. Why a challenge?
• Computational methods do not work for all types of data
• Hard (cognitively) to link observations
• Gaps in the data (sparsity)
• Data is not necessarily linked!
[ by O’Donoghue et al., 2010]

27. BREADTH OF DATA

29. Why a challenge?
• Models become complex!
• Hard to interpret and utilise
• Less reliable findings, high uncertainy
Image from: http://www.visiondummy.com/2014/04/curse-dimensionality-affect-classification/
The curse of dimensionality

30. IMPERFECT DATA

31. This is a straight line that “connects the dots”, i.e., best fits the points.
Small exercise

32. Think of straight lines for these four sets

34. DYNAMIC
DATA (& CONCEPTS)

35. … Internet search behaviour changed during pH1N1,
particularly in the categories “influenza complications”
and “term for influenza.”The complications associated
with pH1N1, the fact that pH1N1 began in the summer
rather than winter, and changes in health-seeking
behaviour ….
13 February 2013,
http://www.nature.com/news/when-google-got-flu-wrong-1.12413

36. GENERATING
PLAUSABLE & USEFUL
HYPOTHESES

37. http://www.tylervigen.com/

38. HOW WE (TRY TO) DEAL WITH
THESE? -- EXAMPLES

40. STUDY 1 – MULTIVARIATE GEOGRAPHICAL DATA
Attribute
Signatures:
DynamicVisual
Summaries for
Analyzing Multivariate
Geographical Data
CagatayTurkay,Aidan Slingsby,
Helwig Hauser, JoWood, Jason
Dykes, InfoVis 2014
UK Census of Population in 2001 and 2011 for the
181,000 Output Areas (OA)
for 41 indicators

41. How all the variables
vary over space (and time)?
question is…

42. interactively
generate
visual summaries of change
in all the variables
in response to variation
(location, extent, resolution)
we built methods to:

43. Attribute signatures
It starts with a map …
and an attribute to analyze,
e.g., unemployment rates

44. Attribute signatures
Above baseline

45. Attribute signatures
~ baseline (country average)

46. Attribute signatures
Below baseline

47. Attribute signatures

48. Attribute signatures
A dynamically generated
visual summary!

49. How about several attributes?
Linked
small
multiples

51. Along the Southwest England coast

52. 2011 vs. 2001

53. STUDY 2 – HOW TO CHARACTERIZE CANCER SUBTYPES?
Characterizing
Cancer Subtypes
Using Dual Analysis
in Caleydo
StratomeX
CagatayTurkay,Alexander Lex, Marc Streit,
Hanspeter Pfister, Helwig Hauser, IEEE CG&A
2014

54. Patients(samples)
Genes
Candidate Subtype /
Heat Map
Cancer Subtypes are identified
by grouping datasets based on
• gene activity
• mutations
• or a combination of these
http://caleydo.org/
Colour to show
gene activity

55. There are always many ways to group!
Group A1
Group A2
Group A3
B1
B2
Grouping 3,
Group. C1
Group. C2
Grouping 1, Grouping 2,

56. What are the groups & Why?
Many shared Patients
Clustering 1 Clustering 2
Sample Overlaps
GeneOverlaps??

57. Finding distinctive properties

58. VISUAL ANALYSIS
TO FACILITATE THE
ITERATIVE PROCESS &
HYPOTHESES GENERATION

59. Some remarks on Future
• Infrastructure & speed – better solutions?
• More on “Value of the data”
• More sophisticated analyses
• Predictive analytics on the rise
• More central role for the user (through visualisation)
• New sources of data
–Increasingly you! , health apps, etc.
–Internet of things

60. Data science is a process starting with
formulating a question that can be answered with
data, and iteratively collecting, cleaning,
analysing and modelling the data, while
communicating the answers to the relevant
audience along this iteration.
To conclude ….

61. Some technologies (we use in teaching DS)
• Python
• Pandas for Statistical Computing
• Scikit-learn for machine learning
• R Statistical Computation Software
• Apache Spark
• Tableau Software

62. Further reading …

63. Thank you !
Cagatay.Turkay.1@city.ac.uk
@cagatay_turkay
http://staff.city.ac.uk/cagatay.turkay.1/ More data science?
Data Science MSc @ CITY