The Art of Data Visualisation Challenges, motives, trends, history, use cases and some of my own original work in the field.

1. @dmarcous

2. “Data Is The New Oil”

3. “Data Is The New Soil”
David McCandless

4. The 19th century culture was defined by the novel,
he 20th century culture was defined by the cinema, and the 21st
entury culture will be defined by the interface”
Lev Manovich, Visual arts professor and media theorist

5. Steps of Data Analytics (Science)
●Defining question
●Data Exploration
●Data Gathering
●Data Preparation
●Data Analysis
●Data Visualisation

6. @dmarcous
●@IDF
DBA (DataBase Administrator)
Big Data Professional
●@Google – Waze
Data Wizard - Magic with Data

10. Old Roots

11. Charles Joseph Minard
French civil engineer
Born: March 27, 1781
Died: October 24, 1870

13. Lon
-5 0 5 10
42
44
46
48
50
Lat
@dmarcous

15. Edward Tufte
●Visual Display of Quantitative Information
●Beautiful Evidence

16. Data-Ink Ratio

21. Grammar of graphics
A graphic is a mapping:
from data - to aesthetic attributes (colour, shape,
size) of geometric object (points, lines, bar).
Leland Wilkinson, 1999

22. ggplot2 package
Hadley Wickham
Assistant professor of statistics at Rice University, Houston

23. Technical Problems -
Large Result Set
Real Time results

24. Serving Database
Indexed Results
Cached Results
Slice Data in Memory
Build Graphs In Memory

25. Combined Architecture
●Batch Processing - Hadoop / Spark
●Stream Processing - Storm / Spark Streaming
●Serving / Online Indexing - HBase / solr / elastic / mongo
●Visualise - R / python /+ javascript (D3.js)

26. Dimensions of Data
NumbersNumbers
DistanceDistance
ShapesShapes
LocationLocation
OpacityOpacity SizeSize
WordsWords

27. Amount of data VS Human Readability
We don’t want only matter experts to be able to understand
the visualisation

28. Techniques

29. Case Study 1
●Bush VS Gore election results
●Divided by Ethnicity (white/ black/ hispanic)
●Divided according to size of population

30. Red - Bush
Black - Gore

31. Case Study 2
●Display specific area attack patterns
○ Time of attack
○ Intensity of response (we want to minimize casualties from our side)
●Take seasonality into account (day of week)
●Display level of confidence
●Bonus - Add “automatic pattern discovery” / Recommendation
for attack time

33. Case Study 3
●Most Photographed Places on Earth
●Personalise to specific user taken photos
●Compare specific user photo location to popularity of location

35. Case Study 4
●What customers do with my app?
●Distribution of entry points
●“Paths to success”

38. Case Study 5
●Detect and present Anomalies in data (machine data / user
patterns / irregular traffic etc’)
○ Anomaly = out of ordinary.
○ Learn “normal”
○ Detect Global anomalies
○ Detect normal anomalies
○ Detect change of trends

39. Twitter Breakout Detection

40. Case Study 6
●Assess and display the effect of an event on a certain metric
○ Learn normal pattern
○ Predict according to normal pattern
○ compute residuals (predicted-actual)
●Did the storm affected number of accidents?
●Did a military action affected number of missiles sent?

41. Google Causal Impact

42. Case Study 7
●Build a stock portfolio with minimised risk
●Pick stocks that are doing well (easy…)
●Pick stocks that act different from each other (correlations)

43. Clustering Analysis

44. Interactive Visualisation

46. Data Art

48. data-waze.com

49. Open source is good for me, I will fully embrace it.
Open source is good for me, I will fully embrace it.
Open source is good for me, I will fully embrace it.
Open source is good for me, I will fully embrace it.
Open source is good for me, I will fully embrace it.
Open source is good for me, I will fully embrace it.
Open source is good for me, I will fully embrace it.
Open source is good for me, I will fully embrace it.
Open source is good for me, I will fully embrace it.
Open source is good for me, I will fully embrace it.
Open source is good for me, I will fully embrace it.
Open source is good for me, I will fully embrace it.
Open source is good for me, I will fully embrace it.

51. Contact
●Daniel Marcous
●dmarcous@gmail.com