This document discusses how to do data science and analysis for everyone, including those without advanced technical skills. It provides examples of accessible datasets to use, such as international and Estonian government open data. Tools like Excel, Google Sheets, R, and Python are presented as options for getting, cleaning, exploring, visualizing and publishing data. Specific R libraries and code examples are provided. The document also discusses strategies for presenting results to audiences through static formats, interactive visualizations using D3.js, blogs, social media and working with the Estonian data science community. Newsletters and blogs are recommended for continued inspiration.

1. Data science
for everyone
Taivo Pungas
29.03.2016

3. For everyone?
● Manual/one-off analyses
○ Not production-level code
● Personal interest
● No deep learning
Survey:
● PhD [student]?
● >1 stats/ML class?
● Can code?

4. Finding
interesting data#1

5. select * from users
Job Experience

6. 1. Existing datasets
int’l organisations
Kaggle
opendata.riik.ee
→ R library (WIP)

7. 1. Existing datasets
2. Scrape
kv.ee/2724720 kv.ee/2724721

8. 1. Existing datasets
2. Scrape
check ToS
don’t DoS
KV, Postimees, Auto24, Osta, …
1M Estonian real estate ads

9. 1. Existing datasets
2. Scrape
3. Quantified Self

10. QS: how did I feel today?
very goodvery bad
count
OK

11. Process & tools for
analysis#2

12. Get data
Clean
Explore /
visualise
Publish
My process
Sleep on it
feedback

13. Python
general purpose data processing
Hadleyverse
Rvs
pandas
matplotlib
scikit-learn

14. R libraries: Hadleyverse
Step Libraries
Get data rvest, xml2, readxl
Clean dplyr, tidyr, stringr
Explore / visualise ggplot2
Publish
... and many others
from Hadley Wickham.

15. Alternatives
Excel / Google Sheets
● External data sources
● Google Apps Script
○ Google Translate API
○ Sending e-mail
○ …
● Not easy to reproduce analyses
Tons of other software
R & Python worth the learning curve

16. R is easy: reading data
# Read data
apartments <- read.csv2("data/apartment_rent_tartu.csv",
sep=";", header=TRUE)

17. R is easy: dplyr
library(dplyr)
# Find average price by part of city
apartments %>%
group_by(Linnaosa) %>%
summarise(KeskmineHind=mean(HindKohandatud)) %>%
arrange(desc(KeskmineHind))

18. R is easy: lin. regression
# Build linear model
fit <- lm(HindKohandatud ~ Tube, data=apartments)
summary(fit)

19. Presenting your
results#3

20. interactive > static

21. D3.js: powerful web visualisations

22. Easy to useHard to use
Limited
Powerful
ggplot2
D3.js
D3 derivates
Excel
GSheets
AI

23. How to reach an audience
● Social media
● Start a blog
○ stat24.ee
○ pungas.ee
● Offer free content
○ Newspapers (tip lines)
○ Guest posts on blogs
● Push to Estonian data science community
○ TODO: FB group? Community blog?

24. Putting it together
##

25. Examples
Apartment prices: R + D3.js 18k hits
Salaries of public servants: R + D3.js 38k hits
Study data: R + D3.js 3k hits
Election promise calculator: D3.js 42k hits
Bondora: R
Alcohol deaths: Illustrator

26. News & inspiration
Mailing lists:
Information is Beautiful, Data Science Weekly, Data Elixir
Blogs:
FiveThirtyEight, R-bloggers, Stat24, Mike Bostock

27. Long-term motivation
Flickr: ucirvine / CC BY-NC-ND

28. pungas.eetaivo@
tpungas