Of course, you know what data is. Probably you know what Big Data and small data is. But what's the heck is that buzz about data? Why is it so important today? These are the questions which will be the topic of the session. This session will be beyond the definitions and descriptions. We will talk about data, about different options for data usage, and how we can benefit from data.

1. Data-Driven Future
What to Learn and What to Expect?
Denis Reznik
Data Architect at Intapp Kyiv
Microsoft Data Platform MVP

2. About me
•Denis Reznik
•Kyiv, Ukraine
•Data Architect at Intapp, Inc.
•Microsoft Data Platform MVP
•Co-Founder of Ukrainian Data Community
2 |

3. Agenda
•Data is a new Oil (c)
•Data and Science
•Data in Big Companies
•Data and Application Development
•Data-Driven Future

4. Data is a New Oil
“Data is the new oil. It’s valuable, but if unrefined it
cannot really be used. It has to be changed into gas,
plastic, chemicals, etc to create a valuable entity that
drives profitable activity; so must data be broken
down, analyzed for it to have value.”
(c) Clive Humby, UK Mathemetician

5. Data and Science
•Thousands of years
•Empirical
•Few hundreds of years
•Theoretical
•Last fifty years
•Computational
•“Query the world”
•Last twenty years
•eScience (Data Science)
•“Download the world”

6. Machine Learning
Supervised Learning Unsupervised Learning
Classification Regression

7. Linear Regression
Learning Algorithm
Training Data
h
h - Hypothesis
Ocean
Temperature
Whales
Population

8. DEMO
Linear Regression

9. Data in Big Companies

10. Parallel Processing
Temperature Sensor Datasets (n Items)
Q: How many times temperature was above
the norm during the last week?
A: 5
Time: 2 sec
Algorithmic Complexity: O(n)

11. Parallel Processing
Temperature Sensor Datasets (k Items in each one)
Q: How many times temperature was above
the norm during the last week?
A: 1
Time: 0.5 sec
Algorithmic Complexity: O(n/k)
A: 0 A: 3 A: 4

12. Map-Reduce
A: 1
Map -> COUNT(*) WHERE Value > 40
A: 0 A: 3 A: 4
Reduce -> COUNT(*)
A: 5
Reduce

13. DEMO
Map-Reduce

14. Data and Application Development
source: https://www.youtube.com/watch?v=t6kM2EM6so4

15. Index (B-Tree) - Seek
…
…
1 .. 1M
1 .. 2K 2K+1 .. 4K
1M-2K .. 1M
1 .. 300 301..800 801..1,5K 1,5K+1..2K
SELECT * FROM Users
WHERE Id = 523

16. Index (B-Tree) - Scan
…
…
1 .. 1M
1 .. 2K 2K+1 .. 4K
1M-2K .. 1M
1 .. 300 301..800 801..1,5K 1,5K+1..2K
SELECT * FROM Users

17. Index (B-Tree) - Range Scan
…
…
1 .. 1M
1 .. 2K 2K+1 .. 4K
1M-2K .. 1M
1 .. 300 301..800 801..1,5K 1,5K+1..2K
SELECT * FROM Users
WHERE Id BETWEEN
700 AND 1700

18. Hashtable
John Dow
John Snow
Jack Snack
2
3
1
4
0
John Dow
Hash Function
0
Jack Snack
2
John Snow
0

19. Data-Driven Future
•Data amount is growing and this is cool
•More and more decisions are based on data
•More and more applications are developed
•It is exciting to be a Software Engineer now!

20. Thank you!
Denis Reznik
Twitter: @denisreznik
Email: denisreznik@live.ru
Blog: http://reznik.uneta.com.ua
Facebook: https://www.facebook.com/denis.reznik.5
LinkedIn: http://ua.linkedin.com/pub/denis-reznik/3/502/234