Check out more about this presentation here: https://www.youtube.com/watch?v=eqNToHn4yB0 The webinar was organized by GetinData on 2020. During the webinar we explaned what does it mean to build a data-driven company. Watch more here: https://www.youtube.com/watch?v=eqNToHn4yB0 Speaker: Rafał Małanij ___ Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets. Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries. https://getindata.com​

1. Analytics 101
How to build a data-driven organisation?

2. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
■ 13+ yrs in IT
■ IT Service Management, Project
Management, Business development
■ Cloud Native, DevOps, Data Science, Big
Data, Genomics
■ Involved in:
● PyData Warsaw
● Data Science Summit
● DevOps Days Warsaw
● Cloud Native Warsaw
Rafał Małanij
rafal.malanij@getindata.com

3. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Founded in 2014 by
ex-Spotify engineers.
Focus only on Big Data and
Cloud (from day 1)
Community builders (Big Data
Tech Warsaw organizers)
60+ Big Data engineers

4. © Copyright. All rights reserved. Not to be reproduced without prior written consent.

5. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Digital Transformation

6. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Big Data and AI Executive Survey 2020
by NewVantage Partners

7. © Copyright. All rights reserved. Not to be reproduced without prior written consent.

8. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
“Technology is the engine of digital transformation,
data is the fuel, process is the guidance system, and
organizational change capability is the landing gear.”
https://hbr.org/2020/05/digital-transformation-comes-down-to-talent-in-4-key-areas

9. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
“We live in a society that is data rich and
information poor.”
“In Search of Excellence: Lessons from America's Best-Run Companies”
January 1, 1982, Jr. Robert H. Waterman

10. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data-first mindset

11. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data Democratization
“If you torture the data long enough,
it will confess.”
Ronald H. Coase, Essays on Economics and Economists

12. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data University @ AirBnB

13. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data University @ AirBnB

14. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data Literacy
Data literacy is the ability to read, understand,
create, and communicate data as information.

15. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
http://www.tylervigen.com/spurious-correlations

16. © Copyright. All rights reserved. Not to be reproduced without prior written consent.

17. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Lean startup
■ Build-Measure-Learn
■ Iterative approach
■ MVP approach
■ Continuous deployment
■ Split testing

18. © Copyright. All rights reserved. Not to be reproduced without prior written consent.

19. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Holistics / dbdiagram.io
■ 2 interns
■ 2-month project
■ Brand awareness
■ Lead Generation
■ Feedback
■ Public Road Map
■ 200k users (1yr)!!!

20. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
https://www.holistics.io/blog/launch-a-product-to-200-000-users-with-0-dollars-spent-on-marketing-budget/

21. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
https://about.gitlab.com/handbook/business-ops/data-team/

22. © Copyright. All rights reserved. Not to be reproduced without prior written consent.

23. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Low
Risk
Predictable approach
Established processes
Experimenting, failing and
learning from your failures

24. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
“It's not necessary
to change. Survival
is not mandatory.”
William E. Deming

25. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
■ Self-service analytics
■ Citizen Data Science
■ Continuous Delivery
■ Data Integrity
■ Use case-driven
■ Security
https://marriagebrokerauntie.com/2017/10/27/fail-if-you-have-to-but-fail-fast/

26. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Q&A

27. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Q&A

28. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
“Without data you’re just another person with an opinion” — W.
Edward Deming
https://towardsdatascience.com/do-you-really-have-a-data-strate
gy-ff08795f10ce

29. Analytics 101
How to build a data-driven organisation

30. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Big Data
■ Volume
■ Variety
■ Velocity
■ Veracity
https://upload.wikimedia.org/wikipedia/commons/e/ee/Big_Data.png

31. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
“Big data isn't a one-off project: It's a culture of
collecting, analyzing, and using data.”
Matt Asay, Infoworld.com
■ Not technology issue
■ Start small
■ Ask the right questions
■ Flexible, open data infrastructure

32. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data
Collection
Data
Storage
Processing Delivery
Clickstream
Mobile apps
Product systems
Transaction system
CRM
Call center
Workforce mgmt

33. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data Lake
■ Repository for raw data
■ Various type of data
● Structured
● Semi-structured
● Unstructured
● Binary
■ Historical data
vs.

34. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Continuous
Data
Collection
Automation Security Monitoring Orchestration
Data Lake
Big Data
Processing
Data
Governance
Event
Processing
Feature
engineering
Interactive BI
& Analytics
Data
Discovery
Data Science
Machine
Learning

35. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data Lineage
■ Where it comes from
■ What happened
■ Where they
are used

36. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Competing on Analytics: The New Science
of Winning
by Thomas H. Davenport, Jeanne G. Harris

37. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data Science vs Machine Learning

38. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Machine Learning

39. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Machine Learning

40. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Machine Learning vs A.I.
“Artificial intelligence is
the science and engineering
of making computers behave
in ways that, until recently,
we thought required human
Intelligence.”
Andrew Moore,
Carnegie Mellon University,

41. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Continuous
Data
Collection
Automation Security Monitoring Orchestration
Data Lake
Big Data
Processing
Data
Governance
Event
Processing
Feature
engineering
Interactive BI
& Analytics
Data
Discovery
Data Science
Machine
Learning

42. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
DevOps vs DataOps
Culture
Automation
Lean
Measurement
Sharing
+ Data quality
+ Manufacturing process
https://www.dataopsmanifesto.org/

43. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Continuous
Data
Collection
Automation Security Monitoring Orchestration
Data Lake
Big Data
Processing
Data
Governance
Event
Processing
Feature
engineering
Interactive BI
& Analytics
Data
Discovery
Data Science
Machine
Learning

44. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Continuous
Data
Collection
Automation Security Monitoring Orchestration
Data Lake
Big Data
Processing
Data
Governance
Event
Processing
Feature
engineering
Interactive BI
& Analytics
Data
Discovery
Data Science
Machine
Learning

45. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Competing on Analytics: The New Science
of Winning
by Thomas H. Davenport, Jeanne G. Harris
Technical
competences
Possibilities

46. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Interactive BI
■ Reports
■ Dashboards
■ Drill-down reports
■ SQL-queries
■ Tools: Excel, PowerBi,
QlikView, Tableau, Superset,
Hive, Presto

47. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data Science
■ Transformed and Raw data
■ Machine Learning
■ Tools: Jupyter,
Spark, Scala/Java
R, Python
Tensorflow, etc.

48. © Copyright. All rights reserved. Not to be reproduced without prior written consent.

49. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data Discovery
■ Search tool for data
■ What, where, who?
■ Metadata
■ Popularity score
■ Quality and profiling

50. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Lexikon@Spotify
■ Library for data and insights
■ Knowledge Mgmt tool
● People
● Description, stats
● Tables, Queries
https://engineering.atspotify.com/2020/02/27/how-we-improved-data-discovery-for-dat
a-scientists-at-spotify/

51. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Continuous
Data
Collection
Automation Security Monitoring Orchestration
Data Lake
Big Data
Processing
Data
Governance
Event
Processing
Feature
engineering
Interactive BI
& Analytics
Data
Discovery
Data Science
Machine
Learning

52. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Źródło: “Continuous Analytics: Stream Query
Processing in Practice”, Michael J Franklin,
Professor, UC Berkley, Dec 2009 i
https://www.slideshare.net/JoshBaer/shortening
-the-feedback-loop-big-data-spain-external

53. © Copyright. All rights reserved. Not to be reproduced without prior written consent.

54. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Continuous
Data
Collection
Automation Security Monitoring Orchestration
Data Lake
Big Data
Processing
Data
Governance
Event
Processing
Feature
engineering
Interactive BI
& Analytics
Data
Discovery
Data Science
Machine
Learning

55. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Hidden Technical Debt in Machine Learning Systems -
https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf

56. © Copyright. All rights reserved. Not to be reproduced without prior written consent.

57. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
https://mattturck.com/data2019/

58. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data-driven insurer
■ Customer Data Platform - Customer360
■ Better understanding of your customer - LTV
■ Personalised marketing
■ Personalised offer, recommendation systems
■ Customer engagement, churn
■ Customer journey, lead scoring
■ Marketing costs and sales funnel optimization

59. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data-driven insurer
■ Customer Data Platform - Customer360
■ Internal data (transactions, CRM, call-center)
■ Clickstream
■ External data

60. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data-driven insurer
■ Personalised
products
https://www.the-digital-insurer.com

61. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data-driven insurer
■ Personalised products
■ Lifestyle information
● Wearables
● Mobile apps
■ Healthcare data
● Diabetes monitoring
● Genomics

62. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data-driven insurer
■ Internet of Things
■ Sensors
● Fire
● Water Sensors
■ Telematics
https://www.the-digital-insurer.com

63. © Copyright. All rights reserved. Not to be reproduced without prior written consent.

64. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Dataism
“Dataism declares that the
universe consists of data flows,
and the value of any
phenomenon or entity is
determined by its contribution
to data processing,”
Yuval Noah Harari, “Homo Deus”.

65. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Q&A