Watch our full performance given by our team during the Big Data Technology Warsaw Summit: https://www.youtube.com/watch?v=CBrq7z8ikaM The nature of Big Data projects are nowadays one of its kind - they are not like the data warehousing initiatives in the old days, nor like cloud native applications projects, at least not yet. Variety of technologies, complicated architectures and rapidly changing landscape are just a few challenges that the IT Department is facing in such projects. When you add the number of stakeholders from different departments involved and that Big Data project is sometimes more like an R&D with unpredictable outcome, this makes a mix where the objectives can be easily lost. It is not a surprise that up to 85% of Big Data projects were pure failures (Gartner 2016). In this talk we will share our experience in planning and executing Big Data initiatives in the organisations, with some use cases and good practices in mind Watch our webinar here: https://www.youtube.com/watch?v=CBrq7z8ikaM Speakers: Rafał Małanij Rafał Zalewski Linkedin: https://www.linkedin.com/in/rafalzalewski/ ___ Company: Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets. Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries. https://getindata.com​

1. Managing Big Data
projects in a constantly
changing environment
Rafał Zalewski & Rafał Małanij

2. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
■ Over 12 years' experience in IT Project
Management and Software Development
■ Technical background. Fascinated in cloud
infrastructure and cloud solutions.
■ Certified:
● Agile Project Management AgilePM® Practitioner
● Professional Scrum Master™
● Professional Scrum Product Owner™
● PRINCE2® in Project Management
● ITIL® 4 in IT Service Management
● AWS Certified Solutions Architect Associate
Rafał Zalewski
IT Project Manager
rafal.zalewski@getindata.com

3. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
■ 13+ yrs in IT
■ IT Service Management, Project Management,
Business development
■ Cloud Native, DevOps, Data Science, Big Data,
Genomics
■ Involved in:
● PyData Warsaw
● Data Science Summit
● DevOps Days Warsaw
● Cloud Native Warsaw
Rafał Małanij
rafal.malanij@getindata.com

4. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Telco DWH Model
Source:
http://dwh-models.com/telecommuni
cations/telco-dwh-model-product/

5. © Copyright. All rights reserved. Not to be reproduced without prior written consent.

6. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Novelty / Bet
Advantage
/
Risk

7. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Set of experiments:
1. Research question
2. Data
3. Hypothesis
4. Methodology
a. What data?
b. How to collect / ingest?
c. How to analyse
5. Measure

8. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Hidden Technical Debt in Machine Learning Systems -
https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf

9. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Data Science
Machine
Learning
Engineering
Backend
Engineering
Data
Engineering
DevOps
Data
Visualisation
Data
Analytics

10. © Copyright. All rights reserved. Not to be reproduced without prior written consent.

11. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
2019 2020

12. © Copyright. All rights reserved. Not to be reproduced without prior written consent.

13. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Novelty / Bet
Advantage
/
Risk
Learn
Learn

14. © Copyright. All rights reserved. Not to be reproduced without prior written consent.

15. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Big Data project vs Software development project
Big Data project Software Development project
Less understandable by client More understandable by client
Highly oriented on a data, analysis
and use cases
Highly oriented on covering known
business functionalities
More focused on research More predictable scope
New infrastructure Current infrastructure

16. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
The Client has little knowledge on:
● Big Data available software
● Big Data architecture
What you should do as Project Manager?
● Include trainings in your project scope
● Use small MVP’s where Client can try them out
● Work closely with Client’s analysts and business
Big Data Project Risks

17. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● Unorganized data
● Data storage and retention
● Cost management
● Incompetent analytics
● Data privacy
Big Data Project Risks

18. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● Create regular meetings with clients analysts and business
● Make sure the business understands approach
● Prepare to change when client knowledge increases
● Make sure business sees ROI from your work
● Focus on real use cases
● Work in iterations
● Create regular demos for the client so they see the real
working product and benefits of it
Big Data Project Management – Go With Agile

19. Thank you!