Google Data Studio for business
Your SlideShare is downloading.
×
Check these out next
Recommended
More Related Content
Similar to Optimise your data pipeline without rewriting it - Big data conference Vilnius (20)
Recently uploaded (20)
Optimise your data pipeline without rewriting it - Big data conference Vilnius
- 1. Optimize your data pipeline without rewriting it Magnus Runesson
- 2. “How did you do that?”
- 3. Who am I?
- 4. We are the rails and brains of open banking Trusted by the industry leaders • Natwest • BNP Paribas Fortis • ABN AMRO • PayPal • Klarna • SEB 260 Employees across Europe Local offices in • Sweden • Finland • Denmark • United Kingdom • Germany • Netherlands • Spain • France • Poland 2,500 + banks & FIs connected We’re connected for access to all types of accounts, from banks, neo-banks, credit cards and more, and bring them together for you via a single, beautiful API. Industry Authority Member of EU, Berlin Group & Open Banking advisory boards ISO/IEC 27001 Certified PSD2 Licensed 2,000 + Platform users
- 5. Agenda Introduction The problem The process to success Summary Proprietary & confidential
- 6. It is not fast enough!
- 7. The process
- 8. Visualize
- 9. Database DB to files on storage bucket Load to bigquery Database Google BigQuery
- 10. 00 03 06 09 12 15 18 21 DB to files on storage bucket Load files to bigquery
- 11. Requirements
- 12. Break down
- 13. 00 03 06 09 12 15 18 21 DB to files on storage bucket Load files to bigquery
- 14. Table 1 Table 2 Table 3 Table 4 Table 1 Table 2 Table 3 Tab 4 00 03 06 09 12 15 18 21
- 15. Parallelise
- 16. Table 1 Table 2 Table 3 Table 4 Table 1 Table 2 Table 3 Tab 4 00 03 06 09 12 15 18 21
- 17. Table 1 Table 2 Table 3 Table 4 Table 1 Table 2 Table 3 Tab 4 00 03 06 09 12 15 18 21
- 18. Remove idle time
- 19. Table 1 Table 2 Table 3 Table 4 Table 1 Table 2 Table 3 Tab 4 00 03 06 09 12 15 18 21
- 20. Table 1 Table 2 Table 3 Table 4 Table 1 Table 2 Table 3 Tab 4 00 03 06 09 12 15 18 21
- 21. Find bottleneck
- 22. Table 1 Table 2 Table 3 Table 4 Table 1 Table 2 Table 3 Tab 4 00 03 06 09 12 15 18 21
- 23. Task 1 Task 2 Task 6Task 3 Task 5 Task 4 Time Bottleneck example
- 24. Task 1 Task 2 Task 6Task 3 Task 5 Task 4 Time Bottleneck example - critical path
- 25. Zoom in
- 26. Process recap
- 27. Visualize Understand requirements Break down Parallelise Remove idle time Find bottlenecks Zoom in
- 28. Remember ● Use dependency based scheduling ● Look on the metrics ● Use metrics to make your stand ● Work with the bottleneck once entering the code ● Stop when good enough
- 29. Thank you! (Psst… come work with us!) Magnus.Runesson@tink.se @MRunesson
