Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Dataiku - data driven nyc - april 2016 - the solitude of the data team manager


Published on

Talk at Data Driven NYC about the solitude of the data team manager

Published in: Software
  • Making a living taking surveys at home! I have been a stay at home mom for almost 5 years and I am so excited to be able to still stay home, take care of my children and make a living taking surveys on my own computer! It's so easy to get started and I plan to make enough money each week so that my husband can actuallly quit his second job!!! Thank you so much! ♥♥♥
    Are you sure you want to  Yes  No
    Your message goes here
  • This Single Mother Makes Over $700 per Week Helping Businesses with their Facebook and Twitter Accounts! and Now You Can Too! ▲▲▲
    Are you sure you want to  Yes  No
    Your message goes here

Dataiku - data driven nyc - april 2016 - the solitude of the data team manager

  1. 1. Data Science Software Platform The Solitude Of The Data Team Manager Data Driven NYC 04/11/16
  2. 2. My nerdy background Type Systems Automated Proving Abstract Program Interpretation Functional Programming Garbage Collection and Vms Graph Analytics Chess IA Natural Language Processing 80% Emacs /20% VIM
  3. 3. So to sum it up … 3 I (USED TO) HATE GUIs
  4. 4. …and our software is A supercharged Visual IDE for Data Teams Deployment
  5. 5. Meet HAL Hal AlowneBI Manager Dim’s Private Showroom
  6. 6. Meet Hal’s Boss, DIM Hey Hal ! We need a big data platform like the big guys. Just do what they’re doing! ‟ ” Big Data Copy Cat Project
  7. 7. TECHNOLOGY Disconnect What technologies should I use ?‟ ”
  8. 8. Welcome to Technoslavia ! 8
  9. 9. TOY PLATFORM ANTI-PATTERN 9 Test and Invest in Infrastructure == Skilled People or Go For Cloud / Packaged Infrastructure Your Brand New Hadoop Cluster is perceived as slow, not so used and not reliable
  10. 10. TECHNO MISMATCH ANTI-PATTERN 10 Assume Being Polyglot or Be a Dictator VS VS The Python Clan The R Tribe The Old Elephant Fraternity The New Elephant Club
  11. 11. PREDICTIVE ANALYTICS DEPLOYMENT STRATEGY 11 Website 2000’ winners Companies that were able to release fast "Artificial Intelligence with Data for Internet of Things" 2010’ winners Companies able to put intelligence in production ? Design a way to put “PREDITICTIVE MODELS” IN PRODUCTION
  12. 12. PEOPLE Disconnect Who should I hire ? ‟ ”
  13. 13. Classic Business Intelligence Team Organization Business Leader Data Consumer Line-of-business Data Consumer Business Project Sponsor BI Solution Architect Model Designer ETL Developer Dashboard / Report Designer Specs Dim Big Boss
  14. 14. Data Science Team Organization Business Leader Data Consumer Line-of-business Data Consumer Business Project Sponsor Data Engineer Data Analyst System Engineer / Data Architect Business Needs Data Scientist IT Constraints I.T.
  15. 15. Manage Expectations 15 Data Plumberer Data Engineer Data Scientist Data Waiter Data Cleaner Data Analyst REAL JOB DREAM JOB
  16. 16. Managing Extreme Personalities 16 Data Scientist Highly Creative Passionate Hard to hire? Hard to manage? Want to take Hal’s job?Ambitious Hard to retain?
  17. 17. Paired for Data 17 Data Analyst Discover Patterns Data Engineer Make things work Fight data entropy Fight tech entropy
  18. 18. What do you prefer? 18 One Analyst One Engineer One Data Scientist Four data scientists OR
  19. 19. Two Mindsets Can Coexist CLICKERS CODERS
  20. 20. DATA Disconnect What about data ? ‟ ”
  21. 21. What is the main reason for data project to fail ? 21 > DATA NOT AVAILABLE
  22. 22. BUT FOR ONLY INCREMENTAL GAIN 50 30 20 0% 25% 50% 75% 100% Contribution to the overall project performance Business Goal Definition and Data Feature Engineering Algorithm
  23. 23. How to Get Data if you don’t have it 23 THE CICADA THE SPIDER THE FOX Be Optimistic ! Wait for Open Data Initiatives And data available in the Enterprise Hub / Data Lake ! Create Network ! Create a set of trackers or Addictive Data Collection internally To get Data on your side ! Hunt for Big Problem! Convince the CEO that you can Solve a Business Critical problem And use it as an excuse to get all The data you want !
  24. 24. PRODUCT Disconnect
  25. 25. What is Big Data about ? 25
  26. 26. The Age Of Distributed Intelligence 26 Global, Personalised and Real Time Data Driven Services
  27. 27. Data to Visualize or Data to Automate ? 2013 2014 2015 2016 2017 2018 Moving to a world of automated decision making 27 DATA FOR MORE INSIGHTS DATA FOR AUTOMATED DECISIONS
  28. 28. Involve Product Team 28 Product Feature Personalised Item Ranking Product Feature Notify User Only when Needed Product Feature: Historical Data For Path Optimisation Have Product Management Deeply Involved In the Data Team
  29. 29. Focus on your added value 29 Build by the DataTeam Is the problem at the Core of my BusinessProcess? Is it a common problem / with share data? Can i solve it on my own? Really? Hire Consultant and Learn Build by the Data Team Go for Best of Breed SaaS Solution Build by the Data Team? YesNo No Yes No Yes No Yes
  30. 30. Create an API culture Do not share o Random Piece of Code o Flat File o Email Do share ü Reproductible documentedworkflows ü Clean, documentedAPIs
  31. 31. Did Hal found his solutions ? Technology Data People Product Polyglot on top of open source Find a way to make clickers and coders work together Create an API culture and involve the product teams Hunt for Big Problems and Convince the CEO Is this the end ?‟ ” Hal Alowne BI Manager Dim’s Private Showroom
  32. 32. That was the (romanced) story… data scientists and engineers25 2 locations 1 software by the numbers For a simple software for clickers and coders
  33. 33. Car Sharing Worldwide Leader FlashSales Worldwide Leader One Mission : Never leave Hal ALONE 3,700 Hotels Worldwide 2500 lovely users by the numbers 70 customers
  34. 34. Food for thoughts THANK YOU ! FREE (as in Beer) Software