The Science of Managing Data Scientists
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

The Science of Managing Data Scientists

  • 58,231 views
Uploaded on

Creating great products powered by big data can be challenging. ...

Creating great products powered by big data can be challenging.

Data science work is often ambiguous, which can make results unpredictable and scheduling almost impossible.

Many of the popular software engineering processes just won’t work for these innovative and ambitious projects. Waterfall falls apart; it just doesn’t make sense to define the product before understanding the limitations of the data and technology. And shoehorning data experiments into tight agile sprints is both difficult and doesn’t necessarily lend itself to discoveries that involve a lot inspiration and perspiration before a light bulb moment.

Even with a working process, few teams collaborate truly effectively. Projects that involve machine learning, algorithm development, or other deeply technical endeavors, are filled with advanced math and complicated terminology, which leaves plenty of teams with communication gaps that prevent the synergy realized when working cohesively.

Thankfully there are solutions to these problems! Based on personal experience, and interviews with many other leaders spearheading big data initiatives, this session aims to distill these lessons into actionable strategies you can use to improve process and communication for your own team.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
58,231
On Slideshare
50,365
From Embeds
7,866
Number of Embeds
45

Actions

Shares
Downloads
189
Comments
0
Likes
29

Embeds 7,866

http://es.slideshare.net 2,608
http://www.marketingdistillery.com 2,501
http://fr.slideshare.net 1,050
http://igorsubbotin.blogspot.ru 656
http://de.slideshare.net 596
https://twitter.com 85
https://www.linkedin.com 52
http://translate.googleusercontent.com 51
https://folio-qa-folio.murgler.org 45
http://feedreader.com 36
http://www.linkedin.com 28
http://filigranada.blogspot.com.es 27
http://frpar1web04 22
http://eventifier.co 19
http://localhost 14
https://igorsubbotin.blogspot.com 9
http://feedly.com 5
http://news.google.com 5
http://nuevospowerpoints.blogspot.com 5
http://igorsubbotin.blogspot.com 5
http://demo.azzist.com 4
http://www.newsblur.com 4
http://eventifier.com 4
http://nuevospowerpoints.blogspot.mx 4
https://w-qa-folio.murgler.org 3
http://informatica4a102013.blogspot.com.es 3
http://informatica4a022013.blogspot.com.es 3
http://nuevospowerpoints.blogspot.com.es 3
http://www.iessoller.com 2
http://kapilapshankar.com 2
http://www.ipv6.eventifier.co 1
http://igorsubbotin.blogspot.de 1
http://www.google.com 1
https://www.rebelmouse.com 1
http://www.google.com.au 1
http://pinterest.com 1
http://honyaku.yahoofs.jp 1
http://newsblur.com 1
http://translate.googleuserconten 1
http://digg.com 1
http://ec2-54-243-189-159.compute-1.amazonaws.com 1
http://10.32.0.65 1
http://10.100.0.241 1
http://filigranada.blogspot.com 1
http://informatica4a102013.blogspot.mx 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The Science of Managing Data Scientists Kate MatsudairaTuesday, February 26, 13
  • 2. Tuesday, February 26, 13
  • 3. What are they doing all day?Tuesday, February 26, 13
  • 4. Data science is different http://data.whicdn.com/images/29273643/funny-science-news-experiments-memes-super-science_large.jpgTuesday, February 26, 13
  • 5. Research doesn’t fit the traditional SDLC image src:http://laurajul.dk/wp-content/uploads/2011/10/Screen-shot-2011-10-06-at-00.03.57.pngTuesday, February 26, 13
  • 6. Good help is hard to find (and keep) image src: legoexpress.tumblr.comTuesday, February 26, 13
  • 7. What are they doing all day?Tuesday, February 26, 13
  • 8. Bring transparency image src: www.ideachampions.comTuesday, February 26, 13
  • 9. Communication Logistics TrustTuesday, February 26, 13
  • 10. Tuesday, February 26, 13
  • 11. CommunicationTuesday, February 26, 13
  • 12. Do you speak the same language? image src: abclang.livejournal.comTuesday, February 26, 13
  • 13. What does it mean to be finished? image source: ladywhodoesntlunch.blogspot.comTuesday, February 26, 13
  • 14. Define “quality” image source: http://www.sodahead.com/fun/Tuesday, February 26, 13
  • 15. R Give them a lesson in P semantics What do precision and recall mean?Tuesday, February 26, 13
  • 16. Before Precision: 80% Recall 25%Tuesday, February 26, 13
  • 17. After For the top search terms* accessories appear 25% of the time on the first page. For top search terms the head products are present 90% of the time in the top 3 results, 98% of the time in the top5. ✴ Top search terms are the 1000 most popular queries on our website over the last 30 days.Tuesday, February 26, 13
  • 18. Measure with data that matters image source: http://www.freepatentsonline.com/6971185-0-large.jpgTuesday, February 26, 13
  • 19. This is a really hard problem.Tuesday, February 26, 13
  • 20. We know it is hard...Tuesday, February 26, 13
  • 21. We know it is hard... but we don’t know why it is hard.Tuesday, February 26, 13
  • 22. Use analogies & examplesTuesday, February 26, 13
  • 23. Use analogies & examples The dandelion swayed in the gentle breeze like an oscillating electric fan set on medium.Tuesday, February 26, 13
  • 24. Constructing model lineages for products is really difficult.Tuesday, February 26, 13
  • 25. Macbook Pro Bravia EX523 LCD Macbook ? Bravia ? EX500 LCD Bravia EX620 LED Macbook Air Be specificTuesday, February 26, 13
  • 26. Take it up a level We are building a lineage Product EvolutionTuesday, February 26, 13
  • 27. Show before image src: www.clarkgriswoldcollection.comTuesday, February 26, 13
  • 28. Show before and after image src: www.clarkgriswoldcollection.comTuesday, February 26, 13
  • 29. Tuesday, February 26, 13
  • 30. LogisticsTuesday, February 26, 13
  • 31. How do you create a sense of urgency? image source: http://favim.com/image/503182/Tuesday, February 26, 13
  • 32. What are the reasons behind it? image source: http://dailyfailcenter.com/sites/default/files/fail/explain-this-shit.jpgTuesday, February 26, 13
  • 33. Deadlines in research?Tuesday, February 26, 13
  • 34. They don’t call it research for nothin’ image source: http://sausagetails.com/2012/07/Tuesday, February 26, 13
  • 35. You can’t predict the future image source: maxseesmovies.blogspot.comTuesday, February 26, 13
  • 36. Applying “agile” to R&D image source:http://mousebreath.com/wp-content/uploads/2011/08/funny-farmers.jpgTuesday, February 26, 13
  • 37. Applying “agile” to R&D Backlog of Trips to Hawaii experiments That’s doesn’t sound agile.... image source:http://mousebreath.com/wp-content/uploads/2011/08/funny-farmers.jpgTuesday, February 26, 13
  • 38. Applying “agile” to R&D Regular Demos Backlog of Trips to Hawaii experiments That’s doesn’t sound agile.... image source:http://mousebreath.com/wp-content/uploads/2011/08/funny-farmers.jpgTuesday, February 26, 13
  • 39. Applying “agile” to R&D Regular Demos Defined workflow with iterations Backlog of Trips to Hawaii experiments That’s doesn’t sound agile.... image source:http://mousebreath.com/wp-content/uploads/2011/08/funny-farmers.jpgTuesday, February 26, 13
  • 40. No Start Here Collect Data Ye Do we have the s right data? Data No Do we have the Are we infrastructure to finished or do Build it... analyze the we need Re data? more.....? s ea rch Y es Get results Run experimentsTuesday, February 26, 13
  • 41. Experiments can take a while...   image source: www.wallpapermay.comTuesday, February 26, 13
  • 42. Take tools out of the equation image source: http://www.holesinyoursocks.com/2011/02/14/funny-monday-tool-love-happy-valentines-day/Tuesday, February 26, 13
  • 43. Focus on feeds and files Image source: http://carcat.files.wordpress.com/2009/03/funny-pictures-cat-searches-for-a-file.jpgTuesday, February 26, 13
  • 44. Format & storage standards /prices /full /date=2012-07-01 price-obs.2012-07-01.csv.gz /date=2012-07-02 /inc /date=2012-07-01 2012-07-01T00-10-00.csv.gz 2012-07-01T00-20-00.csv.gzTuesday, February 26, 13
  • 45. Where is your golden set? image source: ads/2011/09/4bc2dd714ecb9eebb3a66d074638.jpegTuesday, February 26, 13
  • 46. Tuesday, February 26, 13
  • 47. TrustTuesday, February 26, 13
  • 48. What are they doing all day?Tuesday, February 26, 13
  • 49. Building trust image source: http://writealoud.com/funny-dinosaur-pictures/Tuesday, February 26, 13
  • 50. Their motivations image src: http://www.fredhoogervorst.com/oni.app/local/upload/03897400db.jpgTuesday, February 26, 13
  • 51. Their motivations Hard problems to solve image src: http://www.fredhoogervorst.com/oni.app/local/upload/03897400db.jpgTuesday, February 26, 13
  • 52. Their motivations Hard problems to solve My work in the wild serving customers image src: http://www.fredhoogervorst.com/oni.app/local/upload/03897400db.jpgTuesday, February 26, 13
  • 53. Their motivations Hard problems to solve Recognition for a job well done Being GOLD! My work in the wild serving customers image src: http://www.fredhoogervorst.com/oni.app/local/upload/03897400db.jpgTuesday, February 26, 13
  • 54. Their motivations Hard problems to solve Recognition for a job well done Open the door to higher-ups Being GOLD! My work in the wild serving customers image src: http://www.fredhoogervorst.com/oni.app/local/upload/03897400db.jpgTuesday, February 26, 13
  • 55. Minimize time-to-ship image source: : http://2smallerwheels.blogspot.com/2011/09/speedy-delivery.htmlTuesday, February 26, 13
  • 56. Let them own the data image source: http://www.cloudproviderusa.com/weekly-dose-of-humor-mo-data-mo-problems/Tuesday, February 26, 13
  • 57. Let me write a program image src: http://funnyfilez.funnypart.com/pictures/FunnyPart-com-geek_on_the_go.jpgTuesday, February 26, 13
  • 58. Algorithms can’t solve everythingTuesday, February 26, 13
  • 59. Algorithms can’t solve everything Even though we wish they could!Tuesday, February 26, 13
  • 60. Manual data can be awesome image source: http://www.funnyjunk.com/funny_pictures/4416618/LOOK+AT+DE+PUPPY/Tuesday, February 26, 13
  • 61. Can you do the impossible? image source: anybody-want-a-peanut.blogspot.comTuesday, February 26, 13
  • 62. Data Miners aren’t magicians image source: http://1.bp.blogspot.com/_zmT48r8jfbE/S1z4t5L7JxI/AAAAAAAAAJU/1vYZmc5lhBU/s400/steve-brooks-magician.jpgTuesday, February 26, 13
  • 63. You CAN make failure look good image source: http://fightthebees.com/wp-content/uploads/2011/11/cow-fail.jpgTuesday, February 26, 13
  • 64. Your challenge:Tuesday, February 26, 13
  • 65. Your challenge: Should you choose to accept it....Tuesday, February 26, 13
  • 66. Your challenge: Take a difficult problem and transform it into a feature Should you choose to accept it....Tuesday, February 26, 13
  • 67. HARD: Value Estimation 1.7GHz dual-core Intel Core i5 $300 256GB flash storage $250 Intel HD Graphics $100 4000 8GB SDRAM $200 Apple premium $200 Estimated Value: $1050Tuesday, February 26, 13
  • 68. EASIER: Comparing ProductsTuesday, February 26, 13
  • 69. Think with your business hat image source: memegenerator.netTuesday, February 26, 13
  • 70. How do you surface ideas & insights? image source: http://static.someecards.com/someecards/usercards/1327680406361_4248272.pngTuesday, February 26, 13
  • 71. Show ‘em image source: neslihandurmusoglu.edublogs.orgTuesday, February 26, 13
  • 72. And show ‘em often image source: http://www.social-science.co.uk/research/Tuesday, February 26, 13
  • 73. Negative results happen image source: http://www.theofrak.com/2012/09/existential-star-wars.htmlTuesday, February 26, 13
  • 74. The journey is the reward image source: failblog.orgTuesday, February 26, 13
  • 75. Tuesday, February 26, 13
  • 76. CommunicationTuesday, February 26, 13
  • 77. Communication LogisticsTuesday, February 26, 13
  • 78. Communication Logistics TrustTuesday, February 26, 13
  • 79. Questions http://katemats.com @katematsTuesday, February 26, 13
  • 80. May the forces of evil get lost on the way to your doorstep. Questions http://katemats.com @katematsTuesday, February 26, 13