JDO 2019: Data Science for Developers - Matthew Renze
•
0 likes•11 views
Data science is revolutionizing the world around us. We’re incorporating artificial intelligence, machine learning, and data-driven decision making into all aspects of business. However, many software developers have yet to learn how to leverage these practices to create better software. In this presentation, we’ll learn how expert developers are using data science to create better software. We’ll learn how to use data analytics, machine learning, and anticipatory design to create more intelligent software. In addition, we’ll learn how to use data from our dev-ops pipeline to improve our software development practices.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to JDO 2019: Data Science for Developers - Matthew Renze
Similar to JDO 2019: Data Science for Developers - Matthew Renze (20)
Recently uploaded
Recently uploaded (20)
JDO 2019: Data Science for Developers - Matthew Renze
- 3. Do you make decisions every day?
- 4. Are you flooded with data? Do you make decisions every day?
- 5. Are you flooded with data? Do you make decisions every day? Are you using data to make decisions?
- 6. What is data science? Why is it important? How do I get started?
- 11. Job Postings for Data Scientists
- 12. Source: Dice Salary Survey 2017 Top-paying Tech Skills Skill 2016 Change Skill 2016 Change
- 16. What is data science?
- 19. Data Knowledge Decision Action
- 20. What Is a Data Scientist? Performs data science More than a scientist More than an analyst More than a developer
- 25. What skills are necessary?
- 26. Data Science Skills Programming Working with data Descriptive statistics Data visualization
- 27. Data Science Skills Programming Working with data Descriptive statistics Data visualization Statistical modeling Handling Big Data Machine learning Deploying to production
- 28. What tools are used?
- 29. Programming Languages 0% 10% 20% 30% 40% 50% 60% 70% SQL Python R Bash JavaScript Java Scala Visual Basic/VBA C++ Matlab Share of Respondents Source: O’Reilly 2017 Data Science Salary Survey
- 30. Relational Databases 0% 5% 10% 15% 20% 25% 30% 35% 40% MySQL Microsoft SQL Server PostgreSQL Oracle SQLite Teradata IBM DB2 Netezza (IBM) Vertica SAP HANA Share of Respondents Source: O’Reilly 2017 Data Science Salary Survey
- 31. Big Data Platforms 0% 5% 10% 15% 20% 25% 30% Spark Hive MongoDB Amazon Redshift Kafka Pig Redis Zookeeper Imapla Hbase Share of Respondents Source: O’Reilly 2017 Data Science Salary Survey
- 32. Spreadsheets, BI, Reporting 0% 10% 20% 30% 40% 50% 60% 70% Excel Power BI QlikView BusinessObjects PowerPivot Cognos Alteryx Microstrategy Adobe Analytics Oracle BI Share of Respondents Source: O’Reilly 2017 Data Science Salary Survey
- 33. How is data science performed?
- 34. The Data Science Process Data
- 35. The Data Science Process Find a question Data
- 36. The Data Science Process Find a question Collect the data Data
- 37. The Data Science Process Find a question Collect the data Prepare the data Data
- 38. The Data Science Process Find a question Collect the data Prepare the data Create a model Data
- 39. The Data Science Process Find a question Collect the data Prepare the data Create a model Evaluate the model Data
- 40. The Data Science Process Find a question Collect the data Prepare the data Create a model Evaluate the model Deploy the model Data
- 41. The Data Science Process Find a question Collect the data Prepare the data Create a model Evaluate the model Deploy the model Data
- 42. The Data Science Process Iterative process Find a question Explore the data Prepare the data Create a model Evaluate the model Deploy the model Data
- 43. The Data Science Process Iterative process Non-sequential Find a question Explore the data Prepare the data Create a model Evaluate the model Deploy the model Data
- 44. The Data Science Process Iterative process Non-sequential Early termination Find a question Explore the data Prepare the data Create a model Evaluate the model Deploy the model Data
- 45. Why is data science important?
- 57. 4% higher productivity 6% higher profits Source: https://hbr.org/2012/10/big-data-the-management-revolution
- 70. How does data science help us build better software?
- 74. Show me sales by gender and marital status. “Show me sales by gender and marital status.” Internet Sales Displaying sum of sales by gender and marital status Marital Status: Married Single Male Female $0k $5k $10k $15k
- 76. Classification Regression Clustering Anomaly Detection
- 78. Area SalePrice
- 79. Income Age
- 89. Hypothesis-Driven Development Hypothesis ExperimentAnalysis Problem: Two features; similar benefits
- 90. Hypothesis-Driven Development Hypothesis ExperimentAnalysis Hypothesis: Users will prefer feature A over feature B Problem: Two features; similar benefits
- 91. Hypothesis-Driven Development Hypothesis ExperimentAnalysis Hypothesis: Users will prefer feature A over feature B Problem: Two features; similar benefits Experiment: Survey 100 users and ask for their preference
- 92. Hypothesis-Driven Development Hypothesis ExperimentAnalysis Experiment: Survey 100 users and ask for their preference Hypothesis: Users will prefer feature A over feature B Problem: Two features; similar benefits Level of Support: At least 60% must prefer feature A
- 93. Hypothesis-Driven Development Hypothesis ExperimentAnalysis Analysis: 80% of users prefer feature A Hypothesis: Users will prefer feature A over feature B Problem: Two features; similar benefits Experiment: Survey 100 users and ask for their preference Level of Support: At least 60% must prefer feature A
- 94. Hypothesis-Driven Development Hypothesis ExperimentAnalysis Analysis: 80% of users prefer feature A Hypothesis: Users will prefer feature A over feature B Problem: Two features; similar benefits Experiment: Survey 100 users and ask for their preference Level of Support: At least 60% must prefer feature A Decision: Only build feature A
- 95. Hypothesis-Driven Development Hypothesis ExperimentAnalysis Problem: Too many software defects
- 96. Hypothesis-Driven Development Hypothesis ExperimentAnalysis Hypothesis: Pair programming will reduce software defects Problem: Too many software defects
- 97. Hypothesis-Driven Development Hypothesis ExperimentAnalysis Experiment: Pair for 4 sprints and track defects Hypothesis: Pair programming will reduce software defects Problem: Too many software defects
- 98. Hypothesis-Driven Development Hypothesis ExperimentAnalysis Experiment: Pair for 4 sprints and track defects Level of Support: 20% decrease in defects Hypothesis: Pair programming will reduce software defects Problem: Too many software defects
- 99. Hypothesis-Driven Development Hypothesis ExperimentAnalysis Analysis: Defects decreased by 30% per sprint Hypothesis: Pair programming will reduce software defects Problem: Too many software defects Experiment: Pair for 4 sprints and track defects Level of Support: 20% decrease in defects
- 100. Hypothesis-Driven Development Hypothesis ExperimentAnalysis Hypothesis: Pair programming will reduce software defects Analysis: Defects decreased by 30% per sprint Decision: Continue pairing Problem: Too many software defects Experiment: Pair for 4 sprints and track defects Level of Support: 20% decrease in defects
- 101. Hypothesis Stories <Hypothesis> We assume that <hypothesis> Will result in<outcome> We will have succeeded when <measurable result>
- 102. Hypothesis Stories Pair Programming Hypothesis We assume that pair programming Will result in less software defects We will have succeeded when we have seen a 20% or greater decrease in defects after 4 sprints.
- 103. Not all hypotheses are testable or should be tested.
- 104. A/B Testing
- 105. A/B Testing
- 106. Feature Toggles New Feature Feature Toggles User Groups
- 107. Feature Toggles New Feature Feature Toggles User Groups
- 108. Code Source Control Build Q/A Deploy Prod DevOps Pipeline
- 109. Code Source Control Build Q/A Deploy Prod DevOps Pipeline
- 110. Code Quality Metrics Source: NDepend
- 111. Source: NDepend
- 112. How complex is our code? Source: NDepend How maintainable is our code? How should we organize our code?
- 116. Which files are modified most frequently? How often are we checking in code? How is our code evolving over time?
- 118. Software Telemetry
- 120. How long to complete a task? Which features are being used? Is someone hacking into our system?
- 122. Code Source Control Build Q/A Deploy Prod DevOps Pipeline
- 123. Code Source Control Build Q/A Deploy Prod DevOps Pipeline
- 124. Advice for Success Get buy-in from leadership Focus on low-hanging fruit Don’t silo data science teams Democratize your data
- 125. Advice for Success Get buy-in from leadership Focus on low-hanging fruit Don’t silo data science teams Democratize your data Embrace smart failure Focus on feedback Embed data collection Avoid the Observer Effect
- 126. How do I get started?
- 127. www.pluralsight.com/authors/matthew-renze Pluralsight Courses Data Science: The Big Picture Data Science with R Exploratory Data Analysis with R Data Visualization with R (3-part) Deep Learning: The Big Picture https://www.pluralsight.com/authors/matthew-renze
- 129. New Online Courses Self-guided Simple Quick Practical
- 130. www.matthewrenze.com
- 131. Feedback Very important to me! What did you like? What could I improve?
- 132. Conclusion
- 133. What data science is Why it is important How to get started
- 135. Are you prepared? Is your team prepared? Is your organization prepared?
- 137. Thank You! Matthew Renze Data Science Consultant Renze Consulting Twitter: @matthewrenze Email: info@matthewrenze.com Website: www.matthewrenze.com