The document discusses the integration of data science and software development, emphasizing the collaborative role of data scientists and developers in using tools like Jupyter notebooks and PixieDust for analytics and application creation. It highlights the benefits of combining these disciplines, such as competitive advantage, real-time decision-making, and reduced complexity. The localcart project example illustrates how data can be utilized for customer recommendations, showcasing the process from data preparation to model deployment.

1. @MargrietGr
The Convergence of Data Science
and Software Development
Margriet Groenendijk
Developer Advocate | IBM
25 April 2018 | IP EXPO Manchester

2. @MargrietGr
Artificial Intelligence
(Big)
Data
Data Science
Deep
Learning
Machine Learning

3. Machine learning
Algorithm selection
@MargrietGr

4. Machine learning
Algorithm selection
Deep learning
Neural network design
@MargrietGr

5. Machine learning
Algorithm selection
Deep learning
Neural network design
Artificial intelligence
Systems architecture
@MargrietGr

6. Data science
Extract insights from data
@MargrietGr

7. Data science
Extract insights from data
Use historical data to train a model
and then make predictions
@MargrietGr

8. @MargrietGr
Output
Loan
Approval
Model
Train
Algorithm
John X
§ credit_score=800
§ age=25
§ income=$900,000
§ works in Oil & Gas
Historical Loans
Label
Approve

9. @MargrietGr
Approve
Output
Reject
James X
§ credit_score=900
§ age=55
§ income=$1,200,000
§ works in Insurance
New Applicant
Loan
Approval
Model

10. Deep Learning – Imaginary celebrities
@MargrietGr https://arxiv.org/abs/1710.10196

11. Data
@MargrietGr https://unsplash.com/photos/vpR0oc4X8Mk

12. Data Scientists vs. Developers
@MargrietGr
Data Scientists Developers
Data Static data Dynamic databases
Code Python, R JavaScript
Platform Notebooks Text editors
Design Models Web apps

13. https://unsplash.com/photos/AtgRjx271ks@MargrietGr

14. Data Science is a Team Sport
@MargrietGr
Extract Data
Data
Engineer
Prepare
Data
Data
Scientist
Build & train
models
Data
Scientist
Evaluate
Business
Analyst
Deploy DevOps
Use models Developer
Monitor DevOps

15. Data Scientist
@MargrietGr
Extract Data
Data
Engineer
Prepare
Data
Data
Scientist
Build & train
models
Data
Scientist
Evaluate
Business
Analyst
Deploy DevOps
Use models Developer
Monitor DevOps

16. Software Developer
@MargrietGr
Extract Data
Data
Engineer
Prepare
Data
Data
Scientist
Build & train
models
Data
Scientist
Evaluate
Business
Analyst
Deploy DevOps
Use models Developer
Monitor DevOps

17. LocalCart
https://github.com/ibm-watson-data-lab/localcart-at-think-conf
@MargrietGr

18. The LocalCart Project
@MargrietGr
Customer behaviour information, such as
demographics, shopping cart values
A recommendation engine to encourage
additional purchases based on past buying
behaviour

19. Extract & Prepare Data
@MargrietGr

20. Explore Data
@MargrietGr

21. Jupyter
Notebooks
@MargrietGr

22. @MargrietGr

23. @MargrietGr

24. @MargrietGr

25. @MargrietGr

26. @MargrietGr

27. @MargrietGr

28. @MargrietGr

29. @MargrietGr

30. notebook.ipynb
@MargrietGr

31. Jupyter
Notebooks
For Data Scientists and
Developers
@MargrietGr

32. https://ibm-watson-data-lab.github.io/pixiedust/

33. PixieDust - open-source Python library for
Jupyter notebook to load and visualize data
@MargrietGr

34. PixieDust - open-source Python library for
Jupyter notebook to load and visualize data
PixieDebugger – first visual debugging tool for
Jupyter notebooks
@MargrietGr

35. PixieDust - open-source Python library for
Jupyter notebook to load and visualize data
PixieDebugger – first visual debugging tool for
Jupyter notebooks
PixieApps - create dashboards in a notebook
@MargrietGr

36. PixieDust - open-source Python library for
Jupyter notebook to load and visualize data
PixieDebugger – first visual debugging tool for
Jupyter notebooks
PixieApps - create dashboards in a notebook
PixieGateway - run charts or PixieApps as
standalone web applications
@MargrietGr

37. The LocalCart Project
@MargrietGr
Customer behaviour information, such as
demographics, shopping cart values
A recommendation engine to encourage
additional purchases based on past buying
behaviour

38. Display data as a table
@MargrietGr

39. Display data as a chart
@MargrietGr

40. Display data as a chart
@MargrietGr

41. Display data as a chart
@MargrietGr

42. @MargrietGr
Customers by age in
a histogram

43. @MargrietGr
Customers
clustered
by gender

44. @MargrietGr
Customers by state in
a map

45. Combine customer
data with
Census open data of
income by zip code
@MargrietGr

46. Combine customer
data with
Census open data of
income by zip code
@MargrietGr

47. The LocalCart Project
@MargrietGr
Customer behaviour information, such as
demographics, shopping cart values
A recommendation engine to encourage
additional purchases based on past buying
behaviour

48. Build, train and deploy a model
@MargrietGr

49. Build, train and deploy a model
1. Prepare data in notebook
2. Build and train model in notebook
3. Deploy to Watson Machine Learning
4. Test from anywhere
@MargrietGr

50. @MargrietGr

51. Test interactively with a PixieApp
@MargrietGr

52. PixieDebugger
@MargrietGr

53. PixieApp example – Weather Forecast
@MargrietGr

54. Continuous
Learning
https://medium.com/ibm-watson-data-lab/keeping-your-machine-
learning-models-up-to-date-f1ead546591b
@MargrietGr

55. @MargrietGr
1. Load training
data
2. Build
model

56. @MargrietGr
1. Load training
data
2. Build
model
3. Deploy model

57. @MargrietGr
1. Load training
data
2. Build
model
3. Deploy model
4. Feedback data

58. @MargrietGr
1. Load training
data
2. Build
model
3. Deploy model
4. Feedback data
5. Feedback
evaluation

59. @MargrietGr
1. Load training
data
2. Build
model
3. Deploy model
4. Feedback data
5. Feedback
evaluation
6. If accuracy too low, retrain
model with all data

60. @MargrietGr
1. Load training
data
2. Build
model
3. Deploy model
4. Feedback data
5. Feedback
evaluation
6. If accuracy too low, retrain
model with all data
7. Deploy new model if
improved accuracy

61. @MargrietGr
The Convergence of Data Science
and Software Development

62. Summary
@MargrietGr
Data scientists use a Python notebook to load, enrich,
analyze data, and create analytics

63. Summary
@MargrietGr
Data scientists use a Python notebook to load, enrich,
analyze data, and create analytics
From the same notebook, developers create a PixieApp
to operationalize these analytics

64. Summary
@MargrietGr
Data scientists use a Python notebook to load, enrich,
analyze data, and create analytics
From the same notebook, developers create a PixieApp
to operationalize these analytics
Developers publish the PixieApp as a web application

65. Summary
@MargrietGr
Data scientists use a Python notebook to load, enrich,
analyze data, and create analytics
From the same notebook, developers create a PixieApp
to operationalize these analytics
Developers publish the PixieApp as a web application
The PixieApp can be viewed interactively by line-of-
business users with no need to access the notebook

66. Summary
@MargrietGr
Data scientists use a Python notebook to load, enrich,
analyze data, and create analytics
From the same notebook, developers create a PixieApp
to operationalize these analytics
Developers publish the PixieApp as a web application
The PixieApp can be viewed interactively by line-of-
business users with no need to access the notebook
Data scientist and developers both work in the same
Cloud

67. Benefits
of bringing the right tools into
the Data Science work flow
@MargrietGr
• Competitive advantage
• Discover new insights
• Real-time decision making
• Reduce complexity and
lower cost
• Accelerate time to market
and deployment of data
science and analytics

68. Benefits
of bringing the right tools into
the Data Science work flow
ONE GOAL Develop data
driven applications
Data science is maturing
NOW is the time for
integration with software
development workflow
@MargrietGr
• Competitive advantage
• Discover new insights
• Real-time decision making
• Reduce complexity and
lower cost
• Accelerate time to market
and deployment of data
science and analytics

69. Thank you!
@MargrietGr
Margriet Groenendijk
mgroenen@uk.ibm.com
@MargrietGr
https://www.linkedin.com/in/
margrietgroenendijk/
Slides
https://www.slideshare.net/MargrietGroenen
dijk/presentations
Blog
https://medium.com/ibm-watson-data-lab
PixieDust
https://ibm-watson-data-
lab.github.io/pixiedust/
IBM Watson Studio
https://www.ibm.com/cloud/watson-studio
IBM Cloud
https://ibm.com/cloud