Download as PDF, PPTX
Experiences from going open source
- 1. EXPERIENCES FROM GOING OPENSOURCE BY ADRIEN BALL
- 2.
- 3. SNIPS END-TO-END SPEECH TOMEANING SOFTWARE Wake word Speech to text Natural Language Understanding
- 4. SNIPS NATURAL LANGUAGE UNDERSTANDING Speechto text Natural Language Understanding
- 5. SNIPS NATURAL LANGUAGE UNDERSTANDING NaturalLanguage Understanding
- 6. OPEN SOURCE ATSNIPS MOTIVATIONS ▸ Transparency ▸ Visibility ▸ Reproducibility
- 8.
- 9. PACKAGING FOR OPENSOURCE OBJECTIVES FOR THE COMMUNITY ▸ Understand ▸ Use ▸ Contribute
- 10. PACKAGING FOR OPENSOURCE REQUIREMENTS ▸ Documentation ▸ Continuous Integration and build automation ▸ APIs and versioning
- 11. PACKAGING FOR OPENSOURCE GOOD PRACTICES ▸ Documentation ▸ Continuous Integration and build automation ▸ APIs and versioning
- 12. PACKAGING FOR OPENSOURCE DOCUMENTATION ▸ Hard and painful to maintain ▸ More documentation => More outdated documentation ▸ Less documentation => Less explanations
- 13.
- 14.
- 15.
- 16.
- 17.
- 18. PACKAGING FOR OPENSOURCE GOOD PRACTICES ▸ Documentation ▸ Continuous Integration and build automation ▸ APIs and versioning
- 19. PACKAGING FOR OPENSOURCE CONTINUOUS INTEGRATION AND BUILD AUTOMATION ▸ Continuous integration: ▸ Always be merging into a branch ▸ Merge frequently ▸ Build automation: ▸ Enforce tests and checks to pass before merging
- 20. PACKAGING FOR OPENSOURCE BUILD AUTOMATION, WHAT FOR ? ▸ the project can be installed or built on the targeted platforms ▸ the code is doing what it is expected to do ▸ you haven't introduced regressions ▸ the documentation is not outdated ▸ automate whatever is error prone, and can be automated
- 21. PACKAGING FOR OPENSOURCE GOOD PRACTICES ▸ Documentation ▸ Continuous Integration and build automation ▸ APIs and versioning
- 22. PACKAGING FOR OPENSOURCE APIS AND VERSIONING ▸ Python: everything is public! ▸ Public API = Conventions + Doc
- 23. PACKAGING FOR OPENSOURCE SEMANTIC VERSIONING 1.3.2
- 24. PACKAGING FOR OPENSOURCE SEMANTIC VERSIONING 1 3 2 major minor patch Bump when you Examples make incompatible API changes - removed function - additional mandatory param - changed returned type Impact on client code no longer works add functionality in a backwards compatible manner - new API - new optional param additional capabilities make backwards compatible bug fixes improved behavior - internal bugs
- 25.
- 26. MACHINE LEARNING ANDOPEN SOURCE SPECIFIC CHALLENGES ▸ Managing resources ▸ Testing a Machine Learning pipeline ▸ Reproducibility ▸ Modularity and Extensibility
- 27. MACHINE LEARNING ANDOPEN SOURCE SPECIFIC CHALLENGES ▸ Managing resources ▸ Testing a Machine Learning pipeline ▸ Reproducibility ▸ Modularity and Extensibility
- 28. MACHINE LEARNING ANDOPEN SOURCE MANAGING RESOURCES Input Output resources ML Pipeline
- 29. MACHINE LEARNING ANDOPEN SOURCE MANAGING RESOURCES Input Output ML Pipeline ▸ Heavier library ▸ Updating the resources requires a release ▸ No user-defined resources
- 30. MACHINE LEARNING ANDOPEN SOURCE MANAGING RESOURCES Input Output resources ML Pipeline
- 31. MACHINE LEARNING ANDOPEN SOURCE MANAGING RESOURCES Input Output resources ML Pipeline
- 32. MACHINE LEARNING ANDOPEN SOURCE MANAGING RESOURCES IN SNIPS NLU
- 33. MACHINE LEARNING ANDOPEN SOURCE MANAGING RESOURCES IN SNIPS NLU
- 34. MACHINE LEARNING ANDOPEN SOURCE SPECIFIC CHALLENGES ▸ Managing resources ▸ Testing a Machine Learning pipeline ▸ Reproducibility ▸ Modularity and Extensibility
- 35. ▸ Traditional testing: ▸Testing in ML ? MACHINE LEARNING AND OPEN SOURCE TESTING A MACHINE LEARNING PIPELINE
- 36. MACHINE LEARNING ANDOPEN SOURCE TESTING A MACHINE LEARNING PIPELINE
- 37. MACHINE LEARNING ANDOPEN SOURCE TESTING DONE WRONG
- 38. MACHINE LEARNING ANDOPEN SOURCE TESTING DONE RIGHT
- 40. MACHINE LEARNING ANDOPEN SOURCE TESTING A MACHINE LEARNING PIPELINE
- 42. MACHINE LEARNING ANDOPEN SOURCE HANDLE RANDOMNESS IN TESTS
- 43. MACHINE LEARNING ANDOPEN SOURCE SPECIFIC CHALLENGES ▸ Managing resources ▸ Testing a Machine Learning pipeline ▸ Reproducibility ▸ Modularity and Extensibility
- 44. MACHINE LEARNING ANDOPEN SOURCE REPRODUCIBILITY FROM A PRODUCT PERSPECTIVE Data Training Evaluation selected data
- 45. Suffering Reproducibility 0% 100% MACHINE LEARNINGAND OPEN SOURCE REPRODUCIBILITY FROM A DEBUGGING PERSPECTIVE
- 46. MACHINE LEARNING ANDOPEN SOURCE REPRODUCIBILITY FOR BENCHMARKS Data + Code 🤓 🤓 🤓 0.95 0.87 0.92 0.98 0.91 0.88 0.89 0.83 0.92
- 47. MACHINE LEARNING ANDOPEN SOURCE REPRODUCIBILITY WITH RANDOM SEEDS
- 48. MACHINE LEARNING ANDOPEN SOURCE REPRODUCIBILITY THROUGH CONFIGURATIONS 56 3.0 True code
- 49. MACHINE LEARNING ANDOPEN SOURCE REPRODUCIBILITY THROUGH CONFIGURATIONS 42 1.5 False code
- 50. MACHINE LEARNING ANDOPEN SOURCE REPRODUCIBILITY THROUGH CONFIGURATIONS 42 1.5 False x y z param_1: param_2: param_3: code config
- 51. MACHINE LEARNING ANDOPEN SOURCE REPRODUCIBILITY THROUGH CONFIGURATIONS Data + Code + Config 🤓 🤓 🤓 0.95 0.87 0.92 0.98 0.91 0.88 0.89 0.83 0.92
- 53. MACHINE LEARNING ANDOPEN SOURCE SPECIFIC CHALLENGES ▸ Managing resources ▸ Testing a Machine Learning pipeline ▸ Reproducibility ▸ Modularity and Extensibility
- 54. MACHINE LEARNING ANDOPEN SOURCE MODULARITY AND EXTENSIBILITY Input Output LogReg SVM PIPELINE AVAILABLE COMPONENTS
- 55. MACHINE LEARNING ANDOPEN SOURCE MODULARITY AND EXTENSIBILITY Input Output LogReg AVAILABLE COMPONENTS SVM PIPELINE
- 56. MACHINE LEARNING ANDOPEN SOURCE MODULARITY AND EXTENSIBILITY Input Output LogReg SVM PIPELINE AVAILABLE COMPONENTS
- 57. MACHINE LEARNING ANDOPEN SOURCE MODULARITY AND EXTENSIBILITY Input Output LogReg SVM PIPELINE AVAILABLE COMPONENTS
- 58. MACHINE LEARNING ANDOPEN SOURCE MODULARITY AND EXTENSIBILITY Input Output LogReg SVM PIPELINE AVAILABLE COMPONENTS
- 59. MACHINE LEARNING ANDOPEN SOURCE MODULARITY AND EXTENSIBILITY FILE SYSTEMRAM
- 63. MACHINE LEARNING ANDOPEN SOURCE REGISTRABLE COMPONENTS Declarative syntax for pipeline and components
- 64. MACHINE LEARNING ANDOPEN SOURCE REGISTRABLE COMPONENTS Components Registry
- 65. EXPERIENCES FROM GOINGOPEN SOURCE TAKEAWAYS ▸ writing tests save you time, not the opposite ▸ test the right things ▸ make your outputs reproducible ▸ use abstractions to improve modularity and clarity
- 66.