Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Semla2019 xiu pdf

122 views

Published on

An Exploratory Study on Machine Learning Model Stores

Published in: Software
  • Be the first to comment

  • Be the first to like this

Semla2019 xiu pdf

  1. 1. An Exploratory Study on the Machine Learning Model Stores Zhen Ming (Jack) Jiang
  2. 2. Machine Learning (ML) & Software Engineering (SE) Software Engineering Machine Learning
  3. 3. Machine Learning (ML) & Software Engineering (SE) Software Engineering Machine Learning .. Bug Prediction Test Case Prioritization Automated Bug Triage .
  4. 4. Machine Learning (ML) & Software Engineering (SE) Software Engineering Machine Learning .. Bug Prediction Test Case Prioritization Automated Bug Triage . Test ML Applications
  5. 5. Machine Learning (ML) & Software Engineering (SE) Software Engineering Machine Learning .. Bug Prediction Test Case Prioritization Automated Bug Triage . Test ML Applications Anything else???
  6. 6. ML Model Stores
  7. 7. ML Model Stores January 2018 June 2018 November 2018 November 2018 April 2019 (beta)
  8. 8. ML Model Stores January 2018 June 2018 November 2018 November 2018 April 2019 (beta) What are the unique SE practices/challenges for building ML applications?
  9. 9. Structure Contents
  10. 10. Structure
  11. 11. What kind of information elements do ML model stores provide? January 2018 June 2018 November 2018 November 2018 April 2019 (beta)
  12. 12. What kind of information elements do ML model stores provide? January 2018 June 2018 November 2018 April 2019 (beta)
  13. 13. What kind of information elements do ML model stores provide? January 2018 June 2018 November 2018
  14. 14. What kind of information elements do ML model stores provide? January 2018 June 2018 November 2018 October 2008 July 2008
  15. 15. Identifying Information Elements
  16. 16. Identifying Information Elements Owner
  17. 17. Identifying Information Elements ML Algorithm Owner
  18. 18. Identifying Information Elements Framework ML Algorithm Owner
  19. 19. Identifying Information Elements Framework Training set ML Algorithm Owner
  20. 20. Identifying Information Elements LicenseFramework Training set ML Algorithm Size Usage Statistics User feedback User Manual Owner
  21. 21. Identifying Information Elements LicenseFramework Training set ML Algorithm Size Usage Statistics User feedback User Manual Owner
  22. 22. Identifying Information Elements LicenseFramework Training set ML Algorithm Size Usage Statistics User feedback User Manual Owner
  23. 23. vs.ML Engineers End Users
  24. 24. To package an ML application for release - Source code -Trained ML model(s) -Training set info Integrating such a model … - API calls, re-train on new data?
  25. 25. To package an ML application for release - Source code -Trained ML model(s) -Training set info Integrating such a model … - API calls, re-train on new data?
  26. 26. Quality representation - Statistics vs. QoS measures (reliability) - Requirements for different ML applications
  27. 27. [Pricing] Subscription-based - Pricing scheme vs. actual usage context Capacity planning for ML applications
  28. 28. Structure Contents
  29. 29. How unique are the ML models provided by each model store? Very few cross-store ML applications - Vendor lock-in - Need for cross-platform frameworks
  30. 30. Same ML Models Different Usage Context
  31. 31. Same ML Models Different Usage Context
  32. 32. Minor Releases Different Product Pages
  33. 33. Minor Releases Different Product Pages
  34. 34. Minor Releases Different Product Pages A new version for an ML application is: - same ML model re-trained on another dataset? or - same dataset with different ML implementations?

×