Successfully reported this slideshow.
Your SlideShare is downloading. ×

“Incorporating Continuous User Feedback to Achieve Product Longevity in Chaotic Environments,” a Presentation from Observa

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 24 Ad

“Incorporating Continuous User Feedback to Achieve Product Longevity in Chaotic Environments,” a Presentation from Observa

Download to read offline

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2022/09/incorporating-continuous-user-feedback-to-achieve-product-longevity-in-chaotic-environments-a-presentation-from-observa/

Erik Chelstad, CTO and Co-founder of Observa, presents the “Incorporating Continuous User Feedback to Achieve Product Longevity in Chaotic Environments” tutorial at the May 2022 Embedded Vision Summit.

In many computer vision applications, a key challenge is maintaining accuracy when the real world is changing. In this presentation, Chelstad explores techniques for designing hardware and software to stave off obsolescence by allowing users to interact with the output of the computer vision system as quickly and cheaply as possible, so that future errors are avoided. He also discusses ways to motivate users to provide useful feedback, and illustrates concepts using a deployed commercial application.

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2022/09/incorporating-continuous-user-feedback-to-achieve-product-longevity-in-chaotic-environments-a-presentation-from-observa/

Erik Chelstad, CTO and Co-founder of Observa, presents the “Incorporating Continuous User Feedback to Achieve Product Longevity in Chaotic Environments” tutorial at the May 2022 Embedded Vision Summit.

In many computer vision applications, a key challenge is maintaining accuracy when the real world is changing. In this presentation, Chelstad explores techniques for designing hardware and software to stave off obsolescence by allowing users to interact with the output of the computer vision system as quickly and cheaply as possible, so that future errors are avoided. He also discusses ways to motivate users to provide useful feedback, and illustrates concepts using a deployed commercial application.

Advertisement
Advertisement

More Related Content

More from Edge AI and Vision Alliance (20)

Recently uploaded (20)

Advertisement

“Incorporating Continuous User Feedback to Achieve Product Longevity in Chaotic Environments,” a Presentation from Observa

  1. 1. Incorporating Continuous User Feedback to Achieve Product Longevity in Chaotic Environments Erik Chelstad CTO & Co-Founder Observa
  2. 2. What is a chaotic environment? • Fast paced • Rapidly changing • Uncontrollable external factors © 2022 Observa, Inc. 2
  3. 3. What is a product and who is a user? • Products • Hardware • Software • Users • Single human • Teams or companies • Autonomous entities © 2022 Observa, Inc. 3
  4. 4. History of Feedback (not a rock band)
  5. 5. Traditional user feedback • Purposes • Bug reporting • Business development • Mollify users • Future strategy • Measure user satisfaction © 2022 Observa, Inc. 5 • Methods • Surveys (polling) • Social media • Customer support • Live chat • Customer success
  6. 6. Traditional user feedback • Problems • Slow • Too late • Focus on negative issues • Not enough data • Qualitative • Reactive – bug fixes © 2022 Observa, Inc. 6
  7. 7. New reasons for getting user feedback • Market fit • Proactive releases make users feel good • Personalization • Users no longer expect products and systems to be static • Stay ahead of competition and disruptors © 2022 Observa, Inc. 7 • Environment fit • Natural experiments • Controlled experiments • New scenarios and usages • Changing environment • New things appearing • Old things disappearing
  8. 8. No excuses for not getting user feedback • Data harvesting • Storage is cheap • Sensors are cheap • Processing is cheap • Users are NOT cheap • Domain specific data is invaluable © 2022 Observa, Inc. 8 • Data collection • Constant feedback is possible • Products NEED some connectivity for updates, etc.
  9. 9. Designing for Feedback (not a section on guitar amps)
  10. 10. Design – When? • 😀 Immediately! • Build into presentation • 🙂 Soon • Within a usage session • 😐 Before too long • Before a user forgets everything © 2022 Observa, Inc. 10
  11. 11. Design – How? • With interaction • Ease of use • Not distracting • Optional • Not good: © 2022 Observa, Inc. 11 • Simple & quick: • Logging off or shutting down
  12. 12. Design – How? • Passive data collection • Did they stay on the results screen for a while or bounce? • Did the user upload the results they got back to HQ? • Are they redoing the same task? © 2022 Observa, Inc. 12 • Did they save the results to their favorites? • Did they mash the button or yell at their device?
  13. 13. Design – How? • Non-human • Did the navigation system make a sudden correction or cause human interaction? © 2022 Observa, Inc. 13 • Did a plotted route result in a stuck robot?
  14. 14. Design – Why? • Why would a person do it? • How rewarded? • Ideas like recognition - your input was used in the latest release and performance fixes for efficiency • Frustration • Happiness © 2022 Observa, Inc. 14 • Why would a machine or corporation do it? • Because their humans told them to • Maximize ROI on fixed costs • Extend useful life • No new training
  15. 15. Bringing it Back In
  16. 16. When to integrate • The other side of the collection coin is that you must actually do something with the feedback! • Make it part of the data pipeline • Automate as much cleaning and validation as possible • Make it part of the update workflow • Review with humans as needed © 2022 Observa, Inc. 16
  17. 17. How to integrate • Use locally • Example - Auto adjust overlap tolerance on panoramic camera based on user accepting/rejecting their own photos © 2022 Observa, Inc. 17 • At the edge • Upload back to servers in batches or at scheduled intervals • Always connected • Can be real-time
  18. 18. Case Studies
  19. 19. Examples from Observa (my company) • Viewing a report allows for corrective input at any time • No need to “correct” the system, just report mistakes • On any chart of a report • By any viewer of the report • Remove from reporting immediately © 2022 Observa, Inc. 19
  20. 20. Examples from Observa (my company) • Low confidence results “passively” come back for human review in the pipeline © 2022 Observa, Inc. 20 Incoming Image IR System Low Confidence Results Label Tool High Priority Review Training Data Reporting High Confidence Results Low Priority Review Training Data
  21. 21. Examples from Observa (my company) • Observa app when field service users are in front of the shelf • Passive if they agree or need to reanalyze © 2022 Observa, Inc. 21
  22. 22. In Conclusion
  23. 23. In Conclusion • User feedback keeps your product relevant • User feedback can be collected cheaply and unobtrusively if you design for it • Your user feedback is unique and unobtainable by competitors • Users are motivated to help you: • Intrinsically • Financially • Emotionally © 2022 Observa, Inc. 23
  24. 24. Resources Natural Experiments https://www.nobelprize.org/uploads/2021/10/advanced- economicsciencesprize2021.pdf The Virtuous Cycle of AI Products https://www.eriktrautman.com/posts/the-virtuous-cycle-of-ai-products Evolution of intelligent data pipelines https://www.technologyreview.com/2021/12/06/1040716/evolution-of- intelligent-data-pipelines/ © 2022 Observa, Inc. 24

×