Downloaded 26 times
Uploaded byChun-Yu Tseng
from ai.backend import python @ pycontw2018
1. The document describes the role of an AI engineer at a computer vision startup called Umbo. It discusses developing and maintaining computer vision services, building machine learning pipelines, and lessons learned. 2. Key responsibilities of an AI engineer include developing and maintaining computer vision services, building machine learning pipelines to improve services and measure model performance, and debugging and refactoring code. 3. The author learned that domain knowledge is important, Python is a unifying language, and collaboration between researchers and engineers is necessary. The future may see more high-quality backend development to support machine learning services.
More Related Content
Similar to from ai.backend import python @ pycontw2018
![Agile x API x Documentation @ NGO [[MOPCON2015]]](https://cdn.slidesharecdn.com/ss_thumbnails/mopcon2015aadngo-151114145910-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
from ai.backend import python @ pycontw2018
- 1. Joe | AIEngineer
- 2. from ai.backend importpython A (backend → AI) story and several thoughts about the future
- 3. Joe Tseng • Backend(AI) engineer • Tainan.py - organizer • PyConTW 2015/2016 - speaker
- 4. Outline About the story UmboComputer Vision intro Develop and maintain CV services Build our machine learning pipelines What I have learned About the future My imagination
- 5. Outline About the story ➡Umbo Computer Vision intro Develop and maintain CV services Build our machine learning pipelines What I have learned About the future My imagination
- 6. Autonomous video securitysystems
- 7. 400 million cameras ref:UK Police Scientific and Development Branch (PSDB) 12 or less cameras per operator - UK Police Scientific Department
- 8. Our Solution Existing cameras UmboLearning Cameras Real-time alerts Umbo Light
- 9. Infra We use alot of cloud services and have our own GPU clusters
- 10. Backend Languages All ourAI team members speak Python ❤ …
- 11. Flow ./enter_dev_env.sh + writecode with unit tests + code review + CI/CD
- 12. Monitoring We log, monitorand measure our services
- 13. Teams Hardware, Dev, CV,Infra, Research, PoC, QA, FAE …
- 14. Develop and maintainall computer vision and machine learning features The definition of an AI Engineer
- 15. 1. Develop andmaintain CV services 2. Build our machine learning pipelines Our jobs
- 16.
- 17. Outline About the story UmboComputer Vision intro ➡ Develop and maintain CV services Build our machine learning pipelines What I have learned About the future My imagination
- 18. • On productionin 2016 • From PoC projects to online services Previously on SmartCloud and Umbo Light
- 19. The Simplified CVArchitecture Media Server CV Forwarder Load Balancer ZMQ Stream Router Stream Manager Stream Manager Stream Manager Load Balancer CV Worker CV Worker CV Worker HTTP Alert Endpoint
- 20.
- 21.
- 22.
- 23. How much canthe backend do?
- 24. 87% How much thebackend can do
- 25. • Infra • LB/ ASG / VPC … • Monitoring • PaperTrail, Cloud Watch … • Development • Add new features • CI/CD • Maintenance • Upgrade packages • Refactor • Debug 87% MS CF LB SR SM SM SM LB CW CW CW AE
- 26. • Infra • LB/ ASG / VPC … • Monitoring • PaperTrail, Cloud Watch … • Development • Add new features • CI/CD • Maintenance • Upgrade packages • Refactor • Debug 87% MS CF LB SR SM SM SM LB CW CW CW AE
- 27. • Stream Router/Managerwas written by Python2 ❤ ❤ • Python2 => Python3 • byte string / decode • isinstance • format • API changes • Results • Python3 ❤ ❤ ❤ • Enable a lot of cool features • F-strings, typing, asyncio, tracemalloc, etc. 87%
- 28. • Refactor StreamManager • Happy to use async/await • Use pipeline pattern • Still use run_in_executor/ ThreadPoolExecutor if needed • Results • Cleaner architecture • Performance boost 87%
- 29. • Refactor CVWorker • Lua => Python • Torch => PyTorch • Use aiohttp server • Results • Easier to maintain/upgrade • High performance • GPU resources are the bottleneck • Python package ecosystem 👍 87%
- 30. • Debug • Usepyflame to profile the program • Use tracemaclloc to find memory usage • Use Valgrind to check your C++ code • Results • Understand more and gotcha! 87%
- 31. Ping me ifyou want to listen this story (and also give us a talk) in Tainan.py 😼
- 32. 13% How much thebackend cannot handle
- 33. • About MachineLearning • Not so easy to learn 13%
- 34. More than 13% Infact
- 35. • Computer Visionand Machine Learning • They are everywhere • The learning curve is steep More than 13% MS CF LB SR SM SM SM LB CW CW CW AE
- 36. You need todelve really deep into Computer Vision and Machine Learning to build a self-improving system
- 37. Outline About the story UmboComputer Vision intro Develop and maintain CV services ➡ Build our machine learning pipelines What I have learned About the future My imagination
- 38. Why machine learningpipelines? Improve services => customers 😆
- 39. How to improveyour services Measure first New models / heuristics approaches Need to do training or redesigning Need more (or specific) labeled data Training! Need to compare with the existing ones Get the best on production … Measure again 👷
- 40.
- 41. • Meet Michelangelo:Uber’s Machine Learning Platform • Machine Learning Pipeline for Real- Time Forecasting @Uber Marketplace Michelangelo (Uber)
- 42. How much canbackend do?
- 43.
- 44. Why 0%? 🐵 withoutdomain knowledge
- 45. Why 87%? With domainknowledge + good communication with others + speaking the same language (Python!)
- 46. Catch a glimpseof our pipelines
- 47. • Can dispatchtasks to • Our own labelers • Third-party partners • MTurkers • Backend: • Flask + plugins • NumPy • Celery/Redis/Pymongo • uWSGI Labeling Platform source: https://github.com/nightrome/cocostuff
- 48. • Can trainyou a lot of new models with different parameters • Backend: • Airflow • Celery Training Pipeline
- 49. • Can visualize •The ground truth data • The evaluation results • Backend: • Flask + plugins • NumPy • SQLAlchemy Visualization Platform
- 50. A faster pipelinemeans a shorter turnaround time! Speed matters for a startup
- 51. Outline About the story UmboComputer Vision intro Develop and maintain CV services Build our machine learning pipelines ➡ What I have learned About the future My imagination
- 52.
- 53.
- 54. Domain Knowledge Have abasic understanding of ML/CV with practical experience
- 55.
- 56. Outline About the story UmboComputer Vision intro Develop and maintain CV services Build our machine learning pipelines What I have learned About the future ➡ My imagination
- 57. from ai.backend import* Python, Golang, Java, C#, Javascript, PHP, Ruby, etc.
- 58. To build yourpipelines, Python is the best default language Pipeline → Service
- 59. Every young engineerknows machine learning But some of them may have not heard of GIL 😥
- 60. High-quality (ai) backendsare needed
- 61.
- 62. • Backend, CV,Front-end, App, QA, FAE, Firmware • joe.tseng@umbocv.com We are hiring