Download to read offline
![“HW design was getting a little bit
boring. […]
The rise of machine learning (deep
learning) has changed all that.”
Pete Warden - https://petewarden.com/2019/04/14/what-machine-learning-needs-from-hardware/](https://image.slidesharecdn.com/mlontheedgecompressed-191212144027/75/Machine-learning-at-the-edge-33-2048.jpg)
More Related Content
Similar to Machine learning at the edge
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
Machine learning at the edge
- 1. NEW GUY 0.93675 MACHINE LEARNING ATTHE EDGE What’s behind the new tech wave? INFO AND DEMO EVENING WED DECEMBER 11TH 16:00-17:30 HTC 27 GREENHOUSE
- 2. ML at theedge is punching above its weight
- 8.
- 9. ML at theedge: “bringing computation to where the data is.”
- 11. What are thepossible applications that would enjoy ML at the edge?
- 13. Machine learning hadto prove itself in the field.
- 15. Deep Learning –quick recap
- 17. “training scales withthe number of researchers, inference scales with the number of applications times the number of users“ Pete Warden - https://petewarden.com/2019/04/14/what-machine-learning-needs-from-hardware/
- 18. Success of MLdepends on computational power.
- 20. …much more thanwhat we can environmentally sustain.
- 22. …much more thanwhat we can intellectually sustain.
- 24. Today, ML productsare cloud based.
- 26. Network bandwidth failsto keep up with the amount of data generated. Latency will continue to be significant.
- 27. Problem #1 Applications arenot autonomous
- 28. Problem #2 Data privacyis lacking
- 29. Problem #3 Latency issubstantial
- 30. Problem #4 Network connectionrequires energy
- 31. ML applications provedthemselves in principle. Chips follow. Because the growth is predictable.
- 32. Result: Huge downwindfrom ASIC industry
- 33. “HW design wasgetting a little bit boring. […] The rise of machine learning (deep learning) has changed all that.” Pete Warden - https://petewarden.com/2019/04/14/what-machine-learning-needs-from-hardware/
- 34. “improvements in latencyand energy usage” “new or enhanced user experiences.” Pete Warden - https://petewarden.com/2019/04/14/what-machine-learning-needs-from-hardware/
- 35.
- 36. How should MLat the edge HW be designed?
- 37.
- 38. Old internal presentationfrom 2017
- 39. DevB AVAILABILITY $250$600 GPU Video Processing $300 €68,51 Nicestart Startup acquired by Intel $100 $50 USA EU? $90 USA EU? NPU
- 40. 2018-2019 has seentremendous growth in AI ASIC
- 41.
- 42.
- 43.
- 44.
- 46. There are waysto adapt ML models to the edge
- 47. Pruning Sparsity (structured) Reduce precision Reducedata movement (compression)
- 48. Software (+dev env)landscape is also complicated. (Old internal presentation from 2017)
- 50. There will beevolution and elimination.
- 51. There are benchmarksfor ML, and it is really hard.
- 52. Communities are forming,and makers are ecstatic.
- 54. “Future of MLis tiny.” Pete Warden - https://petewarden.com/2018/06/11/why-the-future-of-machine-learning-is-tiny/
- 55.
- 56. • https://slideslive.com/38921492/efficient-processing-of-deep-neural- network-from-algorithms-to-hardware-architectures Efficient Processingof Deep Neural Network: from Algorithms to Hardware Architectures by Vivienne Sze · Dec 9, 2019 • Pete Warden and Daniel Situnayake • Nvidia deep learning institute • We arrange workshops @ Grus
- 57.
Editor's Notes
- #8 Emphasis on being maker-level applications here. They were not possible a few years ago on the platforms that they run on.
- #9 But why? My view on devices being company rather than a tool. It is the ultimate collaboration, as we do with humans.
- #12 But why? My view on devices being company rather than a tool. It is the ultimate collaboration, as we do with humans.
- #15 Machines that emulate human intuition and attention That is how I look at ML market, is intuition useful here? Does it require attention?
- #17 What do we have to compute? A lot of multiplications and additions A lot of memory usage Relatively low control operations (less human intervention, programmability) There is training and inference.
- #18 Pete Warden’s friend Does not imply importance. Abundance, proliferation of
- #20 Logarithmic y axis.
- #22 ML training has a great carbon footprint
- #24 Dennard Scaling Moore’s law is a self-fulfilling prophecy. We find a way. We are on the verge of a new inflection. ML HW is the new way. Therefore -> Specialization, thus huuuuge market movement in ASIC industry, and startup community
- #26 200ms ideal, 2 seconds OK
- #27 But why? My view on devices being company rather than a tool. It is the ultimate collaboration, as we do with humans.
- #33 In order to address these limitations, ASIC industry reinvented the computational node.
- #34 … since the requirements seemed well understood and it was mostly just an exercise in iterating on existing ideas. AI required a computational paradigm shift. But we didn’t know to how it would be. We had to track the developments for a innovative leap. We have an idea now. Deep learning, Convolutional, But still experimenting. But experimentation is within budget now and will pay off. Q:Simon Craske from Arm
- #36 …that even though most of the compute for almost all models does go into a handful of common operations, there are hundreds of others that often appear.
- #37 In order to address these limitations, ASIC industry reinvented the computational node.
- #39 But why? My view on devices being company rather than a tool. It is the ultimate collaboration, as we do with humans.
- #41 But why? My view on devices being company rather than a tool. It is the ultimate collaboration, as we do with humans.
- #43 They can be separated wrt to power/performance.
- #47 Algorithm side.
- #49 The research is so fast that tools come and go from the spotlight.
- #50 Fragmented and constantly changing
- #51 But why? My view on devices being company rather than a tool. It is the ultimate collaboration, as we do with humans.
- #52 But how to eliminate? Only by trial and error for now But people are working. MLPerf
- #53 But why? My view on devices being company rather than a tool. It is the ultimate collaboration, as we do with humans.
- #54 Hotchips became ML dominated (requires novel VLSI tech, that is why. Pushes IPC bandwidth, RAM bandwidth, efficient high dimensional tensor ops, thermal and lithographic novelties.) TinyML Hackster, Hackaday, Arduino, Sparkfun
- #55 Tiny Computers are Already Cheap and Everywhere Energy is the Limiting Factor CPUs and Sensors Use Almost No Power, Radios and Displays Use Lots We Capture Much More Sensor Data Than We Use What This All Means For Machine Learning Deep Learning is Compute-Bound and Runs Well on Existing MCUs Deep Learning Can Be Very Energy-Efficient Deep Learning Makes Sense of Sensor Data
- #56 But why? My view on devices being company rather than a tool. It is the ultimate collaboration, as we do with humans.
- #58 But why? My view on devices being company rather than a tool. It is the ultimate collaboration, as we do with humans.