Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

"Embedded Vision Applications Lead Way for Processors in AI: A Market Analysis of Vision Processors," a Presentation from IHS Markit

80 views

Published on

For the full video of this presentation, please visit:
https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/may-2019-embedded-vision-summit-hackenberg

For more information about embedded vision, please visit:
http://www.embedded-vision.com

Tom Hackenberg, Principal Analyst at IHS Markit, presents the "Embedded Vision Applications Lead Way for Processors in AI: A Market Analysis of Vision Processors" tutorial at the May 2019 Embedded Vision Summit.

Artificial intelligence is not a new concept. Machine learning has been used for decades in large server and high performance computing environments. Why is it attracting so much attention, investment and innovation now? The answer, as with most disruptive technologies, is a convergence of technology enablers and market demands that previously were insufficient to drive scale.

In this presentation, Hackenberg examines the market for processors in AI applications. He illuminates key trends in adoption of different classes of processors in a range of key applications, and explains why embedded vision is at the heart of these trends. And he shows how this adoption is driving revenue for different classes of processors in markets including automotive, consumer, data processing, industrial and wired and wireless communications.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

"Embedded Vision Applications Lead Way for Processors in AI: A Market Analysis of Vision Processors," a Presentation from IHS Markit

  1. 1. Copyright © 2019 IHS Markit Embedded Vision Applications Lead Way for Processors in AI A Market Analysis of Vision Processors Presented for 2019 Embedded Vision Summit
  2. 2. Copyright © 2019 IHS Markit Agenda What is meant by Embedded Vision Leads the Way for AI? <Thesis> Introduction to Market Views of AI <Context> • Scope of AI • AI, machine learning, neural networks, deep learning Definitions • Neural networks and deep learning: training vs. inference • AI Market by Hierarchy • AI Market Common Functional Categories • Processor Strategies • Traditional Microcomponents • Graphics and Discrete AI Processors • System-on-Chip (SoC) Field Programmable Gate Arrays & cSoC Processors 2 AI applications focused on Embedded Vision <Body> • Automotive • ADAS, Cockpit Cameras • Consumer • Game Systems, Drones, Home Automation • Data Processing • Cloud-based Training, Social Media & Search Engines, Data Security • Industrial • Machine Vision, Security, Robotics, Medical • Wired Communications • Broadcast & Streaming, Network Infrastructure, Threat Mitigation • Wireless Communications • Smartphones, Tablets, Wearables Summary <Conclusions> 1 2 3 4
  3. 3. Copyright © 2019 IHS Markit What is meant by: “Embedded Vision Leads the Way for Processors in AI”
  4. 4. Copyright © 2019 IHS Markit Embedded Vision Leads the Way for Processors in AI • Computer vision research emerged in the 1950s to ‘mimic’ human vision. • By 2015, Microsoft, Google, and Baidu had all developed neural networking algorithms (running on supercomputers) capable of recognizing images on ImageNet with greater accuracy than the typical human (<5% error rates). 4 Vertical vision-specific analytics spread into many markets employing deep learning algorithms. Within the next year or two, virtually every processor supplier will offer a competitive platform for AI. AI, led today by demand for vision applications, is the next great transformative technology.
  5. 5. Copyright © 2019 IHS Markit Scope of AI
  6. 6. Copyright © 2019 IHS Markit AI, machine learning, neural networks, deep learning Definitions 6 Artificial intelligence (AI) refers to the body of science that studies how to enable machines to perform independent problem solving, inference, learning, knowledge representation and decision making Machine learning is a set of algorithms that gives machines the ability to automatically find and learn patterns by feeding them with data without explicit programming Neural Network and Deep Learning refer to computational models that try to emulate the structure and workings of a human brain, including training and inference Traditional Compute Instructions Raw Data Process Output Machine Learning Desired Output Training Data Process Output Update Operation Artificial Intelligence Machine Learning Neural Networks Deep Neural Networks
  7. 7. Copyright © 2019 IHS Markit Neural networks and deep learning: Training vs. inference 7 Training • In this phase, the network learns and increases its accuracy by comparing its solution to desired output, fed by enormous datasets. The model is built with characteristics, the weight of those characteristics and identifying incorrect results to re-evaluate these features. Inference • In this phase, the network uses the trained model to provide output based on comparing new data to the previously trained characterizations. Training: Advanced algorithms & deep learning Inference: Not safety critical, and as backup Cloud Data Centers Training: Low-to-high algorithms Inference: Process orchestration Industrial PC, Gateway, Server, DC, PLC Training: Very limited to certain devices and apps Inference: Broadest range of implementations End-Point Devices
  8. 8. Copyright © 2019 IHS Markit AI Market by Hierarchy 8 Latency Processingperformance Energyconsumption Development requirements for AI solutions Privacyandsecurity NetworkReliability Billions of Nodes Thousands of Nodes Millions of Nodes CLOUD Data Centers EDGE End-Point Devices EDGE Network Computing At the top of the pyramid, processing performance demands are high, but application nodes are few. Resources to provide performance, privacy and security are costly, MPUs, GPUs and discrete custom ML processors are common. At the edge, resources are limited, low-power and reduced latency can be critical. Resource optimized SoCs are common solutions. Converting NN to scalar for MCUs is a strategy being explored.
  9. 9. Copyright © 2019 IHS Markit AI Market Common Functional Categories 9 General Cloud-based Machine Learning Perception and Vision Processing Natural Language Processing Autonomous Control
  10. 10. Copyright © 2019 IHS Markit Processor Strategies
  11. 11. Copyright © 2019 IHS Markit Processor Strategies Microprocessor (MPU) • High performance good for brute force processing • On boards optimized for discrete cutting edge components • Complex instruction sets targeting a wide range of applications AI Target - training and cloud-based services Digital Signal Processor (DSP) & Microcontroller (MCU) • DSP with Harvard Architecture targets real-time applications • MCU targets low-power low-cost control functions AI Target – simple scalar ML for real-time data acquisition and control Early Microcomponent Trends AI Tools & SW converting ML to Scalar • Traditional processors like MPUs and MCUs are ubiquitous. SW tools are being developed to map trained neural networks to traditional scalar processing elements. Examples include • STM STM32Cube.AI • Arm Helium • TensorFlow Lite Machine Learning as a Service (MLaaS) - Examples include • AmazonML • Microsoft AzureML • Google Prediction API Traditional Micro component You may be surprised to learn that the vast majority of the AI functions are still processed on a standard MPU!
  12. 12. Copyright © 2019 IHS Markit Processor Strategies Graphics Processing Unit & General Purpose GPU • The GPU market is mature with significant install base. • Over half of all GPUs used for gaming and graphic design. • GPGPU applications are growing faster than graphics. GPUs can provide 100x performance improvement. Discrete Machine Learning Coprocessor • Discrete ML processors are an emerging market. • Most notable are the ASICs developed by cloud services providers such as the Tensor Processor developed by Google. AI processor design startups are attracting capital investment at unprecedented rates. SIMD limitations • Single instruction multiple data (SIMD). • SIMD processors described here are almost exclusively used as coprocessors. • An OS kernel, an application API, & human machine interfaces are inherently threaded to run linearly and would run poorer, if at all, on these SIMD structures. • Most software is abstracted and the machine code is relegated to the best processor for the task. Graphics Processor (GPU) and Discrete AI Coprocessor The transition from GPU to GPGPU and its boost to MPUs alone has been a strong enabler for AI market growth.
  13. 13. Copyright © 2019 IHS Markit Processor Strategies SoCs • Propelled by early demand for smartphones • Accommodates space, power & heat dissipation, cost and other constraints through integrating components and coprocessors Come in many application-specific designs including optimization for vision and artificial intelligence. FPGAs • Reputation as prototype solution, but market is much larger. • Lack of VHDL or Verilog engineers can still be a limitation. Configurability enables wide parallel processing strategies well suited for AI. Configurable SoCs • Xilinx and Intel (Altera) have developed configurable SoCs following the heterogeneous design trends of SoCs. • cSoC use applications processor cores identical to SoCs, but integrate configurable logic similar to an FPGA. • This strategy has adapted to AI demand. Xilinx Adaptive Compute Acceleration Platform (ACAP) is an example with integrated ML acceleration. System-on-Chip (SoC), Field Programmable Gate Array (FPGA) & Configurable SoC As Moore’s Law, based solely on miniaturization becomes unsustainable, the processor market turns to integration.
  14. 14. Copyright © 2019 IHS Markit AI Applications focused on Embedded Vision
  15. 15. Copyright © 2019 IHS Markit Automotive Electronics Automotive Trends • Automobile sales—<100m in 2018. • Vehicle Electronic Control Units—vehicle electronics cost rising from $1200 in 2015 to $1700 by 2023. • Domain Controllers— decentralized electronic control units yielding to centralized domain controllers • Barriers to Entry—AI and vision expertise lowering barriers for new suppliers. Cockpit ADAS Training ITS Cloud ITS & V2I ADAS Advanced Driver Assistance Systems (ADAS) & Cockpit Market *Data Processing Market *Wireless Communications Market
  16. 16. Copyright © 2019 IHS Markit Automotive Electronics Trends directly impacting Processors for AI • ADAS dominates In-vehicle AI market short term. • The primary ADAS AI platforms are SoCs with graphics and other AI subsystems targeting embedded vision. • DCs will reduce ECUs, but AI performance will grow revenue as SoC platforms & ADAS systems become more sophisticated. • Intelligence may still be distributed with near camera imaging and DC control in the long term. • Platforms from suppliers with significant software & tools development are winning solutions. • GPUs are currently exclusively in some Nvidia platforms. 0 500 1,000 1,500 2018 2023 MPU MCU DSP GPU SoC PLD Discrete ML Processor Revenue$(M) Annual Revenue per Processor Class Advanced Driver Assistance Systems (ADAS) and Cockpit Market AI Processor Market for Automotive Source: IHS Markit
  17. 17. Copyright © 2019 IHS Markit Consumer Electronics Consumer Trends • Highly competitive — Cost & time-to-market are barriers for growth of AI in Consumer • Voice Analytics — Besides the “wake word”, almost all voice analytics for HA is processed in the cloud. There is demand for increased local analytics for privacy and security. • Region — The Asian markets are growing significantly more rapidly in Consumer AI markets. Game Consoles Game & Home Automation Hosts Gateways & HA Hubs Cobots & Drones Game Consoles , Home Automation, and Consumer Robots & Drones *Data Processing Market *Wired Communications Market
  18. 18. Copyright © 2019 IHS Markit Consumer Electronics Trends directly impacting Processors for AI • Most advanced consumer AI applications are cloud-based; this is wholly true of training. • The resources required for vision analytics and natural voice speaking are costlier than are typical of the consumer mass market. • MCUs, DSPs and entry level SoCs are common for consumer because of the constraints (cost, area, power, time-to-market, etc.). • For home automation, there is demand for advanced gateways to improve the HMI, privacy and security. The target SoCs include improved graphics and even NN acceleration integrated. • Low-cost AI specific discretes targeting edge nodes are expected to emerge in consumer applications. AI Processor Market for Consumer 0 500 1,000 1,500 2,000 2,500 2018 2023 MPU MCU DSP GPU SoC PLD Discrete ML Processor Revenue$(M) Annual Revenue per Processor Class Game Consoles , Home Automation, and Consumer Robots & Drones Source: IHS Markit
  19. 19. Copyright © 2019 IHS Markit Data Processing Data Processing Trends • Training continues to be the largest AI function for data processing. This is likely to be the case for a very long time. • Despite limited CAPEX for data center & IT, the training of AI is driving growth in the overall server market. More dramatic than server demand, is the increase in BoM. HPC and servers with coprocessors for running AI are increasing penetration rapidly. Social Media & Search Cloud Training Financial ID PC Facial ID Cloud-based Training, Social Media & Search Engines, Data Security
  20. 20. Copyright © 2019 IHS Markit Data Processing Trends directly impacting Processors for AI • Despite marketing espousing the advantages of specialized processors for AI, the vast majority of processor revenue in 2018 was still on MPUs. • The end of Moore’s Law is being felt first in the cutting edge processor markets such as MPUs. • It is only in the last 5-10 years that the total cost of ownership in data centers including systems and power for cooling has driven more interest in performance/watt metrics favoring coprocessors. • GPGPU and its synergy between embedded vision applications and multimedia processing has provided the leverage for significant recent growth of GPUs in data processing. • VLIW, TPUs and discrete ML processors will gain rapidly, especially outside graphics. 0 1,000 2,000 3,000 4,000 5,000 6,000 7,000 2018 2023 MPU MCU DSP GPU SoC PLD Discrete ML Processor Revenue$(M) Annual Revenue per Processor Class AI Processor Market for Data Processing Cloud-based Training, Social Media & Search Engines, Data Security Notes: Primary Drivers are Training, Social Media & Search Engines, Data Security Source: IHS Markit Source: IHS Markit
  21. 21. Copyright © 2019 IHS Markit Industrial Electronics Industrial Trends • Industrial markets are complex, diverse and slow to adopt to technology changes. Time-to-market can take years and equipment expected to last decades. • Some applications may require working in harsh conditions such as infrastructure, military & aerospace, process automation, environmental sciences.Robots & Drones Security Medical Machine Vision Machine Vision, Security, Robots and Drones, Medical
  22. 22. Copyright © 2019 IHS Markit Industrial Electronics Trends directly impacting Processors for AI • Industrial & Commercial applications are diverse and the mix of processors for AI is just as diverse. However, they still follow some patterns: • MPUs & GPUs are used where cutting edge performance, cost, power, and size are not barriers. • MCUs and DSPs are common, but too resource constrained for most neural networking. Small scale sensor hubs controlled by scalar-based ML are an emerging market . • SoCs will tend to be the typical edge node solutions & FGPAs in edge networks • Solutions based on unique constraints in specific applications will be critical. 0 500 1,000 1,500 2,000 2018 2023 MPU MCU DSP GPU SoC PLD Discrete ML Processor Revenue$(M) Annual Revenue per Processor Class Machine Vision, Security, Robots and Drones, Medical AI Processor Market for Industrial Source: IHS Markit
  23. 23. Copyright © 2019 IHS Markit Wired Communications Wired Communication Trends • Data center networks & network security represent coprocessor growth markets for wired communications. • Solutions for AI are at the edge, on-premises for security and privacy, and on the service provider side for network integrity & quality of service. Intelligent gateways are targets for incumbents and emerging providers. Threat Mitigation Broadcast Services Network Infrastructure Broadcast & Streaming, Network Infrastructure, Threat Mitigation
  24. 24. Copyright © 2019 IHS Markit Wired Communications Trends directly impacting Processors for AI • Outside of computers, network infrastructure and security systems consume more MPUs than any other market. • Because, networking has had little association with HMI, there is little demand for GPUs outside of broadband streaming and other multimedia services, Recently, the use of AI for network QoS is driving additional demand for GPUs. • FPGAs, due to their highly parallel logic, are common solutions for packet-forwarding and AI optimization for networking. • SoCs are common, however, tend to be optimized for packet forwarding. NextGen SoCs and discrete coprocessors include a good deal more AI- specific acceleration such as integrated TPUs. 0 500 1,000 2018 2023 MPU MCU DSP GPU SoC PLD Discrete ML Processor Revenue$(M) Annual Revenue per Processor Class Broadcast & Streaming, Network Infrastructure, Threat Mitigation AI Processor Market for Wired Communications Source: IHS Markit
  25. 25. Copyright © 2019 IHS Markit Wireless Communications Wireless Communication Trends • The wireless market has matured, replacement rates and new users have slowed dramatically. • The wearables market represents a transformative technology to open new market opportunities. • 5G represents a way of growing the connected device market size, but AI represents the greater opportunity for transforming what is possible. Smartphones & Tablets Wireless Infrastructure Wearables Smartphones & Tablets, Wearables & Wireless Infrastructure
  26. 26. Copyright © 2019 IHS Markit Wireless Communications Trends directly impacting Processors for AI • While the concept of SoC has been around since the MCU, it was the smartphone market that really drove the development of SoCs for applications processors into a $50 billion + market. • MPUs, GPUs and FPGAs are too resource demanding for portable devices, but for wireless infrastructure, the performance and parallel processing is a valuable asset and the constraints are much less limiting • MCUs and DSPs are common in the wireless market, but their effectiveness as an AI processor is limited. • Discrete ML processor development is expected to target wireless applications heavily, but not dislodge SoCs. 0 2,000 4,000 6,000 8,000 10,000 12,000 14,000 16,000 2018 2023 MPU MCU DSP GPU SoC PLD Discrete ML Processor Revenue$(M) Annual Revenue per Processor Class Smartphones & Tablets, Wearables & Wireless Infrastructure AI Processor Market for Wireless Communications Source: IHS Markit
  27. 27. Copyright © 2019 IHS Markit Conclusion
  28. 28. Copyright © 2019 IHS Markit Conclusion Processors Market for AI Summary • Processors used for AI applications are expected to well over triple by 2023 topping $35 billion. • In 2018, the largest processor revenue for AI was from MPUs used to train AI applications, and that will continue to grow. optimized servers with GPUs, FPGAs, discrete ML accelerators and other coprocessors will grow faster than the GP market. • In 2023, the largest portion of processors running inferencing will be SoCs in smartphones, but with significant interest in SoCs for ADAS. • Development in AI in some markets, such as ADAS, is transforming the industry. • Not all machine learning is neural networking. There are strategies for translating the NN code to run a scalar equivalent – MPUs & MCUs. These scalars can still be used for ML. 0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 2018 2023 MPU MCU DSP GPU SoC PLD Discrete ML Processor Revenue$(M) Annual Revenue per Processor Class Key Takeaways Processor Market For AI Summary Source: IHS Markit
  29. 29. Copyright © 2019 IHS Markit 29 Questions? IHS Markit Customer Care CustomerCare@ihsmarkit.com Americas: +1 800 IHS CARE (+1 800 447 2273) Europe, Middle East, and Africa: +44 (0) 1344 328 300 Asia and the Pacific Rim: +604 291 3600 Disclaimer The information contained in this presentation is confidential. Any unauthorized use, disclosure, reproduction, or dissemination, in full or in part, in any media or by any means, without the prior written permission of IHS Markit Ltd. or any of its affiliates ("IHS Markit") is strictly prohibited. IHS Markit owns all IHS Markit logos and trade names contained in this presentation that are subject to license. Opinions, statements, estimates, and projections in this presentation (including other media) are solely those of the individual author(s) at the time of writing and do not necessarily reflect the opinions of IHS Markit. Neither IHS Markit nor the author(s) has any obligation to update this presentation in the event that any content, opinion, statement, estimate, or projection (collectively, "information") changes or subsequently becomes inaccurate. IHS Markit makes no warranty, expressed or implied, as to the accuracy, completeness, or timeliness of any information in this presentation, and shall not in any way be liable to any recipient for any inaccuracies or omissions. Without limiting the foregoing, IHS Markit shall have no liability whatsoever to any recipient, whether in contract, in tort (including negligence), under warranty, under statute or otherwise, in respect of any loss or damage suffered by any recipient as a result of or in connection with any information provided, or any course of action determined, by it or any third party, whether or not based on any information provided. The inclusion of a link to an external website by IHS Markit should not be understood to be an endorsement of that website or the site's owners (or their products/services). IHS Markit is not responsible for either the content or output of external websites. Copyright © 2018, IHS MarkitTM. All rights reserved and all intellectual property rights are retained by IHS Markit.

×