SlideShare a Scribd company logo
How Axelera AI Uses Digital
Compute-in-Memory to
Deliver Fast and Energy-
Efficient Computer Vision
Bram Verhoef
Head of Machine Learning & Co-Founder
Axelera AI
Compute and Intelligence at Different Layers
2
© 2024 Axelera AI
The Edge
New AI Applications Are Emerging at the Edge
3
© 2024 Axelera AI
Retail
Inventory management
Cashier-less checkouts
Security
Traffic control systems
Intelligent surveillance
Agriculture
Crop health monitoring
Automated pest control
Health
Real-time diagnostics tools
Surgical tools & equipment
Industrial
Quality control automation
Worker safety monitoring
Auto
Driver assistance systems
Autonomous driving systems
ΑΙ Is Moving From the Cloud to the Edge
4
© 2024 Axelera AI
Mainframe Client-server Cloud Edge
Centralized Distributed Centralized Distributed
~10M mainframes ~2B PCs ~50B devices Trillions of devices
$$$$ $$$ $$ $
1960 - 1980 1980 - 2005 2005 - Today Tomorrow
Role tomorrow:
Training and data
storage
Role tomorrow:
Sensing, inference &
automation
Emerging AI edge applications require performance and
accuracy, energy efficiency, and low price
5
© 2024 Axelera AI
Fast, Accurate, Energy-Efficient, and Cost-Effective AI Inference
With Digital Compute-In-Memory (D-IMC)
Metis - AI Platform
6
© 2024 Axelera AI
 AI edge inference accelerator
 M.2 module or PCIe card
 Metis AIPU executes all tasks of
an AI workload
 Offload complete network(s)
 Not just individual layers
 Easy-to-use software stack
 Voyager SDK combining
compilation and quantization flow
Host
PCI-E card connected
to host
AI computer vision applications at the
edge
Metis AI Processing Unit (AIPU)
7
© 2024 Axelera AI
 Quad-core System-on-Chip
 RISC-V controlled
 Security
 PCIe 3.0 4x link to host
 LPDDR4x
 Large on-chip SRAM capacity
 AI-Core powered by D-IMC
 52.4 TOPS @ INT8
(209.6 TOPS aggregate)
 15 TOPS/W energy efficiency
AI
Core
AI Core
AI Core
AI Core
RISC-V System Controller
L2
Memory
LPDDR4x
Security PCIe 3.0
(x4)
Digital In-Memory Computing (D-IMC)
8
© 2024 Axelera AI
4 weight sets
 SRAM-based D-IMC
 Interleaved weight-storage and
compute units in an extremely
dense fashion
 Immune to noise and memory
non-idealities affecting analog
IMC precision
 INT8 activations / weights, with INT32
accumulation to maintain
full precision
 Technology commensurate with CMOS
scaling to low lithography nodes
D-IMC Differentiating Improvements
9
© 2024 Axelera AI
1. Stores multiple weight sets in computational memory
• Enhances IMC storage density
• Allows accumulation up to 16k inputs
• Enables simultaneous processing
and weight reloading
2. Activity gating and clock gating
• Maintains high energy efficiency at low utilization
3. Ensures full-precision accumulation
• Negligible accuracy loss compared to FP32
• Use of post-training quantization;
no need for retraining
AI Core – Key Components
10
© 2024 Axelera AI
 Matrix-Vector Multiplier (MVM)
 D-IMC based
 512 inputs x 512 outputs (4 weight sets)
 INT8 inputs and weights
 Data Processing Unit (DPU)
 Element-wise vector operations
 Apply activation functions
 Depth-Wise Processing Unit (DWPU)
 Depth-wise convolution
 Pooling and Up-sampling
 4 MiByte L1 SRAM
 RISC-V control core
NoC (Network on chip)
AI Core – Deployment Scenarios
11
© 2024 Axelera AI
 A single AI core
 Can execute all layers of a neural network
 Eliminates need for external interactions
 MVM
 Flexibile deployment of multiple AI
cores
 Manage different neural networks
independently
− In multi-network applications
 Jointly tackle a workload to enhance throughput
 Work on same neural network to reduce latency
RISC-V System Controller
L2
32MB
LPDDR4x
Security PCIe 3.0
(x4)
AI Core
AI Core
AI Core
AI Core
Network 1 Network 2
Network 3
Software Development Flow
12
© 2024 Axelera AI
Tensor
ops
Image
ops
Host Non-NN code
 eGPU (Intel/Mali)
 VA-API
 CPU SIMD
Model
Post-processing
ML Model
 Weights
 Dataset
 Metrics
Model Zoo
Sample
Pipelines
Trained Model
PyTorch
ONNX
TensorFlow
Compilation
ML Pipeline
Definition
Performance &
Accuracy Evaluation
Application &
Runtime Integration
Model
Pre-processing
Metis ML code
 Quantization
 Graph optimization
 Lowering
Inference Pipeline
Business Logic
Application Image
Processing
Input Stream(s)
Image Stream
Axelera Inference
Element
Metadata
Inference Pipeline
(GStreamer)
Runs on host CPU/GPU (x86 / ARM)
Runs on Metis
Voyager Build Environment Voyager Runtime Environment
Metis AIPU SoC Performance
13
© 2024 Axelera AI
Deviation from
FP32 accuracy
92 FPS/W
354 FPS/W
YOLOv5s on Metis – Demo Preview
14
© 2024 Axelera AI
496 FPS
YoloV5s
inference
@640x640
Running YoloV5s on 24 Streams on a Single Metis
Chip
15
© 2024 Axelera AI
24 RTSP streams
15FPS/stream
1 Metis Chip
Product Line-Up
16
© 2024 Axelera AI
Modules Cards Boards Systems
Metis M.2 ​
159 USD
AI acceleration to systems
with an M.2 2280M slot
where space is at a
premium
Metis PCIe​
212 USD​
PCIe cards with 1x or 4x
Metis AIPUs for Edge
Servers where AI
performance and flexibility
is a priority
Single Board
Computer
Price upon request
ARM ​(Rockchip RK3588)​
For stand-alone and compact
form factor embedded systems
Partner products
Price upon request
x86 Edge Servers, Industrial
PC’s
Ready to use devices for edge or
near edge processing where out-
of-the-box systems are needed
Evaluation Kits to get stated
17
© 2024 Axelera AI
Dell Precision 3460XE
Advantech ARC-3534
Lenovo ThinkStation P360
Advantech MIC-770
Industrial PC Industrial PC
Edge Server PC
Edge Server PC
Firefly ITX-3588J
Embedded ARM
Metis Evaluation Kits
Edge Host
Systems
Dell Precision 3460XE SFF Core i7
LENOVO ThinkStation P360 ULTRA Core i5
Advantech ARC-3534B Core i5, Industrial PC
Advantech MIC-770v3W Core i5, Industrial PC
Firefly ITX-3588J, 8-core ARM, embedded
AI Acceleration Axelera Metis PCIe, 214 TOPS (int8)
PCIe PCIe 3.0 (x4), HHHL size, 64 x 168 x 40 (mm)
ML frameworks
PyTorch / ONNX / TensorFlow (via ONNX)
Axelera Voyager SDK
Neural Networks
Detection: YOLOv5s / m / l / YOLOv7 / SSD-MobileNetV2
Classification: Resnet-50 / MobileNetV2 / and more
Pre-compiled optimized models and compiler supported
OS Ubuntu Desktop v22.04, v20.04 (w/ Docker)
 Metis AIPU SoC is an innovative and advanced digital
compute-in-memory inference solution for optimized AI
computer vision applications
 Metis delivers fast, energy-efficient, cost-effective and
accurate AI inference
 Voyager SDK supports deep learning out-of-the-box
Summing Up: Powerful, Efficient and Cost-Effective AI
18
© 2024 Axelera AI
Metis evaluation kits available now to get started
 https://www.axelera.ai
 Products: https://www.axelera.ai/ai-acceleration-hardware-
products
 Metis: https://www.axelera.ai/metis-aipu
 Voyager SDK: https://www.axelera.ai/ai-software
 Evaluation Kits: https://www.axelera.ai/metis-evaluation-kit
Resources
19
© 2024 Axelera AI
20
© 2024 Axelera AI
Thank You!
Visit us at the Axelera booth (#510)!!!

More Related Content

Similar to “How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-efficient Computer Vision,” a Presentation from Axelera AI

“The Future of AI is Here Today: Deep Dive into Qualcomm’s On-Device AI Offer...
“The Future of AI is Here Today: Deep Dive into Qualcomm’s On-Device AI Offer...“The Future of AI is Here Today: Deep Dive into Qualcomm’s On-Device AI Offer...
“The Future of AI is Here Today: Deep Dive into Qualcomm’s On-Device AI Offer...
Edge AI and Vision Alliance
 
"The Xilinx AI Engine: High Performance with Future-proof Architecture Adapta...
"The Xilinx AI Engine: High Performance with Future-proof Architecture Adapta..."The Xilinx AI Engine: High Performance with Future-proof Architecture Adapta...
"The Xilinx AI Engine: High Performance with Future-proof Architecture Adapta...
Edge AI and Vision Alliance
 
Deeplearningusingcloudpakfordata
DeeplearningusingcloudpakfordataDeeplearningusingcloudpakfordata
Deeplearningusingcloudpakfordata
Ganesan Narayanasamy
 
Superior Cloud Economics with Power Systems
Superior Cloud Economics with Power Systems Superior Cloud Economics with Power Systems
Superior Cloud Economics with Power Systems
IBM Power Systems
 
5 Things to Know about Virtualization on Compact PCI Serial
5 Things to Know about Virtualization on Compact PCI Serial5 Things to Know about Virtualization on Compact PCI Serial
5 Things to Know about Virtualization on Compact PCI Serial
MEN Mikro Elektronik GmbH
 
5 Things to Know about Virtualization on Compact PCI Serial
5 Things to Know about Virtualization on Compact PCI Serial5 Things to Know about Virtualization on Compact PCI Serial
5 Things to Know about Virtualization on Compact PCI Serial
MEN Micro
 
High Performance Object Storage in 30 Minutes with Supermicro and MinIO
High Performance Object Storage in 30 Minutes with Supermicro and MinIOHigh Performance Object Storage in 30 Minutes with Supermicro and MinIO
High Performance Object Storage in 30 Minutes with Supermicro and MinIO
Rebekah Rodriguez
 
Arm Neoverse market update_05122020.pdf
Arm Neoverse market update_05122020.pdfArm Neoverse market update_05122020.pdf
Arm Neoverse market update_05122020.pdf
Paul Yang
 
IBM Power Systems at FIS InFocus 2019
IBM Power Systems at FIS InFocus 2019IBM Power Systems at FIS InFocus 2019
IBM Power Systems at FIS InFocus 2019
Paula Koziol
 
20230614 LinuxONE Distinguished_Recognition ISSIP_Award_Talk.pptx
20230614 LinuxONE Distinguished_Recognition ISSIP_Award_Talk.pptx20230614 LinuxONE Distinguished_Recognition ISSIP_Award_Talk.pptx
20230614 LinuxONE Distinguished_Recognition ISSIP_Award_Talk.pptx
International Society of Service Innovation Professionals
 
Drive Data Center Efficiency with SuperBlade, Powered by AMD EPYC™ and Instinct™
Drive Data Center Efficiency with SuperBlade, Powered by AMD EPYC™ and Instinct™Drive Data Center Efficiency with SuperBlade, Powered by AMD EPYC™ and Instinct™
Drive Data Center Efficiency with SuperBlade, Powered by AMD EPYC™ and Instinct™
Rebekah Rodriguez
 
Netronome Corporate Brochure
Netronome Corporate BrochureNetronome Corporate Brochure
Netronome Corporate Brochure
Netronome
 
Ibm Power System E850 pod03108 usen
Ibm Power System E850  pod03108 usenIbm Power System E850  pod03108 usen
Ibm Power System E850 pod03108 usen
Diego Alberto Tamayo
 
IoT based Industrial Gateway (IoT-SDK) built around Sitara™ AM437x processors...
IoT based Industrial Gateway (IoT-SDK) built around Sitara™ AM437x processors...IoT based Industrial Gateway (IoT-SDK) built around Sitara™ AM437x processors...
IoT based Industrial Gateway (IoT-SDK) built around Sitara™ AM437x processors...
Mistral Solutions
 
Partner Keynote: Intel - The New Frontier of Cloud Computing
Partner Keynote: Intel - The New Frontier of Cloud ComputingPartner Keynote: Intel - The New Frontier of Cloud Computing
Partner Keynote: Intel - The New Frontier of Cloud Computing
Amazon Web Services
 
CCNA (R & S) Module 02 - Connecting Networks - Chapter 7
CCNA (R & S) Module 02 - Connecting Networks - Chapter 7CCNA (R & S) Module 02 - Connecting Networks - Chapter 7
CCNA (R & S) Module 02 - Connecting Networks - Chapter 7
Waqas Ahmed Nawaz
 
White Box Hardware Challenges in the 5G & IoT Hyperconnected Era
White Box Hardware Challenges in the 5G & IoT Hyperconnected EraWhite Box Hardware Challenges in the 5G & IoT Hyperconnected Era
White Box Hardware Challenges in the 5G & IoT Hyperconnected Era
Charo Sanchez
 
Vmware certified IBM Servers
Vmware certified IBM ServersVmware certified IBM Servers
Vmware certified IBM Servers
TTEC
 
Presentation1.pptx
Presentation1.pptxPresentation1.pptx
Presentation1.pptx
MuhammadTalha563553
 
Cisco connect montreal 2018 compute v final
Cisco connect montreal 2018   compute v finalCisco connect montreal 2018   compute v final
Cisco connect montreal 2018 compute v final
Cisco Canada
 

Similar to “How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-efficient Computer Vision,” a Presentation from Axelera AI (20)

“The Future of AI is Here Today: Deep Dive into Qualcomm’s On-Device AI Offer...
“The Future of AI is Here Today: Deep Dive into Qualcomm’s On-Device AI Offer...“The Future of AI is Here Today: Deep Dive into Qualcomm’s On-Device AI Offer...
“The Future of AI is Here Today: Deep Dive into Qualcomm’s On-Device AI Offer...
 
"The Xilinx AI Engine: High Performance with Future-proof Architecture Adapta...
"The Xilinx AI Engine: High Performance with Future-proof Architecture Adapta..."The Xilinx AI Engine: High Performance with Future-proof Architecture Adapta...
"The Xilinx AI Engine: High Performance with Future-proof Architecture Adapta...
 
Deeplearningusingcloudpakfordata
DeeplearningusingcloudpakfordataDeeplearningusingcloudpakfordata
Deeplearningusingcloudpakfordata
 
Superior Cloud Economics with Power Systems
Superior Cloud Economics with Power Systems Superior Cloud Economics with Power Systems
Superior Cloud Economics with Power Systems
 
5 Things to Know about Virtualization on Compact PCI Serial
5 Things to Know about Virtualization on Compact PCI Serial5 Things to Know about Virtualization on Compact PCI Serial
5 Things to Know about Virtualization on Compact PCI Serial
 
5 Things to Know about Virtualization on Compact PCI Serial
5 Things to Know about Virtualization on Compact PCI Serial5 Things to Know about Virtualization on Compact PCI Serial
5 Things to Know about Virtualization on Compact PCI Serial
 
High Performance Object Storage in 30 Minutes with Supermicro and MinIO
High Performance Object Storage in 30 Minutes with Supermicro and MinIOHigh Performance Object Storage in 30 Minutes with Supermicro and MinIO
High Performance Object Storage in 30 Minutes with Supermicro and MinIO
 
Arm Neoverse market update_05122020.pdf
Arm Neoverse market update_05122020.pdfArm Neoverse market update_05122020.pdf
Arm Neoverse market update_05122020.pdf
 
IBM Power Systems at FIS InFocus 2019
IBM Power Systems at FIS InFocus 2019IBM Power Systems at FIS InFocus 2019
IBM Power Systems at FIS InFocus 2019
 
20230614 LinuxONE Distinguished_Recognition ISSIP_Award_Talk.pptx
20230614 LinuxONE Distinguished_Recognition ISSIP_Award_Talk.pptx20230614 LinuxONE Distinguished_Recognition ISSIP_Award_Talk.pptx
20230614 LinuxONE Distinguished_Recognition ISSIP_Award_Talk.pptx
 
Drive Data Center Efficiency with SuperBlade, Powered by AMD EPYC™ and Instinct™
Drive Data Center Efficiency with SuperBlade, Powered by AMD EPYC™ and Instinct™Drive Data Center Efficiency with SuperBlade, Powered by AMD EPYC™ and Instinct™
Drive Data Center Efficiency with SuperBlade, Powered by AMD EPYC™ and Instinct™
 
Netronome Corporate Brochure
Netronome Corporate BrochureNetronome Corporate Brochure
Netronome Corporate Brochure
 
Ibm Power System E850 pod03108 usen
Ibm Power System E850  pod03108 usenIbm Power System E850  pod03108 usen
Ibm Power System E850 pod03108 usen
 
IoT based Industrial Gateway (IoT-SDK) built around Sitara™ AM437x processors...
IoT based Industrial Gateway (IoT-SDK) built around Sitara™ AM437x processors...IoT based Industrial Gateway (IoT-SDK) built around Sitara™ AM437x processors...
IoT based Industrial Gateway (IoT-SDK) built around Sitara™ AM437x processors...
 
Partner Keynote: Intel - The New Frontier of Cloud Computing
Partner Keynote: Intel - The New Frontier of Cloud ComputingPartner Keynote: Intel - The New Frontier of Cloud Computing
Partner Keynote: Intel - The New Frontier of Cloud Computing
 
CCNA (R & S) Module 02 - Connecting Networks - Chapter 7
CCNA (R & S) Module 02 - Connecting Networks - Chapter 7CCNA (R & S) Module 02 - Connecting Networks - Chapter 7
CCNA (R & S) Module 02 - Connecting Networks - Chapter 7
 
White Box Hardware Challenges in the 5G & IoT Hyperconnected Era
White Box Hardware Challenges in the 5G & IoT Hyperconnected EraWhite Box Hardware Challenges in the 5G & IoT Hyperconnected Era
White Box Hardware Challenges in the 5G & IoT Hyperconnected Era
 
Vmware certified IBM Servers
Vmware certified IBM ServersVmware certified IBM Servers
Vmware certified IBM Servers
 
Presentation1.pptx
Presentation1.pptxPresentation1.pptx
Presentation1.pptx
 
Cisco connect montreal 2018 compute v final
Cisco connect montreal 2018   compute v finalCisco connect montreal 2018   compute v final
Cisco connect montreal 2018 compute v final
 

More from Edge AI and Vision Alliance

“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
Edge AI and Vision Alliance
 
“How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentatio...
“How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentatio...“How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentatio...
“How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentatio...
Edge AI and Vision Alliance
 
“Meeting the Critical Needs of Accuracy, Performance and Adaptability in Embe...
“Meeting the Critical Needs of Accuracy, Performance and Adaptability in Embe...“Meeting the Critical Needs of Accuracy, Performance and Adaptability in Embe...
“Meeting the Critical Needs of Accuracy, Performance and Adaptability in Embe...
Edge AI and Vision Alliance
 
“Build a Tiny Vision Application in Minutes with the Edge App SDK,” a Present...
“Build a Tiny Vision Application in Minutes with the Edge App SDK,” a Present...“Build a Tiny Vision Application in Minutes with the Edge App SDK,” a Present...
“Build a Tiny Vision Application in Minutes with the Edge App SDK,” a Present...
Edge AI and Vision Alliance
 
“The Importance of Memory for Breaking the Edge AI Performance Bottleneck,” a...
“The Importance of Memory for Breaking the Edge AI Performance Bottleneck,” a...“The Importance of Memory for Breaking the Edge AI Performance Bottleneck,” a...
“The Importance of Memory for Breaking the Edge AI Performance Bottleneck,” a...
Edge AI and Vision Alliance
 
“Intel’s Approach to Operationalizing AI in the Manufacturing Sector,” a Pres...
“Intel’s Approach to Operationalizing AI in the Manufacturing Sector,” a Pres...“Intel’s Approach to Operationalizing AI in the Manufacturing Sector,” a Pres...
“Intel’s Approach to Operationalizing AI in the Manufacturing Sector,” a Pres...
Edge AI and Vision Alliance
 
“Transforming Enterprise Intelligence: The Power of Computer Vision and Gen A...
“Transforming Enterprise Intelligence: The Power of Computer Vision and Gen A...“Transforming Enterprise Intelligence: The Power of Computer Vision and Gen A...
“Transforming Enterprise Intelligence: The Power of Computer Vision and Gen A...
Edge AI and Vision Alliance
 
“Challenges and Solutions of Moving Vision LLMs to the Edge,” a Presentation ...
“Challenges and Solutions of Moving Vision LLMs to the Edge,” a Presentation ...“Challenges and Solutions of Moving Vision LLMs to the Edge,” a Presentation ...
“Challenges and Solutions of Moving Vision LLMs to the Edge,” a Presentation ...
Edge AI and Vision Alliance
 
“Implementing Transformer Neural Networks for Visual Perception on Embedded D...
“Implementing Transformer Neural Networks for Visual Perception on Embedded D...“Implementing Transformer Neural Networks for Visual Perception on Embedded D...
“Implementing Transformer Neural Networks for Visual Perception on Embedded D...
Edge AI and Vision Alliance
 
“A Cutting-edge Memory Optimization Method for Embedded AI Accelerators,” a P...
“A Cutting-edge Memory Optimization Method for Embedded AI Accelerators,” a P...“A Cutting-edge Memory Optimization Method for Embedded AI Accelerators,” a P...
“A Cutting-edge Memory Optimization Method for Embedded AI Accelerators,” a P...
Edge AI and Vision Alliance
 
“Efficiency Unleashed: The Next-gen NXP i.MX 95 Applications Processor for Em...
“Efficiency Unleashed: The Next-gen NXP i.MX 95 Applications Processor for Em...“Efficiency Unleashed: The Next-gen NXP i.MX 95 Applications Processor for Em...
“Efficiency Unleashed: The Next-gen NXP i.MX 95 Applications Processor for Em...
Edge AI and Vision Alliance
 
“Optimized Vision Language Models for Intelligent Transportation System Appli...
“Optimized Vision Language Models for Intelligent Transportation System Appli...“Optimized Vision Language Models for Intelligent Transportation System Appli...
“Optimized Vision Language Models for Intelligent Transportation System Appli...
Edge AI and Vision Alliance
 
“Image Signal Processing Optimization for Object Detection,” a Presentation f...
“Image Signal Processing Optimization for Object Detection,” a Presentation f...“Image Signal Processing Optimization for Object Detection,” a Presentation f...
“Image Signal Processing Optimization for Object Detection,” a Presentation f...
Edge AI and Vision Alliance
 
“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...
“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...
“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...
Edge AI and Vision Alliance
 
"Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr...
"Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr..."Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr...
"Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr...
Edge AI and Vision Alliance
 
“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...
“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...
“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...
Edge AI and Vision Alliance
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...
“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...
“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...
Edge AI and Vision Alliance
 
“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...
“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...
“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...
Edge AI and Vision Alliance
 
“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...
“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...
“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...
Edge AI and Vision Alliance
 

More from Edge AI and Vision Alliance (20)

“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
 
“How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentatio...
“How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentatio...“How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentatio...
“How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentatio...
 
“Meeting the Critical Needs of Accuracy, Performance and Adaptability in Embe...
“Meeting the Critical Needs of Accuracy, Performance and Adaptability in Embe...“Meeting the Critical Needs of Accuracy, Performance and Adaptability in Embe...
“Meeting the Critical Needs of Accuracy, Performance and Adaptability in Embe...
 
“Build a Tiny Vision Application in Minutes with the Edge App SDK,” a Present...
“Build a Tiny Vision Application in Minutes with the Edge App SDK,” a Present...“Build a Tiny Vision Application in Minutes with the Edge App SDK,” a Present...
“Build a Tiny Vision Application in Minutes with the Edge App SDK,” a Present...
 
“The Importance of Memory for Breaking the Edge AI Performance Bottleneck,” a...
“The Importance of Memory for Breaking the Edge AI Performance Bottleneck,” a...“The Importance of Memory for Breaking the Edge AI Performance Bottleneck,” a...
“The Importance of Memory for Breaking the Edge AI Performance Bottleneck,” a...
 
“Intel’s Approach to Operationalizing AI in the Manufacturing Sector,” a Pres...
“Intel’s Approach to Operationalizing AI in the Manufacturing Sector,” a Pres...“Intel’s Approach to Operationalizing AI in the Manufacturing Sector,” a Pres...
“Intel’s Approach to Operationalizing AI in the Manufacturing Sector,” a Pres...
 
“Transforming Enterprise Intelligence: The Power of Computer Vision and Gen A...
“Transforming Enterprise Intelligence: The Power of Computer Vision and Gen A...“Transforming Enterprise Intelligence: The Power of Computer Vision and Gen A...
“Transforming Enterprise Intelligence: The Power of Computer Vision and Gen A...
 
“Challenges and Solutions of Moving Vision LLMs to the Edge,” a Presentation ...
“Challenges and Solutions of Moving Vision LLMs to the Edge,” a Presentation ...“Challenges and Solutions of Moving Vision LLMs to the Edge,” a Presentation ...
“Challenges and Solutions of Moving Vision LLMs to the Edge,” a Presentation ...
 
“Implementing Transformer Neural Networks for Visual Perception on Embedded D...
“Implementing Transformer Neural Networks for Visual Perception on Embedded D...“Implementing Transformer Neural Networks for Visual Perception on Embedded D...
“Implementing Transformer Neural Networks for Visual Perception on Embedded D...
 
“A Cutting-edge Memory Optimization Method for Embedded AI Accelerators,” a P...
“A Cutting-edge Memory Optimization Method for Embedded AI Accelerators,” a P...“A Cutting-edge Memory Optimization Method for Embedded AI Accelerators,” a P...
“A Cutting-edge Memory Optimization Method for Embedded AI Accelerators,” a P...
 
“Efficiency Unleashed: The Next-gen NXP i.MX 95 Applications Processor for Em...
“Efficiency Unleashed: The Next-gen NXP i.MX 95 Applications Processor for Em...“Efficiency Unleashed: The Next-gen NXP i.MX 95 Applications Processor for Em...
“Efficiency Unleashed: The Next-gen NXP i.MX 95 Applications Processor for Em...
 
“Optimized Vision Language Models for Intelligent Transportation System Appli...
“Optimized Vision Language Models for Intelligent Transportation System Appli...“Optimized Vision Language Models for Intelligent Transportation System Appli...
“Optimized Vision Language Models for Intelligent Transportation System Appli...
 
“Image Signal Processing Optimization for Object Detection,” a Presentation f...
“Image Signal Processing Optimization for Object Detection,” a Presentation f...“Image Signal Processing Optimization for Object Detection,” a Presentation f...
“Image Signal Processing Optimization for Object Detection,” a Presentation f...
 
“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...
“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...
“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...
 
"Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr...
"Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr..."Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr...
"Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr...
 
“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...
“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...
“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...
“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...
“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...
 
“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...
“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...
“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...
 
“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...
“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...
“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...
 

Recently uploaded

Semantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software DevelopmentSemantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software Development
Baishakhi Ray
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
Matthias Neugebauer
 
Sonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdfSonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdf
SubhamMandal40
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
SynapseIndia
 
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
bellared2
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
sunilverma7884
 
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
FIDO Alliance
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
Brian Pichman
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
maigasapphire
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
DianaGray10
 
kk vathada _digital transformation frameworks_2024.pdf
kk vathada _digital transformation frameworks_2024.pdfkk vathada _digital transformation frameworks_2024.pdf
kk vathada _digital transformation frameworks_2024.pdf
KIRAN KV
 
Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)
Debmalya Biswas
 
Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024
siddu769252
 
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
FIDO Alliance
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 
Patch Tuesday de julio
Patch Tuesday de julioPatch Tuesday de julio
Patch Tuesday de julio
Ivanti
 
Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10
ankush9927
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
shyamraj55
 

Recently uploaded (20)

Semantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software DevelopmentSemantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software Development
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
 
Sonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdfSonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdf
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
 
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
 
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
 
kk vathada _digital transformation frameworks_2024.pdf
kk vathada _digital transformation frameworks_2024.pdfkk vathada _digital transformation frameworks_2024.pdf
kk vathada _digital transformation frameworks_2024.pdf
 
Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)
 
Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024
 
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 
Patch Tuesday de julio
Patch Tuesday de julioPatch Tuesday de julio
Patch Tuesday de julio
 
Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
 

“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-efficient Computer Vision,” a Presentation from Axelera AI

  • 1. How Axelera AI Uses Digital Compute-in-Memory to Deliver Fast and Energy- Efficient Computer Vision Bram Verhoef Head of Machine Learning & Co-Founder Axelera AI
  • 2. Compute and Intelligence at Different Layers 2 © 2024 Axelera AI The Edge
  • 3. New AI Applications Are Emerging at the Edge 3 © 2024 Axelera AI Retail Inventory management Cashier-less checkouts Security Traffic control systems Intelligent surveillance Agriculture Crop health monitoring Automated pest control Health Real-time diagnostics tools Surgical tools & equipment Industrial Quality control automation Worker safety monitoring Auto Driver assistance systems Autonomous driving systems
  • 4. ΑΙ Is Moving From the Cloud to the Edge 4 © 2024 Axelera AI Mainframe Client-server Cloud Edge Centralized Distributed Centralized Distributed ~10M mainframes ~2B PCs ~50B devices Trillions of devices $$$$ $$$ $$ $ 1960 - 1980 1980 - 2005 2005 - Today Tomorrow Role tomorrow: Training and data storage Role tomorrow: Sensing, inference & automation Emerging AI edge applications require performance and accuracy, energy efficiency, and low price
  • 5. 5 © 2024 Axelera AI Fast, Accurate, Energy-Efficient, and Cost-Effective AI Inference With Digital Compute-In-Memory (D-IMC)
  • 6. Metis - AI Platform 6 © 2024 Axelera AI  AI edge inference accelerator  M.2 module or PCIe card  Metis AIPU executes all tasks of an AI workload  Offload complete network(s)  Not just individual layers  Easy-to-use software stack  Voyager SDK combining compilation and quantization flow Host PCI-E card connected to host AI computer vision applications at the edge
  • 7. Metis AI Processing Unit (AIPU) 7 © 2024 Axelera AI  Quad-core System-on-Chip  RISC-V controlled  Security  PCIe 3.0 4x link to host  LPDDR4x  Large on-chip SRAM capacity  AI-Core powered by D-IMC  52.4 TOPS @ INT8 (209.6 TOPS aggregate)  15 TOPS/W energy efficiency AI Core AI Core AI Core AI Core RISC-V System Controller L2 Memory LPDDR4x Security PCIe 3.0 (x4)
  • 8. Digital In-Memory Computing (D-IMC) 8 © 2024 Axelera AI 4 weight sets  SRAM-based D-IMC  Interleaved weight-storage and compute units in an extremely dense fashion  Immune to noise and memory non-idealities affecting analog IMC precision  INT8 activations / weights, with INT32 accumulation to maintain full precision  Technology commensurate with CMOS scaling to low lithography nodes
  • 9. D-IMC Differentiating Improvements 9 © 2024 Axelera AI 1. Stores multiple weight sets in computational memory • Enhances IMC storage density • Allows accumulation up to 16k inputs • Enables simultaneous processing and weight reloading 2. Activity gating and clock gating • Maintains high energy efficiency at low utilization 3. Ensures full-precision accumulation • Negligible accuracy loss compared to FP32 • Use of post-training quantization; no need for retraining
  • 10. AI Core – Key Components 10 © 2024 Axelera AI  Matrix-Vector Multiplier (MVM)  D-IMC based  512 inputs x 512 outputs (4 weight sets)  INT8 inputs and weights  Data Processing Unit (DPU)  Element-wise vector operations  Apply activation functions  Depth-Wise Processing Unit (DWPU)  Depth-wise convolution  Pooling and Up-sampling  4 MiByte L1 SRAM  RISC-V control core NoC (Network on chip)
  • 11. AI Core – Deployment Scenarios 11 © 2024 Axelera AI  A single AI core  Can execute all layers of a neural network  Eliminates need for external interactions  MVM  Flexibile deployment of multiple AI cores  Manage different neural networks independently − In multi-network applications  Jointly tackle a workload to enhance throughput  Work on same neural network to reduce latency RISC-V System Controller L2 32MB LPDDR4x Security PCIe 3.0 (x4) AI Core AI Core AI Core AI Core Network 1 Network 2 Network 3
  • 12. Software Development Flow 12 © 2024 Axelera AI Tensor ops Image ops Host Non-NN code  eGPU (Intel/Mali)  VA-API  CPU SIMD Model Post-processing ML Model  Weights  Dataset  Metrics Model Zoo Sample Pipelines Trained Model PyTorch ONNX TensorFlow Compilation ML Pipeline Definition Performance & Accuracy Evaluation Application & Runtime Integration Model Pre-processing Metis ML code  Quantization  Graph optimization  Lowering Inference Pipeline Business Logic Application Image Processing Input Stream(s) Image Stream Axelera Inference Element Metadata Inference Pipeline (GStreamer) Runs on host CPU/GPU (x86 / ARM) Runs on Metis Voyager Build Environment Voyager Runtime Environment
  • 13. Metis AIPU SoC Performance 13 © 2024 Axelera AI Deviation from FP32 accuracy 92 FPS/W 354 FPS/W
  • 14. YOLOv5s on Metis – Demo Preview 14 © 2024 Axelera AI 496 FPS YoloV5s inference @640x640
  • 15. Running YoloV5s on 24 Streams on a Single Metis Chip 15 © 2024 Axelera AI 24 RTSP streams 15FPS/stream 1 Metis Chip
  • 16. Product Line-Up 16 © 2024 Axelera AI Modules Cards Boards Systems Metis M.2 ​ 159 USD AI acceleration to systems with an M.2 2280M slot where space is at a premium Metis PCIe​ 212 USD​ PCIe cards with 1x or 4x Metis AIPUs for Edge Servers where AI performance and flexibility is a priority Single Board Computer Price upon request ARM ​(Rockchip RK3588)​ For stand-alone and compact form factor embedded systems Partner products Price upon request x86 Edge Servers, Industrial PC’s Ready to use devices for edge or near edge processing where out- of-the-box systems are needed
  • 17. Evaluation Kits to get stated 17 © 2024 Axelera AI Dell Precision 3460XE Advantech ARC-3534 Lenovo ThinkStation P360 Advantech MIC-770 Industrial PC Industrial PC Edge Server PC Edge Server PC Firefly ITX-3588J Embedded ARM Metis Evaluation Kits Edge Host Systems Dell Precision 3460XE SFF Core i7 LENOVO ThinkStation P360 ULTRA Core i5 Advantech ARC-3534B Core i5, Industrial PC Advantech MIC-770v3W Core i5, Industrial PC Firefly ITX-3588J, 8-core ARM, embedded AI Acceleration Axelera Metis PCIe, 214 TOPS (int8) PCIe PCIe 3.0 (x4), HHHL size, 64 x 168 x 40 (mm) ML frameworks PyTorch / ONNX / TensorFlow (via ONNX) Axelera Voyager SDK Neural Networks Detection: YOLOv5s / m / l / YOLOv7 / SSD-MobileNetV2 Classification: Resnet-50 / MobileNetV2 / and more Pre-compiled optimized models and compiler supported OS Ubuntu Desktop v22.04, v20.04 (w/ Docker)
  • 18.  Metis AIPU SoC is an innovative and advanced digital compute-in-memory inference solution for optimized AI computer vision applications  Metis delivers fast, energy-efficient, cost-effective and accurate AI inference  Voyager SDK supports deep learning out-of-the-box Summing Up: Powerful, Efficient and Cost-Effective AI 18 © 2024 Axelera AI Metis evaluation kits available now to get started
  • 19.  https://www.axelera.ai  Products: https://www.axelera.ai/ai-acceleration-hardware- products  Metis: https://www.axelera.ai/metis-aipu  Voyager SDK: https://www.axelera.ai/ai-software  Evaluation Kits: https://www.axelera.ai/metis-evaluation-kit Resources 19 © 2024 Axelera AI
  • 20. 20 © 2024 Axelera AI Thank You! Visit us at the Axelera booth (#510)!!!