Danilo Pau, Claudio Marchisio
System Research and Applications
Agrate Brianza
Components for
Neural Networks
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Turing Test
• “I propose to consider the question: can machines think ?”
• Because thinking is difficult to define, "Are there imaginable digital
computers which would do well in the imitation game?"
• “The imitation game could then be played with the machine in
question … mimicking digital computer and the interrogator would
be unable to distinguish them”
2
Turing Test
Rosenblatt’s
Perceptron Winter of AI
Universal
Aproximation
Theorem
The rise of AI
Google AI
Assistant
Robots
powered by
Artificial
Intelligence ?
1950 1957 1989 1990 2006 2018 20501936
Turing
Machine
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Turing Test
• “I propose to consider the question: can machines think ?”
• Because thinking is difficult to define, "Are there imaginable digital
computers which would do well in the imitation game?"
• “The imitation game could then be played with the machine in
question … mimicking digital computer and the interrogator would
be unable to distinguish them”
2
Turing Test
Rosenblatt’s
Perceptron Winter of AI
Universal
Aproximation
Theorem
The rise of AI
Google AI
Assistant
Robots
powered by
Artificial
Intelligence ?
1950 1957 1989 1990 2006 2018 20501936
Turing
Machine
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Turing Test
• “I propose to consider the question: can machines think ?”
• Because thinking is difficult to define, "Are there imaginable digital
computers which would do well in the imitation game?"
• “The imitation game could then be played with the machine in
question … mimicking digital computer and the interrogator would
be unable to distinguish them”
2
Turing Test
Rosenblatt’s
Perceptron Winter of AI
Universal
Aproximation
Theorem
The rise of AI
Google AI
Assistant
Robots
powered by
Artificial
Intelligence ?
1950 1957 1989 1990 2006 2018 20501936
Turing
Machine
The effects on the Applications
Centralized
Intelligence
Systems
Performance reduction,
cascade effects
on Applications
Faults, Errors,
Uncertainty,
Malfunctioning,
Intrusions
Changes:
Nonstationary,
seasonality, periodicity
5
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
• Definition of Cyber Physical Systems
6
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Cyber-Physical SystemsPhysical
Domain
Cyber
Domain
Object
Domain
Information
Transmission
Actuation
and Control
7
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
• Definition of Cyber Physical Systems
• Current Limitations
8
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Voice Recognition: Challenging the Could 9
Average Person
• 16000 utterances */day
@163 words/minute +
;
• ≈ 98 minutes speech/day.
Natural
Language
Processing
Audio coding@
128Kbps
≈94 MB/day
≈94 TB/day
≈1 PB/day
≈9 PB/day
1 Million People
10 Million
People
100 Million
People
Cloud based Voice
Recognition means
huge data bandwidth
and computation
capabilities
Source:
1. Are Women Really More Talkative Than Men? ResearchGate
2. What is the Average Speaking Rate? SixMinutes
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
10
Courtesy Dday.it
Courtesy Tecnoandroid.it
….
Source: Wired, Aug 2017
Voice Recognition: Challenging the Could
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
11
https://www.theguardian.com/environment/2017/dec/11/tsunami-of-data-could-consume-fifth-global-electricity-by-2025
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Object domain under attack
DDoS, screwing Dyn, Oct 2016
• A group called “New World Hackers“ has
claimed responsibility for the attack.
• It was based on Mirai code
• Attackers were refrigerators, thermostats,
and toasters.
• From 09:30 to 18:00 ET, Dyn’s servers
were attacked in three DDoS waves.
• Cyberattack, affected Twitter, Amazon,
Reddit, Netflix, and more since they used
Dyn DNS provider.
12
https://readwrite.com/2016/10/22/the-internet-of-things-was-used-in-fridays-ddos-attack-pl4/
New World Hackers, Mirai code
Refrigerators,
Thermostats,
toasters
Dyn Servers
Twitter, Amazon,
Reddit, Netflix
… pinging …. … pinging ….
… denial ….
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
13
https://www.bloomberg.com/news/articles/2019-04-10/is-anyone-listening-to-you-on-alexa-a-global-team-reviews-audio
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
• Definition of Cyber Physical Systems
• Current Limitations
• Opportunities
14
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Computers vs Embedded µControllers 15
Olivetti M24 23
• Intel 8086
• 8 MHz
• 128 KB RAM
• 16 KB ROM
• 1.84 W
• 360 $
• STM32 MCU L4
• 80 MHz
• 128 KB RAM
• 1 MB Flash
• < 20mW
• < 4 €
STM32 L4+
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
STM 6-axis IMU evolution
SiP 3D
Digital Accelerometer and Gyroscope
Embedded Sensors
Credit http://semieurope.omnibooksonline.com/2014/semicon_europa/International_MEMS_Forum/13_Romain_Fraux_System_Plus_Consulting.pdf
16
• First accelerometer (1923)
• Credits: McCollum and Peters
• Resistance Bridge type, with
carbon rings in a tension-
compression Wheatstone
• Dimensions: ~ 28 c63
• Price: $420 ($6,275 today)
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Digital Camera Kodak’s Steven Sasson 73 17
• 50ms to capture an image
• 10K pixels, black and white
• 23s to record it on tape
• 3.6 Kg
• Electronic still camera,
• US patent 4131919 A
OpenMV Cam
STM32H743VI ARM Cortex M7
400 MHz, 1MB RAM, 2 MB Flash
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Local Processing
• STM32F446 32-bit ARM Cortex-M4F MCU
• 180MHz, 2MB FLASH
• STM32F446 product line provides from 256-
Kbyte to 512-Kbyte Flash, 128-Kbyte SRAM
• DSP Instructions
Inertial and Environmental
• LSM6DSM iNEMO 6DoF Gyro + Acc
• LSM303AGR e-Compass
• 4 X MP34DT06J Microphones
• LPS22HB MEMS Pressure sensor
• HTS221 Humidity & Temperature Sensor
Sensing
Processing
Wireless
• BlueNRG-MS, Bluetooth Low Energy
Network Processor supporting Bluetooth 4.2
core specification
Connectivity
Intelligent Acoustic Sensing Unit
STEVAL-BCNKT01V1
18
https://www.st.com/en/evaluation-tools/steval-bcnkt01v1.html
STEVAL-BCNST01V1
CoinStation
130mAh LiPo Battery
(UN38.3 Certified)
ST-Link SWD
Programming Cable
STEVAL-BCNCS01V1
Core System
Plastic Case
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
100s Billions of Sensors 19
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
https://www.slideshare.net/KNIMESlides/advanced-analytics-for-the-internet-of-things-restocking-rental-bike-stations?from_action=save
Physical
Domain
Unit
SensorsSensors
Comm
Unit
SensorsSensors
Comm
Unit
SensorsSensors
Comm
Gate
way
Unit
SensorsSensors
Comm
Gate
way
Gate
way
Server
Cloud
Application
Artificial IntelligenceScalability
Responsiveness
Intelligent units
Designing Intelligent and distributed CPS 20
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
• Definition of Cyber Physical Systems
• Current Limitations
• Opportunities
• STM32CubeAI
21
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Can STM32 run Artificial Neural Network ? 22
88 9:6;<=>?@ABCDEF
GHIBCDEF/F
K
LM6;<=L ?N M O?NP:O
QRSTRUV RWXYZ[T/R
1,034,722BCDEF
13,6GBCDEF/F
K
16,384FbcECdF
16,000FbcECdF/R
efGcgh i f9@?jM@?:NLkSlmUVn
e0Gcgh i o=?pq@LkSlmUVn
r*stu i
K vwtu
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Key Steps Behind AI Applications 23
2
Collect, clean, label Data
5
Run Field trials
1
Define AI application
4
Convert ANNs into
optimized code
for STM32 MCU
Neural Network (ANN) Model Creation Deployment Mode
3
Build ANN Topologies
Train ANN Models
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Sound Recognition
Requires Distributed Sensing Nodes
24
Doppler effect of cars
approaching/leaving
1
1
2
2
Installed at the side of streets
1
Define AI application
2
Collect, clean, label Data
Managed
by
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Deep Learning Frameworks
Popularity
25
https://www.kdnuggets.com/2018/09/deep-learning-framework-power-scores-2018.html
3
Build ANN Topologies
Train ANN Models
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
26Keras Model Example
3
Build ANN Topologies
Train ANN Models
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Key Steps Behind AI Applications 27
2
Collect, clean, label Data
1
Define AI application
Neural Network (ANN) Model Creation Deployment Mode
4
Convert ANNs into
optimized code
for STM32 MCU
3
Build ANN Topologies
Train ANN Models
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Neural Networks
Available Now for STM32
28
Embedded Solution
Optimized Neural Network
Code generated for STM32
Deep
Learning
SW Solution
Choose your IDE
Compiler and Debugger
Framework Independent
Sensors and OS Agnostic
Interoperability
Pre-trained Neural Network models
Deep Learning framework dependent
TodayTomorrow
AI Hub
Multiple Neural
Networks
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Making AI Accessible Now 29
World 1st
Cortex-M MCU
World 1st
Cortex-M
Ultra-low-power
1st High Perf.
120 MHz, 90nm
1st High Perf.
Cortex-M4
168 MHz
Entry Cost
STM32F0
Cortex-M0
1st Mixed Signal
DSP + Analog
STM32F3 Cortex-M4
Entry Cost
Ultra-low-power
World 1st
Cortex-M7
Leadership
Ultra-low-power
Cortex-M4
273 ULPBench™
#1 ULP
#1
Performance
2020 CoreMark
Ultra-low-power
Excellence
Dual-core,
multi-protocol
and open radio
Introduction of M33
Excellence in ULP
with more security
Mainstream
Cortex-M0+ MCUs
Efficiency at its best!
20182007 2009 2010 2011 2012 2013 2014 2015 2016 20192017
First STM32 MPU
Dual Cortex-A7 + Cortex-M4
STM32 meets Linux
Compatible with
STM32Cube.AI ecosystem
Compatible with Partner
Machine Learning ecosystems
More than 60,000 customers Over 4 Billion STM32 shipped since 2008
Leader in Arm® Cortex®-M 32-bit General Purpose MCU
• Based on single or dual ARM Cortex M core
• Broadest global portfolio
• > 1000 products available
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
STM32Cube.AI: select STM32
- Loading a pre-trained
NN model
- Possibility to compress
the weights to gain Flash
space
- Analyze the NN to
provide RAM/Flash
usage
30
After analysis:
list of compatible
STM32
Minimum RAM /
Flash for the
selected NN
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
- Loading several pre-
trained NN model
- Compress weights
- Analyze the NNs to
provide RAM/Flash
usage
- Validate on desktop or
on target
31
Validate each NN
layer
Configure
STM32 pin
(standard
CubeMX feature)
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
STM32Cube.AI: project generation 32
Generate project
for your favorite
IDE
Run on target
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
STM32Cube.AI Architecture
This optimized STM32 neural network model can be included into the user project
(using any user IDE) and can be compiled and ported onto the final device for field trials
Neural
Network
Exporter
DL Framework
Independent
Neural Network
Representation
NN Layers
Library
for STM32
Neural
Networks
API’s
Code
Generator
33
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
X-Cube-AI Generated files 34
Customer application file
Artificial Neural Network
topology and API
Artificial Neural Network data
file (weights/bias parameters)
Once the code is generated we
have:
Validation application
Always generated/needed
Needed only if NN validation
on target is required
Needed in case we start from
a blank project.
Neural network
computing kernel
provided as a
static library
precompiled for
the given STM32
P/N selected
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
We brought Demo here
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
ST and Bluewind @ Embedded World 2019 36
https://www.youtube.com/watch?v=94tT9nlv5B8&list=PLnMKNibPkDnFpi3vFzkbH32yblwN3XVwv&index=6
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Speech to Intent on MCU 37https://www.youtube.com/watch?v=WadKhfLyqTQ
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
Audio Scene Classification (ASC)
Audio Example in FP-AI-SENSING1 Package
38
Embedded audio Data stored on the device
SD card for future learning
Inference result
displayed on mobile app
Inferences running
on the microcontroller
Embedded audio
pre-processing
NN & example
dataset provided
Indoor, Outdoor, In vehicle
3 classes
Labelling controlled
by smartphone application
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
STM32Cube.AI Roadmap 39
2019 Apr Jun
• Floating Point
Support
• Quantization
• TensorFlow for MCU
• Command line interface
• UI Improvements
• Additional layers
2020
• Introduction
• Additional layers
• Debug improvements
+microcontrollers
+
IoTNode
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
www.st.com/STM32CubeAI
This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/

2019-06-14:5 - Componenti per reti neurali

  • 1.
    Danilo Pau, ClaudioMarchisio System Research and Applications Agrate Brianza Components for Neural Networks This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 2.
    This work islicensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/ Turing Test • “I propose to consider the question: can machines think ?” • Because thinking is difficult to define, "Are there imaginable digital computers which would do well in the imitation game?" • “The imitation game could then be played with the machine in question … mimicking digital computer and the interrogator would be unable to distinguish them” 2 Turing Test Rosenblatt’s Perceptron Winter of AI Universal Aproximation Theorem The rise of AI Google AI Assistant Robots powered by Artificial Intelligence ? 1950 1957 1989 1990 2006 2018 20501936 Turing Machine
  • 3.
    This work islicensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/ Turing Test • “I propose to consider the question: can machines think ?” • Because thinking is difficult to define, "Are there imaginable digital computers which would do well in the imitation game?" • “The imitation game could then be played with the machine in question … mimicking digital computer and the interrogator would be unable to distinguish them” 2 Turing Test Rosenblatt’s Perceptron Winter of AI Universal Aproximation Theorem The rise of AI Google AI Assistant Robots powered by Artificial Intelligence ? 1950 1957 1989 1990 2006 2018 20501936 Turing Machine
  • 4.
    This work islicensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/ Turing Test • “I propose to consider the question: can machines think ?” • Because thinking is difficult to define, "Are there imaginable digital computers which would do well in the imitation game?" • “The imitation game could then be played with the machine in question … mimicking digital computer and the interrogator would be unable to distinguish them” 2 Turing Test Rosenblatt’s Perceptron Winter of AI Universal Aproximation Theorem The rise of AI Google AI Assistant Robots powered by Artificial Intelligence ? 1950 1957 1989 1990 2006 2018 20501936 Turing Machine
  • 5.
    The effects onthe Applications Centralized Intelligence Systems Performance reduction, cascade effects on Applications Faults, Errors, Uncertainty, Malfunctioning, Intrusions Changes: Nonstationary, seasonality, periodicity 5 This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 6.
    • Definition ofCyber Physical Systems 6 This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 7.
    Cyber-Physical SystemsPhysical Domain Cyber Domain Object Domain Information Transmission Actuation and Control 7 Thiswork is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 8.
    • Definition ofCyber Physical Systems • Current Limitations 8 This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 9.
    Voice Recognition: Challengingthe Could 9 Average Person • 16000 utterances */day @163 words/minute + ; • ≈ 98 minutes speech/day. Natural Language Processing Audio coding@ 128Kbps ≈94 MB/day ≈94 TB/day ≈1 PB/day ≈9 PB/day 1 Million People 10 Million People 100 Million People Cloud based Voice Recognition means huge data bandwidth and computation capabilities Source: 1. Are Women Really More Talkative Than Men? ResearchGate 2. What is the Average Speaking Rate? SixMinutes This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 10.
    10 Courtesy Dday.it Courtesy Tecnoandroid.it …. Source:Wired, Aug 2017 Voice Recognition: Challenging the Could This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 11.
    11 https://www.theguardian.com/environment/2017/dec/11/tsunami-of-data-could-consume-fifth-global-electricity-by-2025 This work islicensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 12.
    Object domain underattack DDoS, screwing Dyn, Oct 2016 • A group called “New World Hackers“ has claimed responsibility for the attack. • It was based on Mirai code • Attackers were refrigerators, thermostats, and toasters. • From 09:30 to 18:00 ET, Dyn’s servers were attacked in three DDoS waves. • Cyberattack, affected Twitter, Amazon, Reddit, Netflix, and more since they used Dyn DNS provider. 12 https://readwrite.com/2016/10/22/the-internet-of-things-was-used-in-fridays-ddos-attack-pl4/ New World Hackers, Mirai code Refrigerators, Thermostats, toasters Dyn Servers Twitter, Amazon, Reddit, Netflix … pinging …. … pinging …. … denial …. This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 13.
    13 https://www.bloomberg.com/news/articles/2019-04-10/is-anyone-listening-to-you-on-alexa-a-global-team-reviews-audio This work islicensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 14.
    • Definition ofCyber Physical Systems • Current Limitations • Opportunities 14 This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 15.
    Computers vs EmbeddedµControllers 15 Olivetti M24 23 • Intel 8086 • 8 MHz • 128 KB RAM • 16 KB ROM • 1.84 W • 360 $ • STM32 MCU L4 • 80 MHz • 128 KB RAM • 1 MB Flash • < 20mW • < 4 € STM32 L4+ This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 16.
    STM 6-axis IMUevolution SiP 3D Digital Accelerometer and Gyroscope Embedded Sensors Credit http://semieurope.omnibooksonline.com/2014/semicon_europa/International_MEMS_Forum/13_Romain_Fraux_System_Plus_Consulting.pdf 16 • First accelerometer (1923) • Credits: McCollum and Peters • Resistance Bridge type, with carbon rings in a tension- compression Wheatstone • Dimensions: ~ 28 c63 • Price: $420 ($6,275 today) This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 17.
    Digital Camera Kodak’sSteven Sasson 73 17 • 50ms to capture an image • 10K pixels, black and white • 23s to record it on tape • 3.6 Kg • Electronic still camera, • US patent 4131919 A OpenMV Cam STM32H743VI ARM Cortex M7 400 MHz, 1MB RAM, 2 MB Flash This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 18.
    Local Processing • STM32F44632-bit ARM Cortex-M4F MCU • 180MHz, 2MB FLASH • STM32F446 product line provides from 256- Kbyte to 512-Kbyte Flash, 128-Kbyte SRAM • DSP Instructions Inertial and Environmental • LSM6DSM iNEMO 6DoF Gyro + Acc • LSM303AGR e-Compass • 4 X MP34DT06J Microphones • LPS22HB MEMS Pressure sensor • HTS221 Humidity & Temperature Sensor Sensing Processing Wireless • BlueNRG-MS, Bluetooth Low Energy Network Processor supporting Bluetooth 4.2 core specification Connectivity Intelligent Acoustic Sensing Unit STEVAL-BCNKT01V1 18 https://www.st.com/en/evaluation-tools/steval-bcnkt01v1.html STEVAL-BCNST01V1 CoinStation 130mAh LiPo Battery (UN38.3 Certified) ST-Link SWD Programming Cable STEVAL-BCNCS01V1 Core System Plastic Case This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 19.
    100s Billions ofSensors 19 This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/ https://www.slideshare.net/KNIMESlides/advanced-analytics-for-the-internet-of-things-restocking-rental-bike-stations?from_action=save
  • 20.
    Physical Domain Unit SensorsSensors Comm Unit SensorsSensors Comm Unit SensorsSensors Comm Gate way Unit SensorsSensors Comm Gate way Gate way Server Cloud Application Artificial IntelligenceScalability Responsiveness Intelligent units DesigningIntelligent and distributed CPS 20 This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 21.
    • Definition ofCyber Physical Systems • Current Limitations • Opportunities • STM32CubeAI 21 This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 22.
    Can STM32 runArtificial Neural Network ? 22 88 9:6;<=>?@ABCDEF GHIBCDEF/F K LM6;<=L ?N M O?NP:O QRSTRUV RWXYZ[T/R 1,034,722BCDEF 13,6GBCDEF/F K 16,384FbcECdF 16,000FbcECdF/R efGcgh i f9@?jM@?:NLkSlmUVn e0Gcgh i o=?pq@LkSlmUVn r*stu i K vwtu This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 23.
    Key Steps BehindAI Applications 23 2 Collect, clean, label Data 5 Run Field trials 1 Define AI application 4 Convert ANNs into optimized code for STM32 MCU Neural Network (ANN) Model Creation Deployment Mode 3 Build ANN Topologies Train ANN Models This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 24.
    Sound Recognition Requires DistributedSensing Nodes 24 Doppler effect of cars approaching/leaving 1 1 2 2 Installed at the side of streets 1 Define AI application 2 Collect, clean, label Data Managed by This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 25.
    Deep Learning Frameworks Popularity 25 https://www.kdnuggets.com/2018/09/deep-learning-framework-power-scores-2018.html 3 BuildANN Topologies Train ANN Models This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 26.
    26Keras Model Example 3 BuildANN Topologies Train ANN Models This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 27.
    Key Steps BehindAI Applications 27 2 Collect, clean, label Data 1 Define AI application Neural Network (ANN) Model Creation Deployment Mode 4 Convert ANNs into optimized code for STM32 MCU 3 Build ANN Topologies Train ANN Models This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 28.
    Neural Networks Available Nowfor STM32 28 Embedded Solution Optimized Neural Network Code generated for STM32 Deep Learning SW Solution Choose your IDE Compiler and Debugger Framework Independent Sensors and OS Agnostic Interoperability Pre-trained Neural Network models Deep Learning framework dependent TodayTomorrow AI Hub Multiple Neural Networks This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 29.
    Making AI AccessibleNow 29 World 1st Cortex-M MCU World 1st Cortex-M Ultra-low-power 1st High Perf. 120 MHz, 90nm 1st High Perf. Cortex-M4 168 MHz Entry Cost STM32F0 Cortex-M0 1st Mixed Signal DSP + Analog STM32F3 Cortex-M4 Entry Cost Ultra-low-power World 1st Cortex-M7 Leadership Ultra-low-power Cortex-M4 273 ULPBench™ #1 ULP #1 Performance 2020 CoreMark Ultra-low-power Excellence Dual-core, multi-protocol and open radio Introduction of M33 Excellence in ULP with more security Mainstream Cortex-M0+ MCUs Efficiency at its best! 20182007 2009 2010 2011 2012 2013 2014 2015 2016 20192017 First STM32 MPU Dual Cortex-A7 + Cortex-M4 STM32 meets Linux Compatible with STM32Cube.AI ecosystem Compatible with Partner Machine Learning ecosystems More than 60,000 customers Over 4 Billion STM32 shipped since 2008 Leader in Arm® Cortex®-M 32-bit General Purpose MCU • Based on single or dual ARM Cortex M core • Broadest global portfolio • > 1000 products available This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 30.
    STM32Cube.AI: select STM32 -Loading a pre-trained NN model - Possibility to compress the weights to gain Flash space - Analyze the NN to provide RAM/Flash usage 30 After analysis: list of compatible STM32 Minimum RAM / Flash for the selected NN This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 31.
    - Loading severalpre- trained NN model - Compress weights - Analyze the NNs to provide RAM/Flash usage - Validate on desktop or on target 31 Validate each NN layer Configure STM32 pin (standard CubeMX feature) This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 32.
    STM32Cube.AI: project generation32 Generate project for your favorite IDE Run on target This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 33.
    STM32Cube.AI Architecture This optimizedSTM32 neural network model can be included into the user project (using any user IDE) and can be compiled and ported onto the final device for field trials Neural Network Exporter DL Framework Independent Neural Network Representation NN Layers Library for STM32 Neural Networks API’s Code Generator 33 This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 34.
    X-Cube-AI Generated files34 Customer application file Artificial Neural Network topology and API Artificial Neural Network data file (weights/bias parameters) Once the code is generated we have: Validation application Always generated/needed Needed only if NN validation on target is required Needed in case we start from a blank project. Neural network computing kernel provided as a static library precompiled for the given STM32 P/N selected This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 35.
    We brought Demohere This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 36.
    ST and Bluewind@ Embedded World 2019 36 https://www.youtube.com/watch?v=94tT9nlv5B8&list=PLnMKNibPkDnFpi3vFzkbH32yblwN3XVwv&index=6 This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 37.
    Speech to Intenton MCU 37https://www.youtube.com/watch?v=WadKhfLyqTQ This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 38.
    Audio Scene Classification(ASC) Audio Example in FP-AI-SENSING1 Package 38 Embedded audio Data stored on the device SD card for future learning Inference result displayed on mobile app Inferences running on the microcontroller Embedded audio pre-processing NN & example dataset provided Indoor, Outdoor, In vehicle 3 classes Labelling controlled by smartphone application This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 39.
    STM32Cube.AI Roadmap 39 2019Apr Jun • Floating Point Support • Quantization • TensorFlow for MCU • Command line interface • UI Improvements • Additional layers 2020 • Introduction • Additional layers • Debug improvements +microcontrollers + IoTNode This work is licensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/
  • 40.
    www.st.com/STM32CubeAI This work islicensed under the Creative Commons Attribution- NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/