© 2019 The MathWorks, Inc.
Developing Game-Changing
Embedded Intelligence
Fabrizio Sara, Managing Director
Francesca Perino, application Engineering team

© 2019 The MathWorks, Inc.
Optional
Image Area
MathWorks is the
leading provider of technical
computing software
◼ Founded in 1984
◼ Revenues $1B in 2018
◼ 4000 employees worldwide
◼ More than 3 million users in over 180 countries
MATLAB®
SIMULINK®
Technical Computing Simulation and
Model-Based Design

© 2019 The MathWorks, Inc.
Aerospace and
Defense Automotive Biological Sciences
Biotech and
Pharmaceutical Communications
Electronics Energy Production Financial Services Industrial Machinery Medical Devices
Metals, Materials,
Mining Neuroscience Railway Systems Semiconductors Software and Internet
Serving customers across diverse industries

© 2019 The MathWorks, Inc.
Why Do These Customers Care
About Embedded Intelligence?

© 2019 The MathWorks, Inc.
Digital Transformation of the industry is everywhere

© 2019 The MathWorks, Inc.
Motivations for Embedded Intelligence
▪ Increasingly individualized products
▪ Autonomous machines that do not require costly
programming to meet new requirements
▪ Intelligent products that collect data to optimize
processes and develop new products
▪ Opportunities for innovative business models
and services
“Sample-size 1”
“Smart products”
“Servitization”

© 2019 The MathWorks, Inc.
Software in Everything
The application –
what the software does
– is the critical point.

© 2019 The MathWorks, Inc.
Artificial Intelligence
The capability of a machine to imitate
intelligent human behavior

© 2019 The MathWorks, Inc.
Artificial Intelligence
The capability of a machine to match or exceed
intelligent human behavior

© 2019 The MathWorks, Inc.
Artificial Intelligence Today
The capability of a machine to match or exceed
intelligent human behavior
by training a machine to learn the desired behavior

© 2019 The MathWorks, Inc.
Types of AI
Unsupervised Learning
(unlabeled data)
Clustering
Group and interpret data
based only on input data
Supervised Learning
(labeled data)
Classification Regression
Develop predictive model
based on input and output data
Machine Learning

© 2019 The MathWorks, Inc.
BMW Uses Machine Learning to Detect Oversteering
Challenge
Develop automated software to detect oversteering
Solution
Develop, automatically train, and evaluate
supervised machine learning classifiers
Results
▪ Oversteering identified with >98% accuracy
▪ True negative rate of 96%
(detected vehicle not oversteering)
▪ Code was automatically generated to an ECU
“With little previous experience
with machine learning, we
completed a working ECU
prototype capable of detecting
oversteering, in just three weeks.”
– Tobias Freudling, BMW Group
A BMW M4 oversteering on a test track.

© 2019 The MathWorks, Inc.
Deep Learning
▪ Performance,
especially for
vision systems
▪ Newer networks and
algorithms support
other time-series data,
e.g., speech, audio,
biomedical
Source: ILSVRC Top-5 Error on ImageNet
Human
Accuracy

© 2019 The MathWorks, Inc.
Diverse Applications of Deep Learning
Shell: Tag recognition to
identify machinery at location
Veoneer: Lidar based
sensor verification
Genentech: Digital
pathology of tumors
Ritsumeikan University:
Reducing radiation risk in CT imaging
Musashi Seimitsu: Detecting
abnormalities in automotive parts

© 2019 The MathWorks, Inc.
Developing a Deep Learning Network from Scratch
Predict and assess
network accuracy
Trained network
Test images
Train network
Training images
…
Deploy results
Boat
Plane
Car
Train
Probability
Create and configure
network layers
…
• Good knowledge of
neural networks
• Skill to configure
complex options
• Thousands/millions
of training images
• Dedicated GPUs

© 2019 The MathWorks, Inc.
Replace final layers
…
New layers to learn
features specific
to your data set
Load
pretrained network
Early layers that
learned low-level
features (edges,
blobs, colors)
Last layers
that learned
task-specific
features
…
Predict and assess
network accuracy
Trained network
Test images
Train network
Training images
…
Deploy results
Boat
Plane
Car
Train
Probability
Developing a Deep Learning Network with Transfer Learning
• General network
structure is set
• Fewer layers to
learn
• Hundreds of training
images, not millions

© 2019 The MathWorks, Inc.
Deep Learning Frameworks: Fragmented
Runtimes
Converters
Frameworks
https://onnx.ai/
→ Interoperable

© 2019 The MathWorks, Inc.
AI for Classification, Inference, Prediction
Unsupervised Learning
(data not labeled)
Clustering
Group and interpret data
based only on input data
Supervised Learning
(labeled data)
Classification Regression
Deep Learning
Develop predictive model
based on input and output data
Machine Learning

© 2019 The MathWorks, Inc.
AI for Predictive Maintenance
• Measure the wear of each blade
• Predict and fix failures before they happen
• Can’t rely on failures in the field for data

© 2019 The MathWorks, Inc.
Simulations to Synthesize Data for Training

© 2019 The MathWorks, Inc.
Reinforcement Learning
(interaction data)
Decision
Making
Control
Unsupervised Learning
(unlabeled data)
Clustering
Group and interpret data
based only on input data
Supervised Learning
(labeled data)
Classification Regression
Deep Learning
Develop predictive model
based on input and output data
Learn series of actions based on
observations and reward signal
AI for Controls: Reinforcement Learning
Machine Learning

© 2019 The MathWorks, Inc.
Reinforcement Learning
(interaction data)
Decision
Making
Control
Deep Learning
Learn series of actions based on
observations and reward signal
AI for Controls: Reinforcement Learning
Reinforcement learning
▪ Learning through trial & error,
maximizing a predefined reward
▪ Uses large numbers of simulations
▪ Can leverage deep learning as well
Machine Learning

© 2019 The MathWorks, Inc.
Gaining Efficiency in Embedded Systems
▪ Transform
the Data
▪ Optimize the
Target
▪ Simplify the
Algorithm
▪ Recast the
Problem
▪ Filtering Data in the Fourier Domain
▪ Fixed-Point Processor vs. Floating-Point
▪ Reducing Cyclomatic Complexity

© 2019 The MathWorks, Inc.
Gaining Efficiency in Embedded Intelligence
▪ Transform
the Data
▪ Optimize the
Target
▪ Simplify the
Model
▪ Recast the
Problem
Instantaneous frequency
Spectral entropy
ECG
Atrial
Fibrillation
LSTM
Normal
Spectrogram
Data dimensionality reduced from 1x9000 to 2x512
LSTM
Atrial
Fibrillation
Normal
ECG
50%
94%

© 2019 The MathWorks, Inc.
Gaining Efficiency in Embedded Intelligence
▪ Transform
the Data
▪ Optimize the
Target
▪ Simplify the
Model
▪ Recast the
Problem
GPU
Fastest
FPGA/ASIC
Lowest Power
CPU
Lowest Cost
Easily Available

© 2019 The MathWorks, Inc.
Gaining Efficiency in Embedded Intelligence
▪ Transform
the Data
▪ Optimize the
Target
▪ Simplify the
Model
▪ Recast the
Problem
TensorRT,
cuDNN
MKL-
DNN
ARM
Compute
Library Layer
NetworkOptimization
Automatic
Code
Generation
Pre-trained DNN

© 2019 The MathWorks, Inc.
Gaining Efficiency in Embedded Intelligence
▪ Transform
the Data
▪ Optimize the
Target
▪ Simplify the
Model
▪ Recast the
Problem
Pre-trained DNN
TensorRT,
cuDNN
MKL-
DNN
ARM
Compute
Library Layer
NetworkOptimization
Automatic
Code
Generation
Re-architect
network
Weight
compression
Weight
Quantization Simpler DNN
User
logic

© 2019 The MathWorks, Inc.
Gaining Efficiency in Embedded Intelligence
▪ Transform
the Data
▪ Optimize the
Target
▪ Simplify the
Model
▪ Recast the
Problem
Wavelet Scattering Transforms
▪ For 1-D or n-D data
▪ Automatically provides features that are
low-variance representations of inputs
▪ 1x9000 signal yields features that are
269x9 (a ~73% reduction)
▪ Low compute power/memory need
(scattering network is 3 layers)
enables running on CPU
Texture Spectrum
1st order
scattering
2nd order
scatterin
g

© 2019 The MathWorks, Inc.
But Embedded Intelligence
Is More Than the “AI”
That’s the AI Models

© 2019 The MathWorks, Inc.
Integrating AI Models in Subsystem Simulation

© 2019 The MathWorks, Inc.
AI Must Also Integrate with Other Algorithms and Components
Perception
Localization Planning
Controls
Connectivity
Deep Learning
3D Map RRT* Path Planner
ROS Integration

© 2019 The MathWorks, Inc.
Integrating AI Algorithms in System Simulation

© 2019 The MathWorks, Inc.
But Embedded Intelligence
Needs More Than Embedded Systems
That’s the Embedded Systems

© 2019 The MathWorks, Inc.
OT Infrastructure
Data
Ingestion
Local
Communications
Long-Range
Communications
Edge
Management
Edge systems
Integration
Distributing the Tasks for Embedded Intelligence
Smart assets IT Systems

© 2019 The MathWorks, Inc.
OT InfrastructureEdge systems
Distributing the Training and Implementation
Smart assets IT Systems
Training the inference model
Applying the inference model
Data
Ingestion
Local
Communications
Long-Range
Communications
Edge
Management
Integration

© 2019 The MathWorks, Inc.
Valueofdatatodecisionmaking
Time
Make Timely Decisions Where They Have Most Value
OT InfrastructureEdge systems
Seconds Minutes Hours Days MonthsMilliseconds
Smart assets IT Systems
Data
Ingestion
Local
Communications
Long-Range
Communications
Edge
Management
Integration
Real-time decisionsHard real-time control Time-sensitive decisions Big Data processing on historical data

© 2019 The MathWorks, Inc.
Time-sensitive decisions Big Data processing on historical data
Time
Speed
ValueofdatatodecisionmakingDevelopment for Fast and Deterministic Systems
Seconds Minutes Hours Days MonthsMilliseconds
Edge systems
Model-Based Design
Multi-domain system
modeling
Parameter
estimation
VPOC
PPOC
QPOC
PSIM
QSIM
Automatic code
generation
CODE GENERATIO
MCU DSP FPGA ASIC
VHDL, VerilogC, C++ Structured Text
PLC
OT Infrastructure IT SystemsSmart assets
Data
Ingestion
Local
Communications
Long-Range
Communications
Edge
Management
Integration
Real-time decisionsHard real-time control
Edge Processing
Model-Based Design,
code generation
Model-Based Design
with automatic code
generation

© 2019 The MathWorks, Inc.
Real-time decisionsHard real-time control
Scope
Valueofdatatodecisionmaking
Time
Stream Processing Hadoop/Spark, and other
enterprise IT integration
Deployment to OT/IT Enterprise Infrastructure
Seconds Minutes Hours Days MonthsMilliseconds
OT InfrastructureSmart assets Edge systems
Machine Learning
and Deep Learning
OptimizationBig, Unstructured Data
Enterprise System Integration
(on-prem/cloud)
IT Systems
Data
Ingestion
Local
Communications
Long-Range
Communications
Edge
Management
Integration
Time-sensitive decisions Big Data processing on historical data

© 2019 The MathWorks, Inc.
Speed Scope
Valueofdatatodecisionmaking
Time
Time-sensitive decisions Big Data processing on historical dataReal-time decisions
Combining Approaches Often is the Solution
Seconds Minutes Hours Days MonthsMilliseconds
Hard real-time control
Smart assets OT InfrastructureEdge systems IT Systems
Stream Processing Hadoop/Spark, and other
enterprise IT integration
Edge Processing
Model-Based Design,
code generation
Model-Based Design
with automatic code
generation
Data
Ingestion
Local
Communications
Long-Range
Communications
Edge
Management
Integration

© 2019 The MathWorks, Inc.
Smart City Example: BuildingIQ
Adaptive building energy management

© 2019 The MathWorks, Inc.
Data
Ingestion
Local
Communications
Long-Range
Communications
Edge
Management
Integration
Smart City Example: BuildingIQ
Robustness analysisHVAC strategy updated for next 12 hours
Multi-objective optimization
for energy efficiency
• Time of Use Energy Price
• Demand Forecast
• Predicted Weather
BuildingIQ Cloud
HVAC real-time
closed-loop control
Current building
condition
Data preprocessing
Tuned setpoints on
each HVAC system
Supervisory
control applied
Supervisory
Control
Operations
Optimization
HVAC BMS
Machine learning models
of building, BMS, comfort
Reduced HVAC energy
consumption by 10–25%
IT Systems

© 2019 The MathWorks, Inc.
A Complex Collection of Tools, Platforms and Protocols
Smart assets OT InfrastructureEdge systems IT Systems
Azure Stream
Analytics
TCP/IP
Rest APIs
Analyst/Engineer
Data
Ingestion
Local
Communications
Long-Range
Communications
Edge
Management
Integration

© 2019 The MathWorks, Inc.
MATLAB Deep Learning Container
for NVIDIA GPU Cloud
A Combination of Strategies
Automatic CUDA
code generation
Analyst/Engineer
Smart assets OT InfrastructureEdge systems IT Systems
Data
Ingestion
Local
Communications
Long-Range
Communications
Edge
Management
Integration
NVIDIA GPU Cloud

© 2019 The MathWorks, Inc.
With a Combination of Collaborators
OT team Solution
Architects
Embedded
system
engineers
Analyst/Engineer
Smart assets OT InfrastructureEdge systems IT Systems
Data
Ingestion
Local
Communications
Long-Range
Communications
Edge
Management
Integration

© 2019 The MathWorks, Inc.
Local
Communications
Edge
Management
Data
Ingestion
Long-Range
Communications Integration
Internet of Things?
Smart assets OT InfrastructureEdge systems IT Systems

© 2019 The MathWorks, Inc.
Internet of Things?
Smart assets OT InfrastructureEdge systems IT Systems
V
A
L
U
E
Optimize
Control
Predict
Analyze
Monitor
Local
Communications
Edge
Management
Data
Ingestion
Long-Range
Communications Integration

© 2019 The MathWorks, Inc.
AI-Driven Systems for Embedded Intelligence
System-on-Chip (SOC) 
Network-on-Chip (NoC) 
System-in-Package (SiP) 
System on Networks
Smart assets OT InfrastructureEdge systems IT Systems
Local
Communications
Edge
Management
Data
Ingestion
Long-Range
Communications Integration

© 2019 The MathWorks, Inc.
AI-Driven Systems for Embedded Intelligence
“AI” models.– and much more
Runs on embedded systems.– and distributed systems
Targets you select.– and platforms you don’t control
Collaboration in engineering.– and across your organization

© 2019 The MathWorks, Inc.
Looking Ahead: Key Issues
▪ Repeatability
▪ Suitability
• Applications: Analyze and Predict? Or also Optimize and Control?
• System’s requirement: “COULD…”, “SHOULD…”, or “MUST…”?
• Enable engineers (not only data scientists) to understand AI
• Tools for AI-driven system design on Asset, Edge, and Enterprise IT/OT
• Processes when including AI models in a system (Agile? Handoff?)
▪ Explainability
• Why AI works… When AI can work (and when it cannot)
• How to describe when and how to use AI, so society can be comfortable

© 2019 The MathWorks, Inc.
Where Will We Be in 2030?
▪ It depends
• Can we create tools that enable us to characterize the allowed
input environment to AI models that gives us confidence
that we know when and how they will work?
• If so, the use of AI models will grow tremendously
• If not, its use will still be numerous, but lessened

© 2019 The MathWorks, Inc.
Where Will We Be in 2030?
▪ It depends
• Can we create tools that enable us to characterize the allowed
input environment to AI models that gives us confidence
that we know when and how they will work?
• If so, the use of AI models will grow tremendously
• If not, its use will still be numerous, but lessened
▪ A key objective for the next 10 years
• Work out how to characterize where a model works
with complete accuracy

© 2019 The MathWorks, Inc.
Enjoy the conference!
Visit MathWorks at Booth