1) Image recognition and computer vision technologies can enable various smart city applications like crowd behavior analysis, traffic analysis, and thermal signature tracking. 2) Autonomous systems that use computer vision and machine learning can perceive their environment and act independently to help during disasters by providing survivors and emergency personnel with locating information. 3) MATLAB provides tools for computer vision, machine learning, and deep learning that can help develop prototypes and applications for smart cities from idea to product.

1. 1© 2017 The MathWorks, Inc.
Impact of Image Recognition in Smart Cities
Paola Jaramillo Application Engineer, MathWorks, Benelux
Ionut Barbu Application Engineer, MathWorks, Benelux
Digipolis Meetup
September 21, Antwerpen

2. 2
Agenda
▪ Case Study 1: Image recognition in Smart Systems (SERS)
▪ Case Study 2: Computer vision for autonomous systems (CNH)
▪ Relevant computer vision applications for Smart Cities:
▪ Crowd behavior analysis using:
▪ Live Face tracking on thermal imaging using Deep Learning and Regression
▪ Multiobject detection and tracking using Kalman Filters
▪ Thermal signature tracking
▪ Traffic Analysis using rapid prototyping of IoT systems

3. 3

4. 4
What it means to have SERS for building Smart Cities?
SERS challenges:
• Optimized response time and
infrastructure.
• Automated and in real-time.
SERS enabler:
Cyber-physical systems (CPS)
connecting the networked world
with the physical world.
SERS description
2017 Newsletter
SERS has been featured at the White House in June 2014 as an exemplary project.
Download the code!

5. 5
Autonomous systems play a key role in SERS:
Providing the survivors
and the emergency
personnel with information
to locate and assist each
other during a disaster.

6. 6
Provides the ability of a system
to act independently
of direct human control
under unrehearsed conditions.
Computer vision and Machine Learning,
enable these systems
to perceive and be aware of the environment.
Why Autonomous Technology?
Act
Perceive
Decide
& Plan
Sense

7. 7
Example: CNH IntelliFill System
Control Algorithms
Perceive
Act
Sense
Decide
& Plan
The New Holland IntelliFill™ awarded with gold medal for technical innovation.

8. 8
• Driver Input
• Visualization
• Computer
Vision
• Controls
Example: CNH IntelliFill System
Act
Perceive
Decide
& Plan
Sense
Computer vision and
controls algorithms
Vehicle Display Controller
ECU
ActuatorsEmbedded Coder
3D Cameras
CAN

9. 9
Access Data
Extract Features
Integrate
Extensive workflow support for Computer Vision and Machine Learning
… with industry
proven solutions
… from
idea to product
… enabling
application experts
Explore Models, Train, Evaluate

10. 10
Learn about Face recognition, Simplify Deep Learning training using Transfer Learning
Relevant applications for Smart Cities
Live face detection and tracking
Dataset obtained from Kaggle: Facial keypoints detection

11. 11
Relevant applications for Smart Cities
Crowd behavior analysis
https://nl.mathworks.com/help/vision/examples/motion-based-multiple-object-tracking.html

12. 12
Relevant applications for Smart Cities
Thermal signature tracking
https://nl.mathworks.com/videos/thermal-analysis-with-matlab-and-flir-cameras-98177.html

13. 13
Relevant applications for Smart Cities
Traffic Analysis
MATLAB Example, Simulink Example, Implementation on Hardware and ThingSpeak
From idea, to prototype
MATLAB, Simulink and ThingSpeak
Smart
Connected
Devices
Algorithm
Development
Analytic IoT
Platform

14. 14
Relevant applications for Smart Cities
Surveillance
Acknowledgement University of Cambridge: http://mi.eng.cam.ac.uk/research/projects/VideoRec/CamVid/

15. 15
Capture Images Directly into MATLAB
Image Acquisition and Hardware Support Packages
Hardware support

16. 16
Big Data Capabilities in MATLAB
Memory and Data Access
• Datastores
• Datastore
• imageDataStore
• …
Platforms
• Desktop (Multicore, GPU)
• Clusters
• Cloud
• …
Programming Constructs
• Parallel Processing
• GPU Arrays
• …

17. 17
Sparse
Techniques
Dense
Techniques
SURF HOG Image PixelsBag of Words
Definition: features are distinctive representations of data.
• Often invariant to changes in scale, rotation, illumination, etc.
• Features are more compact than storing pixel data
• Feature selection depends on the nature of the problem
Extracting Relevant Information from Data: Engineering Features!

18. 18
Many Options for Features and Machine Learning!
Feature Extraction
Computer Vision System Toolbox
• BRISK, FREAK, SURF
• Histogram of Oriented Gradients (HoG)
• Using box filters (integral images)
• Bag of visual words
• Color-based features
• Frequency-domain features
Machine Learning
Statistics and Machine Learning Toolbox
• SVM
• Decision trees
• Ensemble Learners
• Bagged trees
• k-NN
• Discriminant analysis
• Bayes classifiers
Many permutations and combinations to fit the needs of your problem!

19. 19
Many Options for Deep Learning!
Deep Learning
Neural Network Toolbox
• LSTM and DAG Networks
• Ground Truth labeling and Semantic Segmentation
• Train in multi-GPUs and the cloud
• Inspect features using activations
• Automatically convert MATLAB models to CUDA code
MATLAB makes Deep Learning easy and accessible!

20. 20
Access Data
Extract Features
Integrate
Extensive workflow support for Computer Vision and Machine Learning
… with industry
proven solutions
… from
idea to product
… enabling
application experts
Explore Models, Train, Evaluate

21. 21
MathWorks Overview
Chart: Earth’s topography on a Miller cylindrical projection,
created with MATLAB and Mapping Toolbox.
● Headquarters:
Natick, Massachusetts U.S.
● Other U.S. Locations:
California; Michigan;
Texas; Washington, D.C.
● Europe:
France, Germany, Italy,
Netherlands, Spain, Sweden,
Switzerland, United Kingdom
● Asia-Pacific:
Australia, China, India,
Japan, Korea
● Worldwide training
and consulting
● Distributors serving more
than 20 countries
Revenue & profitability
 ~$850M revenue in 2016
 More than 60% from outside the U.S.
 Profitable every year since its founding
Facts
 Founded in 1984
 Privately held, no external funding
 3500 employees worldwide
 #19 “Best company to work for”
#4 Software company, GlassDoor 2015
Customers
 Millions of users in 175+ countries

22. 22
Connecting you to thousands of users: MATLAB Central
▪ Community for MATLAB and Simulink users
– 200k daily visits
▪ File Exchange
– Access more than 29k+ free files including functions,
apps, examples, and models
▪ MATLAB Answers
– Ask programming questions or search
160k+ community-answered questions
▪ Blogs
– Read commentary from engineers who design, build,
and support MATLAB and Simulink
▪ ThingSpeak
– The open data platform for the Internet of Things
▪ Cody
– Improve MATLAB skills through interactive game play
Based on average 2016 data

23. 23
• Highly Skilled Resources
• Over 100 engineers, all MS+
• High customer satisfaction
• > 70% of issues resolved in 24 hours
• Extensive Online Help:
Don’t hesitate to contact us!
• Submit a service request:
www.mathworks.nl/support/servicerequests/create.html
• Starting with R2012b, submit directly from MATLAB
Toolstrip (see screenshot)
• Contact us directly at +31 40-215-6700, option 4,
followed by option 2
World class technical support ready to help

24. 24
Migration Planning
Component
Deployment
Full Application
Deployment
ContinuousImprovement
Accelerating return on investment with
Consulting Services
A global team of experts supporting every stage of tool and process integration
Supplier InvolvementProduct Engineering TeamsAdvanced EngineeringResearch
Advisory Services
Process Assessment
Jumpstart
Process and Technology
Standardization
Process and Technology
Automation

25. 25
Exploit the full potential of MathWorks products with
Training Services
Flexible delivery options:
▪ Public training available worldwide
▪ Onsite training with standard or customized courses
▪ Web-based training with live, interactive instructor-led courses
▪ Self-paced interactive online training
More than 50 course offerings:
▪ Introductory and intermediate training on MATLAB and
Simulink.
▪ Specialized courses in machine learning, signal processing,
and computer vision.
Public
On-Site

26. 26© 2017 The MathWorks, Inc.
Thank you!
Paola.Jaramillo@mathworks.nl Signal Processing, Computer Vision, Automated Driving Systems
Ionut.Barbu@mathworks.nl Data analytics, Big data, Application Deployment to Enterprise IT Systems.