October 4, 2016
Santa Clara Convention Center
Mission City Ballroom
Open Software eXpansion
Fabrizio Rovati
Agenda
• STM32 Open Development Environment
• Open Software eXpansion
• Audio
• Sensors
• RF
• Frameworks
• Ecosystem
• Conclusions
2
STM32 Open Development Environment
• STM32 Open Development Environment
• Open Software eXpansion
• Audio
• Sensors
• RF
• Frameworks
• Ecosystem
• Conclusions
3
Lowering the Barriers for Developers 4
Market
Device
Prototype Integrated Boards Final
Device
Idea
Application
Test SW
Field
Test
Commercial
SW
Fast, affordable prototyping with development continuity to final device
STM32 Open Development Environment 5
Developer community and support
Compatibility with free and commercial Development Environments
STM32 Nucleo
development boards
STM32 Nucleo
expansion boards
Sensors – motion,
environment, light ..
Ultra-low power
connectivity
ULP Memories and
NFC Tags
Analog and mixed
signal components
Power and energy
management
STM32Cube
expansion software
STM32Cube
development software
Function packs
Set of function examples for some of the most common application cases
Fast, affordable prototyping & development
pre-integrated ST components and SW
Cloud
Connectivity
Sensing
Safety &
Security
STM32 ODE X-CUBE packages 6
Tools & IDEs
Application
Middleware
Hardware
Abstraction
Hardware
IAR EWARN, Keil MDK-ARM, GCC-based IDEs
(e.g. AC6 System Workbench for STM32)
Sample applications
STM32Cube
middleware
STM32Cube Expansion
middleware
STM32Cube Expansion
HAL
STM32Cube Hardware Abstraction Layer (HAL)
STM32 Nucelo expansion boards (X-NUCLEO)
STM32 Nucelo development boards
Open Software eXpansion
• STM32 Open Development Environment
• Open Software eXpansion
• Audio
• Sensors
• RF
• Frameworks
• Ecosystem
• Conclusions
7
OpenSoftwareX 8
Open Software eXpansion environment
Middleware
open.MEMS open.AUDIO open.RF
open.FRAMEWORK
STM32 Nucleo
development
boards
STM32 ODE
STM32 Nucleo
expansion
boards
STM32Cube
software
STM32Cube
expansion
software
+
HARDWARE SOFTWARE DRIVERS and HAL
Application examples
Open = Open Licensing
• Added value binary libraries for use with ST products
• Normally provided in binary form
• Free, automated and quick access to development license
• Clear path to production license
9
OpenSoftwareX licensing framework
License types
• Two different licensing type supported:
• EVALUATION license
• The license is NODE-LOCKED, i.e. the licensed library is usable on a specific STM32 only.
• The request/authorization of this license type can be initiated and completed by the licensee
autonomously via a licensing server.
• The licensee is given a unique KEY for the specific device
10
Enabling rapid evaluation of software IPs
through a simple automated email-based activation process
OpenSoftwareX licensing framework
License types
• Two different licensing type supported:
• PRODUCTION license
• The license is NODE-FREE, i.e. the licensed library is usable on any STM32-based mass–
produced devices
• Licensee agrees on the PRODUCTION license Limited License Agreement
• Licensee will get a new NODE-FREE KEY
• No need to rework the application, just insert the new key
11
OpenSoftwareX
License Wizard
12
Node-free
license request flag
License type field
Select library
Identify board
Generate request
Email request
Licensee info
OR… license directly via the APP! 13
Option menu
ST BlueMS APP
OR… license directly via the APP! 14
License status page
ST BlueMS APP
OR… license directly via the APP!
ST BlueMS APP
15
OR… license directly via the APP! 16
ST BlueMS APP
OR… license directly via the APP! 17
license enabled!
ST BlueMS APP
Audio
• STM32 Open Development Environment
• Open Software eXpansion
• Audio
• Sensors
• RF
• Frameworks
• Ecosystem
• Conclusions
18
Acoustic Processing - MEMS Microphones
Open.Audio
19
Sound Source
Localization
osxAcousticSL
Beamforming
osxAcousticBF
Acoustic Echo
Cancellation
osxAcousticEC
Using a MEMS microphone
array, it estimates the angle of
arrival of audio signal
Small and compact directional
virtual microphone based on
ST MEMS microphone array
Based on the Speex.org Open
Source libraries
Support for µ4 (micro-4)
The smallest microphone array architecture ever!
Sensors
• STM32 Open Development Environment
• Open Software eXpansion
• Audio
• Sensors
• RF
• Frameworks
• Ecosystem
• Conclusions
20
Open.MEMS for developers
Sensor Fusion
21
osxMotionFX – inputs/outputs
osxMotionFX – GUI osxMotionFX – features
• Magnetometer calibration routine
• Gyroscope bias compensation
• Dynamic distortion (hand jitter),
measured by the accelerometer
• Power saving
Open.MEMS for developers
Motion and Gesture processing
Activity Recognition
osxMotionAR
Carry Position Determination
osxMotionCP
Gesture Recognition
osxMotionFX
Detects
Walking, Fast Walking, Running,
Biking, Driving, Stationary
Detects
In Hand, Near Head, Shirt Pocket,
Trouser Pocket, On Desk, Arm Swing
Detects
gestures such as Glance, Pick-up,
Wake-up
Optimized for low power and always on experience
Uses Machine Learning based models to achieve higher accuracy
22
RF
• STM32 Open Development Environment
• Open Software eXpansion
• Audio
• Sensors
• RF
• Frameworks
• Ecosystem
• Conclusions
23
Bluetooth Low Energy Profiles
List of profiles supported by OSXSmartConnPS
24
• Slave profiles (peripheral role):
• Alert Notification Client
• Blood Pressure Sensor
• Find Me Locator
• Find Me Target
• Glucose Sensor
• Health Thermometer
• Heart Rate
• Phone Alert Client
• Proximity Monitor
• Proximity Reporter
• Time Client
• Time Server
• Master profiles (central role):
• Heart Rate Collector
• Time Client
• Find Me Locator
• Blood Pressure Collector
• Health Thermometer Collector
• Alert Notification Client
• Glucose Collector
Test it with STM32 BLE Profiles App
STM32 BLE Profiles
6LoWPAN
• Source code available in the osxContiki6LP package
• This package is based on STM32Cube and it provides a porting of the
Contiki OS and IPv6/6LoWPAN protocol stack 3.0, for STM32 L1 and F4
families, using the SPIRIT1 sub-1GHz radio transceiver
• Support for mesh networking by
the means of the standard RPL
protocol over IPv6 based protocol
stack, a key technology for large
wireless sensors networks
deployments
25
Sensor
Node
Sensor
Node
Sensor
Node
Sensor
Node
6LoWPAN
Border Router
IPv6 Host
Frameworks
• STM32 Open Development Environment
• Open Software eXpansion
• Audio
• Sensors
• RF
• Frameworks
• Ecosystem
• Conclusions
26
Open.Framework applications
• BLUEMICROSYSTEM
• Bluetooth Low Energy & sensor applications
• BlueVoiceLink
• Bluetooth Low Energy & audio applications
27
BLUEMICROSYSTEM
• A family of fully compatible applications, tailored to your needs
28
Function BMS1 BMS2 BMS3
Microcontroller
(NUCLEO-F401RE or NUCLEO-L476RG)   
Bluetooth Low Energy
(X-NUCLEO-IDB04A1 or X-NUCLEO-IDB05A1)   
Motion & Environmental sensors
(X-NUCLEO- IKS01A1)   
NFC secure pairing & TAG
(X-NUCLEO-NFC01A1) 
Ambient light & proximity, touchless UI
(X-NUCLEO-6180XA1) 
MEMS microphones
(X-NUCLEO-CCA02M1) 
www.st.com/BLUEMICROSYSTEM
ST BlueMS APP
• APP SDK available on GitHub in source code (project name BlueSTSDK)
• Both iOS and Android version
• https://github.com/STMicroelectronics-CentralLabs
• APP available on Google Play and Apple Store
29
ST BlueMS
BLUEMICROSYSTEM
sensing & actuating
30
Environmental page LSM6DS3/DSM EventsLed StatusAccelerometer plot
BLUEMICROSYSTEM
leverage HW features
31
LSM6DS3/DSMEvents
Pedometer Wake Up OrientationDouble Tap
BLUEMICROSYSTEM: Open.MEMS libraries 32
osxMotionAR osxMotionCPosxMotionGR osxMotionFX
BLUEMICROSYSTEM: Console & debug 33
Serial Console (stdout/stderr) Debug Console (stdin/stdout/stderr)
BLUEMICROSYSTEM: Touchless UI 34
BLUEMICROSYSTEM: Secure pairing 35
BLUEMICROSYSTEM: Firmware upgrade 36
BLUEMICROSYSTEM: Firmware upgrade 37
BLUEMICROSYSTEM
supports SensorTile
• The connectable multi-sensor node
38
Sensing, Tracking and Monitoring
Embedded Processing Unit
Motion GamingAlgorithms Augmented livingAudio
BlueVoiceLink SDK
Voice over Bluetooth Low Energy
39
Real-time voice streaming for remotely controlled applications
(addressing the RCU and wearable market)
BlueVoice Integration in
BLUEMICROSYSTEM
40
Cloud-based
ASR Service
8 kHz
ADPCM
osxBlueVoice 2.0.0
BLUEMICROSYSTEM2 V XYZ
ST BlueMS – ASR Demo 41
Hold on & speak!
Release
KO
OK
Ecosystem
• STM32 Open Development Environment
• Open Software eXpansion
• Audio
• Sensors
• RF
• Frameworks
• Ecosystem
• Conclusions
42
No soldering.
No components purchasing.
No hardware production plan needed!
Diolan SensiBLE
Much Smaller than STM32 Nucleo dev. Boards
+
+
+ +
43
Download App Insert Battery Explore ST BLUEMS App
Environmental Page Sensor Fusion
SensiBLE
SensiBLE & OSX SW: 100% compatibility
BLUEMICROSYSTEM1 pre-
flashed
ST BlueMS
1 2 3
44
Diolan SensiBLE
• Zero effort and time spent from transition from
prototype to deployment!
• I can demonstrate final form factor and have a first
batch of production in no time!
• Ready off-the-shelf solution at the right price
• no skills needed or no ROI to justify to develop own
board
45
• Zero SW development costs & time!
• Zero SW support costs & time for the standard SW
package!
• Fast & inexpensive development cycle but high
potential returns!
• Implicit promotion by the OSX
• Low volumes but with very high market base –
thousands of potential customers
Ideal for first low-volume trials or productions for customers
Opens door for further opportunities
• If I need a fast modification or design service, I have
professional and skilled support at hand
• If a want to scale up production volume, I can make
optimized deals
• Possibility to promote design customization skills
• A sizable part of the trials/small production customers
can grow exponentially very soon
Developer’sview
Partner’sview
Takeaways
• ST has all the building blocks adapted to the different needs of diverse
applications
• STM32 ODE is the right environment for fast affordable developments
• Open Software eXpansion provides added value libraries and end-to-end
applications in sensors, audio and RF field
• Exploit the ecosystem to cover the “last mile” to product introduction
46
We are full of more ideas…. stay tuned for further developments!!!
Thank You

Track 2 session 4 - st dev con 2016 - opensoftwarex

  • 1.
    October 4, 2016 SantaClara Convention Center Mission City Ballroom Open Software eXpansion Fabrizio Rovati
  • 2.
    Agenda • STM32 OpenDevelopment Environment • Open Software eXpansion • Audio • Sensors • RF • Frameworks • Ecosystem • Conclusions 2
  • 3.
    STM32 Open DevelopmentEnvironment • STM32 Open Development Environment • Open Software eXpansion • Audio • Sensors • RF • Frameworks • Ecosystem • Conclusions 3
  • 4.
    Lowering the Barriersfor Developers 4 Market Device Prototype Integrated Boards Final Device Idea Application Test SW Field Test Commercial SW Fast, affordable prototyping with development continuity to final device
  • 5.
    STM32 Open DevelopmentEnvironment 5 Developer community and support Compatibility with free and commercial Development Environments STM32 Nucleo development boards STM32 Nucleo expansion boards Sensors – motion, environment, light .. Ultra-low power connectivity ULP Memories and NFC Tags Analog and mixed signal components Power and energy management STM32Cube expansion software STM32Cube development software Function packs Set of function examples for some of the most common application cases Fast, affordable prototyping & development pre-integrated ST components and SW Cloud Connectivity Sensing Safety & Security
  • 6.
    STM32 ODE X-CUBEpackages 6 Tools & IDEs Application Middleware Hardware Abstraction Hardware IAR EWARN, Keil MDK-ARM, GCC-based IDEs (e.g. AC6 System Workbench for STM32) Sample applications STM32Cube middleware STM32Cube Expansion middleware STM32Cube Expansion HAL STM32Cube Hardware Abstraction Layer (HAL) STM32 Nucelo expansion boards (X-NUCLEO) STM32 Nucelo development boards
  • 7.
    Open Software eXpansion •STM32 Open Development Environment • Open Software eXpansion • Audio • Sensors • RF • Frameworks • Ecosystem • Conclusions 7
  • 8.
    OpenSoftwareX 8 Open SoftwareeXpansion environment Middleware open.MEMS open.AUDIO open.RF open.FRAMEWORK STM32 Nucleo development boards STM32 ODE STM32 Nucleo expansion boards STM32Cube software STM32Cube expansion software + HARDWARE SOFTWARE DRIVERS and HAL Application examples
  • 9.
    Open = OpenLicensing • Added value binary libraries for use with ST products • Normally provided in binary form • Free, automated and quick access to development license • Clear path to production license 9
  • 10.
    OpenSoftwareX licensing framework Licensetypes • Two different licensing type supported: • EVALUATION license • The license is NODE-LOCKED, i.e. the licensed library is usable on a specific STM32 only. • The request/authorization of this license type can be initiated and completed by the licensee autonomously via a licensing server. • The licensee is given a unique KEY for the specific device 10 Enabling rapid evaluation of software IPs through a simple automated email-based activation process
  • 11.
    OpenSoftwareX licensing framework Licensetypes • Two different licensing type supported: • PRODUCTION license • The license is NODE-FREE, i.e. the licensed library is usable on any STM32-based mass– produced devices • Licensee agrees on the PRODUCTION license Limited License Agreement • Licensee will get a new NODE-FREE KEY • No need to rework the application, just insert the new key 11
  • 12.
    OpenSoftwareX License Wizard 12 Node-free license requestflag License type field Select library Identify board Generate request Email request Licensee info
  • 13.
    OR… license directlyvia the APP! 13 Option menu ST BlueMS APP
  • 14.
    OR… license directlyvia the APP! 14 License status page ST BlueMS APP
  • 15.
    OR… license directlyvia the APP! ST BlueMS APP 15
  • 16.
    OR… license directlyvia the APP! 16 ST BlueMS APP
  • 17.
    OR… license directlyvia the APP! 17 license enabled! ST BlueMS APP
  • 18.
    Audio • STM32 OpenDevelopment Environment • Open Software eXpansion • Audio • Sensors • RF • Frameworks • Ecosystem • Conclusions 18
  • 19.
    Acoustic Processing -MEMS Microphones Open.Audio 19 Sound Source Localization osxAcousticSL Beamforming osxAcousticBF Acoustic Echo Cancellation osxAcousticEC Using a MEMS microphone array, it estimates the angle of arrival of audio signal Small and compact directional virtual microphone based on ST MEMS microphone array Based on the Speex.org Open Source libraries Support for µ4 (micro-4) The smallest microphone array architecture ever!
  • 20.
    Sensors • STM32 OpenDevelopment Environment • Open Software eXpansion • Audio • Sensors • RF • Frameworks • Ecosystem • Conclusions 20
  • 21.
    Open.MEMS for developers SensorFusion 21 osxMotionFX – inputs/outputs osxMotionFX – GUI osxMotionFX – features • Magnetometer calibration routine • Gyroscope bias compensation • Dynamic distortion (hand jitter), measured by the accelerometer • Power saving
  • 22.
    Open.MEMS for developers Motionand Gesture processing Activity Recognition osxMotionAR Carry Position Determination osxMotionCP Gesture Recognition osxMotionFX Detects Walking, Fast Walking, Running, Biking, Driving, Stationary Detects In Hand, Near Head, Shirt Pocket, Trouser Pocket, On Desk, Arm Swing Detects gestures such as Glance, Pick-up, Wake-up Optimized for low power and always on experience Uses Machine Learning based models to achieve higher accuracy 22
  • 23.
    RF • STM32 OpenDevelopment Environment • Open Software eXpansion • Audio • Sensors • RF • Frameworks • Ecosystem • Conclusions 23
  • 24.
    Bluetooth Low EnergyProfiles List of profiles supported by OSXSmartConnPS 24 • Slave profiles (peripheral role): • Alert Notification Client • Blood Pressure Sensor • Find Me Locator • Find Me Target • Glucose Sensor • Health Thermometer • Heart Rate • Phone Alert Client • Proximity Monitor • Proximity Reporter • Time Client • Time Server • Master profiles (central role): • Heart Rate Collector • Time Client • Find Me Locator • Blood Pressure Collector • Health Thermometer Collector • Alert Notification Client • Glucose Collector Test it with STM32 BLE Profiles App STM32 BLE Profiles
  • 25.
    6LoWPAN • Source codeavailable in the osxContiki6LP package • This package is based on STM32Cube and it provides a porting of the Contiki OS and IPv6/6LoWPAN protocol stack 3.0, for STM32 L1 and F4 families, using the SPIRIT1 sub-1GHz radio transceiver • Support for mesh networking by the means of the standard RPL protocol over IPv6 based protocol stack, a key technology for large wireless sensors networks deployments 25 Sensor Node Sensor Node Sensor Node Sensor Node 6LoWPAN Border Router IPv6 Host
  • 26.
    Frameworks • STM32 OpenDevelopment Environment • Open Software eXpansion • Audio • Sensors • RF • Frameworks • Ecosystem • Conclusions 26
  • 27.
    Open.Framework applications • BLUEMICROSYSTEM •Bluetooth Low Energy & sensor applications • BlueVoiceLink • Bluetooth Low Energy & audio applications 27
  • 28.
    BLUEMICROSYSTEM • A familyof fully compatible applications, tailored to your needs 28 Function BMS1 BMS2 BMS3 Microcontroller (NUCLEO-F401RE or NUCLEO-L476RG)    Bluetooth Low Energy (X-NUCLEO-IDB04A1 or X-NUCLEO-IDB05A1)    Motion & Environmental sensors (X-NUCLEO- IKS01A1)    NFC secure pairing & TAG (X-NUCLEO-NFC01A1)  Ambient light & proximity, touchless UI (X-NUCLEO-6180XA1)  MEMS microphones (X-NUCLEO-CCA02M1)  www.st.com/BLUEMICROSYSTEM
  • 29.
    ST BlueMS APP •APP SDK available on GitHub in source code (project name BlueSTSDK) • Both iOS and Android version • https://github.com/STMicroelectronics-CentralLabs • APP available on Google Play and Apple Store 29 ST BlueMS
  • 30.
    BLUEMICROSYSTEM sensing & actuating 30 Environmentalpage LSM6DS3/DSM EventsLed StatusAccelerometer plot
  • 31.
  • 32.
    BLUEMICROSYSTEM: Open.MEMS libraries32 osxMotionAR osxMotionCPosxMotionGR osxMotionFX
  • 33.
    BLUEMICROSYSTEM: Console &debug 33 Serial Console (stdout/stderr) Debug Console (stdin/stdout/stderr)
  • 34.
  • 35.
  • 36.
  • 37.
  • 38.
    BLUEMICROSYSTEM supports SensorTile • Theconnectable multi-sensor node 38 Sensing, Tracking and Monitoring Embedded Processing Unit Motion GamingAlgorithms Augmented livingAudio
  • 39.
    BlueVoiceLink SDK Voice overBluetooth Low Energy 39 Real-time voice streaming for remotely controlled applications (addressing the RCU and wearable market)
  • 40.
    BlueVoice Integration in BLUEMICROSYSTEM 40 Cloud-based ASRService 8 kHz ADPCM osxBlueVoice 2.0.0 BLUEMICROSYSTEM2 V XYZ
  • 41.
    ST BlueMS –ASR Demo 41 Hold on & speak! Release KO OK
  • 42.
    Ecosystem • STM32 OpenDevelopment Environment • Open Software eXpansion • Audio • Sensors • RF • Frameworks • Ecosystem • Conclusions 42
  • 43.
    No soldering. No componentspurchasing. No hardware production plan needed! Diolan SensiBLE Much Smaller than STM32 Nucleo dev. Boards + + + + 43
  • 44.
    Download App InsertBattery Explore ST BLUEMS App Environmental Page Sensor Fusion SensiBLE SensiBLE & OSX SW: 100% compatibility BLUEMICROSYSTEM1 pre- flashed ST BlueMS 1 2 3 44
  • 45.
    Diolan SensiBLE • Zeroeffort and time spent from transition from prototype to deployment! • I can demonstrate final form factor and have a first batch of production in no time! • Ready off-the-shelf solution at the right price • no skills needed or no ROI to justify to develop own board 45 • Zero SW development costs & time! • Zero SW support costs & time for the standard SW package! • Fast & inexpensive development cycle but high potential returns! • Implicit promotion by the OSX • Low volumes but with very high market base – thousands of potential customers Ideal for first low-volume trials or productions for customers Opens door for further opportunities • If I need a fast modification or design service, I have professional and skilled support at hand • If a want to scale up production volume, I can make optimized deals • Possibility to promote design customization skills • A sizable part of the trials/small production customers can grow exponentially very soon Developer’sview Partner’sview
  • 46.
    Takeaways • ST hasall the building blocks adapted to the different needs of diverse applications • STM32 ODE is the right environment for fast affordable developments • Open Software eXpansion provides added value libraries and end-to-end applications in sensors, audio and RF field • Exploit the ecosystem to cover the “last mile” to product introduction 46 We are full of more ideas…. stay tuned for further developments!!!
  • 47.