uTensor is a neural network inference library for microcontrollers (MCUs) that automatically generates C++ code from trained TensorFlow models. It includes a command line tool that takes a trained TensorFlow model file as input and outputs C++ header and source files to run the model on MCUs. The tool currently supports generating code for multilayer perceptrons (MLPs) and convolutional neural networks (CNNs). The presentation demonstrated an end-to-end example of training a CNN model in TensorFlow, saving it as a protocol buffer file, and using the uTensor tool to generate C++ code to run inference on the model.

1. uTensor
Neural Network Inference Library for MCU
github.com/uTensor
uTensor Team

2. FOSDEM presentation by Neil Tan

3. About Me
● Data Scientist @ Pinkoi
● Python Programmer
○ utensor_cgen

4. Agenda
● Tensorflow Toward the Edge
● Automatic Code Generator
○ CNN demo

5. Tensorflow v.s uTensor
Tensorflow uTensor

6. We Are All Set!

7. From This

8. To This

16. 我

17. Automatic Code Generator (utensor-cli)
github.com/uTensor/utensor_cgen

18. utensor-cli simple_mnist.pb --output-nodes=y_pred

20. The Code

22. End-to-End Example
1. MLP: end_to_end.ipynb
2. CNN: deep_cnn.ipynb

23. Basic Pipeline
1. Training with Tensorflow
2. Save trained model (.pb file)
3. uTensor cli
a. model.pb ---> uTensor cpp/hpp files

24. Status Quo
● Cli support
○ Multi layer Perceptron (MLP)
○ Convolution Neural Network (CNN)
● Performance/Memory Optimization
○ Memory Optimization: Reference Counting
○ Graph Optimization
■ Dropout layer removal
?

25. CNN Demo

27. Guide

28. Q & A