第 1 回 Jetson ユーザー勉強会

JETSON ユーザー勉強会
MAY 2019

2
アジェンダ
➢ Jetson Platformのご紹介
• Jetson採用事例
• Jetsonファミリラインアップ
• Jetson Nanoの概要
➢ Jetson Nano Getting Started
• 各種のドキュメント、リソース
• システムセットアップ
• 電力効率 & パフォーマンスモニター
• GPIO & アクセサリ
➢ DeepStreamとTensorRTの概要
➢ Jetson NanoによるDeepStreamデモンストレーション

JETSON TX1 →
JETSON TX2 4 GB
7 - 15W
1 – 1.3 TFLOPS (FP16)
JETSON TX2 8GB | Industrial
7 – 15W
1.3 TFLOPS (FP16)
JETSON AGX XAVIER
10 – 30W
10 TFLOPS (FP16) | 32 TOPS (INT8)
JETSON NANO
5 - 10W
0.5 TFLOPS (FP16)
JETSON ファミリ
エッジでのAIから自律動作マシンまで
同一のソフトウェアが使用可能
エッジでのAI 完全自律マシン

6
JETSON NANO
GTC 2019 にて発表
開発者キット
モジュール

JETSON NANO スペック
機能仕様
GPU 128 基の NVIDIA CUDA® コアを実装した NVIDIA Maxwell™ アーキテクチャ
CPU クアッドコア ARM® Cortex®-A57 MPCore プロセッサ
メモリ 4 GB 64 ビット LPDDR4
ストレージ 16 GB eMMC 5.1 フラッシュ
ビデオエンコード 4K @ 30 (H.264/H.265)
ビデオデコード 4K @ 60 (H.264/H.265)
カメラ 12 レーン (3x4 または 4x2) MIPI CSI-2 DPHY 1.1 (1.5 Gbps)
コネクティビティギガビットイーサネット
ディスプレイ HDMI 2.0 または DP1.2 | eDP 1.4 | DSI (1 x2) の 2 つ同時
UPHY 1 x1/2/4 PCIE、1x USB 3.0、3x USB 2.0
I/O 1x SDIO / 2x SPI / 6x I2C / 2x I2S / GPIOs

JETSON NANO RUNS MODERN AI
0
9
0
48
0 0 0 0 0 0
16
0
5
11
2
0
5
0.6
5
36
11 10
39
7
2
25
18
15 14
0
10
20
30
40
50
Resnet50 Inception v4 VGG-19 SSD
Mobilenet-v2
(300x300)
SSD
Mobilenet-v2
(960x544)
SSD
Mobilenet-v2
(1920x1080)
Tiny Yolo Unet Super
resolution
OpenPose
Img/sec
Inference
Coral dev board (Edge TPU) Raspberry Pi 3 + Intel Neural Compute Stick 2 Jetson Nano Not supported/DNR
TensorFlow TensorFlow TensorFlowTensorFlow PyTorch MxNet Darknet Caffe PyTorch Caffe

9
NsightDeveloperTools
DeepStream SDK
Modules
Depth
Estimation
Path
Planning
Object
Detection
Gesture
Recognition
Ecosystem
Modules
Pose
Estimation
Autonomous
Navigation
CUDA / Linux for Tegra
JetPackSDK
TensorRT
cuDNN
VisionWorks
OpenCV
libargus
Video API
GraphicsComputer Vision Accel. ComputingDeep Learning
Drivers
Ecosystem
cuBLAS
cuFFT
Vulkan
OpenGL
SensorsMultimedia
Isaac Robotics Engine
https://docs.nvidia.com/jetson/jetpack/introduction/index.html
https://docs.nvidia.com/jetson/l4t/index.html
https://docs.nvidia.com/cuda/index.html
Jetson Nano Jetson AGX XavierJetson TX1/TX2
JETSON SOFTWARE

より多くのエッジデバイスにAIを
ホーム / サービスロボット
175 billion hours per year
on household chores (US)
マシンビジョン / AOI
1 trillion product units per
year require visual inspection
ネットワークビデオレコーダー
(NVR)
200 million 1080p streams
AIOT
80% of Enterprise IOT
projects will use AI by 2022

11
JETSON NANO GETTING STARTED

12
JETSON NANO RESOURCES
https://developer.nvidia.com/embedded/twodaystoademo https://github.com/NVIDIA-AI-IOT/jetbot https://devtalk.nvidia.com/default/board/371/jetson-nano/
https://elinux.org/Jetson_Nano#Ecosystem_Products_and_Sensorshttps://elinux.org/Jetson_Nanohttps://developer.nvidia.com/embedded-computing
TUTORIALS PROJECTS DEVELOPER FORUM
DOCUMENTATION & SOFTWARE DOWNLOADS JETSON WIKI NANO ACCESSORIES

13
• Device is booted from a MicroSD card
• 16GB UHS-1 recommended minimum
• Download the SD card image from NVIDIA.com
• Flash the SD card image with Etcher program
• Insert the MicroSD card into the slot located on the
underside of the Jetson Nano module
• Connect keyboard, mouse, display, and power supply
• Board will automatically boot when power is applied
• Green power LED will light
http://www.nvidia.com/JetsonNano-Start
SYSTEM SETUP

14
• 5V⎓2A Micro-USB charger
• Adafruit #1995
• 5V⎓4A DC barrel jack adapter
• Adafruit #1466
• 5.5mm OD x 2.1mm ID x 9.5mm length
• Place a jumper on header J48
• J41 Expansion Header, pins 2/4
• Up to 5V⎓3A per pin (5V⎓6A total)
• Power over Ethernet (PoE)
• Standard PoE supply is 48V
• Use a PoE hat or 5V regulator
POWER SUPPLIES
• J40 Button Header can disable Auto Power-On
• Manual Power-On / Reset
• Enter Recovery Mode

15
Different power mode presets: 5W and 10W
Default mode is 10W
Users can create their own presets, specifying clocks and online cores in /etc/nvpmodel.conf
< POWER_MODEL ID=1 NAME=5W >
CPU_ONLINE CORE_0 1
CPU_ONLINE CORE_1 1
CPU_ONLINE CORE_2 0
CPU_ONLINE CORE_3 0
CPU_A57 MAX_FREQ 918000
GPU MAX_FREQ 640000000
EMC MAX_FREQ 1600000000
NVIDIA Power Model Tool
sudo nvpmodel –q (for checking the active mode)
sudo nvpmodel –m 0 (for changing mode, persists after reboot)
sudo jetson_clocks (to disable dynamic scaling & lock clocks to max for active
mode)
Power Mode 10W† 5W
Mode ID 0 1
Online CPU Cores 4 2
CPU Max Frequency (MHz) 1428 918*
GPU Max Frequency (MHz) 921 640*
Memory Max Freq. (MHz) 1600 1600
† Default Mode is 10W (ID:0)
* Rounded at runtime to closest discrete freq. available
POWER MODES

16
Run sudo tegrastats to launch the performance/utilization monitor:
RAM 1216/3963MB (lfb 330x4MB) IRAM 0/252kB(lfb 252kB)
CPU [27%@102,36%@307,6%@204,35%@518] EMC_FREQ 19%@204 GR3D_FREQ 0%@76 APE 25
PLL@25C CPU@29.5C PMIC@100C GPU@27C AO@34C thermal@28C POM_5V_IN 1532/1452
POM_5V_GPU 0/20 POM_5V_CPU 241/201
Memory
Memory
CPU
GPU
Thermal Power
Memory Used / Total Capacity
Bandwidth % @ Frequency (MHz)
Utilization / Frequency (MHz)
Utilization / Frequency (MHz)
Zone @ Temperature (°C) Current Consumption (mW) / Average (mW)
Refer to the L4T Developer Guide for more options and documentation on the output.
PERFORMANCE MONITOR

17
NsightDeveloperTools
DeepStream SDK
Modules
Depth
Estimation
Path
Planning
Object
Detection
Gesture
Recognition
Ecosystem
Modules
Pose
Estimation
Autonomous
Navigation
CUDA / Linux for Tegra
JetPackSDK
TensorRT
cuDNN
VisionWorks
OpenCV
libargus
Video API
GraphicsComputer Vision Accel. ComputingDeep Learning
Drivers
Ecosystem
cuBLAS
cuFFT
Vulkan
OpenGL
SensorsMultimedia
Isaac Robotics Engine
https://docs.nvidia.com/jetson/jetpack/introduction/index.html
https://docs.nvidia.com/jetson/l4t/index.html
https://docs.nvidia.com/cuda/index.html
Jetson Nano Jetson AGX XavierJetson TX1/TX2
JETSON SOFTWARE

18
• Similar 40-pin header to rPI, 3.3V logic levels
• Adafruit Blinka + SeeedStudio Grove support
• Jetson.GPIO Python library
• Compatible API with rPI.GPIO
• Docs & samples in /opt/nvidia/jetson-gpio/
• sysfs I/O access from /sys/class/gpio/
• Map GPIO pin echo 38 > /sys/class/gpio/export
• Set direction echo out > /sys/class/gpio/gpio38/direction
• Bit-banging echo 1 > /sys/class/gpio/gpio38/value
• Unmap GPIO echo 38 > /sys/class/gpio/unexport
• Query status cat /sys/kernel/debug/gpio
• https://www.kernel.org/doc/Documentation/gpio/sysfs.txt
• C/C++ programs (and other languages) can use same sysfs files
• I2C – libi2c for C/C++ and Python
USING GPIO
sysfs GPIO Name Pin Pin Name sysfs GPIO
3.3V 5.0V
I2C_2_SDA 5.0V
I2C_2_SCL GND
gpio216 AUDIO_MCLK UART_2_TX
GND UART_2_RX
gpio50 UART_2_RTS I2S_4_SCLK gpio79
gpio14 SPI_2_SCK GND
gpio194 LCD_TE SPI_2_CS1 gpio232
3.3V SPI_2_CS0 gpio15
gpio16 SPI_1_MOSI GND
gpio17 SPI_1_MISO SPI_2_MISO gpio13
gpio18 SPI_1_SCK SPI_1_CS0 gpio19
GND SPI_1_CS1 gpio20
I2C_1_SDA I2C_1_SCL
gpio149 CAM_AF_EN GND
gpio200 GPIO_PZ0 LCD_BL_PWM gpio168
gpio38 GPIO_PE6 GND
gpio76 I2S_4_LRCK UART_2_CTS gpio51
gpio12 SPI_2_MOSI I2S_4_SDIN gpio77
GND I2S_4_SDOUT gpio78
J41 Expansion Header
1 2
3
5
4
6
7
9
8
10
11 12
13 14
15 16
17 18
2019
21
23
22
24
2625
27 28
3029
31
33
32
34
35 36
38
4039
37

19
JETSON NANO ACCESSORIES
Printable Enclosures
Carriers
Sensors & Cameras
GPIO Hats
5V FansBattery Packs
eLinux.org/Jetson_Nano

DEEPSTREAMとTENSORRTの概要

21
画像認識アプリケーションの基本構成要素

22
Perception infra - Jetson, Tesla server (Edge and cloud)
Linux, CUDA
Analytics infra - Edge server, NGC, AWS, Azure
DeepStream SDK
Video/image capture and processing
Plugins Development and Deployment
RTSP Communications
DNN inference with TensorRT
3rd party libraries
Reference applications & orchestration recipes
Plugin templates for custom IP integration
TensorRT
Multimedia APIs/
Video Codec SDK
Imaging & Dewarping
library
Metadata & messaging NVDocker Message bus clients
Multi-camera
tracking lib
DEEPSTREAM SOFTWARE STACK
Multi-GPU applications and containers
Message broker and connection to IOT services
H264/
H265 Accelerated decode

23
DEEPSTREAM パイプラインアーキテクチャ

24

25

26
NVIDIA ハードウェア対応プラグイン
DeepStream プラグイン
プラグイン名称機能
gst-nvvideocodecs H.265 & H.264 ビデオデコーダー
gst-nvstreammux ストリーム・アグリゲーター – マルチプレクサとバッチ処理
gst-nvinfer 物体検出、及び識別の為の TensorRT ベースの推論
gst-nvtracker リファレンス KLT トラッカーの実装
gst-nvosd バウンディング・ボックスやオーバーレイ・テキスト表示の為の OSD API
gst-tiler 複数のソース映像を 1画面に表示するタイリング表示機能
gst-eglglessink NVIDIA ハードウェア対応 X11 / EGL ベースのレンダラープラグイン
gst-nvvidconv 映像のスケーリング、フォーマット変換、ローテーション
Gst-nvdewarp 360度カメラ入力の補正
Gst-nvmsgconv メタデータの生成
Gst-nvmsgbroker クラウドへのメッセージング

27
NVIDIA TensorRT
From Every Framework, Optimized For Each Target Platform
TESLA V100
DRIVE PX 2
TESLA P4
JETSON TX2
NVIDIA DLA
TensorRT

28
TENSORRT DEPLOYMENT WORKFLOW
TensorRT Optimizer
(platform, batch size, precision)
TensorRT Runtime Engine
Optimized Plans
Trained Neural
Network
Step 1: Optimize trained model
Plan 1
Plan 2
Plan 3
Serialize to disk
Step 2: Deploy optimized plans with runtime
Plan 1
Plan 2
Plan 3 Embedded Automotive Data center

29
concat
max pool
input
next input
3x3 conv.
relu
bias
1x1 conv.
relu
bias
1x1 conv.
relu
bias
1x1 conv.
relu
bias
concat
1x1 conv.
relu
bias
5x5 conv.
relu
bias
LAYER & TENSOR FUSION
max pool
input
next input
3x3 CBR 5x5 CBR 1x1 CBR
1x1 CBR
TensorRT Optimized Network
• Vertical Fusion
• Horizonal Fusion
• Layer Elimination
Network Layers
before
Layers
after
VGG19 43 27
Inception
V3
309 113
ResNet-152 670 159

30
JETSON NANOによるDEEPSTREAMデモンストレーション

31
DEEPSTREAM デモダイアグラム
GST-NvInfer
(Car-Detect)
Gst-
uridecode
GST-
NvTracker
(Car-Color)
(Car-Model)
GST-NvInfer
(Car-Make)
GST-
NvEglglessinkGST-OSD GST-Tiler
Gst-
uridecode
GST-NvInfer
(Car-Detect)
VIDEO
DECODE
STREAM MUX
PRIMARY
DETECTOR
OBJECT
TRACKER
SECONDARY
CLASSIFIERS
ON SCREEN
DISPLAY
TILER RENDERER

第 1 回 Jetson ユーザー勉強会

第 1 回 Jetson ユーザー勉強会

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to 第 1 回 Jetson ユーザー勉強会

Similar to 第 1 回 Jetson ユーザー勉強会 (20)

More from NVIDIA Japan

More from NVIDIA Japan (20)

Recently uploaded

Recently uploaded (8)

第 1 回 Jetson ユーザー勉強会