The document discusses skeleton tracking with TensorFlow. It describes PoseNet, a TensorFlow.js model for real-time pose estimation. PoseNet takes an input image and returns predicted keypoints and heatmaps for each detected pose. It works by outputting heatmaps for each keypoint and offset values, then uses the heatmaps and offsets to extract the predicted keypoint locations and estimate poses. The document provides example output formats for PoseNet and discusses factors like model size, accuracy, and processing speed.

1. LANCASTER AI
SKELETON TRACKING WITH TENSORFLOW
Richard Everts
Co-founder, Sugey
www.sugeyone.com
rich@sugeyone.com
@sugeyone

2. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
WHAT ARE WE TALKING ABOUT?

3. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
WHAT ARE WE TALKING ABOUT?

4. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
THE TOOLS

5. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
SIMILAR TO OBJECT DETECTION

6. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
POSENET - TENSORFLOW.JS VARIATION
17 Total Points
0 nose
1 leftEye
2 rightEye
3 leftEar
4 rightEar
5 leftShoulder
6 rightShoulder
7 leftElbow
8 rightElbow
9 leftWrist
10 rightWrist
11 leftHip
12 rightHip
13 leftKnee
14 rightKnee
15 leftAnkle
16 rightAnkle

7. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
RETURNING JSON OBJECT SHAPES
6 - 7 million
120 million
{
"score": 0.32371445304906,
"keypoints": [
{ // nose
"position": {
"x": 301.42237830162,
"y": 177.69162777066
},
"score": 0.99799561500549
},
{ // left eye
"position": {
"x": 326.05302262306,
"y": 122.9596464932
},
"score": 0.99766051769257
},
{ // right eye
"position": {
"x": 258.72196650505,
"y": 127.51624706388
},
"score": 0.99926537275314
},
...
]
}
[
{ // pose #1
"score": 0.42985695206067,
"keypoints": [
{ // nose
"position": {
"x": 126.09371757507,
"y": 97.861720561981
},
"score": 0.99710708856583
},
...
]
},
{ // pose #2
"score": 0.13461434583673,
"keypositions": [
{ // nose
"position": {
"x": 116.58444058895,
"y": 99.772533416748
},
"score": 0.9978438615799
},
...
]
},
...
]

8. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
THE POSENET MODEL
6 - 7 million
120 million

9. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
HEATMAPS AND OFFSETS
6 - 7 million
120 million

10. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
HEATMAPS AND OFFSETS
6 - 7 million
120 million

11. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
STRIDES AND ACCURACY
6 - 7 million
120 million

12. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
6 - 7 million
120 million
- Special thanks to Dan Oved and his Medium article on OpenPose
and using Tensorflow.js