Paper: http://ceur-ws.org/Vol-2670/MediaEval_19_paper_58.pdf Youtube: https://www.youtube.com/watch?v=KgTUlU_gALI Pierre-Etienne Martin, Jenny Benois-Pineau, Boris Mansencal, Renaud Peteri and Julien Morlier , Siamese Spatio-temporal convolutional neural network for stroke classification in Table Tennis games. Proc. of MediaEval 2019, 27-29 October 2019, Sophia Antipolis, France. Abstract: This work presents a Table Tennis stroke classification approach through a siamese spatio-temporal convolutional neural network - SSTCNN. The videos are recorded at 120 frames per second with players performing in natural conditions. The frames are extracted, resized and processed to compute the optical flow. From the optical flow, a region of interest - ROI - is inferred. The SSTCNN is then fed by RGB and optical flow ROIs stream to give a probabilistic classification over all the table tennis strokes. Presented by Pierre-Etienne Martin

1. Benchmarking Initiative for Multimedia Evaluation
MediaEval
MediaEval 2019
10th Anniversary Workshop
27-29 October 2019
EURECOM, Sophia Antipolis, France

2. ediaEval 2019
by:
Pierre-Etienne Martin
Jenny Benois-Pineau
Boris Mansencal
Renaud Péteri
Julien Morlier
2
Siamese Spatio-Temporal
Convolutional Neural
Network for stroke
classification in Table
Tennis games

3. Runs submitted
3
Accuracies in %
Rank Team Run1 Run2 Run3 Run4
1
Univ. Bordeaux -
LaBRI
19.2 17.2 17.8 22.9

4. Data construction
4
➢ RGB extraction
➢ Optical Flow computation
○ DIS
○ Deep Flow
➢ ROI calculation from Flow
Dis * Foreground Deep Flow
[1] P.-E. Martin et al., “Optimal choice of motion estimation methods for fine-grained action classification with 3D convolutional networks,”
in IEEE ICIP, 2019.

5. Data Normalization
5
➢ Normalization which takes into account the distribution of the maximum flow values in each
direction of each frame among the training set
[1] P.-E. Martin et al., “Optimal choice of motion estimation methods for fine-grained action classification with 3D convolutional networks,”
in IEEE ICIP, 2019.
Max Normalization Ours ‘Normal’

6. Data augmentation
6
➢ Spatial augmentation centered in the ROI center
○ translation
○ rotations
○ zooms
○ flip
➢ RGB channel swap
➢ Temporal augmentation with probabilistic random extraction of 100 successive frames within the
boundaries of the stroke

7. Model
7
➢ Cuboides of size (W x H x T) = (120 x 120 x 100)
➢ Trained from scratch with 250 epochs on split train dataset
➢ Trained with full dataset using number of epochs with best performances
SSTCNN Architecture
[2] P.-E. Martin et al., “Sport action recognition with siamese spatio-temporal cnns: Application to table tennis,” in IEEE CBMI, 2018.

8. Runs submitted
8
Accuracies in %
Rank Team Run1 Run2 Run3 Run4
1
Univ. Bordeaux -
LaBRI
19.2 17.2 17.8 22.9
?

9. Runs submitted
9
Accuracies in %
Train dataset Split Full
Flow DIS DeepFlow DIS DeepFlow
Runs 19.2 17.2 17.8 22.9

10. Best run
10
➢ 20 strokes

11. Evaluation
11
➢ Hand
○ Forehand
○ Backhand

12. Evaluation
12
➢ Type
○ Service
○ Offensive
○ Defensive

13. Evaluation
13
➢ Hand + Type

14. Conclusion
14
➢ Need negative samples
➢ Multi label or/and cascade
○ Forehand / Backhand
○ Offensive / Defensive / Services
○ Type of stroke
[3] P.-E. Martin et al., “Fine-Grained Action Detection and Classification in Table Tennis with Siamese Spatio-Temporal Convolutional
Neural Network,” in IEEE ICIP, 2019.

15. LaBRI - Building A30
351 cours de la Libération
33405, Talence
pierre-etienne.martin@u-bordeaux.fr
https://www.labri.fr/projet/AIV/CRISP_presentation.php
+33 5 40 00 38 80
www.linkedin.com/in/p-e-martin
@P_eMartin
Merci