Presentation for Paris Big Data and Machine Learning Meetup. Summary: Quick description of the process currently used at Footbar to start recognizing football gestures from only one acceleremoter attached on the calf of the football player. Please feel free to contact us if you're interested by our data or our project, we would be glad to hear from you.

1. FOOTBAR
From accelerometer data
to football gesture
recognition.
Sébastien BENOIT A LA GUILLAUME
sebastien@footbar.com

2. A BIT OF CONTEXT
• Device: 3-axis accelerometer attached around the
calf with a strap.

Data retrieval through USB.

1 hour data = 1 MB

3. FOOTBAR TODAY

4. FOOTBAR TODAY

5. OUR AVAILABLE DATA
Video feed Raw accelerometer data

6. WHAT DO WE WANT ?
• Know the player’s activity at any given time knowing only its acceleration.
• First questions to answer before even thinking about machine learning :

How to defne an activity / a gesture ?

How to slice (temporally speaking) our acceleration data to guess
the right class for the right gesture ?

7. • Create a similarity with NLP feld :

Slice the raw data into « elementary » windows, representing words.

Create sentences made of successive words, ie gestures.

Classify gestures from their included elementary windows.
OUR APPROACH
Footbar NLP
Elementary gesture Word
Player’s gesture /
activity (e.g. shot,
walk...)
Sentence
Player gestures’
classification
Sentence classification

8. OUR APPROACH

Process for characterizing a new report :

Slice it into elementary windows

Classify every elementary window

Slice the report into bigger windows (successive elementary windows)

Classify every slice

9. DATA SLICING
« Smart slicing » :
Universal slicing
strategy for easier
elementary gesture
recognition.
Here, when the
fltered norm
crosses 0.

10. EXAMPLE : WALK
Crash
Stance
Takeoff
Freefall

11. CREATE A DATABASE – FROM SCRATCH

Where to start ?

Easily recognizable gestures, at elementary level too : walk.

For every player : Annotate some walk at elementary level (and macro
level too)

Learn from these annotated elementary windows and classify others in
the signal.

Amongst newly predicted elementary windows, look for macro walk
pattern (crash – stance – takeof – freefall).

When there’s a match, check on video that the player is walking.

Repeat this for other gestures.

12. ELEMENTARY GESTURE RECOGNITION

An elementary gesture = around 10 (200 ms) * 3 (axes) values of
acceleration.

Our data representation : 10 features (max, mean, std, duration…).

Pros : Fast to compute, same length for all windows.

Cons : No axis dependant feature.

Classifcaton algorithm used : Random forests.

Pros : Fast to compute, robust.

Cons : Can be painful to understand when classifcation goes wrong.

13. ELEMENTARY TO REAL GESTURE
A frst approach : Bag of words modelisation of real gestures.
Example : someone walking.
Word Occurences
Crash 2
Stance 2
Takeoff 2
Freefall 2

14. ELEMENTARY TO REAL GESTURE
Real gesture classifcation :
Use bag of words representation as input of a classifer.
Adapt to diferent lengths of gestures (e.g. walk of 6s and walk of 2s).

Use relative frequency.

15. SO FAR

Elementary gestures :

> 100k gestures.

17 diferent elementary gestures.

F1-score of 0.95 for walk related gestures, 0.65 for shot gestures.

Player gestures :

> 10k gestures.

4 diferent gestures : rest, walk, run and shot.

16. PERSPECTIVES
So many things to explore…

Work around the features, the classifcation algorithms.

Recognize and annotate more gestures.

Add other player’s data for team interactions.

Classify other « level » of activities. (example : player on the pitch /
not on the pitch, player defending / attacking).
➢
Start looking at deep learning possibilities. (recurrent neural
networks seem natural in a football context.)

17. YOUR TURN!
Contact :
sebastien@footbar.com
sylvain@footbar.com
Interested by our project?
We are open to any ideas. If you’re interested in getting an eye (or
even your hands!) on our data, please contact us!

18. QUESTIONS
Contact :
sebastien@footbar.com
sylvain@footbar.com