My daily work place is at Trondheim University College, Faculty of Informatics and e-Learning.
One particular problem with ski jumping is that the number of ski jumps is limited to only a few ski jumps per hour. The ski jumper has to get back to the hill.
Human Movement Science Program at NTNU posesses much valuable knowledge about human movement patterns. They also have the Norwegian national ski jumping in for regular training sessions. Indoor ski jumps concist mainly of take-off training and landing in a mattress on the floor. These may be extended into virtual ski jumps simulating the real effects in the hill.
I just wanted to show a picture of the hill for those of you who do not know how it looks. Up to the left you can see a 3D model of the ski jumping hill that we use for the visualizations.
In our case we use three identical cameras. It is neccesary to have at least two cameras to recover the 3D information. A combination of many pixels and high frame rate will have great effect on the location accuracy in 3D.
As you can see this can result in large amounts of video data (about 30 – 35 Mbyte per second of data). RAID solutions or hard disks with increased rotational speed can handle this. Another solution is to limit the video stream in time and capture the entire stream to memory before transfering it to the hard disk.
Human body markers must be easy separable from the rest of the image and can for instance be white spots of reflective material on the jumping suit and skies. Naturally robust features can be found studying image features like edges, corners and gradients. Marking manually is too time consuming.
Position in one camera can be lost and 3D coordinate still recovered from minimum two cameras. Blur effect caused by the fast movement of the ski jumper compared against the limited frame rate of 30 fps.
Is an old field originating from land surveying. Ideally the cameras should be placed 90 degrees perpendicular to each other. More practically in a pyramid shape (about 45 degrees). The features are then much easier detectable and comparable.
On site calibration is done every time the system is being mounted for use. Describes the linear relationship between camera coordinates (pixels) and object coordinates (cm).
All the movements made by the ski jumper must be allowable. The jumper can not twist his head five times or spin his leg through the right leg.
Transcript of "PPT"
Three-dimensional Motion Capture, Modelling and Analysis of Ski Jumpers Atle Nes CSGSC 2005 Trondheim, April 28th
Overview <ul><li>Project description </li></ul><ul><li>What kind of data are we interested in? </li></ul><ul><li>Capturing data: Image acquisition, Camera system </li></ul><ul><li>Processing data: Feature points, Motion capture, Photogrammetry </li></ul><ul><li>Interpreting data: Visualization, Motion analysis </li></ul><ul><li>Conclusion </li></ul>
Project description <ul><li>Task: Design a computer system that can capture and study the motion of ski jumpers in 3D. </li></ul><ul><li>Goal: The results will be used to give feedback to the ski jumpers that can help them to increase their jumping lengths. </li></ul>
Data collection <ul><li>Will be gathered and analyzed in close cooperation with Human Movement Science Program at NTNU. </li></ul><ul><li>Data: </li></ul><ul><li>Mainly from outdoor ski jumps captured at Granåsen ski jumping hill here in Trondheim. </li></ul><ul><li>Also from indoor ski jumps captured at Dragvoll sports facilities. </li></ul>
Image acquisition <ul><li>Video sequences are captured simultanuously from multiple video cameras. </li></ul><ul><li>Two decisive camera factors: </li></ul><ul><li>Spatial resolution (pixels) </li></ul><ul><li>Time resolution (frame rate) </li></ul>
Camera equipment <ul><li>3 x AVT Marlin F080b </li></ul><ul><li>IEEE1394 Firewire, DCAM </li></ul><ul><li>8-bit greyscale w/ max resolution 1024x768x15fps or 640x480x30fps </li></ul><ul><li>Extra trigger cable/signal Video capture synchronization. </li></ul><ul><li>Different camera lenses Capture the same area from different distances. </li></ul><ul><li>Optical fibre Extends the distance from computer to cameras in the hill, keeping the transmission speed. </li></ul>
Feature points <ul><li>Robust feature points: </li></ul><ul><li>Human body markers (easy detectable) </li></ul><ul><li>Naturally robust features (more difficult). </li></ul><ul><li>Want to have automatic detection of robust feature points using simple image processing techniques. </li></ul>
Motion capture <ul><li>Localizing, identifying and tracking identical feature points in both sequences of video images as well as accross different camera views . </li></ul><ul><li>Synchronized video streams ensures good 3D coordinate accuracy. </li></ul>
Tracking w/ missing data <ul><li>Occluded features Redundancy using multiple cameras with different views. </li></ul><ul><li>Probability theory Guess the point position based on feature point velocity. </li></ul><ul><li>Another problem Blur effect </li></ul>?
Photogrammetry <ul><li>Matching corresponding feature points from two or more cameras allows us to calculate the exact position of that feature point in 3D. </li></ul><ul><li>Good camera placement is important for good triangulation capabilities (3D coordinate accuracy). </li></ul>
Camera calibration <ul><li>Coordinate system On site calibration using known coordinates in the ski jumping arena. </li></ul><ul><li>Direct Linear Transformation (DLT) by Abdel-Aziz and Karara in 1971. </li></ul><ul><li>Lens distortion (unlinear) </li></ul><ul><li>Intelligent removal of the worst calibration points (sources of error). </li></ul>
Visualization <ul><li>Feature point tracks are connected back onto a dynamic model of the ski jumper. </li></ul><ul><li>Dynamic model of ski jumper is combined with static model of ski jump arena. </li></ul>
Motion analysis <ul><li>Done in close cooperation with Human Movement Science Program </li></ul><ul><li>Extract movements that have greatest influence on the result. </li></ul><ul><li>Using statistical tools and prior knowledge about movements </li></ul><ul><li>Project some movements to unseen 2D views. </li></ul>
Related applications <ul><li>Medical: </li></ul><ul><li>Diagnosis of infant spontaneous movements for early detection of possible brain damage (cerebral palsy). </li></ul><ul><li>Diagnosis of adult movements (walk), for determination of cause of problems. </li></ul>
Related applications 2 <ul><li>Sports: </li></ul><ul><li>Study top athletes for finding optimal movement patterns. </li></ul><ul><li>Surveillance: </li></ul><ul><li>Crowd surveillance and identification of possible strange behaviour in a shopping mall or airport. </li></ul>
Conclusion <ul><li>I have presented an overview of a system that can capture, visualize and analyze ski jumpers in a ski jumping hill. </li></ul><ul><li>Remains to see how well such a system can perform and if it can help the ski jumpers improve their skills. </li></ul>