Fish4Knowledge: large scale coral reef fish monitoring using undersea computer vision methods.
•
2 likes•3,786 views
Talk given at the 9th Indo_Pacific Fish Conference, Okinawa, June 27th 2013 fish4knowledge.eu pdf available from http://www.cwi.nl/~lynda/talks/2013/Fish4Knowledge130627.pdf
Recommended
More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
Fish4Knowledge: large scale coral reef fish monitoring using undersea computer vision methods.
- 1. Fish4Knowledge Large scale coral reef fish monitoring using undersea computer vision methods Lynda Hardman CWI (Centrum Wiskunde&Informatica), Amsterdam, NL Centre for Mathematics & Computer Science
- 2. THE FISH4KNOWLEDGE PROJECT 1. Collecting Video Data • Collecting videos • Detecting fish • Recognizing fish species • Recognizing fish behaviors • Evaluating video analysis accuracy 2. Exploring Video Data • Exploring fish counts • Investigating species composition • Checking potential biases 2fish4knowledge.eu
- 3. THE FISH4KNOWLEDGE PROJECT 1. Collecting Video Data • Collecting videos • Detecting fish • Recognizing fish species • Recognizing fish behaviors • Evaluating video analysis accuracy 2. Exploring Video Data • Exploring fish counts • Investigating species composition • Checking potential biases 3fish4knowledge.eu
- 4. COLLECTING VIDEO DATA Multiple video streams: 9 Cameras, >20,000 hours of videos (Dec. 2012) Terabyte-scale data platform High-performance servers for data access 4fish4knowledge.eu
- 5. DETECTING FISH Detection of fish in each frame Description of fish contour, color, texture... (>30 features) Tracking of single fish over several frames Handling partial and total occlusions Over 400 million fish detections 5fish4knowledge.eu
- 6. RECOGNIZING FISH SPECIES Detection of body parts (head, tail…) Construction of a fish model for each species 15 species detected (96% of total number of fish)
- 7. RECOGNIZING FISH BEHAVIORS Detection of behaviors by analyzing trajectories Relation to background objects (feeding, hiding…) Relation to other fish (pairing, grouping, solitary…) Fish Trajectories Description of scene’s background
- 8. THE FISH4KNOWLEDGE PROJECT 1. Collecting Video Data • Collecting videos • Detecting fish • Recognizing fish species • Recognizing fish behaviors • Evaluating video analysis accuracy 2. Exploring Video Data • Exploring fish counts • Investigating species composition • Checking potential biases 8fish4knowledge.eu
- 9. EVALUATING VIDEO ANALYSIS ACCURACY 9fish4knowledge.eu Comparison with manual detections by experts Fish detection game to encourage crowd sourcing
- 10. EVALUATING VIDEO ANALYSIS ACCURACY Certainty Scores indicate the quality of each detection These can be used to filter out low-quality fish Score > 0.8 Fish Counts for 5 Certainty Score Thresholds Score > 0.2 Score > 0.4 Score > 0.5 Score > 0.6 Certainty Score: 0.4 Certainty Score: 0.9Certainty Score: 0.75 Certainty Score: 0.6
- 11. THE FISH4KNOWLEDGE PROJECT 1. Collecting Video Data • Collecting videos • Detecting fish • Recognizing fish species • Recognizing fish behaviors • Evaluating video analysis accuracy 2. Exploring Video Data • Exploring fish counts • Investigating species composition • Checking potential biases 11fish4knowledge.eu
- 14. THE FISH4KNOWLEDGE PROJECT 1. Collecting Video Data • Collecting videos • Detecting fish • Recognizing fish species • Recognizing fish behaviors • Evaluating video analysis accuracy 2. Exploring Video Data • Exploring fish counts • Investigating species composition • Checking potential biases 14fish4knowledge.eu
- 15. SPECIES COMPOSITION DEPENDS ON LOCATION
- 16. Amphiprionclarkii, Camera 38 Plectroglyphidodondickii, Camera 39
- 17. SPECIES COMPOSITION DEPENDS ON LOCATION
- 18. THE FISH4KNOWLEDGE PROJECT 1. Collecting Video Data • Collecting videos • Detecting fish • Recognizing fish species • Recognizing fish behaviors • Evaluating video analysis accuracy 2. Exploring Video Data • Exploring fish counts • Investigating species composition • Checking potential biases 18fish4knowledge.eu
- 19. SPECIES COMPOSITION DEPENDS ON LOCATION Plectroglyphidodondickii, Camera 38 Amphiprionclarkii, Camera 38 Plectroglyphidodondickii, Camera 39 Amphiprionclarkii, Camera 39
- 20. THE FISH4KNOWLEDGE PROJECT 20fish4knowledge.eu http://fish4knowledge.eu/people.htm University of Edinburgh (United Kingdom) Universitàdi Catania (Italy) Centrum Wiskunde&Informatica (Netherlands) National Applied Research Laboratories (Taiwan)
Editor's Notes
- Good afternoon! As you may have noticed, I am not Bas Boom, but Lynda Hardman, and I’m not a marine biologist, I’m a computer scientist.I am at this conference, because I am a member of a European project that has spent the last 3 years building a system for automatically counting fish in fish videos.
- Here you can see the habitats for camera 38, on the left, and camera 39, on the right (also based near Kenting).