This document discusses how AI and data science can help protect biodiversity by detecting, identifying, and counting animals using computer vision models. It provides examples of projects that use deep learning to identify salmon from underwater videos, classify species from photos shared on iNaturalist, estimate penguin counts from images, and detect elephants spotted via aerial surveys to help conservation efforts. The document emphasizes the importance of having adequate training data and combining automated methods with human inputs and validation to accurately protect threatened species.

1. AI for Social Good: Saving the planet with
Data Science
Ganes Kesari, Gramener
@kesaritweets

2. 2Gramener
Introduction
Co-founder & Head of
Analytics
100+ Clients
Insights as Stories
Help apply & adopt
Analytics
Our data science platform,
Gramex is now open-sourced!
“Simplify Data Science
for all”

3. 3Gramener
What ails our Biodiversity?
Now Extinct!
Movie copyright: Blue Sky Studios
Image by AngieToh from Pixabay
Now Extinct…

4. 4Gramener
Can we save our Biodiversity?
Source: Image1 ; Image2; Image3; Image4

5. 5Gramener
Spotting, Identifying and Counting Animals to Save them
Live Case Studies from Gramener’s partnership with Microsoft AI for
Earth
https://www.microsoft.com/en-us/ai/ai-for-earth
Climate Agriculture Biodiversity Water

6. 6Gramener
The path to protecting animal species
Detect

7. 7Gramener
Mugshots in the Forest
Source: Giphy (https://media.giphy.com/media/1k10MvscEjChyyHtmQ/giphy.mp4)

8. 8Gramener
Sample Camera trap pictures

9. 9Gramener
Input Output
Identify features to
teach model
Traditional Machine Learning
Person
Name
9
Deep Learning
Input Output
Model automatically identifies
features to learn
Person
Name
https://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr14.pdf
Deep Learning to the rescue

10. 10Gramener
Not very different from how we learn
Training ..Versatile Detection!

11. 11Gramener
Detecting animals: Approach
https://towardsdatascience.com/build-your-first-deep-learning-classifier-using-tensorflow-dog-breed-example-964ed0689430

12. 12Gramener
Detecting animals: Approach

13. 13Gramener
The path to protecting animal species
Detect Identify

14. 14Gramener
Where have the Salmon Gone?
Source: Giphy (https://media.giphy.com/media/QM5GJO6J8lDfa/giphy.mp4)

15. 15Gramener
Underwater Salmon snapshots

16. 16Gramener
Approach

17. 17Gramener
The Model in Action
Video

18. 18Gramener
The path to protecting animal species
Detect Identify
…from videos
…from pictures

19. 19Gramener
Can we Classify All species on Earth?
Source: Giphy (https://media.giphy.com/media/CFk1wEH7Cke0E/giphy.mp4)

20. 20Gramener
iNaturalist: Fostering Citizen Science
https://play.google.com/store/apps/details?id=org.inaturalist.android&hl=en

21. 21Gramener
Building the Model
Inception v4

22. 22Gramener
The path to protecting animal species
Detect Identify Count

23. 23Gramener
Let’s Save the Penguins
Source: Giphy (https://giphy.com/gifs/push-bwLowbhUWm2lO)

24. 24Gramener
Tracking the Penguins
https://www.zooniverse.org/projects/penguintom79/penguin-watch/about/research
~100 cameras in 16 locations, and hourly
images over years
Crowd-sourced annotations to identify
the penguins

25. 25Gramener
Crowd-sourced Annotations
https://www.zooniverse.org/projects/penguintom79/penguin-watch/classify

26. 26Gramener
Counting Crowds – Bounding boxes vs Density estimates
Occlusion
Density Difference
Perspective Distortion
Camera angle
Preserve spatial information
Localize count
Handle scale variations
No longer looking for a head!

27. 27Gramener
Approach
https://arxiv.org/pdf/1707.09605.pdf
Input an image Density Map Estimate the countSplit into 9 patches

28. 28Gramener
Approach
• High-level prior to classify image into buckets
• Density estimation to create the density map
• NC6 v3 virtual machine with V100 GPU card
• Trained for 200 epochs, MAE for the model : ~10.5
https://arxiv.org/pdf/1707.09605.pdf Work done by Gramener in partnership with Microsoft AI for Earth

29. 29Gramener
Results
https://arxiv.org/pdf/1707.09605.pdf

30. 30Gramener
The path to protecting animal species
Detect Identify Count Protect

31. 31Gramener
Saving elephants in the Wild
Source: Giphy (https://media.giphy.com/media/1AHZzdXVTYDtJVTO5a/giphy.mp4)

32. 32Gramener
Can you spot the elephants?
https://www.savetheelephants.org/project/tsavo-aerial-defence/

33. 33Gramener
Approach
Region Proposal Network Region of Interest Pooling Regional CNN
FASTER RCNN

34. 34Gramener
Model Results
https://www.savetheelephants.org/project/tsavo-aerial-defence/

35. 35Gramener
Join the Revolution!
AnnotateShare your pictures Spread the word

36. 36Gramener
6 Deep Learning Takeaways: When the Rubber hits the Road
Scout for the
right Data
Watch out for
labelling
Educate on
Accuracy
Augment with
human inputs
Build into a
workflow
Budget refresh
& upkeep

37. Session deck with model details, Github repos & bonus case study at
@kesaritweetsgramener.com @kesari
gkesari.com/strata

38. Rate today’s session
Session page on conference website O’Reilly Events App