This new feature is a continuation of and improvement on our previous Image Processing release. Now, Object Detection lets you go a step further with your image data and allows you to locate objects and annotate regions in your images. Once your image regions are defined, you can train and evaluate Object Detection models, make predictions with them, and automate end-to-end Machine Learning workflows on a single platform. To make that possible, BigML enables Object Detection by introducing the regions optype. As with any other BigML feature, Object Detection is available from the BigML Dashboard, API, and WhizzML for automation. Object Detection is extremely helpful to tackle a wide range of computer vision use cases such as medical image analysis, quality control in manufacturing, license plate recognition in transportation, people detection in security surveillance, among many others.

1. Democratizing Object Detection

2. BigML Release: Object Detection
BigML, Inc 2
Object Detection
Enter questions into chat box. We will answer some via chat and others
at the end of the session.
QUESTIONS
info@bigml.com @bigmlcom
Atakan Cetinsoy
VP of Predictive Applications.
MODERATOR
Charles Parker Ph.D.
VP of Machine Learning Algorithms.
SPEAKER
https://bigml.com/releases/object-detection
RESOURCES

3. BigML Release: Object Detection
BigML, Inc 3
Object Detection is a fundamental Computer Vision task that
involves localizing and classifying objects-of-interest in an image.
Introducing Object Detection

4. BigML Release: Object Detection
BigML, Inc 4
Object Detection
truck
car
Built on the foundation of BigML Image Processing, Object Detection allows you to localize
and classify objects in a fast and scalable manner.
Image Classi
fi
cation vs. Object Detection
Image Classi
fi
cation

5. BigML Release: Object Detection
BigML, Inc 5
Maturity of DL
techniques
Decreasing
compute costs
Abundance
of image data
High-speed
processing
(frames/sec.)
Next-Gen Tools
Object Detection - Why now?

6. BigML Release: Object Detection
BigML, Inc 6
Object Detection is Everywhere!
Healthcare
Agriculture
Manufacturing
Sports Transportation
Retail

7. BigML Release: Object Detection
BigML, Inc 7
Get plate numbers and states from distinct highway toll car images
Toll Video Cameras
1
Individual Car Images
2
Plate Detection
3
BigML Object Detection
Plate Extraction
4
5 Characters Recognition
6 State Classi
fi
er
BigML Object Detection
Character Images Classi
fi
cation
Image Classi
fi
cation
7
Automatic
Payment
After detecting plate
number and state,
automatic payment
is possible
Object Detection in Transportation

8. BigML Release: Object Detection
BigML, Inc 8
Smart Video Review Framework: Find, Group, Label, and Review
Find interesting events
Flags
Chants
Group events
Experts review clusters
and assign their own labels
1 2
Find anomalous events
Cluster similar events
Smoke bombs / Flares
Label event groups
3
4
Assign labels to
known events
Models learn new
labels from expert’s
feedback
Object Detection for Public Safety

9. BigML Release: Object Detection
BigML, Inc 9
Object Detection without BigML
Image Data
Labeled Image Data
Labeling Tool Modeling Tool
Predictions
Con
fi
gured
Machine w/ GPU
Modeling Hardware
or Service
Trained Model
LABELERS
ML ENGINEER
IT DEPARTMENT

10. BigML Release: Object Detection
BigML, Inc 10
Scaling Object Detection without BigML
Image Data
Labeled Image Data
Trained Model
Con
fi
gure
d
Machine w/ GPU
Labeling Tool Modeling Tool
Modeling
Hardware or
Predictions
Con
fi
gure
d
Machine w/ GPU
Con
fi
gure
d
Machine w/ GPU
Con
fi
gure
d
Machine w/ GPU
Con
fi
gure
d
Machine w/ GPU
Con
fi
gured
Machine w/ GPU
Modeling
Hardware or
Modeling
Hardware or
Modeling
Hardware or
Modeling Hardware
or Service
MLOps / Reporting
Trained Model
Trained Model
LABELERS
ML ENGINEER
IT DEPARTMENT

11. BigML Release: Object Detection
BigML, Inc 11
Scalable Object Detection with BigML
LABELERS
ML ENGINEER
Image Data
Predictions

12. BigML Release: Object Detection
BigML, Inc 12
BigML Composability
ML Work
fl
ow
Object Detection
BigML Resources High-Level Algorithm 1-Click Execution

13. BigML Release: Object Detection
BigML, Inc 13
BigML Resource Synergies

14. BigML Release: Object Detection
BigML, Inc 14
Handling Images the BigML Way
• ALL the tasks for generating insights from
image data ON A SINGLE PLATFORM:
From labeling to inference, evaluation, and
predictions.
• Streamlined image dataset management
with composite sources.
• Automatic handling of infrastructure
concerns.
• Your choice: Code and no code.
Solving image data-driven business problems with remarkable ease of use

15. BigML Release: Object Detection
BigML, Inc 15
What Object Detection Is
• Simultaneously locating and classifying specific sub-areas
(returned as xmin, ymin, xmax, ymax) in a given image.
• Many uses, but especially in:
• Manufacturing defect detection
• Medical image analysis
• Autonomous navigation
• Security (monitoring) applications

16. BigML Release: Object Detection
BigML, Inc 16
What Object Detection Is
• Identification of a specific area in the image that contains
an object and identification of the class of the object.
• The “score” given to the box is a product of the probability
of both of these factors. Note that even if the model
thinks there’s an object in a particular region, it can still fail
if the model has trouble deducing the class of the object.
• While we typically visualize the prediction as a box, this is
just for convenience.
• The parameters xmin, ymin, xmax, ymax could also specify an ellipse
• The boundaries of the object are unlikely to be pixel-accurate, even for
objects that are axis-aligned rectangles
• Better to think of the returned values is “approximate location” and
“approximate size”

17. BigML Release: Object Detection
BigML, Inc 17
What Object Detection Isn’t
• Object detection tells you the class for a specific
window of the image. If you want to classify the whole
image, you want “image classification” (available on
BigML).
• Object detection does not give an idea of exactly
which pixels in the image are in the detected object.
This is done using one of the various flavors of “image
segmentation” algorithms.
• Object detection on its own cannot find specific parts
of the object being detected. This is “keypoint
detection”, or for articulated objects (like people),
“pose estimation”. Though it is possible to label and
detect parts separately with object detection.

18. BigML Release: Object Detection
BigML, Inc 18
New Optype: Regions
Type Description Example
Numeric A continuous numeric value 0,75
Categorical One of a small set of possible values Cat
Datetime A String specifying a date, time, or both 4-22-2022 11:24am
Text An unstructured string of free text “Hello, cats!”
Image A
fi
le containing an image
Path The path of the image
fi
le in the zip
fi
le containing it cat/image1.jpg
Regions A list of
fi
ve-tuples, [label, xmin, ymin, xmax, ymax]
[[“cat”, 0.5, 0.1, 0.6, 0.9],
[“dog”, 0.2, 0.2, 0.8, 0.8]]

19. BigML, Inc BigML Release: Object Detection
Regions Labeling
19

20. BigML Release: Object Detection
BigML, Inc 20
The More Things Change . . .
Training Dataset
Test Dataset
Zip
fi
le with Images
Train
Evaluate

21. BigML, Inc BigML Release: Object Detection
Training An Object Detector
21

22. BigML Release: Object Detection
BigML, Inc 22
Under The Hood
• Break the image into a rough grid of cells.
• Centered at each cell, consider a number of
“bounding box proposals”.
• For each box in each cell, try to predict the
following things simultaneously:
• Whether or not there’s a object inside.
• The class of the object.
• Adjustments needed for the height/width/x-center/y-
center to make the box fit properly around the object.
• The shapes of the proposals are learned
directly from the data.

23. BigML Release: Object Detection
BigML, Inc 23
Some Caveats
• The usual caveats about feeding images to
machine learning models still apply.
• Example: Blurring.
• Example: Negative images.
• The bounding box proposals are learned
from the sizes given in your data.
• The boxes consider the entire image when
making predictions, not just the content of
the box proposal itself.

24. BigML Release: Object Detection
BigML, Inc 24
What if we Need More Power?
• For the public deployment, we use a small object detection architecture which performs reasonably well on most
problems.
• There are, however, other architectures that can give a significant bump in accuracy.
• They need significantly more data to train properly.
• They are slower for both training and testing.
• We don’t make these available on our public deployment, but contact us about these options in private
deployments.

25. BigML, Inc BigML Release: Object Detection
Making Predictions with Your Detector
25

26. BigML Release: Object Detection
BigML, Inc 26
Evaluations - A Little Different
• In classification problems, we can construct a “confusion matrix”
with true positives, true negatives, false positives, and false
negatives.
• For object detection, we don’t typically count “true negatives” as we
don’t give the model credit for predicting “nothing” for the vast
majority of locations, which would dominate metrics that consider
them.
• So we must rely on metrics that don’t involve true negatives, which
are based on precision, recall, and their combinations (e.g., average
precision; the precision averaged over several levels of recall).
• We can still change the score threshold for the model, and see how
that impacts the count of true positives, false positives, and false
negatives.

27. BigML Release: Object Detection
BigML, Inc 27
What is a “Correct” Prediction?
• Clearly, if the boxes align exactly, the prediction is correct, but they usually don’t.
• For each possible match, we compute a quantity called “Intersection over Union” or IOU.
• If the IOU is greater than some threshold, we consider the prediction correct.
• We do this for several thresholds in the evaluation.

28. BigML, Inc BigML Release: Object Detection
Object Detection Evaluation
28

29. BigML Release: Object Detection
BigML, Inc
Object Detection Model
Classi
fi
er
29
A Simple Work
fl
ow
Anomaly Detector
Object 1 Object 2
Object Classes Anomaly Scores
• What if you have a basic list of things you would
like to detect, but you know it’s not exhaustive?
• This might cause low scores for rare / never
seen objects.
• Solution: Train an object detector to detect a
generic “thing”.
• Crop out the detected objects, then passed the
crops to both an anomaly detector and a
classifier.
• If it’s a known object, you’ll get a high-
confidence class. If it’s a strange object, you will
get a high anomaly score.

30. BigML Release: Object Detection
BigML, Inc 30
Object Detection the BigML Way
• Everything is immutable, traceable, and composable
• All of your models are always available and never change
• You can always see exactly the training data that generated
the model
• Everything is available via the API and downloadable
• Image dataset creation and bounding box labeling can be
done via the API
• Models are live via API endpoint as soon as they are created
• Models can be downloaded and run locally