This document discusses photo optical character recognition (OCR). It describes the photo OCR problem and pipeline, which includes text detection, character segmentation, and character classification. It also discusses techniques for getting more training data through artificial data synthesis, such as introducing distortions to real data, and analyzing what part of the photo OCR pipeline to focus on improving next through ceiling analysis.

1. Application example:
Photo OCR
Problem description
and pipeline
Machine Learning

2. Andrew Ng
The Photo OCR problem

3. Andrew Ng
Photo OCR pipeline
1. Text detection
2. Character segmentation
3. Character classification
N
A T

4. Image Text detection
Character
segmentation
Character
recognition
Photo OCR pipeline
Engineers

5. Application example:
Photo OCR
Sliding windows
Machine Learning

6. Andrew Ng
Text detection Pedestrian detection
Aspect ratio of rectangles is same
Aspect ratio of rectangles is different

7. Andrew Ng
Positive examples
Supervised learning for pedestrian detection
pixels in 82x36 image patches
Negative examples

8. Andrew Ng
Sliding window detection

9. Andrew Ng
Sliding window detection

10. Andrew Ng
Sliding window detection

11. Andrew Ng
Sliding window detection

12. Andrew Ng
Text detection

13. Andrew Ng
Text detection
Positive examples Negative examples

14. Andrew Ng
Text detection
[David Wu]
We can discard boxes who's height and width are
almost same as text should have more width
as compared to the height. But it might lead to
Skipping some text.

15. Andrew Ng

16. Andrew Ng
1D Sliding window for character segmentation
Positive examples Negative examples

17. Andrew Ng
Photo OCR pipeline
1. Text detection
2. Character segmentation
3. Character classification
N
A T

18. Application example:
Photo OCR
Getting lots of
data: Artificial
data synthesis
Machine Learning

19. Andrew Ng
Character recognition
N
I
A
Q
T
A

20. Andrew Ng
Artificial data synthesis for photo OCR
Real data
Abcdefg
Abcdefg
Abcdefg
Abcdefg
Abcdefg
[Adam Coates and Tao Wang]

21. Andrew Ng
Artificial data synthesis for photo OCR
Real data Synthetic data
[Adam Coates and Tao Wang]

22. Andrew Ng
Synthesizing data by introducing distortions
[Adam Coates and Tao Wang]

23. Andrew Ng
Synthesizing data by introducing distortions: Speech recognition
Original audio:
Audio on bad cellphone connection
Noisy background: Crowd
Noisy background: Machinery
[www.pdsounds.org]

24. Andrew Ng
Synthesizing data by introducing distortions
Distortion introduced should be representation of the type of
noise/distortions in the test set.
Audio:
Background noise,
bad cellphone connection
Usually does not help to add purely random/meaningless noise
to your data.
intensity (brightness) of pixel
random noise
[Adam Coates and Tao Wang]

25. Andrew Ng
Discussion on getting more data
1. Make sure you have a low bias classifier before expending the
effort. (Plot learning curves). E.g. keep increasing the number
of features/number of hidden units in neural network until
you have a low bias classifier.
2. “How much work would it be to get 10x as much data as we
currently have?”
- Artificial data synthesis
- Collect/label it yourself
- “Crowd source” (E.g. Amazon Mechanical Turk)

26. Andrew Ng
Discussion on getting more data
1. Make sure you have a low bias classifier before expending the
effort. (Plot learning curves). E.g. keep increasing the number
of features/number of hidden units in neural network until
you have a low bias classifier.
2. “How much work would it be to get 10x as much data as we
currently have?”
- Artificial data synthesis
- Collect/label it yourself
- “Crowd source” (E.g. Amazon Mechanical Turk)

27. Application example:
Photo OCR
Ceiling analysis: What
part of the pipeline to
work on next
Machine Learning

28. Andrew Ng

29. Andrew Ng
Another ceiling analysis example
Face recognition from images
(Artificial example)
Logistic regression
Face detection
Camera
image
Eyes segmentation
Nose segmentation
Mouth
segmentation
Preprocess
(remove background)
Label

30. Andrew Ng

31. Andrew Ng

32. Andrew Ng