This document proposes an active deep learning approach to CAPTCHA recognition using a small initial training set. A convolutional neural network is trained on CAPTCHAs containing 6 digits. During testing, samples that are classified correctly but with high uncertainty are added back to the training set to improve the model in subsequent rounds of learning, without needing human labels. The method is evaluated on CAPTCHAs generated using different configurations, and performance is shown to improve significantly with each round of active learning by selecting additional uncertain samples for retraining.

1. CAPTCHA Recognition
with Active Deep
Learning
Group 26
N96041119 陳柏皓
N96041101 黃偉鑫

2. Directory
• Introduction
• A Deep CNN for CAPTCHA Recognition
• Active Learning
• Experimental Evaluation
• Conclusion

3. Introduction
• A CAPTCHA (Completely Automated PublicTuring test
to tell Computers and Humans Apart) is an automated
test to identify whether the user is a human or not.

4. Introduction
• Recently, researchers have started investigating
automated methods to solve CAPTCHAs.
• Many of these existing solutions first perform
character segmentation and then recognition.

5. Introduction
• However, they can not solve newer, more
challenging CAPTCHAs, where the letters are
skewed so that they can not be separated by
vertical lines.

6. Introduction
• One successful approach proposed recently by
Goodfellow et al. uses a deep Convolutional
Neural Network (CNN).
• However, a major requirement for training a deep
CNN is a very large training data set , and for
CAPTCHA recognition, there is usually only a
small annotated training data set available.

7. Introduction
• To address these problems, we propose in this
paper an approach that is based on Active
Learning.
• The idea here is to start learning with a
comparably small training set and to add new
training samples in every subsequent learning
round.

8. Introduction
• We do not need a human supervisor to acquire
the ground truth labels for training.
• If the CAPTCHA is not solved we don't have that
information, but we will show how learning can
be done from the correctly predicted samples
only.

9. A Deep CNN for CAPTCHA
Recognition
• We focus on CAPTCHAs with 6 digits. Each digit is
represented by 62 neurons in the output layer.
• We define a bijection Θ(x) that maps a character x ∈ {‘0’,...‘9’,
‘A’,..., ‘Z’, ‘a’,..., ‘z’} to an integer l ∈ {0, ...61}:

10. A Deep CNN for CAPTCHA
Recognition
• For a digit xi the neuron index n is computed as n
= i · 62 + Θ(xi), where i ∈ {0, ..., 5} is the index of
the digit.
• The output layer has 6 · 62 = 372 neurons.

11. A Deep CNN for CAPTCHA
Recognition
• Example output of the network for the CAPTCHA
“qnnivm

12. A Deep CNN for CAPTCHA
Recognition
• Network Design
• Train the network with the SGD algorithm
• learning rate changes by the rule

13. Active Learning
• Obtaining the uncertainty
• Sort the correctly classified test samples in each learning round by predicti
uncertainty.

14. Experimental Evaluation
• Dataset Generation
• Use the Cool PHP CAPTCHA framework to generate CAPTCHAs
• Black and white images
• Disabled shadows and the line through the text
• Not use dictionary words

15. Experimental Evaluation
• Pick new training samples from the correctly classified ones.
• Take all of them, or only pick 5 ·103 samples based on their
uncertainty: Either with the highest uncertainty, the lowest
uncertainty, or randomly.

16. Experimental Evaluation
• Increasing the number of samples in the training set requires
more storage and longer training time.
• Suggest to use only the selected samples at each iteration to
re-train the network.

17. Conclusion
• Uses initially a very small set of images to train a deep
CNN.
• Significantly improved with the correctly classified but
uncertain test samples.
• Learning with misclassified samples should yield better
results, however not possible in practice.