This document summarizes an effort to classify immigration documents through machine learning in order to streamline the visa application process. The goals were to accurately identify documents, provide real-time feedback on missing documents, and reduce intensive document oversight. Data was extracted from over 50,000 immigration documents using OCR and various preprocessing techniques. Classification algorithms like SVC and Random Forest were tested, with SVC performing best at handling imbalanced classes. The initial model achieved good performance on some document types but poorer on others like passports. Future work outlined improving the model through techniques like identifying outliers, document similarity, and incorporating more document types and samples.

1. IMMIGRATION SIMPLIFIED
Classifying Immigration Documents for Bridge
US
By Baolin Liu

2. About Bridge US:
• Immigration service provider that
streamlines the US visa application
process for employers.
• Works with companies like Getaround,
Allegiant, and American Family Insurance.
• Over 50k immigration related documents
managed.

3. Motivation
• Typical process includes over 20 files
• Identifying documents takes time
• Files are misplaced
• No immediate feedback

4. Goals:
• Accurately identify documents
• Guide users to provide what’s missing
• Real-time feedback
• less intensive document oversight

5. The Data:

6. Data Collect and “Munging”
• Text Extraction with Textract (OCR)
• Regex commands
• Stemmed the words
• PySpark
Before: After:

7. Classification Algorithm
• 2 pathways:
-Selectable text
-Image text
• Assigned probability
http://cheaper-than-tuition.com/shop/fake-diploma/

8. Modeling: SVC
• Linear Kernel
• Handled imbalanced classes
• Dealt with low samples.
http://scikit-learn.org/stable/auto_examples/svm/plot_separating_hyperplane_unbalanced.html

9. Modeling: Random Forest
http://blog.yhat.com/posts/random-forests-in-python.html
• Repeated sampling, or “bootstrapping”
• “Wisdom of the Crowd”

10. Confusion Matrix
SVC: 48 misclassified
Random Forest: 60 misclassified

11. ROC Plots

12. Cons for both models:
• SVC: False high confidence
• Random Forest: Bad at low samples
Winner: SVC-imbalanced classes more important

13. Moving code to
production:
• Classify only the first page
• Classify the document as an Image
• Took out spell corrector
• Model Persistence, pickling
• Py files to launch from command line
• AWS

14. Production Testing
Data:
• 1664 PDF files from 87 case files
Initial Findings:
• more classes: job offer, wage surveys, tax returns, org charts, W-2
Did Well:
• government forms, pay stubs and resumes
Did Moderate:
• Transcripts, Diplomas, Birth Certificates, Marriage Certificates
Did Poor:
• Passports

15. Preliminary Data Engineering
Architecture
Reporting Bucket
Case files, monitored for changes Temp Folder, Classify Files
https://www.eyemovic.com/blog_it/6749.php
https://www.conceptdraw.com
Boto, the S3 interface
Sends details of this bucket
Keep “Misclassified” Documents

16. Future Work:
• Retrain the model
• Apply statistics
• Add to Data Engineering Architecture
• Apply an Image classifier

17. Retrain Model:
• Use boxplot to identify outliers
of low probability that don’t
belong to that class
• only below *minimum*
• Convert all documents into
matrix form
• Use Dot product to find
document similarity (0,1)
• document similarity = hashvector
* hashvector.T
http://stackoverflow.com/questions/8897593/similarity-between-two-text-documents

18. Retrain Model(cont):
• Use heat map or correlation matrix to identify new
groups

19. Retrain Model(cont)
• Look at the file contents, give the documents a label, repeat to validate

20. Retrain Model(cont):
• Use statistics to identify sample size for class
• Bigger standard deviation, more samples(randomized) to
needed capture the variance
myweb.facstaff.wwu.edu

21. Final Thoughts
• No one else has this technology in
immigration
• Used all my skillsets
• Learned a whole new set of skills