The document discusses the development of AI models for recognizing handwritten math expressions, detailing the process from problem input to feedback for students. It emphasizes the creation of high-quality synthetic datasets for training, allowing for precise control over image characteristics and efficient product development cycles. Additionally, it mentions the open-sourcing of a dataset on Kaggle, comprising 100,000 labeled images to support research and further advancements in this area.

1. Let the Product Lead:
R&D Paradigms for
Recognition Models of
Handwritten Math
Expressions
Quinn Lathrop
quinn.lathrop@gmail.com

2. AI Products and Solutions
• Global organization
• Research, Development, Engineering, Product, Design
• We own the product/feature from ideation to delivery
• Achieve broad innovation goals by developing B2C products first
Contributors to this work:
Zac Hancock, Jiamin He, Michael Chifala, Sungjin Nam, Bill Vander Lugt, Mounika Kakarla, Teddy Ampian, Luke Tuguluke, JB
DeVries, Leslie Satterfield, Claudia Cassidy, Douglas Cobb, Brian LoPiccolo, Joey Ashcroft, Ashley Fallon, Holly Smith, Tim
Stewart, JD Corbin, Tejawini Nallagatla, Jakob Vendegna, Wes Galbraith, Randall Barnhart, Eric Kattwinkel, Johann Larusson,
Jason Fournier, Michal Okulski, Piotr Kabacinski, Kacper Lodzikowski

3. Aida Calculus
Check This Problem Flow:
• Student inputs their problem
by taking a picture of their
handwritten work
• Student receives step-by-step
feedback
• Personalized hints and
tutoring
Video

5. Handwriting Recognition of
Math Expressions
y = ( 2 x – 3 ) ( x^2 - 5 )^3

6. Typical Data -> Model Flow
Waterfall:
• Collect dataset
• Iterate on model
At a certain point of time
the dataset is fixed
Real data is not perfect

7. Product Driven R&D
• Iteratively build a synthetic generation capability
towards requirements
• Control over the distribution of math
expressions, characters, location of characters,
specific visual qualities of the math, image noise,
and image augmentations
• Control every pixel of image
• Can create millions of perfectly labelled images in
hours

8. Math Expression: y=(2x-3)(x^2-5)^3
Background
Extra Marks
Bleed Through
Our synthetic data builds these features from the ground up:
• Math Expression
• Font and writing utensil
• Character-specific distortions/augmentations
• Backgrounds
• Other visual noise
• Simulated photo – angle, distance, quality, shadows

11. Benefits: Product
Development Cycles
When other product features depend on a developing AI capability, it is
important to integrate and iterate early and often
• Alpha
• Beta
• MVP

12. Benefits: Exact Control
and Visibility of the
Population Distributions

13. Benefits: User Behavior
Drives Backlog
Example: computer screens
• Can generate data and quickly ask the question: Does our model architecture
support this expansion in scope?

14. Benefits: Modeling
Because we have 100% correct pixel-level tagging, a
range of Object Detection models are available.
Tagging bounding boxes and masks comes at no
additional data collection cost

15. Open Sourcing Dataset on Kaggle
100,000 images with ground truth Latex, Bounding Boxes, and Masks
https://www.kaggle.com/aidapearson/ocr-
data

16. Thank you!
Open Sourced Kaggle Dataset
https://www.kaggle.com/aidapearson/ocr-data
Quinn Lathrop
quinn.lathrop@gmail.com