This document summarizes research on math symbol recognition from handwritten text. It discusses preprocessing techniques like normalization and smoothing. Features like curvature, direction, and aspect ratio are extracted. A random forest classifier is trained on these features, achieving an accuracy of 78.3%. Minimum spanning tree segmentation is used to group strokes into symbols. This reduces the search space compared to all possible combinations. Finally, a line of sight parsing graph models connectivity between symbols based on whether they can see each other, further reducing the search space during recognition. Code for the full system is available on GitHub.

1. Math Symbol Recognition
By
Suhas.Pillai

2. Overview
● Preprocessing
● Features
● Random Forest
● Minimum Spanning Tree Segmentation.
● Parsing
● Code

3. Stroke Preprocessing
Math Expression Pre
Processed
Strokes
Normalization Smoothing

4. Features
● Curvature
● Start direction
● Last direction
● Aspect Ratio
● Delta_x,Delta_y [1]
● Histogram of points
● Writing Angle and Delta Writing Angle
● Crossing features [2]
[1] Han Su, “On Line Handwriting Recognition Using Hidden Markov Models”
[2] Davila, Kenny, Stephanie Ludi, and Richard Zanibbi. "Using off-line features and synthetic data for on-line handwritten math symbol
recognition."Frontiers in Handwriting Recognition (ICFHR), 2014 14th International Conference on. IEEE, 2014.

5. PCA Features
● Stroke points interpolated to image of 64 * 64 pixels
Original image Interpolated Binary Dilation Gaussian Smoothing
image

6. Classification Results
Random Forest Classifier.
No of Trees 60 100
Accuracy (Test) 77.52 % 78.30%

7. Minimum Spanning Tree Segmentation
● Calculate Centroid of every stroke [1]
● Calculate Euclidean distance between strokes
● Construct Graph between all the strokes, where weight of edges is the
euclidean distance
● Construct a minimum spanning tree using Kruskal’s algorithm.
● Use Dynamic Programming to find different possible ways to combine strokes
to symbol in an efficient way.
● Use classifier to get probability estimate, i.e the set of strokes belong to a
class with highest probability.
● Sum the probability of different symbols in the partition set, and choose the
partition with highest cost value.
[1] Nicholas.E Matsakis, “Recognition of Handwritten Mathematical Expressions”

8. Minimum Spanning Tree Segmentation
Advantages of using MST.
● No need to consider combination of strokes, which do not have an edge
between them. Thus, reduces the search space and is more computationally
efficient.
● Following figure is the MST for an expression, green lines are connection
between stokes

9. Segmentation Results
Training Testing

10. Parsing
● Line of Sight Graph for parsing [1], which is a method from computational
geometry.
● Idea is that the object (i.e central object or an eye ) can only be connected to
other object, if it can see other object.
● If an object’s view is blocked, then the central object or eye , cannot have an
edge to that object.
● Compared to fully connected graphs, we only see potential
candidates,thereby reducing search space.
[1] Lei Hu Phd Dissertation “Features and Algorithms for Visual Parsing of Handwritten Mathematical
Expression”

11. Code
● Code available on github
https://github.com/suhaspillai/Handwritten-Recognition-of-Math-Symbols.git

12. Thank You
Questions?