A novel transform for fast detection of naturally curved items in digital images is described in this article. This general purpose image transform is defined to suit platforms with limited memory and processing footprints by utilizing only additions and simple shift and bitwise operations. We present this unique algorithmic approach in application to real world problems of iris detection and handwriting recognition systems as typical applications in such devices. The new approach is tested on several data sets and the experiments show promising and superior performance compared to existing techniques. https://www.researchgate.net/publication/290691433_Systems_and_methods_for_obtaining_structural_information_from_a_digital_image

1. Magdi Mohamed, 2017:06:02 1 / 81
Gradient Direction Transform
Magdi Mohamed
QCT, Multimedia and Standards R&D, Computer Vision Systems, 2017:06:02

2. Magdi Mohamed, 2017:06:02 2 / 81
Agenda
 Motivations
 Existing Solutions
 Gradient Direction Transform - Definition & Characteristics
 Iris Detection Application
 Handwriting Recognition Application
 Future Thoughts
 Summary & Conclusions

3. Magdi Mohamed, 2017:06:02 3 / 81
Optical Eye Tracking System Diagram – Motivation for Idea
Hough
Transform
Processing
And / Or
Other Methods
(Timm & Barth)
Frame
Normalization
Size
Light
Pose
Image
Preprocessing
Binarization
Noise Filtering
Smoothing
Image
Segmentation
Morphological
Operations,
Clustering,
&
Relaxation
Techniques
Feature
Extraction
Edge
Operations:
Detection
Linking
Thinning
Analysis &
Classification
Search
&
Image
Interpretation
Tasks
Post-
processing
Spatial
Relationships,
Sanity Checks
&
Accept / Reject
Iris detection is essential for several applications including gaze tracking

4. Magdi Mohamed, 2017:06:02 4 / 81
Overview of Techniques
Practical solutions are usually engineering combination of multiple techniques

5. Magdi Mohamed, 2017:06:02 5 / 81
Hough Transform for Iris Detection – Existing Solution
β
α
(α1, β1)
Hough Space for Circles
(α− x1 ) 2 + ( β − y1 ) 2 = r2
x
y Image Space
( x - α1 ) 2 + ( y – β1 ) 2 = r 2
(x1,y1)
Iris Detection Requirements
Deals with variations in light and reflections
Deals with partially occluded, missing, and noisy features
No special markers or makeup required
Features realtime processing
Quantifies action codes
Hough method can detect multiple curves and is resilient to noisy inputs

6. Magdi Mohamed, 2017:06:02 6 / 81
Other Methods: Timm & Barth – Existing Solution
{timm,barth}@inb.uni-luebeck.de
Proposing a faster and novel extension of this method

7. Magdi Mohamed, 2017:06:02 7 / 81
Gradient Direction Transform: New Idea
after initializing the transform matrix to zeros, for each gradient vector in the
input image region of interest, increment / decrement the value of the cells in
the transform matrix inline with the gradient vector accordingly

8. Magdi Mohamed, 2017:06:02 8 / 81
Gradient Direction Transform: Solution for Iris Detection
Pre-Processing
Blur-Convolution
Vertical-Edge Horizontal-Edge
Gradient Direction Transform
Weighted Gradient Direction
Thinning
Find Eye
Otsu Dynamic Threshold
Edge-Convolution
i-image
n-image
b-image e-image
o-imagev-image h-image
g-image
w-image h-space
iris-location
Hough Transform for Elliptical Shapes
t-image

9. Magdi Mohamed, 2017:06:02 9 / 81
Bresenham’s Algorithm – Implementation Details

10. Magdi Mohamed, 2017:06:02 10 / 81
Bresenham’s Algorithm – Implementation Details

11. Magdi Mohamed, 2017:06:02 11 / 81
Bresenham’s Algorithm – Implementation Details

12. Magdi Mohamed, 2017:06:02 12 / 81
Gradient Direction Transform: Implementation Details

13. Magdi Mohamed, 2017:06:02 13 / 81
References
• Hough:
Method and means for recognizing complex patterns
https://docs.google.com/viewer?url=patentimages.storage.googleapis.com/pdfs/US3069654.pdf
• Ballard:
Generalizing Hough transform to recognize arbitrarily shapes
http://www.cs.utexas.edu/~dana/HoughT.pdf
• Mohamed & Nasir:
Method and system for parallel processing of Hough transform computations
https://docs.google.com/viewer?url=patentimages.storage.googleapis.com/pdfs/US7406212.pdf
• Timm & Barth:
Accurate eye center localization by means of gradients
http://cjee.lakeheadu.ca/public/journals/22/TiBa11b.pdf
• Bresenham:
A rasterizing algorithm for drawing curves
http://members.chello.at/easyfilter/bresenham.pdf

14. Magdi Mohamed, 2017:06:02 14 / 81
Major Characteristics – Complexities & Capabilities
 Face detection takes ~ 30 ms (24 x 24)
 Eye corner detection takes ~ 12 ms (256 x 256)
 Iris detection time complexity for region of interest (C x R)
 T&B = K1 (C * R)
2
(K1 ~ cost for normalized floating point vector dot product)
 GDT = K2 (C * R) * C (K2 ~ cost for integer addition and bit-wise operations, C>R)
 T&B ignores sign of vector in squaring dot products to avoid square root
computations
 GDT is capable of efficient
 Consideration of sign of vectors at no extra cost (inward/outward directions)
 Extension of gradient normal-vector (by choice) to other directions such as gradient
tangent-vector that may suit describing other (binary/gray/color) image analysis tasks

15. Magdi Mohamed, 2017:06:02 15 / 81
Timm & Barth Versus Gradient Direction Transform
Method
Size (C x R)
Conventional
T&B
T1 (Seconds)
Novel
GDT
T2 (Seconds)
Speedup
Ratio
T1 / T2
040 x 030 0001.291 0000.053 024.538
080 x 060 0022.630 0000.345 065.672
160 x 120 0394.155 0002.689 146.580
240 x 180 2063.608 0013.426 153.705
320 x 240 6603.897 0036.665 180.116
Measured Speedup Ratio = T1 / T2

16. Magdi Mohamed, 2017:06:02 16 / 81
Sample Image 320x240

17. Magdi Mohamed, 2017:06:02 17 / 81
Input Image 320x240

18. Magdi Mohamed, 2017:06:02 18 / 81
Gradient Image (Gx)

19. Magdi Mohamed, 2017:06:02 19 / 81
Gradient Image (Gy)

20. Magdi Mohamed, 2017:06:02 20 / 81
Gradient Direction Transform 320x240 (In Grey)

21. Magdi Mohamed, 2017:06:02 21 / 81
Gradient Direction Transform 320x240

22. Magdi Mohamed, 2017:06:02 22 / 81
Gradient Direction Transform 320x240

23. Magdi Mohamed, 2017:06:02 23 / 81
Gradient Direction Transform 320x240

24. Magdi Mohamed, 2017:06:02 24 / 81
Timm & Barth Transform 320x240

25. Magdi Mohamed, 2017:06:02 25 / 81
Timm & Barth Transform 320x240

26. Magdi Mohamed, 2017:06:02 26 / 81
Timm & Barth Transform 320x240

27. Magdi Mohamed, 2017:06:02 27 / 81
Iris Detection – Major Challenges
 Eye boarder localization
 Occlusion
 Eyelashes
 Glasses
 Makeup
 Shadows
 Resolution
 Non-circularity
 Wrinkles
 Redness
 Water
 Abrupt motion
 Blinks
 Differences between left and right eyes
 Pose, color variations, and light reflections
 Age sensitive
 Manual annotation and evaluation metrics
 Strong/Weak eye concerns for gaze

28. Magdi Mohamed, 2017:06:02 28 / 81
Iris Detection - Notes On Used Data
 The data is not constructed particularly for evaluating iris detection algorithms
 It contains illegitimate images, and less accurate annotation for iris centers
 It contains some faces with sun glasses
 It contains many cases with corneal reflections from strong light sources
 It significantly represents almost frontal faces
 It contains mostly colored images
 It represents different categories (age groups, makeup, styles, …)
 Since it is the only annotated data available to us, we used it to construct our
facial landmark detector, and to evaluate the iris detector
 Ideally, a balanced data set with “casual” iris positions, not mostly looking at the
camera, with more accurate manual annotation of iris centers is needed

29. Magdi Mohamed, 2017:06:02 29 / 81
Iris Detection – Experiment # 1 Setup
 Using 15800 (mostly frontal view) face images with 40 landmarks per face
manually annotated as Ground Truth,
 Using Omron Face Detector,
 Using Omron Eye Corner Locator,
 Conducting a Blind Test to Compare GDT Versus T&B Solution,
 Compute normalized Cumulative Error Distribution (CED) for each estimator

30. Magdi Mohamed, 2017:06:02 30 / 81
Iris Detection – Experiment # 1 Performance Summary

31. Magdi Mohamed, 2017:06:02 31 / 81
Iris Detection – Experiment # 2 Setup
 Using 15800 (mostly frontal view) face images with 40 landmarks per face
manually annotated as Ground Truth,
 Using Omron Face Detector,
 Using Omron Eye Corner Locator,
 Conducting a Blind Test to Compare GDT Versus Omron Solution,
 Compute normalized Cumulative Error Distribution (CED) for each estimator

32. Magdi Mohamed, 2017:06:02 32 / 81
Iris Detection – Experiment # 2 Performance Summary

33. Magdi Mohamed, 2017:06:02 33 / 81
Iris Detection – Experiment # 3 Setup
 Using 15800 (mostly frontal view) face images with 40 landmarks per face
manually annotated as Ground Truth,
 Using Omron Face Detector,
 Using Supervised Newton Facial Landmark Detection method with Qualcomm
feature descriptor (HSG) and training algorithm,
 Conducting 5 fold cross validation for evaluation,
 Compute normalized Cumulative Error Distribution (CDF) for each estimator

34. Magdi Mohamed, 2017:06:02 34 / 81
Iris Detection – Experiment # 3 Performance Summary

35. Magdi Mohamed, 2017:06:02 35 / 81

36. Magdi Mohamed, 2017:06:02 36 / 81

37. Magdi Mohamed, 2017:06:02 37 / 81

38. Magdi Mohamed, 2017:06:02 38 / 81

39. Magdi Mohamed, 2017:06:02 39 / 81

40. Magdi Mohamed, 2017:06:02 40 / 81

41. Magdi Mohamed, 2017:06:02 41 / 81

42. Magdi Mohamed, 2017:06:02 42 / 81

43. Magdi Mohamed, 2017:06:02 43 / 81

44. Magdi Mohamed, 2017:06:02 44 / 81

45. Magdi Mohamed, 2017:06:02 45 / 81

46. Magdi Mohamed, 2017:06:02 46 / 81

47. Magdi Mohamed, 2017:06:02 47 / 81

48. Magdi Mohamed, 2017:06:02 48 / 81

49. Magdi Mohamed, 2017:06:02 49 / 81

50. Magdi Mohamed, 2017:06:02 50 / 81

51. Magdi Mohamed, 2017:06:02 51 / 81

52. Magdi Mohamed, 2017:06:02 52 / 81

53. Magdi Mohamed, 2017:06:02 53 / 81

54. Magdi Mohamed, 2017:06:02 54 / 81

55. Magdi Mohamed, 2017:06:02 55 / 81

56. Magdi Mohamed, 2017:06:02 56 / 81
Handwriting Recognition Using BAR Features
A Character Image and the Feature Image for the Foreground Horizontal Direction Pseudo-Code for Computing the Bar Transform on the Foreground
BAR transform is an existing technique proven to provide superior performance

57. Magdi Mohamed, 2017:06:02 57 / 81
Handwriting Recognition Using BAR Features
An Upper Case “B” and the Foreground and Background Bar Transform Feature Images Corresponding to the East-West Directions
BAR Feature Vector Uses Both Foreground & Background BAR Transforms

58. Magdi Mohamed, 2017:06:02 58 / 81
GDT for Handwriting (Normal Direction)

59. Magdi Mohamed, 2017:06:02 59 / 81
GDT for Handwriting (Tangent Direction)

60. Magdi Mohamed, 2017:06:02 60 / 81
Handwriting Recognition Using GDT Features

61. Magdi Mohamed, 2017:06:02 61 / 81
Handwriting Recognition Using GDT Features

62. Magdi Mohamed, 2017:06:02 62 / 81

63. Magdi Mohamed, 2017:06:02 63 / 81

64. Magdi Mohamed, 2017:06:02 64 / 81
Testing

65. Magdi Mohamed, 2017:06:02 65 / 81
Testing

66. Magdi Mohamed, 2017:06:02 66 / 81
Testing

67. Magdi Mohamed, 2017:06:02 67 / 81
Testing

68. Magdi Mohamed, 2017:06:02 68 / 81
Testing

69. Magdi Mohamed, 2017:06:02 69 / 81
Testing

70. Magdi Mohamed, 2017:06:02 70 / 81
Testing

71. Magdi Mohamed, 2017:06:02 71 / 81
Testing

72. Magdi Mohamed, 2017:06:02 72 / 81
Testing

73. Magdi Mohamed, 2017:06:02 73 / 81
Testing

74. Magdi Mohamed, 2017:06:02 74 / 81
Testing

75. Magdi Mohamed, 2017:06:02 75 / 81
Testing

76. Magdi Mohamed, 2017:06:02 76 / 81
Testing

77. Magdi Mohamed, 2017:06:02 77 / 81
Testing

78. Magdi Mohamed, 2017:06:02 78 / 81
Testing

79. Magdi Mohamed, 2017:06:02 79 / 81
Testing

80. Magdi Mohamed, 2017:06:02 80 / 81
Testing

81. Magdi Mohamed, 2017:06:02 81 / 81
Testing