T.Q. Pham, S.W. Perry and P.A. Fletcher, DICTA, 2009.

1. Australian R&D with global impact
Paper 55 Session3B 12:10PM-12:30PM on Thu 3 Dec
Paper fingerprinting using
alpha-masked image matching
Tuan Pham, Stuart Perry, and Peter Fletcher
Canon Information Systems Research Australia
Copyright CISRA Slide 1 Printed 30 November 2009

2. Australian R&D with global impact
Overview
1. Paper FingerPrint (PFP)
Random structure of paper
Uniqueness
2. Alpha-masked image matching
Normalized correlation
Image inpainting
3. PFP robustness
Evaluation
Possible improvements
Copyright CISRA Slide 2 Printed 30 November 2009

3. Australian R&D with global impact
The The original
Paper fingerprint coupon fingerprint
Intrinsic characteristic of a
piece of paper that uniquely
describes itself
Document authentication
application
Paper Fingerprint System
Match Strength
Not Original Threshold Original
Comparison
Copyright CISRA Slide 3 Printed 30 November 2009

4. Australian R&D with global impact
Paper FingerPrinting (PFP) using a scanner
Office
paper
Copyright CISRA Slide 4 Printed 30 November 2009

5. Australian R&D with global impact
PFP matching using cross-correlation
Paper FingerPrint is a 256x256 8-bit grayscale scan at 600dpi
PFP match is determined based on correlation peak strength:
Peak strength > 10 → match
Peak strength < 10 → non-match
Using extreme value theory, the false alarm rate is about 1x10-50
___ Fisher-Tippett
distribution fit
Non-Match PFP strengths Matching PFP strengths
Copyright CISRA Slide 5 Printed 30 November 2009

6. Australian R&D with global impact
Correlation is not robust against change
We swap 7.5% of pixels around → match strength drops below 10
Printing also decreases match strength & increases false matches
Some
printed text Low correlation due to the
creates a
false negative
influence of the printed
areas
Same sheet of paper
Same Same
printed text printed text High correlation due to
creates a creates a
false positive false positive
the influence of the
printed areas
Different sheets of paper
Copyright CISRA Slide 6 Printed 30 November 2009

7. Australian R&D with global impact
Solution 1: alpha-masked correlation [Fitch et al]
Use weight α to mask out change areas during least-square matching:
E ( x0 , y0 ) = ∑ ( f1 ( x, y ) − f 2 ( x − x0 , y − y0 ) ) α1 ( x, y )α 2 ( x − x0 , y − y0 )
2
x, y
= α1 f12 ⊗ α 2 − 2α1 f1 ⊗ α 2 f 2 + α1 ⊗ α 2 f 22
If images f1 & f2 have zero mean, alpha-masked α f ⊗ α 2 f2
EN = −2 1 1
correlation can be simplified to normalized correlation: α1 ⊗ α 2
Paper 1 Cross-correlation
Peak strength = 12.44
Same
printed
texts
Alpha-masked correlation
Paper 2 Peak strength = 4.04
α
Copyright CISRA Slide 7 Printed 30 November 2009

8. Australian R&D with global impact
Solution 2: inpainting followed by correlation
Scanned paper with printed texts Filled-in with mean value Smooth inpainting
How well do these different solutions compare to each other?
Alpha-masked Normalized Mean-filled Inpainting
Match 46.32 44.64 44.62 28.34
Non-match 8.16 5.84 5.88 5.36
Ratio 5.67 7.63 7.59 5.28
Normalized correlation is most discriminative, followed by mean-filled correlation
Copyright CISRA Slide 8 Printed 30 November 2009

9. Australian R&D with global impact
Fill factor experiment: synthetic mask
Mask is successively eroded to reduce the fill-factor
50
100
alpha-masked correlation [8]
45
90 normalized correlation (section 2.3)
40 mean-filled correlation (section 3.3)
fill = 0.77 80
inpainting correlation (section 3.3)
PFP 1
35
70 correlation of non-matching pairs
match strength
30
60
fill = 0.58
25
50
PFP 2
20
40
fill = 0.45 15
30
…
10
20
Paper 1
after printing 5
10
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
fill = 0.05
mask fill factor
Copyright CISRA Slide 9 Printed 30 November 2009

10. Australian R&D with global impact
Match strength vs. fill factor from a real document
The document is scanned twice, 256x256 image patches are matched
25
50
alpha-masked correlation [8]
40
20
fill = 0.94
match strength
30
15
fill = 0.86
20
10
fill = 0.73
…
10
5
0
0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
fill = 0.16
mask fill factor
Copyright CISRA Slide 10 Printed 30 November 2009

11. Australian R&D with global impact
Improve PFP by multiple orientation scans
Reflection from paper consists of diffuse and specular components
These components can be separated by photometric stereography
paper scanned at 0º paper scanned at 180º
+ _
+ +
diffuse reflectance specular reflectance
Copyright CISRA Slide 11 Printed 30 November 2009

12. Australian R&D with global impact
Improve PFP robustness by double-sided scan
Verify the Paper FingerPrint on both sides.
Front and back side must correlate
The displacement between front and back side is fixed
Front side Back side Correlation of
scanned at 0° scanned at 0° front and back
Copyright CISRA Slide 12 Printed 30 November 2009

13. Australian R&D with global impact
Conclusions
Conventional scanners capture internal structure of
paper: Paper FingerPrint (PFP)
PFPs are very unique and can be used for authentication
PFPs can be matched even after printing
PFPs can be made more robust using more than one
scans
Copyright CISRA Slide 13 Printed 30 November 2009

14. Australian R&D with global impact
Thank you
tuan.pham@cisra.canon.com.au
Copyright CISRA Slide 14 Printed 30 November 2009