Paper fingerprinting using alpha-masked image matching


Published on

T.Q. Pham, S.W. Perry and P.A. Fletcher, DICTA 2009.
Digital Image Computing: Techniques and Applications,
Melbourne, Australia, 2009.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Paper fingerprinting using alpha-masked image matching

  1. 1. Paper fingerprinting using alpha-masked image matching Tuan Q. Pham† Stuart W. Perry Peter A. Fletcher Canon Information Systems Research Australia (CiSRA) 1 Thomas Holt drive, North Ryde, NSW 2113, Australia. † corresponding author: Abstract arrangement of fibres and no two regions on the same sheet of paper have the same arrangement. The randomness of this In this paper, we examine the problem of authenticating arrangement can be exploited to create a unique signature for paper media using the unique fibre structure of each piece the sheet of paper. A recent article described the creation of of paper (the so-called ”paper fingerprint”). In particular, unique signatures for a variety of common objects such as we look at methods to authenticate paper media when text paper, coated cardboard packaging and matt-finish plastic has been printed over the authentication zone. We show how cards using the diffuse reflection from a laser focussed alpha-masked correlation [8] can be applied to this problem onto a small region of the object [3]. A unique signature and develop a modification to alpha-masked correlation could be obtained even when the object had been soaked that is more closely matched to the requirements of this in water and then dried, baked in an oven, crumpled and problem and produces an improvement in performance. We creased, or written over with a ballpoint pen or thick marker. also investigate two methods of pixel inpainting to remove An object could be authenticated by cross-correlating the printed text or marks from the authentication zone and allow signature obtained from the object with those in a database ordinary correlation to be performed. We show that these of signatures of authentic objects. methods can perform as well as alpha-masked correlation. The basis of this idea has been around for a while. It can Finally two methods of improving the robustness to forgery be traced back as least as far as US patents filed in 1981 are investigated. [9] and 1984 [10]. In these patents the inventor discloses the idea of using information about inherent irreproducible 1. Introduction randomness within an object imaged by an optical system to form a unique signature for the object. More recently it has When a reasonable facsimile of an object can be created been noted that a coherent light source is not required for at a cost less than the value of the object, the object a unique signature to be computed for paper [16]. Indeed, may be become a target for forgery. The concept of an the earlier work [9], [10] does not assume a coherent light object’s ”value” can be defined in many different ways. source. By simply imaging the paper surface illuminated Paper currency is an example of an object with a cost to by incoherent light at high resolution, sufficient information manufacture that is often small compared to the value society about the random structure of the paper surface can be places on the object. Even plain paper documents whose obtained to create a highly unique signature for the paper cost is almost negligible to produce can have a very large [16]. The authors have found that the requirements for value due to the importance of the information printed on the imaging a piece of common office paper to obtain a highly document. In recent times the availability of cheap printing unique signature are easily met by many common consumer technology has further reduced the cost of manufacture document scanners. In addition, experiments we have per- for paper objects, increasing the temptation of forgery and formed indicate that the signatures obtained are robust to improving the quality of forgeries. damage to the paper such as light staining, wrinkling, and There have been many different strategies employed to wetting followed by drying. The use of a desktop scanner protect objects from forgery. One strategy is to make the for this application provides a number of benefits including equipment required for manufacture of the authentic object removing the requirement for specialised equipment to read so expensive that forgery is discouraged. For example, the the signatures. In this work the signature we use is simply a addition of holograms, special paper and/or inks or finely greyscale 8-bit 256 by 256 pixel image of the paper surface printed details. captured by the scanner at 600dpi. In [3] and [16], cross- Another approach that has gained interest recently is correlation was used to match an obtained signature with a security based on the inherent randomness present in many previously stored signature, and this approach can be used to objects. An ordinary piece of office paper when viewed at match the above image signatures from a desktop scanner. the micrometer scale is a highly random arrangement of However, simply cross-correlating an image of the surface fibres and filler material. Every sheet of paper has a different of the paper does not provide a signature authentication
  2. 2. system that is robust to marks, writing or printing across 2.1. Fitch’s alpha-masked correlation the signature zone. Such marks can obscure the signature of interest in the cross-correlation operation and increase the Suppose the two ROIs from images f1 and f2 are related likelihood of an authentic object being declared unauthentic. by a translation (x0 , y0 ) and unknown noise n: There are many applications of this technology where it would be useful to allow the user to print additional in- f2 (x − x0 , y − y0 ) = f1 (x, y) + n(x, y), formation onto the document after the signature is collected Fitch et al. [8] define a weighted error function: and still have the object be recognised as authentic. In [2], a method was described where the image of a signature zone E(x0 , y0 ) = (n(x, y))2 α1 (x, y)α2 (x − x0 , y − y0 ) was filtered before comparing the signature with that in a x,y 2 database using cross-correlation. The filtering was intended = f1 (x, y) − f2 (x − x0 , y − y0 ) to remove the effect of any marks within the signature x,y zone, such as damage or printed text. This method has a α1 (x, y)α2 (x − x0 , y − y0 ) (1) disadvantage that the filtered pixels are still included in the cross-correlation calculations and hence the filtered pixels which can be written as a summation of three correlations: degrade the accuracy of the signature matching. In fact the 2 2 method presented in [2] would only be effective when the E(x0 , y0 ) = α1 f2 ⊗ α2 − 2α1 f1 ⊗ α2 f2 + α1 ⊗ α2 f2 area of damage or printed text was small compared to the where (f ⊗ g)(x0 , y0 ) = x,y f (x, y)g(x − x0 , y − y0 ) size of the signature zone. Unfortunately, [2] contains almost denotes the correlation between f and g. Once the error no implementation information about the method presented function is calculated, the translation between the two im- and no actual numerical results. For this reason, a direct ages can be determined by finding the coordinates (x0 , y0 ) comparison between this method and the presented approach at which E(x0 , y0 ) is minimum. is not possible. To further simplify the representation of the error func- m In this paper we present signature matching algorithms tion, let us make the following definition: Cmn = α1 f1 ⊗ n that are robust to marks, handwriting or text printed over the α2 f2 . With this notation, the error function can be expressed signature region either before the initial reference signature as: is collected, or subsequently. We do this by using several E = C20 − 2C11 + C02 modified signature matching algorithms which are based on cross-correlation, but are tailored to exclude pixels deter- A problem with the above error function E is that the sum mined to correspond to printed text from the calculation. in (1) contains a different number of terms for each (x0 , y0 ), depending on how often α1 (x, y) and α2 (x − x0 , y − y0 ) are In section 2 we review the alpha-masked correlation of zero at the same time. Thus, [8] normalizes the error function Fitch, et al. [8], and present a modification to the alpha- by the area of overlap between α1 and α2 : masked correlation algorithm to better account for back- ground illumination variation. Experiments are performed 1 Eα = (C20 − 2C11 + C02 ) (2) to show how alpha-masked correlation can be applied to the C00 problem of paper authentication. In section 3, alternative algorithms to solve this problem using pixel inpainting are 2.2. Correlation vs matching: peaks vs troughs presented and experiments performed. Section 4 presents two methods to improve the robustness to forgery of the Image matching as described above is presented as a proposed methods. Section 5 presents the conclusions. least squares estimation problem, in which the best match between two images is regarded as the position at which the squared, normalized differences between two images is 2. Correlation-based alpha-masked image at a minimum, with a perfect match achieving a value of matching zero. This is at variance with the view of image matching by correlation, in which the best match between two images is regarded as the position with the maximum correlation Alpha-masked image matching is a technique to match value. In the case of image matching, finding the best match sub-regions of two images. The Regions Of Interest (ROI), is equivalent to finding the minimum value in an image. In defined by alpha masks α1 and α2 , allow registration of the case of correlation, finding the best match is equivalent non-rectangular objects. The alpha mask 0 < α(x, y) < 1 to finding the maximum value in an image. can be seen as a weighting map, where a weight of zero If one attempts to estimate the statistical significance of at (x, y) means that the pixel at (x, y) should be removed a match, then a maximum correlation value is easier to from consideration. When pixels have differing α-values, it interpret than a least squares error, as the magnitude of means that some are considered more important than others. the correlation peak can be compared against an assumed
  3. 3. (a) paper 1 before printing (b) paper 2 before printing distribution of non-matching correlation values, allowing statistical inferences to be drawn from the data. For example, 3.33 (3.46) a normal distribution could be used, as the correlation image contains both positive and negative values, unlike the match image, which contains only positive values. A simplistic method is used to convert the least-squares 3.89 error value returned from image matching to something (2.91) similar to a magnitude value from a correlation process. 23.16 (3.06) 25.64 (3.31) The least-squares image is simply negated, normalized to remove any DC offset, and divided by the root-mean-square 4.08 value of the image. This results in an matching image with (3.36) a standard deviation of 1, and as long as the width of the correlation peak is small compared to the size of the correlation image, the peak strength can be analyzed in terms of the number of standard deviations away from the mean value. The correlation image around the peak may also be 2.02 (6.22) processed using quadratic interpolation to estimate a sub- (c) paper 1 after printing (d) paper 2 after printing pixel accurate peak position and peak magnitude. Figure 1. Paper matching using alpha-masked correla- 2.3. Normalized correlation of background sub- tion versus (cross correlation). tracted images Image matching using the alpha-masked correlation for- peak at the matching offset rather than a trough like Eα . mula (2) is not robust to different gains and offsets between As a result, a strong peak in EN signifies a good match the images being matched due to the mean squared differ- between the two input images. ence in (1). To make the algorithm robust to different gains and offsets, the input images should be normalized to have 2.4. Paper matching using alpha-masked correla- zero mean and unit standard deviation over the masked area tion before correlation. This is easily done for images of paper because blank paper usually has a flat intensity distribution. To demonstrate that a paper can be uniquely identified The offset is removed first by subtracting from f1 and f2 even after substantial marking, we perform a simple paper their background intensities (average intensity of the non- matching experiment. The experiment involves two different printed area). The gain is then normalized by the standard pieces of paper scanned before and after text printing. The deviation over the same background area. 600dpi paper scans are subjected to alpha-masked corre- For more complicated images that do not have a flat lation and cross-correlation. We show that while different ¯ background, a background image f can be estimated using sheets of paper with the same printed text could be mis- normalized convolution [15]: classified as matching by standard correlation, alpha-masked ¯ (αf ) ∗ k f= (3) correlation correctly classifies them as non-matching. α∗k Figure 1a and 1b show two 256 × 256 scans of two blank where k is a blur kernel with large support such as Gaussian sheets of paper. The printed versions of these sheets of blur of σ = 5 pixels and ∗ is the convolution operator. paper are in Figure 1c and 1d. The intensities of these 8-bit After background subtraction, the terms C20 /C00 and images are linear stretched between [200 255] so that the C02 /C00 in (2) are simply variance estimates of the pa- paper texture is more visible in Figure 1. There is a small per background. Although these estimates do vary due to translation between images 1a and 1c and 1b and 1d due different areas of overlap C00 , the variation is small if to the feeding mechanism of the scanner. Also due to the the overlapping area of the two masks does not change same positioning error, the text in Figure 1c and 1d appear substantially over all x0 , y0 offsets. As a result, C20 /C00 slightly shifted, even though the same text was printed. The and C02 /C00 can safely be removed from the error function alpha masks are constructed by thresholding the images (2), leaving a single term in the normalized correlation error at intensity 160 followed by two iterations of 8-connected function: binary erosion. α1 f1 ⊗ α2 f2 C11 We performed alpha-masked correlation and cross- EN = = . (4) α1 ⊗ α2 C00 correlation between every image pair and the correlation Compared to Eα in (2), EN requires only half the number peak strengths defined in section 2.2 are displayed in be- of correlations to compute. EN also produces a correlation tween the paper images in Figure 1 (the cross-correlation
  4. 4. peak strengths are displayed within brackets). Zoomed-in Laplacian image, is stored as the base level of the pyramid, versions of the correlation images are also displayed below while the low-pass image l0 is losslessly downsampled the peak strength numbers. The contrast of these correlation (requires band-limitedness). The downsampled image f1 is images has been enhanced to visualize the correlation peaks again subjected to low- and high-pass decomposition. This more effectively. process is repeated recursively until a desired number of The most striking difference between alpha-masked cor- pyramid levels are obtained (3 levels as shown in Figure 2). relation and cross-correlation occurs between image pairs Because the Laplacian pyramid is generated using lossless from the same paper before and after printing. While alpha- operations, the original image can be reconstructed perfectly masked correlation produces high peaks around 25 for from the Laplacian pyramid as shown in Figure 2b. Although matching papers, cross-correlation peaks remain low around the Laplacian pyramid requires 33% more storage than a sin- 3. The correlation images below these numbers (in bold in gle image, it allows separate manipulation of different band- Figure 1) also confirm very sharp peaks when using alpha- pass images. This is especially useful in texture generation masked correlation. The cross-correlation peaks, on the other or infilling, where the texture is different at different scales. hand, are barely visible. Cross-correlation peak values of 3.06 and 3.31 between the matching pairs are no better than 3.2. Alpha-masked Laplacian pyramid the peak values of non-matching pairs. Another interesting result is the correlation between the The concept of Laplacian pyramid can be extended to two different pieces of paper after printing. While alpha- images with uncertain data. The data uncertainty is specified masked correlation correctly gives a small correlation of by an alpha mask 0 ≤ α ≤ 1, where zero corresponds to 2.02, cross-correlation produces a visible peak at an appre- invalid data and one for valid data. Rather than the low- ciable strength of 6.22 (in italic in Figure 1). This visible pass operator BLUR in Figure 2a, Normalised Convolution peak is a result of correlation of the printed text, not the (NC) [15] is used to smooth out an alpha-masked image by texture of the papers. Cross-correlation is therefore not a kernel k: robust to printing. (α0 f0 )∗k l0 = h0 = α0 (f0 −l0 ) α1 = {α0 ∗ k} ↓ (5) α0 ∗ k 3. Alpha-masked image matching using in- where f0 and α0 are the input image and its alpha-mask, painting l0 and h0 are the low- and high-pass decompositions, α1 is a generated alpha-mask for the downsampled image f1 , ∗ Image inpainting is a technique to fill missing pixels of and ↓ denote a convolution and a downsampling operator an image with plausible intensities. The missing pixels may respectively. be lost during transmission or they may cover unwanted As can be seen in Figure 3, the normalised convolution objects to be erased from the image. In a typical solution operation imports valid intensities into the masked areas. It [1], the image is decomposed into texture and structure does so in a similar fashion to image morphology where the components. Texture synthesis is then used to fill in the masked regions are successively eroded and filled with local texture component, whereas anisotropic diffusion is used mean intensities. Note that in (5) the high-pass image hi to transport intensities along iso-contours into the missing has to be multiplied with the mask αi of the same pyramid regions of the structure component. level to invalidate the influence of the masked pixels. The This paper does not introduce a new inpainting solution. Laplacian pyramid generated this way is compatible with a Rather, a smooth infilling of masked images followed by normal Laplacian pyramid. The difference happens in the cross-correlation is presented as an alternative to alpha- reconstructed image, where invalid pixels are automatically masked correlation. Laplacian-based smooth inpainting [5] filled with mean intensities from surrounding valid pixels. minimizes sudden intensity changes at mask boundaries. 3.2.1. Implementation details. To avoid the need for a Spectrum distortions are therefore minimized. The inpainted lossless subsample operation during pyramid construction, images can then be matched reasonably well even when the the high-pass image hi is computed slightly differently from masking area is large. (5): hi = αi (fi − {fi+1 } ↑) (6) 3.1. Laplacian pyramid where ↑ denotes an upsampling operator. In this paper, A Laplacian pyramid [4] is an over-complete representa- we use a upsampling/downsampling factor of two. The tion of an image using a set of band-pass images at succes- blur kernel is a separable Burt-Adelson [4] 5-tap filter sively reduced dimensions. The decomposition of an input [0.05 0.25 0.4 0.25 0.05] (which approximates a Gaussian image into its Laplacian pyramid is illustrated in Figure 2a. filter of σ = 1). The Laplacian images are successively The input image f0 is decomposed into a high-pass image h0 generated until the alpha mask αi no longer contains any and a low-pass image l0 . The high-pass image h0 , a.k.a. the zero pixels.
  5. 5. (a) Laplacian pyramid construction from an image (b) Image reconstruction from a Laplacian pyramid Figure 2. Laplacian pyramid as an over-complete representation of an image. BLUR SUBSAMPLE α 0 α 1 SUBSAMPLE NC NC SUBSAMPLE f2 f1 l1 _ + (a) Lena with mask (b) Inpainted result of (a) f0 l0 _ + h1 Alpha-masked Laplacian pyramid h0 Figure 3. α-masked Laplacian pyramid construction. (c) Barbara with mask (d) Inpainted result of (c) Figure 4. Smooth inpainting of natural images. 3.2.2. Smooth inpainting results. The results of smooth image inpainting for two natural images can be seen in Figure 4. Compared to the images with average intensity correlation sometimes produces better peak detection than infilling on the left, the inpainted images on the right look alpha-masked correlation. Infilling using average local in- much more pleasing. The text masks are almost invisible. tensity performs equally well as normalized correlation with Because image inpainting transports neighboring intensities background subtraction. This is understandable since both into the masked areas, low-frequency details are recon- infilling and background subtraction essentially fills the structed very well. The missing of information is only masks with local average intensities. detectable at textured areas like the fur of Lena’s hat or the stripe pattern of Barbara’s clothes. 3.3.1. Inpainted paper matching. Two inpainting results for a scan of a printed paper using mean intensity infill- 3.3. Alpha-masked image matching using inpaint- ing and smooth inpainting are shown in Figure 5. The ing 256×256 central region of Figure 5a is matched against the blank paper signatures in Figure 1a-b. Due to optical dot The alpha-masked images can be inpainted before match- gain [11], the intensities around printed texts are slightly ing to avoid extra correlations in the alpha-masked and darker than the background. Smooth inpainting in Figure normalized correlation. By matching two scans of the same 5b transports these dark intensities into the masked area, paper before and after text printing, we show that inpainted whereas mean infilling in Figure 5a uses the correct average
  6. 6. 50 alpha−masked correlation [8] 45 normalized correlation (section 2.3) mean−filled correlation (section 3.3) 40 fill = 0.77 inpainting correlation (section 3.3) 35 correlation of non−matching pairs match strength 30 fill = 0.58 25 20 (a) mean intensity infilling (b) smooth Laplacian inpainting 15 fill = 0.45 Figure 5. Paper 1 with printed text after inpainting. … 10 5 alpha-masked normalized mean-filled inpainting 0 match 23.16 22.32 22.31 14.17 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 non-match 4.08 2.92 2.94 2.68 fill = 0.05 mask fill factor ratio 5.67 7.63 7.59 5.28 (a)mask (b) match strength versus mask fill factor Table 1. Matching a printed paper (Fig. 1c) with two Figure 6. Correlation results under fill-factor stress test. fingerprints (match = Fig. 1a, non-match = Fig. 1b). containing Japanese test was scanned twice on an EPSON 10000XL scanner. Japanese text was chosen for this experi- background intensity. This gives rise to a lower fingerprint ment because compared to English text, the different spatial match strength for the smooth inpainted image compared to frequency characteristics of the three Japanese character sets that of the mean-filled image (first row of Table 1). However, (kanji, hiragana, and katakana) would be likely to produce both infilling methods produce substantially higher match a wider spread of mask fill factors on which to test the strengths than the correlation peak strengths of non-matching proposed method. To mitigate the effects of fixed pattern papers. Infilling is therefore an effective yet inexpensive scanner noise on the result, each scan was performed at a alpha-masked image matching method. different position on the scanner platen. Both scans were collected as 8bit greyscale images with a resolution of 3.3.2. Robustness against mask fill factor. One big advan- 600dpi. The scans were roughly aligned by eye; however tage of alpha-masked image matching over other signature no attempt at electronic correction of residual rotational verification techniques such as [2] is the ability to handle a misalignment between the images was made. Each scan was large area of alteration from the original signature. In Figure divided into 240 256×256 pixel image patches and each 6, we investigate the performance of alpha-masked image patch in the first scan was matched against the corresponding matching at different levels of the fill factor of the mask. patch in the second scan using alpha-mask correlation [8]. The masks in Figure 6a are successively eroded to reduce The match strength and mask fill factor for each patch is their fill factors (defined as ratio of the valid area in white shown in Figure 7 below. It should be noted that each point over the area of the whole mask). This mask is used with on Figure 7 represents a different part of the document, and the printed image in Figure 1c to match against two blank two iterations of 8-connected binary erosion of the printed signatures in Figure 1a-b. The resulting match strengths are content was used to generate different fill-factors. plotted against fill factors in Figure 6b as continuous lines This experiment was performed using a C implementation for matching pair and dashed lines for non-matching pair. of the normalised alpha-masked correlation method on a It is interesting to see that alpha-masked image matching computer running Microsoft Windows XP, Service Pack 3. still produces a high matching strength after more than 80% The computer had two Intel Xeon 5060 Dual Core CPUs of the original image has been masked out (continuous lines running at 3.20GHz, with 3.25 GBytes of RAM. The 240 in Figure 6). Normalized correlation and mean-filled correla- match operations took a total of 161.48 seconds, giving tion performs equally well, both are much better than alpha- an average time of 0.67 seconds per match operation. No masked correlation and inpainted correlation. The perfor- special optimisation of the code to take advantage of the mance of alpha-masked correlation, while being preferable multiple CPUs or multiple cores was performed. for large fill factors (as shown in Table 1), degrades quickly as the fill factor decreases. All four methods in Figure 4. Improving the robustness of paper finger- 6b also produces consistently low correlation between non- printing matching paper pairs (dashed lines in Figure 6). An additional experiment was performed to test the Being a pure image-based method, the paper verification proposed method’s robustness to mask fill factor. A page technique using correlation is susceptible to attack if a forger
  7. 7. (a) paper scanned at 0º (b) paper scanned at 180º 25 alpha−masked correlation [8] 20 fill = 0.94 match strength 15 fill = 0.86 + _ 10 + + fill = 0.73 … 5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 mask fill factor (e) (f) fill = 0.16 (a)ROI (b) match strength versus mask fill factor (c) diffuse reflectance (d) specular reflectance Figure 7. Paper matching on a Japanese document. Figure 8. Diffuse and specular components of paper. manages to print a desired paper signature onto a fake propose to match the two reflectance components sepa- medium. While we believe that it is difficult to reproduce rately to improve robustness against forgery. The specular paper textures at different resolutions, we want to improve reflectance image could also be turned into a surface depth the robustness of our paper fingerprinting technique. This map using shape from shading [13]. While it is possible section presents two such methods to address the robustness to forge a 3D surface using molds or very fine topology to forgery problem. creation tools, the forgery needs to have a matching diffuse reflectance at precisely the same alignment with the 3D 4.1. Multiple orientation paper fingerprinting topology. This makes it more difficult for a document to be forged at a cost of one extra scan of the candidate document. The reflection of light from most turbid materials consists of two major components: the surface and the subsurface 4.2. Double-sided paper fingerprinting reflectance. Surface reflection, a.k.a. specular reflection, is the mirror-like reflection of light at the paper surface. Due to Another way to improve the robustness of paper finger- the roughness of paper, this surface reflection is more diffuse printing is to verify its signatures on both sides. During a than the specular reflection from a smooth surface (see registering phase, the signatures for both sides of a document Figure 8f). Subsurface reflection, a.k.a. diffuse reflection, are collected. These signatures are stored in a database to- is the reflection of light from within the paper after light gether with the displacement between the signatures. When penetrates the surface and scatters within the substrate (see an object is presented for authentication, the signatures on Figure 8e). both sides of the document are scanned. These candidate The diffuse and specular reflectance components can be signatures must match the signatures in the database and separated from an image of a paper using photometric stere- the alignment of the signatures in the database must also ography [14]. As illustrated in Figure 8a-b, two scans from be the same before a match is declared. This uses the fact opposite orientations (i.e. 180◦ difference in illumination that the rear side fingerprint can be detected when the front angle) obtained by the same scanner setting are required. side of the paper is scanned and visa-versa (see Figure 9). Because shadows in one image correspond to specular reflec- This allows accurate alignment determination between the tion in the other image, the specular reflectance is roughly front and back signatures. With the double-side approach, cancelled when the two aligned images are added together. a forger is required to match signatures on both sides of The diffuse reflectance is derived in Figure 8c as the sum of the document and match the relative positions between Figure 8a and 8b. The specular reflectance in Figure 8d is the signatures on both sides. This greatly complicates the the difference of Figure 8a and 8b. As expected, the specular forgery process while still using only an inexpensive scanner image looks shiny, whereas the diffuse image looks dull and during verification. contains pores from the filler material within the paper. A very similar diffuse reflectance image is obtained from two 5. Conclusion aligned scans at 90◦ and 270◦ orientation. Different from [6], which bundles the diffuse and specular In this paper we have looked at the problem of matching reflectance into a feature vector for matching purposes, we paper media in the presence of printed text, or other mark-
  8. 8. published and unpublished patent applications [7], [12]. References [1] M. Bertalmio, L. Vese, G. Sapiro, and S. Osher. Simultaneous structure and texture image inpainting. IEEE Trans. on IP, (a) front side at 0◦ (b) back side at 0◦ (c) correlation of a&b 12(8):882-889, 2003. Figure 9. Matching different sides of paper at 1200dpi. [2] Y. Boutant, D. Labelle, and H. Seux. Method for recognition and tracking of fibrous media and applications of such a method, particularly in the computer field. US patent applica- tion 2005/0257064, filed in Apr. 2005. ings, on either one or both of the paper media to be matched. We examined Fitch’s formulation of alpha-masked correla- [3] J. D. R. Buchanan, R. P. Cowburn, A.-V. Jausovec, D. Petit, tion [8], and adapted this method to the problem of interest. P. Seem, G. Xiong, D. Atkinson, K. Fenton, D. A. Allwood, and M. T. Bryan. Forgery: ’fingerprinting’ documents and In particular, Fitch’s method is modified to better handle packaging. Nature, 436:475, 2005. the problem non-uniform background illumination and is re- normalised. An experiment was performed that showed that [4] P. J. Burt and E. H. Adelson. The Laplacian pyramid as alpha-mask correlation is effective at matching paper with a compact image code. IEEE Trans. on Communications, printed text present, whilst ignoring correlations amongst the 32(4):532-540, 1983. printed text that might otherwise create false matches. Our [5] P. J. Burt and E. H. Adelson. Method for compensating for modified version of alpha-mask correlation was shown to void-defects in images. US patent 4698843, RCA Corporation, out-perform the basic alpha-mask correlation in some cases. Oct. 1987. As alternatives to the alpha-masked correlation approach, we investigate two methods of pixel inpainting; mean intensity [6] W. Clarkson, T. Weyrich, A. Finkelstein, N. Heninger, J. A. Halderman, and E.W. Felten. Fingerprinting blank paper using in-filling of pixels identified as text and an approach based commodity scanners. In Proc. IEEE Symposium on Security on the Laplacian pyramid decomposition of the paper image. and Privacy, 2009. After in-painting text or markings on the paper image, ordinary cross-correlation may then be used to compute a [7] S. E. Ecob, S. W. Perry, and S. A. Rudkin. Authenticating match strength for the paper media. We performed a number partially transparent medium. Australian patent application AU2007254655, Canon Kabushiki Kaisha, filed in Dec. 2007. of experiments and found that Laplacian pyramid inpainted images followed by ordinary cross-correlation perform no [8] A. J. Fitch, A. Kadyrov, W. J. Christmas, and J. Kittler. Fast worse than alpha-masked correlation in this problem. In robust correlation. IEEE Trans. on IP, 14(8):1063-1073, 2005. fact, the Laplacian pyramid inpainted augmented method performed better in some cases. The mean-filled pixel in- [9] R. N. Goldman. Non-counterfeitable document system. US patent 4423415, Light Signatures Inc., Jun. 1981. painting method performed even better than the Laplacian method, equalling the performance of our modified version [10] R. N. Goldman. Verification system for document substance of alpha-masked correlation. and content. US patent 4568936, Light Signatures, Dec. 1984. In addition we investigated methods to further improve the security of paper authentication. The first method involved [11] S. Gustavson. Dot gain in color halftones. Ph.D. disertation no. 492, Link¨ ping University, Sweden, 1997. o scanning the paper from two different directions and de- composing the scans into a specular and diffuse component. [12] S. J. Hardy, K. G. Larkin, S. Farrar, P. A. Fletcher, and T. Q. The specular component is mostly a function of the 3D Pham. Alpha-masked image matching. US patent application surface structure of the paper, while the diffuse component 2009/0003601, Canon Kabushiki Kaisha, filed in Jun. 2008. is mostly a function of the internal structure of the paper. [13] B. K. P. Horn and M. J. Brooks (eds). Shape from shading. By separating these components, two pieces of paper can MIT press, 1989. be matched on the internal and surface structure separately, improving the robustness to forgery. The second method [14] P.-A. Johansson and P. Hansson. Method of determining an made use of the fact that for many ordinary office papers, illuminated surface. US patent 6757065, STFI, 2004. the fingerprint collected on one side of the paper can be [15] H. Knutsson and C.-F. Westin. Normalized and differential detected in the image collected on the other side of the paper. convolution: methods for interpolation and filtering of incom- By collecting signatures from both sides of the paper, and plete and uncertain data. In Proc. of CVPR, 1993. measuring the through paper displacement of the signatures, a greater robustness to forgery can be achieved. [16] H. Sugino, K. Ito, and T. Shimizu. Document verification system. US patent 6948657, Fuji Xerox Co. Ltd., Apr. 2003. Aspects of this work are the subject of a number of