Detailled description of the paper "Robust image alignment for tampering detection" published on TIFS (2012)

1. 31/05/2017
1
ROBUST IMAGE ALIGNMENT FOR
TAMPERING DETECTION
Image Processing Lab – http://iplab.dmi.unict.it
University of Catania
Battiato, S., Farinella, G. M., Messina, E., & Puglisi, G.
IEEE Transactions on Information Forensics and Security, Vol. 7, No. 4,
pp. 1105-1117. DOI: 10.1109/TIFS.2012.2194285 - 2012
Outline
• Introduction and Motivations
• Related Works
• Proposed Approach
• Experimental Results
• Conclusions

2. 31/05/2017
2
Introduction and Motivations (1)
• Different episodes make questionable the use of visual
content as evidence material [PT11, HF06].
[PT11] “Photo tampering throughout history,” www.cs.dartmouth.edu/farid/research/digitaltampering/
[HF06] H. Farid, “Digital doctoring: how to tell the real from the fake,” Significance, vol. 3, no. 4, pp.
162–166, 2006.
= +
Introduction and Motivations (2)
Tampering localization is the process of localizing the
regions of the image that have been manipulated for
malicious purposes to change the semantic meaning of
the visual message.
In order to create a more heroic portrait of himself, Benito Mussolini had the
horse handler removed from the original photograph [PT11].
[PT11] “Photo tampering throughout history,” www.cs.dartmouth.edu/farid/research/digitaltampering/
Tampering

3. 31/05/2017
3
Introduction and Motivations (3)
Sender Receiver
hashhash
Classic hash
techniques
(e.g., MD5, SHA1)
Classic hashing techniques are unsuitable, even in case of slight manipulations which
does not change the visual content (e.g., image compression); just a small change in
the image (even a single bit) will, with overwhelming probability, results in a
completely different hash code.
Introduction and Motivations (4)
An image hash is a distinctive signature which represents the
visual content of the image in a compact way (usually just few
bytes).
Different solutions have been recently proposed in literature. Most of
them share the same basic scheme:
i. a hash code based on the visual content is attached to the
image to be sent;
ii. the hash is analyzed at destination to verify the reliability of the
received image.
Fundamental requirement: image signature should be as “compact"
as possible.
Assumption: signature is sent upon request throught a trust
authentication server which encrypt it in order to guarantee integrity
during transmission.

4. 31/05/2017
4
Introduction and Motivations (5)
• The image hash should be robust against “allowed
operations” (e.g., scaling, image compression, etc.) and
at the same time it should differ from the one
computed on a different/tampered image.
Original Compressed Scaled
Tampered
Introduction and Motivations (6)
• Although the importance of the binary decision task
related to the image authentication, this is not always
sufficient.
• In the application context of Forensic Science is
fundamental to provide scientific evidences through
the history of the possible manipulations (geometric
and photometric) applied to the original image to obtain
the one under analysis.

5. 31/05/2017
5
Introduction and Motivations (7)
Moreover, in order to perform tampering localization, the receiver
should be able to filter out all the geometric transformations
(e.g., rotation, scaling) added to the tampered image by aligning the
received image the one at the sender.
The alignment should be done in a semi-blind way: at destination
one can use only the received image and the image hash to deal with
the alignment problem since the reference image is not available.
Motivations and Aims of Our Work
The challenging task of recovering the geometric
transformations occurred on a received image from its
signature motivates our work.
The main problem addressed is the design of a robust
forensic hash component for Image alignment to better
perform tampering detection.

6. 31/05/2017
6
Destination
Authentication
Algorithms
Related Works: Basic Scheme
Source
Image Hash
(e.g., Alignment
component)
Untrusted Connection
Tampering
Detection
Reliable
Image
Original
Source
Image Received
Image
Trusted
Connection
Related Works (1)
• Fundamental requirement: image signature should be as
“compact" as possible.
Despite different robust alignment techniques have been
proposed by computer vision researchers, these
techniques are unsuitable in the context of forensic
hashing.
• To fit the underlying requirements, authors of [LVW10]
have proposed to exploit information extracted through
Radon transform and scale space theory in order to
estimate the parameters of the geometric
transformations.
[LVW10] W. Lu, A. L. Varna, and M. Wu, “Forensic hash for multimedia information,” in Proceedings of the
IS&T-SPIE Electronic Imaging Symposium - Media Forensics and Security, 2010, vol.7541.

7. 31/05/2017
7
Related Works (2)
Θ=45
shift along the angle axis=45
Radon
Trasform
Radon
Trasform
become
[LVW10] W. Lu, A. L. Varna, and M. Wu, “Forensic hash for multimedia information,” in Proceedings of the
IS&T-SPIE Electronic Imaging Symposium - Media Forensics and Security, 2010, vol.7541.
Related Works (3)
σ=0.5
σ=0.5
Radon
Trasform
Radon
Trasform
become
[LVW10] W. Lu, A. L. Varna, and M. Wu, “Forensic hash for multimedia information,” in Proceedings of the
IS&T-SPIE Electronic Imaging Symposium - Media Forensics and Security, 2010, vol.7541.

8. 31/05/2017
8
Related Works (4)
• To make more robust the alignment phase with respect to
manipulations such as cropping and tampering, an image
hash based on robust invariant features has been proposed
in [LW10].
• The above technique extended the idea previously proposed
in [RS07] by employing the Bag of Features (BOF) model to
represent the features to be used as image hash.
• The exploitation of the BOF representation is useful to
reduce the space needed for the image signature, by
maintaining the performances of the alignment component.
[LW10] W. J. Lu and M. Wu, “Multimedia forensic hash based on visual words,” in Proceedings of the IEEE
International Conference on Image Processing, 2010, pp. 989–992.
[RS07] S. Roy and Q. Sun, “Robust hash for detecting and localizing image tampering,” in Proceedings of the
IEEE International Conference on Image Processing, 2007, pp. 117–120.
Related Works (5)
[LW10] W. J. Lu and M. Wu, “Multimedia forensic hash based on visual words,” in Proceedings of the IEEE
International Conference on Image Processing, 2010, pp. 989–992.
22 
11 
33 
44 
55 
11   nn
nn 
11 
22 
33 
44 
55 
11   nn
nn 
hs hr
Ransac (σ)
SIFT points with highest contrast value
+
+
Send
Send
Receive
Receive
Shared Vocabulary
(obtained by clustering
of SIFT points extracted
from a training set)
Match:
same ID and
single occurrence
Ransac (θ)

9. 31/05/2017
9
Related Works (6)
• In [BS11] a more robust approach based on a cascade of
estimators has been introduced.
• The method in [BS11] is able to better handle the
replicated matchings in order to make a more robust
estimation of the orientation parameter.
• Moreover, the cascade of estimators allows a higher
precision in estimating the scale factor outperforming the
approach in [LW10].
[LW10] W. J. Lu and M. Wu, “Multimedia forensic hash based on visual words,” in Proceedings of the IEEE
International Conference on Image Processing, 2010, pp. 989–992.
[BS11] S. Battiato, G. M. Farinella, E. Messina, and G. Puglisi, “Understanding geometric manipulations of images
through BOVW-based hashing,” in International Workshop on Content Protection & Forensics (CPAF 2011), 2011.
Related Works (7)
[LW10] W. J. Lu and M. Wu, “Multimedia forensic hash based on visual words,” in Proceedings of the IEEE
International Conference on Image Processing, 2010, pp. 989–992.
22 
11 
33 
44 
55 
11   nn
nn 
11 
22 
33 
44 
55 
11   nn
nn 
hs hr
SIFT points with highest contrast value
+
+
Send
Send
Receive
Receive
Match:
same ID and
single occurrence
Ransac (σ)
Ransac (θ)
Shared Vocabulary
(obtained by clustering
of SIFT points extracted
from a training set)

10. 31/05/2017
10
Related Works (8)
[LW10] W. J. Lu and M. Wu, “Multimedia forensic hash based on visual words,” in Proceedings of the IEEE
International Conference on Image Processing, 2010, pp. 989–992.
[BS11] S. Battiato, G. M. Farinella, E. Messina, and G. Puglisi, “Understanding geometric manipulations of images
through BOVW-based hashing,” in International Workshop on Content Protection & Forensics (CPAF 2011), 2011.
Approach of [BS11]
22 
11 
33 
44 
55 
11   nn
nn 
11 
22 
33 
44 
55 
11   nn
nn 
hs hr
22 
11 
33 
44 
55 
11   nn
nn 
11 
22 
33 
44 
55 
11   nn
nn 
hs hr
Approach of [LW10]
Ransac (σ)
Ransac (θ)
Learned from previous works…
• The exploitation of the BOF representation is
useful to reduce the space needed for the image
signature, by maintaining the performances of the
alignment component.
• Handle replicated matchings help to make more
robust the parameters estimation phase
(especially for rotation angle estimation).
• Cascade approach (filtering) help to make more
robust the parameters estimation phase
(especially for the scale factor estimation).
22 
11 
33 
44 
55 
11   nn
nn 
11 
22 
33 
44 
55 
11   nn
nn 

11. 31/05/2017
11
What we have done more…
Feature selection problem
Transformation model (e.g.,
we wish to estimate traslation
too)
Check the robustness on a
bigger and challenging dataset
by considering different image
transformation
Consider realistic tampering
samples
ID
ID
2
1
7.0
30




Ty
Tx
Sender Receiver
Features Selection:
Ordering by contrast values
SIFT features extraction
SIFT ordering by contrast values
and selection of top n SIFT
SIFT
[θ,λ,(x,y)]
label (id)
1
2
Associate id value of the
closest prototype belonging
to the shared codebook
3
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
The signature is
composed of all the
quadruple [id,θ,x,y]
associated to the
selected SIFT features
4
h
Signature
Generation
Process

12. 31/05/2017
12
Features Selection:
Spatial Distribution and Ordering by Contrast Values
SIFT features extraction
cluster-based SIFT
ordering by contrast
values and selection
of top one for each
cluster SIFT
[θ,λ,(x,y)]
label (id)
Associate id value of the
closest prototype belonging
to the shared codebook
The signature is
composed of all the
quadruple [id,θ,x,y]
associated to the
selected SIFT features
2
3
5
h
Signature
Generation
Process
Spatial
Clustering
4
1
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
Previous Approaches Proposed solution
Features Selection: Example
Original Image Pattern
Different matches are preservedAll are wrong matches
Sender Receiver Sender Receiver
SpatialDistributionand
OrderingbyContrastValues
OrderingbyContrastValues

13. 31/05/2017
13
ProposedApproach: Alignment
Alignment
•Trasformation Model
•Voting procedure (with
Pre-Filtering) into the
space of parameters.
•Scale estimation on
realiable information
Image Alignment (1)
The alignment is performed by employing a similarity transformation of
keypoint pairs corresponding to hashes entries matched through their ID.
The aim of the aligment component is the estimation of the quadruple
hs hr
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
 yx TT ˆ,ˆ,ˆ,ˆ 

14. 31/05/2017
14
hs hr
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
Each pair of coordinates (xs ,ys) and (xr ,yr) (matched entries) can be used together with (4) and
(5) to represent two lines in the parameters space α x Tx x Ty
Image Alignment (2)
Initial parameters estimation
Matched keypoints:
Image Alignment (3)
 yx TT
~
,
~
,~The initial estimation of is obtained (using a voting procedure) in
corrispondence of the bin with the maximum number of intersection
between lines generated in corrispondence of matched keypoints by using
(4) and (5).
hs hr
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 

xT
yT
Initial parameters estimation

15. 31/05/2017
15
Image Alignment (4)
Prefiltering (to discard ouliers)
For each triplet of the quantized parameter space, we consider only
filtered matchings.
 yx TT ,,
Differences between
dominant orientations
of matched entries
We consider only those
matching pairs such that:
Matched keypoints:
hs hr
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
In our experiment we
consider only matchings
with a small initial error by
setting tα = 3.5
Algorithm for initial parameters estimation
Image Alignment (5)
It is worth noting that the proposed
registration method, by combining
equations of similarity transformation,
obtains a considerable reduction of
computational complexity and memory
usage.
The combination of the equations
involved into the similarity transformation
model allows us to use a 3D histogram to
estimate four parameters instead of a 4D
histogram as in the case of a naive
implementation based on classic Hough
Transform.

16. 31/05/2017
16
for each pair belonging to the
selected bin we consider the following traslation
vectorshs hr
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
Image Alignment (6)
Refinement
Taking into account the following equations
Final traslation vector estimation:
Only matchings which have generated the lines
intersecting into the selected bin are exploited
(blue).
For each pair belonging to the
selected bin we consider the following equations:
hs hr
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
Image Alignment (7)
Refinement
Final orientation angle estimation:
where and are the ones in the
equations (6) and (/).
Only matchings which have generated the lines
intersecting into the selected bin are exploited
(blue).

17. 31/05/2017
17
hs hr
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
Image Alignment (8)
Refinement
Only matchings which have generated the lines
intersecting into the selected bin are exploited
(blue).
Final scale factor estimation:
For each pair belonging to the
selected bin we consider the following equations:
Once is obtained (see the previous slide) the
following equation (derived from (8) and (9) by
considering (6) and (7) is used to estimate
So, taking into account the pairs belonging to
the selected bin, as well as the quantized
parameter vector obtained from the
initial estimation by voting procedure, we
obtain the final registration parameters:
hs hr
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
Image Alignment (9)
Refinement
Only matchings which have generated the lines
intersecting into the selected bin are exploited
(blue).

18. 31/05/2017
18
Experimental Dataset (1)
In order to cope with scene variability the tests have been performed
considering a subset of the fifteen scene category benchmark dataset
[LSP06] and the dataset DBForgery 1.0 [BM09].
[LSP06] S. Lazebnik, C. Schmid, and J. Ponce, “Beyond bags of features: Spatial pyramid matching for recognizing
natural scene categories,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2006,
pp. 2169–2178.
[BM09] S. Battiato, G. Messina "Digital Forgery Estimation into DCT Domain - A Critical Analysis", ACM Multimedia
2009, Multimedia in Forensics (MiFor'09), October 2009, Beijing, China.
Ten images have been randomly
sampled from each scene category
(average size: 244x272 pixels).
29 different images tampered with
different parameter settings
(average size: 470x500 pixels)
+
• The training set used in the experiments is built through a
random selection of 179 images from the previous
mentioned datasets.
• The test set consists of 21330 images generated through the
application of different manipulations on the training images.
Experimental Dataset (2)
Operations Parameters
Rotation (α) 3, 5, 10, 30, 45 degrees
Scaling (σ) factor = 0.5, 0.7, 0.9, 1.2, 1.5
Orizontal Traslation (Tx) 5, 10, 20 pixels
Vertical Traslation (Ty) 5, 10, 20 pixels
Cropping 19%, 28%, 36%, of entire image
Tampering block size 50x50
Malicious Tampering block size 50x50
Linear Photometric Transformation (a*I+b)
a = 0.90, 0.95, 1, 1.05, 1.10
b = -10, -5, 0, 5, 10
Compression JPEG Q=10
Seam Carving 10%, 20%, 30%
Realistic Tampering [BM09]
Various combinations of above operations

19. 31/05/2017
19
Image Alignment – Results (1)
Proposed approach
Number of SIFT 15 30 45 60
Unmatched Images 5.18% 1.90% 1.12% 0.83%
Spatial Clustering without with without with without with without with
Mean Error α 1.3826 1.9911 0.8986 0.8627 0.6661 0.6052 0.5658 0.4518
Mean Error σ 0.0462 0.0593 0.0306 0.0302 0.0241 0.0200 0.0208 0.0164
Mean Error Tx 2.7672 3.3191 1.8621 1.9504 1.5664 1.5626 1.4562 1.4227
Mean Error Ty 2.6650 3.2428 1.9409 2.0750 1.7009 1.7278 1.6008 1.5944
Image Alignment – Results (2)
[LW10] W. J. Lu and M. Wu, “Multimedia forensic hash based on visual words,” in Proceedings of the IEEE
International Conference on Image Processing, 2010, pp. 989–992.
[BS11] S. Battiato, G. M. Farinella, E. Messina, and G. Puglisi, “Understanding geometric manipulations of images
through BOVW-based hashing,” in International Workshop on Content Protection & Forensics (CPAF 2011), 2011.
Unmatched Images
Number of SIFT 15 30 45 60
Lu et al. [LW10] 7.87% 2.77% 1.52% 1.16%
Battiato et al. [BS11] 0.86% 0.48% 0.25% 0.08%
Proposed approach without spatial clustering 3.00% 1.35% 0.87% 0.73%
Proposed approach with spatial clustering 2.53% 0.64% 0.18% 0.10%

20. 31/05/2017
20
Image Alignment – Results (3)
[LW10] W. J. Lu and M. Wu, “Multimedia forensic hash based on visual words,” in Proceedings of the IEEE
International Conference on Image Processing, 2010, pp. 989–992.
[BS11] S. Battiato, G. M. Farinella, E. Messina, and G. Puglisi, “Understanding geometric manipulations of images
through BOVW-based hashing,” in International Workshop on Content Protection & Forensics (CPAF 2011), 2011.
Mean Error α
Number of SIFT 15 30 45 60
Unmatched Images 10.99% 3.85% 2.02% 1.56%
Lu et al. [LW10] 7.3311 7.9970 7.8600 7.4125
Battiato et al. [BS11] 3.4372 2.4810 2.4718 1.9581
Proposed approach without spatial clustering 1.1591 0.8206 0.5485 0.4634
Proposed approach with spatial clustering 1.7933 0.8288 0.5735 0.4318
Mean Error σ
Number of SIFT 15 30 45 60
Unmatched Images 10.99% 3.85% 2.02% 1.56%
Lu et al. [LW10] 0.0619 0.0680 0.0625 0.0592
Battiato et al. [BS11] 0.0281 0.0229 0.0197 0.0179
Proposed approach without spatial clustering 0.0388 0.0281 0.0214 0.0183
Proposed approach with spatial clustering 0.0541 0.0287 0.0195 0.0161
Image Alignment – Results (5)
[LW10] W. J. Lu and M. Wu, “Multimedia forensic hash based on visual words,” in Proceedings of the IEEE
International Conference on Image Processing, 2010, pp. 989–992.
[BS11] S. Battiato, G. M. Farinella, E. Messina, and G. Puglisi, “Understanding geometric manipulations of images
through BOVW-based hashing,” in International Workshop on Content Protection & Forensics (CPAF 2011), 2011.

21. 31/05/2017
21
Image Alignment – Results (6)
[LW10] W. J. Lu and M. Wu, “Multimedia forensic hash based on visual words,” in Proceedings of the IEEE
International Conference on Image Processing, 2010, pp. 989–992.
[BS11] S. Battiato, G. M. Farinella, E. Messina, and G. Puglisi, “Understanding geometric manipulations of images
through BOVW-based hashing,” in International Workshop on Content Protection & Forensics (CPAF 2011), 2011.
Malicious Manipulation
Unmatched Images
Number of SIFT 15 30 45 60
Lu et al. [LW10] 90.50% 87.71% 81.01% 73.74%
Battiato et al. [BS11] 68.72% 54.19% 29.61% 9.50%
Proposed approach without spatial clustering 87.15% 86.03% 74.86% 64.25%
Proposed approach with spatial clustering 0% 0% 0% 0%
Mean Error α
Number of SIFT 15 30 45 60
Lu et al. [LW10] 85.6844 79.9884 88.4555 97.4700
Battiato et al. [BS11] 86.9447 92.0451 92.5144 91.8478
Proposed approach without spatial clustering 35.6087 33.6800 42.5111 38.5156
Proposed approach with spatial clustering 1.2458 0.0000 0.0000 0.0000
Mean Error σ
Number of SIFT 15 30 45 60
Lu et al. [LW10] 0.2868 0.2934 0.2920 0.3482
Battiato et al. [BS11] 0.3141 0.3453 0.3505 0.3493
Proposed approach without spatial clustering 0.8249 0.7891 0.9284 0.7706
Proposed approach with spatial clustering 0.0193 0.0005 0.0002 0.0006
Unmatched Images
45 60
20.67% 21.79%
1.68% 1.12%
14.53% 13.97%
0.00% 0.00%
Mean Error α
45 60
14.2848 13.4975
34.3832 36.8093
7.0392 9.6364
0.0000 0.0000
Mean Error σ
45 60
0.0538 0.0564
0.1288 0.1608
0.1141 0.1815
0.0003 0.0004
patch size 50x50 patch size 25x25
Image Alignment – Results (7)
Malicious Manipulation (50x50)
[LW10] W. J. Lu and M. Wu, “Multimedia forensic hash based on visual words,” in Proceedings of the IEEE
International Conference on Image Processing, 2010, pp. 989–992.
[BS11] S. Battiato, G. M. Farinella, E. Messina, and G. Puglisi, “Understanding geometric manipulations of images
through BOVW-based hashing,” in International Workshop on Content Protection & Forensics (CPAF 2011), 2011.
Number of SIFT 45 60
Unmatched Images 92.74% 89.39%
Mean Error α σ α σ
Lu et al. [LW11] 81.1994 0.2750 88.2215 0.3126
Battiato et al. [BS10] 96.5480 0.4163 88.3088 0.3058
Proposed approach without spatial clustering 32.3846 0.7285 34.9474 0.6213
Proposed approach with spatial clustering 0.0000 0.0001 0.0000 0.0009

22. 31/05/2017
22
Realistic Tampering (DBForgery1.0) [BM09]
Image Alignment – Results (8)
[BM09] S. Battiato, G. Messina "Digital Forgery Estimation into DCT Domain - A Critical Analysis", ACM
Multimedia 2009, Multimedia in Forensics (MiFor'09), October 2009, Beijing, China.
Proposed approach without sp
Proposed approach with spa
Proposed approach without sp
Proposed approach with spa
Proposed approach without sp
Proposed approach with spa
Proposed approach without sp
Proposed approach with spa
Number of SIF
UnmatchedNumber of SIFT 15 30 45 60
Unmatched 3.45% 0.00% 0.00% 0.00%
Spatial Clustering without with without with without with without with
Mean Error α 0.1071 0.0000 0.0690 0.0000 0.0000 0.0000 0.0000 0.0000
Mean Error σ 0.0065 0.0002 0.0014 0.0002 0.0006 0.0006 0.0003 0.0003
Mean Error Tx 0.5675 0.0171 0.1699 0.0104 0.0106 0.0140 0.0092 0.0130
Mean Error Ty 0.3718 0.0270 0.0283 0.0219 0.0178 0.0172 0.0144 0.0138
Image Alignment – Results (9)
Cropping
Number of SIFT 15 30 45 60
Unmatched 2.05% 0.74% 0.00% 0.00%
Spatial Clustering without with without with without with without with
Mean Error α 1.5494 1.5399 0.2758 0.2720 0.1750 0.1266 0.1695 0.0782
Mean Error σ 0.0416 0.0427 0.0143 0.0089 0.0078 0.0064 0.0088 0.0061
Mean Error Tx 2.3241 2.1522 0.6974 0.6862 0.5512 0.4864 0.5444 0.4527
Mean Error Ty 1.9857 1.9840 0.7036 0.6820 0.5474 0.4773 0.5158 0.4660

23. 31/05/2017
23
The presented results have been obtained
employing the following formulas:
hs hr
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
222 yx
333 yx
444 yx
555 yx
111 yx
111  nnn yx
nnn yx 
Image Alignment – On going work
Refinement: Affine Transformation
Only matchings which have generated the lines
intersecting into the selected bin are exploited
(blue).
feydxy
cbyaxx
isisir
isisir


,,,
,,,
Extention to Affine Trasformation Model
S. Battiato, G. M. Farinella, E. Messina, G. Puglisi, “Robust Image Alignment for Tampering Detection”,
IEEE Transaction on Information Forensics and Security, 2012
Operations Parameters
Anisotropic Scaling (σx or σy) 0.7, 0.9, 1.2
Shear (k) 0.05, 0.1, 0.15
Image Alignment – Results (11)Refinement: Affine Transformation
Unmatched Mean Error α Mean Error σ Mean Error Tx Mean Error Ty
Proposed approach
without spatial
clustering
0.0675
0.4331 0.0155 1.2072 1.3249
Proposed approach
with spatial clustering
0.2287 0.0097 1.0831 1.2095
Proposed approach
with spatial clustering
and affine estimation
0.2076 0.0088 1.2077 1.3144
Proposed approach with spatial clustering and affine estimation
Unmatched 0.0824
Mean Error α 0.2093
Mean Error σx 0.0109
Mean Error σy 0.0069
Mean Error k 0.0274
Mean Error Tx 1.2210
Mean Error Ty 1.2742
S. Battiato, G. M. Farinella, E. Messina, G. Puglisi, “Robust Image Alignment for Tampering Detection”,
IEEE Transaction on Information Forensics and Security, 2012

24. 31/05/2017
24
Alignment - Complexity Comparison
[LW10] W. J. Lu and M. Wu, “Multimedia forensic hash based on visual words,” in Proceedings of the IEEE
International Conference on Image Processing, 2010, pp. 989–992.
[BS11] S. Battiato, G. M. Farinella, E. Messina, and G. Puglisi, “Understanding geometric manipulations of images
through BOVW-based hashing,” in International Workshop on Content Protection & Forensics (CPAF 2011), 2011.
ProposedApproach: Tampering
Tampering Detection
Hash for
tampering
Hash for
tampering
=
?
Alignment
•Trasformation Model
•Voting procedure (with
Pre-Filtering) into the
space of parameters.
•Scale estimation on
realiable information

25. 31/05/2017
25
Tampering Detection (1)
Representation of each block is based on histogram of oriented gradients (HOG)
magnitudo
orientation
Finally, the histogram is normalized and quantized.
(ϑ1,ρ1)
(ϑ2,ρ2)
:
:
(ϑL,ρL)
4 bins
For each pixel of the block the
magnitudo ρ and the
orientation θ are computed
Create the gradient
image and divide it into
blocks of 32x32 pixels
Each pixel of the block
votes for a bin using its
magnitude
Tampering Detection (2)
Each block representation is part of the image signature, so it has to be as small as
possible. We compared two different solutions:
Uniform quantization Non-uniform quantization
Each bin is quantized using a
fixed number of bits. In our
tests, we used 3 bits. So, 12
bits are required to encode a
single histogram.
Sequence of 12 bits
3bits
For each image with N
blocks, 12*N bits are needed
to encode the hash.
It uses a precomputed shared vocabulary of
histograms of oriented gradients, making a
clustering (through k-means) considering all the
histogram of gradients extracted from the whole scene
category dataset.
. . .(1) (2) (3) (k)
The sequence of ids is the image hash representation.
In this case the histogram centroids are not quantized.
Each block is hence associated to an ID
corresponding to the closed centroid.
[id1,id2,…idk]

26. 31/05/2017
26
(1)
Tampering Detection (3)
Similarity
Tampering Detection – Results (1)
[LW10] W. J. Lu and M. Wu, “Multimedia forensic hash based on visual words,” in Proceedings of the IEEE
International Conference on Image Processing, 2010, pp. 989–992.
[BS11] S. Battiato, G. M. Farinella, E. Messina, and G. Puglisi, “Understanding geometric manipulations of images
through BOVW-based hashing,” in International Workshop on Content Protection & Forensics (CPAF 2011), 2011.

27. 31/05/2017
27
Tampering Detection – Results (2)
Conclusions
• The assessment of the reliability of the content of an image received
through the Internet is an important issue in nowadays society.
• Image alignment is a fundamental step to perform tampering detection
and further analysis which are commonly used to establish the integrity of
a received image.
• In this work a robust image registration component which exploits an
image signature based on the Bag of Features paradigm and voting
procedure into the parameter space of the employed tranformation model
has been proposed.
• The proposed image hash encodes the spatial distribution of features to
better deal with highly texturized and contrasted patches.
• Comparative tests performed on a representative dataset show that the
proposed approach outperforms recently appeared techniques by
obtaining a significant margin in terms of registration accuracy and
tampering detection.

28. 31/05/2017
28
Future Works
should concern :
• a more in depth analysis to establish the minimal number of SIFT
needed to guarantee an accurate estimation of the geometric
transformations
• other ad-hoc attacks (which conceal true local features)
Recent works
• Cai-Ping Yan et al. - Multi-scale image hashing using adaptive local feature
extraction for robust tampering detection -Signal Processing Volume 121, April
2016, Pages 1–16
• Abstract. The main problem addressed in this paper is the robust tampering
detection of the image received in a transmission under various content-preserving
attacks. To this aim the multi-scale image hashing method is proposed by
using the location-context information of the features generated by adaptive
and local feature extraction techniques. The generated hash is attached to the
image before transmission and analyzed at destination to filter out the geometric
transformations occurred in the received image by image restoration firstly. Based
on the restored image, the image authentication using the global and color hash
component is performed to determine whether the received image has the same
contents as the trusted one or has been maliciously tampered, or just different. After
regarding the received image as being tampered, the tampered regions will be
localized through the multi-scale hash component. Lots of experiments are
conducted to indicate that our tampering detection scheme outperforms the existing
state-of-the-art methods and is very robust against the content-preserving attacks,
including both common signal processing and geometric distortions.

29. 31/05/2017
29
Recent works
• Ouyang, J., Liu, Y. & Shu, H. Robust hashing for image
authentication using SIFT feature and quaternion Zernike
moments - Multimed Tools Appl (2017) 76: 2609.
doi:10.1007/s11042-015-3225-x
• Abstract. A novel robust image hashing scheme based on
quaternion Zernike moments (QZMs) and the scale invariant
feature transform (SIFT) is proposed for image authentication.
The proposed method can locate tampered region and detect the
nature of the modification, including object insertion, removal,
replacement, copy-move and cut-to-paste operations. QZMs
considered as global features are used for image authentication while
SIFT key-point features provide image forgery localization and
classification. Proposed approach performance were evaluated on the
color images database of UCID and compared with several recent
and efficient methods. These experiments show that the proposed
scheme provides a short hash length that is robust to most common
image content-preserving manipulations like large angle rotations,
and allows us to correctly locating forged image regions as well as
detecting types of forgery image.
Prof. Sebastiano Battiato
Dipartimento di Matematica e Informatica
University of Catania, Italy
Image Processing Lab - http://iplab.dmi.unict.it
battiato@dmi.unict.it
Contact Information