Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Digital Image Forensics: camera fingerprint and its robustness


Published on

Politecnico di Milano

Published in: Education

Digital Image Forensics: camera fingerprint and its robustness

  1. 1. Polo Regionale di Como Facoltà di Ingegneria dell’Informazione Laurea Magistrale in Ing. InformaticaCamera Fingerprint and its robustness12 gennaio 2010Francesco ForestieriGiorgio Zinetti
  2. 2. 1. Forensics Overview2. Application of Image Forensics3. Image Acquisition4. Type os Image Sensor5. Camera Fingerprint6. PRNU7. PRNU Robustness8. Conclusions9. References 2
  3. 3. Forensic science is the application of a broadspectrum of sciences to answer questions of interest to a legal system. This may be in relation to a crime or a civil action. In modern use, the term "forensics" in the the place of "forensic science" can be considered correct as term "forensic" is effectively a synonym for "legal" or "related to courts". 3
  4. 4. Digital forensics (sometimes Digital forensic science) is a branch Forensic scienceencompassing the recovery and investigation of material found in digital devices, Digital Forensics Computer Network Multimedia Forensics Forensics Forensics Image Video Audio 4
  5. 5. Digital ForensicsDIGITAL IMAGE FORENSICS Computer Network Multimedia Forensics Forensics ForensicsFor a digital image: Image Video Audio• Wich camera brand took the picture?• Is it forgered or manipulated?• How it is capturede? Digital camera? Digital scanner o camcorder?• Is a computer graphic rendering? In a court a digital images and videos are not easily acceptable because Is difficult to establish their integrety. To avoid this problem we can use: 5
  6. 6. Forgery Identification Is it real ? 6
  7. 7. Fake photo Original photo 7
  8. 8. 8
  9. 9. Estimate geometrical processing Visual representation of the detected cropping and scaling parameters . The gray frame shows the original image size, while the blue frame shows the image size after cropping and before resizing. 9
  10. 10. Distinction between scan and digital imageIllegal copy of a digital content:• Cinema recaptured video by a camcorder• Scannered book 10
  11. 11. Device Identification Device Identification • Which camera brand took this photo? • Is a natural image or a computer grapich rendering? Device Fingerprint Linking Matching 11
  12. 12. Photo-Response NonUnifomity (PRNU) is an intrinsic property of all digital imagingsensor due to slight variations among indivisual pixels in their ability to convertphotons to electorns. Consecuently every sensor cast a weak noise-like pattern ontoevery image it takes and this pattern play the role of sensor fingerprint. Human Fingerprint Camera Fingerprint 12
  13. 13. Device LinkingTo prove that two images were taken by the same device Deviceor to know which camera brand took the photo. Identification Device Fingerprint Linking Matching 13
  14. 14. Fingerprint matchingCorresponds to the situation in which we need to Device Identificationdecide whether or not two estimates of twopotentially different fingerprints are identical.The presence of camera fingerprint in an image is Device Fingerprintalso indicative of the fact that the image under Linking Matchinginvestigation is natural and not a computer rendering Is it a natural photo? 14
  15. 15. The process of acquiring a digital image is quite complex and varies greatly acrossdifferent camera models, some basic elements are common to most cameras. light …0101011010100101… 15
  16. 16. LightProjectionAmplification and 1. The light cast by the camera optics is Storing quantization of projected onto the pixel grid of the energy imaging sensors. Sharpening Signal Adjusting Color and gamma Interpolation correction Colors and demosaiking 16
  17. 17. LightProjectionAmplification and 2. The charge generated through interaction Storing quantization of of photons with silicon is amplified and energy quantized Sharpening Signal Adjusting Color and gamma Interpolation correction Colors and demosaiking 17
  18. 18. LightProjectionAmplification and 3. The signal from each color channel is Storing quantization of adjusted for gain (scaled) to achieve proper energy white balance. Sharpening Signal Adjusting Color and gamma Interpolation correction Colors and demosaiking 18
  19. 19. LightProjectionAmplification and 4. Because most sensors cannot register Storing quantization of color, the pixels are typically equipped with energy a color filter that lets only light of one specific color (red, green, or blue) enter the pixel. The array of filters is called the Sharpening color filter array (CFA) Signal Adjusting Color and gamma Interpolation correction Colors and demosaiking 19
  20. 20. LightProjectionAmplification and 5. To obtain a color image, the signal is Storing quantization of interpolated or demosaicked. energy Sharpening Signal Adjusting Color and gamma Interpolation correction Colors and demosaiking 20
  21. 21. LightProjectionAmplification and 6. Then the colors are further adjusted to Storing quantization of display correctly on a computer monitor energy through color correction and gamma correction Sharpening Signal Adjusting Color and gamma Interpolation correction Colors and demosaiking 21
  22. 22. LightProjectionAmplification and 7. Cameras may also employ filtering Storing quantization of techniques, such as denoising or energy sharpening. Sharpening Signal Adjusting Color and gamma Interpolation correction Colors and demosaiking 22
  23. 23. LightProjectionAmplification and 8. Finally, the image is stored in the JPEG or Storing quantization of some other format, which may involve energy quantization. Sharpening Signal Adjusting Color and gamma Interpolation correction Colors and demosaiking 23
  24. 24. Image sensors are devices that convert light in electical signals which will be trasformedin bits that are the costituents of a digital image.In the digital camera world there are basically two types of digital sensors: (Charge-Coupled Devices)Currently the most commonly used image sensor, CCDs capture light onto an array oflight-sensitive diodes, each diode representing one pixel. 24
  25. 25. (Complementary Metal Oxide Semiconductor)Like CCDs, these imagers are made from silicon, but as the name implies, the processthey are made in is called CMOS. This process is today the most common method ofmaking processors and memories, meaning CMOS Imagers take advantage of theprocess and cost advancements created by these other high-volume devices. 25
  26. 26. Due to both design and manufacturing considerations, there are a number ofadvantages that CMOS Imagers have over CCD:• Integration• Manufacturing Cost CMOS CCD inexpensive because CMOS expensive to produce wafers are used for many because of special Cost different types of manufacturing methods semiconductors employed low power consumption consumes upto 100x more Power power than CMOS Noise susceptible to noise high quality, low noise images other circuitry easily technically feasible; other Extended Functionality incorporated on same chip chips are used Fill Factor high low 26
  27. 27. The noise in a taken picture in caused by shot noise and pattern noise.This first one is unavoidable noise caused by fluctuating photons, the latter onestays almost the same when pictures are taken in the same scene and over time.This is why pattern noise can be used for identication. Noises Shot Pattern Noise Noise PRNU FPN 27
  28. 28. pixel indices have been dropped for better readabilityI[i] the quantized signal registered at pixel iY[i] is the incident light intensity at pixel iQ is the combined distortion due to quantization and/or JPEG compressiong is the gain factor (different for each color channel) and is the gammacorrection factor (typically < 0.45) is a combination of the other noise sources The matrix K is a zero-mean noiselike signal responsible for the PRNU K is the sensor fingerprint. 28
  29. 29. In parts of the image that are not dark, the dominant term in the square bracket inis the scene light intensity, Y.By factoring it out and keeping the first two terms in the Taylor expansion ofwe obtainwhere is the ideal sensor output in the absence of any noise or imperfections Is the PRNU term that absorbs also is the modeling noise 29
  30. 30. Device Linking Device Identification Calculate the camera Device Fingerprint reference pattern Linking Matching Calculate the noise of an image Find out the correlation between camera reference pattern and image noise 30
  31. 31. Fast Way: Calculate the camera reference pattern• Average multiple images (approximation)To speed up this process:• Remove the scene content using a denoising filter• Subtract the denoised images with the original one• Average the noise residual Hints: • Denoising filter can be: Median Filter or Wavelet-Based filter • Is useful also average uniformly images like photos of a white paper or a white wall. • The larger the number of images over, the more we suppress random noise and the camera reference pattern is accurate. 31
  32. 32. A Camera ReferencePattern that we havecalculate with MatLab. 32
  33. 33. Apply a denoise filter like before Calculate the camera reference patternSubtract the denoised image with the original oneWe obtain the noise of the image Calculate the noise of an imageWhile the PRNU is unique to the sensor, the other artifacts like colorinterpolation and JPEG compression are shared among cameras of the samemodel or sensor design.Subtracting the image denoised with the original image allow us to suppressall other artifcats and avoid false identification rate. 33
  34. 34. Finding out the Calculate the camera correlation between reference pattern camera reference pattern and the noise of our image we can link the device which takes Calculate the noise of an that photo with the image photo itself. Find out the correlation between camera reference pattern and image noiseWe obtain a Threshold of acceptance and a false identification rate 34
  35. 35. The factor K is thus a very useful forensic quantity, responsible for a uniquesensor fingerprint with the following important properties: • Dimensionality: the fingerprint is stochastic in nature and has a large information content, which makes it unique to each sensor. • Universality: all imaging sensors exhibit PRNU. • Generality: the fingerprint is present in every picture independently of the camera optics, camera settings, or scene content, with the exception of completely dark images. 35
  36. 36. • Stability: It is stable in time and under a wide range of environmental conditions (temperature, humidity, etc.).• Robustness: it survives lossy compression, filtering, gamma correction, and many other typical processing procedures. 36
  37. 37. Remove PRNU Methods: • Destroying the PRNU • Removing the PRNU • Forging the PRNU • Other methods 37
  38. 38. Destroying the PRNU (1)Adding random noise to the picture • Changing Least Significant Bit (LSB) 10100101 10100101 10100101 38
  39. 39. Destroying the PRNU (2) Blurring At least using a 4x4 matrix Sharpening To get the details back into the picture 39
  40. 40. Example of destroying (1) Original Blur4x4 + Sharp3x3 40
  41. 41. Example of destroying (2) Original Blur5x5 + Sharp5x5 41
  42. 42. Removing the PRNU (1) A dark frame (B) is a single picture which was taken with the shutter closed or the lens cap still on. This dark frame consists of the fixed-pattern noise (FPN); noise from the sensor itself, like dead or hot pixels. Subtracting this dark frame from the original input picture (I) will reduce the inherent noise in the picture, and thus, improving the pictures quality. 42
  43. 43. Removing the PRNU (2) FlatField (FF) correction is done by taken multiple pictures (N) that are taken of a, mostly white, at surface. After averaging all the pixels with their neighbours (depicted in the second equation), which essentially blurs the FlatField, the pixels can be compared to all their neighbours to see their individual deviation. These small deviations are then used to correct the entire picture, getting rid of further sensor defects present in the picture. Finding the right (n) can be tricky as it partially depends on the strength of the PRNU. 43
  44. 44. Example of removing Original PRNU removed 44
  45. 45. Forging the PRNU First the original PRNU pattern is removed just like in the removing formula, secondly a new pattern of the target camera is added as the second part of the forge formula. 45
  46. 46. Example of Forging Original PRNU Forged 46
  47. 47. Other Methods Rotating a picture by a few degrees would also rotate the pattern and thus make matching not possible. Flipping also flips the pattern so this also works, however, this might be easily noticeable if there is any text or well-known scenery in the picture. Scaling the picture, due to the new and dierent size of the picture it cannot be compared to any original picture from possibly the same camera. 47
  48. 48. 48
  49. 49. 49
  50. 50. Matching original pictures with multiple cameras.Camera four is the original. 50
  51. 51. Matching pictures with PRNU noise pattern destroyed,using a blur 4x4 and a sharp 3x3 matrix.Camera four is the original. 51
  52. 52. Matching pictures with PRNU noise pattern destroyed,using a blur 5x5 and a sharp 3x3 matrix.Camera four is the original. 52
  53. 53. Matching pictures with PRNU noise patterndestroyed, using a blur 5x5 and a sharp 5x5 matrix.Camera four is the original. 53
  54. 54. Matching pictures with PRNU noise pattern removed,using 10 at-elds.Camera four is the original. 54
  55. 55. Matching pictures with PRNU noise patternremoved, using 30 at-elds.Camera four is the original. 55
  56. 56. removed the PRNU noise pattern with 30 ateld pictures from theoriginal camera and then forge the PRNU noise pattern with both adierent camera and the original camera. 56
  57. 57. Forging the PRNU noise pattern, using 30 flatfield to remove and to forge. 57
  58. 58. The answer is a genuine: yes.ConsequencesEvery method that increases anonymity can, and mostly will, be abused forillegal or controversial activities.It becomes even more interesting when we look at forging a pattern where weimpose that a picture is taken with a different digital camera.The validity of evidence in that case decreases even more. 58
  59. 59. Jessica Fridrich a professor of electrical and computer engineering at Binghamton University (SUNY). She received her Ph.D. in systems science from Binghamton University in 1995 and her M.S. in applied mathematics from Czech Technical University in Prague in 1987.Her main interests are in steganography, steganalysis, digitalwatermarking and digital image forensics.Since 1995, she received 18 research grants; most for projects ondata embedding and steganalysis that lead to more than 80 papersand seven U.S. patents. She is a Member of the IEEE and ACM. • Digital Image Forensics 59
  60. 60. Digital Image Forensics and PRNU• Source Digital Camcorder Identification Using Sensor Photo Response Non-Uniformity• Imaging Sensor Noise as Digital X-Ray for Revealing Forgeries• Digital Camera Identification from Sensor• Determining Image Origin and Integrity Using Sensor NoiseCCD and CMOS••••• 60
  61. 61. Thanks for your attention 61