Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Perceptual image distortion


Published on

Published in: Technology, Art & Photos
  • Be the first to comment

Perceptual image distortion

  1. 1. Perceptual Image Distortion Patrick Teo and David Heeger Stanford UniversityFirst IEEE International Conference on Image Processing, v2, pp 982-986, November 1994 1
  2. 2. Distorted Images with SimilarMean Squared ErrorsMany imaging and image processingmethods are evaluated by how wellthe images they output resemblesome given image. Examples include:image data compression, ditheringalgorithms, flat-panel display andprinter design. In all of these cases, thehuman visual system is the judge ofimage fidelity. Most of these methodsuse the mean squared error (MSE) orroot mean squared error (RMSE)between the two images as a measureof visual distortion. These measuresare popular largely because of theiranalytical tractability. It has long been A commonly used image of Albert Einstein in image processing.accepted that MSE (or RMSE) isinaccurate in predicting perceiveddistortion. This is illustrated in thefollowing paradoxical example. 2
  3. 3. Similar Mean Squared Errors (cont’d) The top two images on the right were created by adding different types of distortions to the original image; the original image is shown below them.Root Mean Squared Error of 8.5 The root mean squared error (RMSE) between each of the distorted images and the original were computed. The root mean squared error is the square root of the average squared difference between every pixel in the distorted image and its counterpart in the original image. The RMSE between the first distorted image and the original is 8.5 while theRoot Mean Squared Error of 9.0 RMSE between the second distorted image and the original is 9.0. Although the RMSE of the first image is less than that of the second, the distortion introduced in the first image is more visible than the distortion added to the second. Thus, the root mean squared error is a poor indicator of perceptual image fidelity. Original 3
  4. 4. A Computational Model ofPerceptual Image DistortionWe have developed a perceptual distortion measure based on a modelof spatial pattern detection.It is important to recognize the relevance of these empirical spatialpattern detection results to developing measures of image integrity. In atypical spatial pattern detection experiment, the contrast of a visualstimulus (called the target) is adjusted until it is just barely detectable.Threshold contrasts of the target are measured over a range of spatialfrequencies, mean luminance, and spatial extents.In some experiments (called contrast masking experiments), the targetis also superimposed on a background pattern (called the masker). Inother experiments (called luminance masking experiments), the targetis superimposed on a brief, bright, uniform background. In either case(contrast or luminance masking), the contrast of the target is adjusted(while the masker is held fixed) until the target is just barely detectable.Typically, a target is harder to detect (i.e., a higher contrast is required)in the presence of a masker.A model that predicts spatial pattern detection is obviously useful inimage processing applications. In the context of image compression, forexample, the target takes the place of quantization error and themasker takes the place of the original image. 4
  5. 5. Perceptual Image Distortion Our model consists of three main parts: a retinal component, a cortical component, and a detection mechanism. The retinal component is responsible for contrast sensitivity and its dependence on mean luminance masking. The cortical component accounts for contrast masking. To compute perceptual image distortion, the reference and distorted images are passed through these two stages of the model independently. At this point, the images have been normalized for the differential sensitivities of the human visual system. The final (detection mechanism) component of the modelA model of perceptual image distortion. compares these two normalized images to give a measure of image fidelity. The final result is an image representing the probability of perceiving a distortion at each position in the distorted image. 5
  6. 6. Model Predictions ofVisible DistortionThe top image on the right is theoriginal image. The two imagesdirectly below it were created byadding different types of distortions tothe original image. The root meansquared error (RMSE) between the leftdistorted image and the original (8.5)is smaller than the root mean squarederror between the right distortedimage and the original (9.0). In spite ofthat, the distortion is more visible inthe left image.The images directly below eachdistorted image are the predictions ofthe perceptual image distortionmodel. Lighter areas indicate regionswhere the distortion is more visiblewhile darker areas indicate regionswhere the distortion is less visible. Themodel correctly predicts that the leftdistorted image is more visiblydistorted than the right. Model predictions: lighter areas indicate greater visible errors. 6
  7. 7. Visible Distortion in JPEG Compressed Images To further validate the models performance, we applied the model to JPEG compressed images. The original image was compressed using the JPEG algorithm at different quality settings. The model was then used to predict the visibility of the distortion between each compressed image and the original. The pairs of images on the left are images compressed using the JPEG algorithm along with the models predictions of the amount of visible distortion when compared with theJPEG qual. setting = 80, RMSE = 9.5, PDM = 1.2 original. The image compressed at a quality setting of 80 is virtually indistinguishable from the original. The models prediction corroborates this observation. The average distortion value computed by the model is 1.2, which indicates that the distortion is slightly above threshold (threshold is set at 1.0). 7
  8. 8. JPEG Compressed Images(cont’d)The image compressed at a qualitysetting of 20 is slightly deteriorated JPEG qual. setting = 20, RMSE=11.4, PDM = 5.7while the image compressed at aquality setting of 10 shows markedblocking artifacts. The modelspredictions agree with these trendsfairly well. JPEG qual. setting = 10, RMSE = 12.9, PDM = 9.8 8
  9. 9. Error Histograms of JPEG Compressed Images The top graph plots a histogram of the squared error differences for individual pixels. The bottom graph plots a histogram of the perceptual distortion predictions of the model for individual pixels. Both histograms have been normalized so that the vertical axis represents fractions of the total number of pixels. The histograms of squared error differences for the different compressed images are very similar to one another. The histograms of perceptual distortion predictions of the different compressed images are dramatically different from one another. It is clear, for example, that the model predicts that the images compressed at quality settings of 20 and 10 (the "middle" and "right" images) are more severely distorted than the image compressed at a Histograms of Squared Error (top) and Perceptual quality setting of 80 (the "left" image).Distortion Measure (bottom) predicted by the model forthe 3 JPEG compressed images: “left” (JPEG quality 80), “middle” (JPEG quality 20), “right” (JPEG quality 10). 9