Btp viewmorph

327 views
275 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
327
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • The two images are first represented as meshes. Then corresponding feature points are selected Using these feature points the two images are mapped from source to destination.
  • There are two types of interpolation available – cubic and biharmonicThese are (cubic) used to find the smoothest curve or surface that passesThrough a set of irregularly spaced data points.
  • The mapping is applied incrementally to shift the features in the original image to those in the final image and vice versa.
  • Btp viewmorph

    1. 1. Face Recognition Using View Morphing
    2. 2. Motivation• View morphing has tremendously been used in special effects.• It allows us to combine the salient features of different images into one.• In surveillance and security systems, where cameras are used to capture the footage of people coming in and out. It is tough to capture the perfect frontal face of any person.
    3. 3. Motivation• We often get faces from certain angle.• In such scenarios it becomes very difficult to perform face recognition on all the faces seen in the footage and get the correct results.
    4. 4. Motivation• Face recognition technology is rapidly growing as evidenced by the new surveillance system.• Here is an example of ‘surveillance system that can recognize a face from 36 million other faces in one second’, from Hitachi.• It processes faces as it records and keep a pre indexed gallery of faces rather than a messy footage.• Super speed is achieved as it compares the faces in real time Source : Hitachi Face Recognition System in Surveillance Systems by pre grouping faces with http://mashable.com/2012/03/23/hitachi-face-recognition/ similar features, so that it can narrow down the field of search quickly.
    5. 5. Motivation• Limitations – The software assumes that people are looking right into the camera or facing not more than 30 degrees of the centre. – Also it requires the people to be close to the camera. Anything smaller than 40 x 40 pixel square face is not recognizable at all.
    6. 6. Research Problem Can we use View Morphing in Recognition systems ?• As we see, for any surveillance system we need a frontal face to perform recognition.• We investigate through our study that whether we can generate a face given the footage images which can be used for face recognition.
    7. 7. Research Problem• Primary focus of the research was to be able to perform view morphing on face images with different poses and expressions to generate intermediate pose and expression images which capture all the salient features of the face.
    8. 8. Datasets Used• For pose variant images we used MULTI PIE database. Sample images from the database
    9. 9. Dataset Used• For expression variant images we used JAFFE database. Sample images from the database
    10. 10. Introduction• View morphing is a technique used for transforming one image into another.• Hollywood films and Disney uses morphing for effects and animation.• Algorithms has been used to generate morphing between images of face of different people as well as between images of face of individuals.• Warping and Cross Fading are the two techniques involved in any kind of morphing.
    11. 11. Introduction• Cross Fading is simple but does not produce smooth transitions, major problem is to warp.• Cross Fading generate a series of frames that are linear combinations of the original images. Example of cross fading, Source : http://odonnell- wiki.cs.uchicago.edu/inde x.php/TenStepCrossfade
    12. 12. Introduction• For smoother transitions it is necessary to use warping.• Warping transforms shape where as cross dissolve transform color only color.• We will now demonstrate the two different approaches that were taken during the project involving different kinds of warping and cross fade techniques.
    13. 13. Approach 1For Pose Variant Face Images
    14. 14. Feature Based Field Morphing Outline of the processInput Image 1 Input Image1& & image 2 image 2Pre Extracting Coordinate CrossProcessing Features Transform Dissolve
    15. 15. Feature Based Field Morphing• Pre Processing – Estimating Fundamental Matrix (F) using 8 Point Algorithm. – Finding Eigen vectors of F , the epipoles. – Rotate the epipolar vectors and compute the transform. – Apply the transform to get the prewarped image.
    16. 16. Estimating F using 8-point algorithm• To calculate F, the user first inputs corresponding feature points from both the images.• These points are used to perform a mapping between the two images.
    17. 17. Estimating F using 8-point algorithm• The fundamental matrix F is defined by x Fx 0 for any pair of matches x and x’ in two images. f11 f12 f13• Let x=(u,v,1)T and x’=(u’,v’,1)T, F f 21 f 22 f 23 f 31 f 32 f 33 each match gives a linear equationuu f11 vu f12 u f13 uv f 21 vv f 22 v f 23 uf31 vf 32 f 33 0
    18. 18. 8-point algorithm f11 f12 f13u1u1´ v1u1´ u1´ u1v1´ v1v1´ v1´ u1 v1 1 f 21u 2 u 2 ´ v2 u 2 ´ u 2 ´ u 2 v2 ´ v2 v2 ´ v2 ´ u 2 v2 1 f 22 0          f 23u n u n ´ vn u n ´ u n ´ u n vn ´ vn vn ´ vn ´ u n vn 1 f 31 f 32 f 33• In reality, instead of solving Af 0, we seek f to minimize Af , least eigenvector of A A.
    19. 19. 8-point algorithm – Problem?• F should have rank 2• To enforce that F is of rank 2, F is replaced by F’ that minimizes F F subject to the rank constraint.• This is achieved by SVD. Let F UΣV, where 1 0 0 1 0 0 Σ 0 2 0 , let Σ 0 2 0 0 0 3 0 0 0 then F UΣ V is the solution.
    20. 20. Image Transformation• After Computing F we now have variables that can be used to transform the image and compute the pre-warped image.• Following is an example from the experiment. The first two images are the original images with features marked for estimating F. The last two images are the resultant pre-warped images.
    21. 21. Feature Morphing Algorithm• Step 1 – Input corresponding feature LINES. – For every line PQ in source image there is a matching line P’Q’ in the destination image.
    22. 22. Feature Morphing Algorithm• Step 2
    23. 23. Multiple Pair of Lines: Problem?
    24. 24. Step 3: Algorithm
    25. 25. Feature Morphing• Step 4 – Cross dissolve. – First image fade out while second image fade in. Blend = (1-t) * Source + t * Destination , 0 <= t<= 1
    26. 26. Results
    27. 27. Results
    28. 28. Results
    29. 29. Approach 2For Expression Variant Face Images
    30. 30. Mesh warping• Mesh warping algorithms relates features with non-uniform mesh in the source and destination images.• The original images are first broken down into small regions then these regions are mapped onto each other to produce a morph.
    31. 31. Mesh WarpingSource : Davis.wpi.edu/~matt/courses/morph
    32. 32. Mesh Warping Algorithm• Two step process – In the first step, we generate the mapping by the process of scattered data interpolation. The interpolation technique that we have attempted to use is the biharmonicsplineinterpolation, built in the Matlab Programming Environment. – In the second step, the mapping is applied incrementally to shift the features from the first image to the second and vice versa. – The two sequences are combined via a linear cross fade to form the final morph.
    33. 33. Process of Generating Mapping Arrays1. Input 2 arrays of size nx 2 containing the coordinates of n corresponding points in the initial and final images: Riand Rf.2. Compute the difference array Subtract Ri fromRfand split the resultant array into column vectors [Hx, Hy] = Rf - Ri.3. Now create 2 arrays equal in size to the original image: Mx and My.4. Create data points in the arrays Mx and My using Hx and Hy 1. Mx(Rx,i, Ry,i) = Hxi 2. My(Rx,i, Ry,i) = Hyi3. Now Apply the interpolation (cubic or biharmonicspline) to assign values in Mx and My.4. Save Mx and My.
    34. 34. The Interpolation Process1. Input number of frames over, which morph should occur, N+1. Also, mapping arrays Mx and Myand initial and final images Ii and If.2. Create array Fn of equal size to images.3. Define ‘frame weighting’W = (n−1)/(N-1)W’= (1-W)4. For all values of (x,y) within Fn. 1. Define a = Mx(x,y)*W ,b= My(x,y)*W 2. Set Fn(x,y) = W*I(x+a,y+b) + W’*I(x-a,y-b)5. Compile all the frames from F1to Fn.
    35. 35. Interface• The Matlab program asks the user to first select the two input images.
    36. 36. Interface• The user then selects the critical points between the two images and saves them into respective .mat files.
    37. 37. Resultant imagesThe images on the top left corner andbottom right corner are nearly theoriginal images.30 intermediate frames weregenerated between the two faces withdifferent expressions.The images shows 12 of the 30intermediate frames.
    38. 38. Resultant imagesThe images on the top left corner andbottom right corner are nearly theoriginal images.30 intermediate frames weregenerated between the two faces withdifferent expressions.The images shows9of the 30intermediate frames.
    39. 39. Resultant imagesThe images on the top left corner andbottom right corner are nearly theoriginal images.30 intermediate frames weregenerated between the two faces withdifferent expressions.The images shows9of the 30intermediate frames.
    40. 40. Experiments – Face Recognition• 4 experiments were performed on the resultant images.• For face recognition we used Local Binary Pattern and Chi Square Distance measure.• LBP is a feature vector used for classification in computer feature.• It divides the image into cells and computes a 8 digit binary number for each cell using the neighboring cells.• A histogram over the cells are computed using the frequency of the each number occurring.
    41. 41. Experiment 1Field Morphing Case 1Scenario Before MorphingData Set Multi PieNumber of Subjects in gallery 30Number of images per subject 2Difference of PoseAngle difference 30 degreesNumber of intermediate images 0generatedTotal number of images in gallery 60Number of test images 30Number of correct matches 22Percentage Accuracy 73.3
    42. 42. Experiment 2Field Morphing Case 2Scenario After MorphingData Set Multi PieNumber of Subjects in gallery 30Number of images per subject 2Difference of PoseAngle difference 30 degreesNumber of intermediate images 20generatedTotal number of images in gallery 600Number of test images 30Number of correct matches 20Percentage Accuracy 66.6
    43. 43. Result of Face RecognitionAfter morphingTop row shows an example ofmismatchBottom row shows anexample of correct match.
    44. 44. Experiment 3Mesh Warping Case 1Scenario Before MorphingData Set JAFFENumber of Subjects in gallery 10Number of images per subject 20Difference of ExpressionAngle difference 0Number of intermediate images 0generatedTotal number of images in gallery 200Number of test images 20Number of correct matches 15Percentage Accuracy 75
    45. 45. Experiment 4Mesh Warping Case 2Scenario After MorphingData Set JAFFENumber of Subjects in gallery 10Number of images per subject 20Difference of ExpressionAngle difference 0Number of intermediate images 30generatedTotal number of images in gallery 300Number of test images 20Number of correct matches 12Percentage Accuracy 60
    46. 46. Result of Face RecognitionAfter MorphingTop row shows an example ofmismatchBottom row shows anexample of correct match.
    47. 47. Analysis – Feature Morphing1. All the salient features of the face were restored in all the frames.2. There were few distortions but those were not important if we see from a face recognition point of view.3. While performing warping, the shape and size of the images change, due to which the LBP features of the similar frames change, leading to low accuracy.4. Also the test images have to warped differently using one of the gallery image so that the feature vector can be compared, which can lead to another variant of shape.5. LBP uses unwanted information from the images to compute the feature vector.
    48. 48. Analysis – Mesh Warping• For JAFFE database and mesh warping algorithm there was no such problem of shape and size of the images.• The only anticipated reason for reduced accuracy was the not so perfect test images.
    49. 49. Conclusion• From the above discussion and analysis we conclude that view morphing which was never used for face recognition purpose before is a valid tool, with few shortcomings which can be removed.• Morphing is a common technique in special effects and is now becoming and important part of computer vision and pattern recognition.• Face recognition can be seen as a new application of the existing view morphing algorithms.
    50. 50. References• View Morphing - Seitz and Dyer, 1996• View Interpolation - Chen and Williams, 1993• Image Morphing - Beier and Neely, 1992• Image Morphing - Mullens and Notley, 2006• Image Morphing: A survey - G. Wolberg, 1998

    ×