Matrix mirrors project


Published on

Published in: Technology, Art & Photos
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Matrix mirrors project

  1. 1. Matrix mirrors – a very short presentationRenato Roque – FEUP, University of PortoThis short text presents very briefly Matrix Mirrors project and some of its results. Matrix Mirrors was driven by aMultimedia Master Thesis in FEUP (2007/2009) and it led as well to a photographic/artistic project with the samename. (please see attached documentation on artistic project).We are looking for ideas in order to continue our work. One main vector for this continuation would be to guaranteethat it would cover not only technical/ scientific/ image processing issues but it could be as well the basis for thecontinuation of the photographic/ artistic project, which was an important result of our previous work.Matrix mirrors – project backgroundSince the 1950’s that a number of psychologists and neurologists believe that Humans havedeveloped specialized mechanisms to learn, memorize and recognize faces. They support that thissophisticated learning and recognition process was an essential achievement for the specie’ssurvival. Figure 1 – Areas in the brain involved in face recognition process (red striped)Concepts and theories from the communication area, such as entropy, sparse coding, redundancyand signal/noise relation were adopted to quantify and try to explain a lot of experimental data,related with visual perception, in particular the perception of human faces. In particularredundancy concept has played a central role in this vision. Horace Barlow was the first to drawthe importance of such concept for the economy in representation and for the speed in recognition.He believed that a mechanism to identify regularities or redundancies was crucial to optimize thisprocess. Most information in a new face will correspond to information in other faces. It isredundancy which allows unsupervised learning process. On the other hand, to be able to recognizesomething new, the system must allow comparing what is being observed with what is usuallyseen.Related to this vision there has been in the computational areas, since the 1990’s, a lot ofinvestigation to use statistical tools for image-vectors, to create automatic recognition applications.There has been a lot of work to develop valid models and efficient algorithms for the processing ofnatural images and in particular human faces.There are already today, for example, a few commercial products making use of this work, forexample to identify a person from a photo or from a video. 1
  2. 2. Matrix mirrors – a photographic projectPhotography has been associated since its invention to something involving mystery and magic. Itscapability of freezing time and space has still today, in spite the mass vulgarization of photos, anaura of witchcraft.Photography has been clearly associated since its invention in the XIXth century with identity.Photographs have been and are still used in all identity documents, like passports and all kinds ofcards. Looking at human portraits trough statistic techniques will allow a reflection around humanidentity and what it means. Concepts such as entropy will allow quantifying answers to questionssuch as: What is common and what is different in each human face? What new information do weget when we know a new face?The knowledge that it might be possible to use mathematical/image processing tools to get, from aportrait database, a set of abstract components, which would allow us to reconstruct any portrait,within or without the database used, just adding those components in the right percentage, wassomething completely magic to add to the magic of photography. Still today, this capability is awonder to us. Thinking that we might have in our brains an equivalent process is still a biggerwonder.Matrix mirrors – some resultsAll computational work which we referred makes use of sophisticated statistical/ data/ imageprocessing techniques like PCA (Principal Component Analysis), ICA (Independent ComponentAnalysis) or NMF (Non negative Matrix Factorization) which allow, using different criteria,decomposing a portrait into a set of components. We could say that these components correspondto portrait’s global abstract features. If we turn them into images they can be seen as ghostly faces.An example can be seen in Figure 2, which shows some of these components, which we calculatedfrom our portrait Data Base (DB) using PCA. Figure 2 – PCA components - Eigen facesIn our work we used such techniques but from a photography point of view. We wanted to evaluateand compare the capability of such techniques, not to produce a correct machine identification of aportrait, comparing coefficients, not to improve the recognition rate of existing algorithms, but toreconstruct recognizable human portraits, using generic components.We photographed 439 persons from the Oporto University and built a 400 facial portraits 200x200pixels database. This resolution was imposed by our computer memory, taking into account thehuge size of the matrixes to deal with. The other 39 portraits were used as a test set to simulate thebehaviour of the system towards new portraits. We calculated statistical components for ourdatabase, using PCA, ICA, NMF and PCA+ICA (a new hybrid statistical system). 2
  3. 3. These components can be added to reconstruct, not only the portraits which were used to calculatethem, but other new portraits as well, using simple formula like: I = I DC + ∑ Ak I k (1) kFirst we proved that, with the right coefficients, all 400 portraits from the database canreconstructed with perfection and we need even only a few components to perform identification:with an hybrid PCA+ICA system (the most performing) we only need less than 20 components tosucceed in identification.We concluded that even new portraits, like the one in Figure 3, which had not been used tocalculate the components, can be reconstructed, although the reconstruction is not perfect this time.In spite of the final error, these reconstructed portraits can be recognized once more making use ofonly about 30 components.Using a questionnaire we demonstrated that with only a few components (about 20 for portraits inthe database and 30 for new portraits) we can reconstruct all portraits, with an error low enough toallow recognition. We observed that an error below 4.5% allows nearly 100% recognition rate.Error in reconstruction was measured using Euclidean distance concept. These values wereobtained with PCA+ICA hybrid system and with a normalized database, where portraits werenormalized to guarantee that both eyes from all portraits are coincident. Other statistical techniqueseven with a non normalized database show similar results, they only require a slight bigger numberof components. Figure 3 – The reconstruction of a new portrait, which was not in database step by stepAnother apparently very significant result was that we observed that a portrait from a specificgroup, for example a woman, can be reconstructed without any visible difference, usingcomponents calculated from a DB of portraits from a different group, for example men. Womencan be reconstructed as well as men from a DB of mens portraits. As well an African face can bereconstructed as well as an European face from an European DB.Matrix mirrors – some statistical resultsUsing our portrait database a few statistical portraits could be calculated and some very interestingresults were obtained. We calculated namely average, standard deviation (SD), skew and kurtosisportraits for the whole database and for attribute related groups. Figure 4 shows the DB averageportrait. 3
  4. 4. Figure 4 – Average portrait for the 400 portraits in the databaseFigure 5 shows average and SD portraits for different groups, which we considered in our work. Figure 5 – Average and SD portraits for different groups From left to right: whole DB, men, women, 118 men, men aged over 50, men aged less than 30We observe clearly that on one hand the whole DB statistic portraits appear to be very similar tomens portraits and that is not due to the fact that we have more men than women in our DB,because we obtain the same result with 118 men, the number of women in our DB, as we can see.On the other hand, according to what one might expect, we can observe that each group appears tolead to specific statistical portraits, where one can observe some groups characteristics.But we decided as well to analyse the evolution of statistical portraits with DB dimension, mixingall kinds of portrits: men, women, old and young people. Figure 6 shows the results for average andSD portraits. 4
  5. 5. Figure 6 – Average and SD portraits for different DB dimensions From left to right: 6, 25, 50, 100, 200 and 400 portraitsIt appears to be very relevant to observe that we need only about 50 persons to get a result which isvery, very similar with the statistical portraits for the whole DB, which could lead us to think thatthis might be the humanitys average and SD portraits!Although each group shows to have its statistical specificities, when we mix different portraits, wecome very fast to average and SD generic portraits.Matrix mirrors – some ideas to continue previous workWe would like to enlarge our portrait DB, in order to make it equally representative of men and women andof different ethnical origins, allowing this way to validate some of the results which we intuited in our work.Already after completion of the Master Thesis we have been trying some interesting tools to performautomatic image classification - cluster analysis of portraits - based upon portraits dissimilarity.Matrix mirrors – Main references[1] Aapo Hyvärinen. 1999. Fast and Robust Fixed-Point Algo-rithms for Independent Component Analysis. . IEEE Trans-actions on Neural Networks 10 (3):626-634.[2] Aapo Hyvärinen, Erkki Oja. 2000. Independent Component Analysis Algorithms and Applications. Neural Networks 13 ((4-5) ):411-430.[3] Barlow Horace. 1989. Unsupervised learning. Neural Compu-tation (1):295–311.[4] Bartlett Marian Stewart. 1998. Face image analysis by unsu-pervised learning and redundancy reduction. Ph.D., Univer-sity of California, San Diego.[5] Bruce Vicky, Young Andy. 1986. Understanding Face Recognition. British Jounal of Psychology (77):305,327.[6] Chih-Jen Lin. 2007. Projected Gradient Methods for Non-negative Matrix Factorization. Neural Computation 19:2756-2779.[7] Draper B., K. Baek, M.S. Bartlett and R. Beveridge. 2003. Recognizing Faces with PCA and ICA. Computer Vision and Image Understanding Volume 91 (Issues 1-2):115-137[8] Edelman, S., B. P. Hiles, H. J. Yang, and N. Intrator. 2001. Probabilistic principles in unsupervised learning of visual structure: human data and a model. Paper read at 15th An-nual Conference on Neural Information Processing Systems (NIPS), Dec 03-08, at Vancouver, Canada.[9] Edelman, S., N. Intrator, and J. S. Jacobson. 2002. Unsuper-vised learning of visual structure. Paper read at 2nd Interna-tional Workshop on Biologically Motivated Computer Vi-sion (BMCV 2002), Nov 22-24, at Tubingen, Germany.[10] Ekman Paul. 1999. Handbook of Cognition and Emotions. John Wiley and Sons Ltd ed. Vol. Facial Expressions.[1] Lee Daniel, Seung H. Sebastian. 2001. Algorithms for non-negative matrix factorization. Adv. Neural Info in Process-ing Systems 13 556-562.[11] Ming-Hsuan Yang. 2002. Detecting Faces in Images: a Survey. IEEE Transactions on Pattern Analysis and Machine Intel-ligence 24 (1).[12] Puga André. 2001. A computational allegory for V1. 2nd Inter-national Symposium on Image and Signal Processing and Analysis at PULA, CROATIA.[13] Wendy S. Yambor, Bruce A. Draper and J. Ross Beveridge. 2002. Analyzing PCA-based Face Recognition Algorithms: Eigenvector Selection and Distance Measures. World Sci-entific Press.[14] Xie Xudong. 2006. Face image analysis and its applications. Ph.D., Hong Kong Polytechnic University (Hong Kong), Hong Kong.[15] Xue Yun. 2007. Non-negative matrix factorization for face recognition. Ph.D., Hong Kong Baptist University (Hong Kong), Hong Kong.[16] Zafeiriou, S., A. Tefas, I. Buciu, and I. Pitas. 2005. Class-specific discriminant non-negative matrix factorization for frontal face verification. Paper read at 3rd International Conference on Advances in Pattern Recognition, Aug 22-25, at Bath, ENGLAND.[17] Zafeiriou, S., A. Tefas, I. Pitas, and Ieee. 2005. Discriminant NMFFaces for frontal face verification. Paper read at IEEE Workshop on Machine Learning for Signal Processing (MLSP), Sep 28-30, at Mystic, CT.[18] Zhao W, Chellappa R., Phillips P. J., Rosenfeld A. 2003. Face recognition: A literature survey. ACM Computing Surveys 35 (4):399-458 5