DE Conferentie 2005 Egon v.d. BroekPresentation Transcript
Multilevel Access to our Cultural Heritage (MACH)
People who participated in the project Egon L. van den Broek (AI, VU Amsterdam / NICI, RU Nijmegen) Thijs Kok (ICIS, RU Nijmegen) Eduard Hoenkamp (NICI, RU Nijmegen) Theo E. Schouten (ICIS, RU Nijmegen) Peter J. Petiet (AI, VU Amsterdam) Louis G. Vuurpijl (NICI, RU Nijmegen)
In addition…The people who worked on the foundation of M4ART Frans Gremmen (RTOG, RU Nijmegen) Maarten A. Hendriks (freelance) Peter Kisters (freelance) Harco C. Kuppens (ICIS, RU Nijmegen) Eva M. van Rikxoort (ISI, UMC Utrecht) Charles M. de Weert (Soc. Fac., RU Nijmegen)
NWO research line ToKeN Two projects within the ToKeN research line: Eidetic VindIT A few of the aims of the projects Access and enhancement of our cultural heritage Combining Computer Science and Cognitive Science Combining (text-based) Information Retrieval (IR) and Content-Based Image Retrieval (CBIR) Launching a CBIR system for the National Gallery of Arts (Het Rijksmuseum)
Key notions of this talk Aim: Access and enhancement of knowledge Domain: Cultural heritage; more specific: The National Gallery of Arts (Het Rijksmuseum) Characteristics: Digital photographs of objects Annotations of these objects; i.e., descriptions of the artist, the material, the content/theme of the object etc. For … both laypersons (i.e., non-experts) and experts How: (text-based) Information Retrieval (IR) and Content-Based Image Retrieval (CBIR) Additionally, facilitating: The user’s understanding of CBIR Comparison of IR and CBIR
Current state-of-the-art in museum access through the Internet Four examples: Virtual Catalogue for Art History The National Gallery of Art (Het Rijksmuseum) The hermitage museum (powered by IBM)
Why this aim in this domain? Attract people to museums through Internet Make museums better accessible for laypersons Make museums better accessible for experts Providing more information through new methods of access Gaining more understanding of CBIR through using it Future aims: Interaction between users and the system through agent technology Extracting information of artists directly from their art
Information Retrieval (e.g., as with Google)
Color-Based Image Retrieval:3 methods of querying Browsing the catalogue Providing an image from the Internet, by way of an URL Uploading a local image from your desktop
From simple to complex user interfaces M4ART can be described using 2 dimensions: 1) Retrieval method; either IR or CBIR 2) Complexity of querying; - straightforward queries - specified/combined queries
Complex IR querying (1) Exclusion of terms Boolean search, using AND and/or OR operators Utilizing specific tags (fields) of the annotations
Complex IR querying (2)
Complex CBIR querying (1) Color space Quantization scheme; i.e., how any colors do you want to distinguish? Distance measure Optionally: include texture
Complex CBIR querying (2)
Gaining understanding … Showing the features … Compare IR and CBIR techniques
Color spaces (and their quantizations) RGB HSV YUV/YIQ CIE XYZ CIE LUV 11 colors ...
Its all only in 11 color categories Most famous: Berlin & Kay (1969) with their book “Basic color terms: Their universals and evolution” But already 15 years before them Brown and Lenneberg (1954) presented their work in the paper: “A study in language and cognition” More recently: Derefeldt, Swartling, Berggrund, and Bodrogi (2004) Moroney (2004) Regier, Kay, and Cook (2005)
The practical use of fundamental research: Color categories People perceive in color categories People talk in color categories Human memory uses color categories Robust to inter and intra personal variability Computationally inexpensive
THE 11 colors
Black Gray White Red Green Blue
Texture defined … Cross and Jain (1983): There is no universally accepted definition for texture Bovik, Clarke, and Geisler (1990): an exact definition of texture either as a surface property or as an image property has never been adequately formulated. Gonzales and Woods (2002) state: No formal definition of texture exists, intuitively this descriptor provides measures of properties such as smoothness, coarseness and regularity.
Texture Analysis Palm (2004) noted: “The integration of color and texture is still exceptional.” Others: Drimbarean and Whelan (2001) and Mäenpää and Pietikäinen (2004) Techniques applied in literature, so far: Global color analysis (e.g., color histograms) Local gray-scale texture analysis (e.g., co-occurrence matrix) Local color-based texture analysis (e.g., color correlogram) Global color analysis combined with local gray-scale texture analysis (e.g., parallel approach) What is used in M4ART: Parallel-sequential texture analysis Global and local color analysis in parallel: the color histogram and the color correlogram combined.
Gray versus Color
Parallel-Sequential texture analysis* * More info? Broek, E. L. van den and Rikxoort, E. M. van (2005). Parallel-sequential texture analysis. Lecture Notes in Computer Science (Advances in Pattern Recognition), 3687, 532-541.
Back to C-BAR: Some last remarks Technical specifications Freely accessible Questions?
Technical specifications W3C HTML 4.01 validated W3C CSS validated Running on an Apache HTTP server PHP 5.1 powered Running on a MySQL database
Freely accessible All feedback = welcome! How to get there … Use Google query on ‘M4ART’ or go through: http://cai.nici.ru.nl/M4ART/