Similarity Search inNon-text Data<br />Pavel Zezula<br />Faculty of Informatics<br />Masaryk University, Brno<br />4.10.20...
Real-Life MotivationThe social psychology view<br />Any event in the history of organism is, in a sense, unique.<br />Reco...
Contemporary Networked MediaThe digital data view<br />Almost everything that we see, read, hear, write, measure, or obser...
Examples with Similarity<br />Does the computer disk of a suspected criminal contain illegal multimedia material?<br />Wha...
Challenge<br />Networked media is getting close to the human “fact-bases”.<br />Similaritydatamanagement is needed to conn...
Limitations:Data Types<br />We have<br />Attributes<br />Numbers, strings, etc.<br />Text (text-based)<br />Documents, ann...
Limitations:Models of Similarity<br />We have<br />Simple geometric models, typically vector spaces<br />We need<br />More...
Limitations:Queries<br />We have<br />Simple query<br />Nearest neighbor<br />Range<br />We need<br />More query types<br ...
Limitations:Implementation Strategies<br />We have<br />Centralized or parallel processing<br />We need<br />Scalable and ...
4.10.2011<br />Searching Session NTK 2011<br />Search Strategy Evolution<br />Scalability<br /><ul><li>data volume - expon...
number of users (queries)
variety of data types
multi-lingual, -feature –modal queries</li></ul>well established<br />cutting-edge<br />research<br />high<br />Determinis...
Word Cloud of Applications<br />4.10.2011<br />Searching Session NTK 2011<br />
Metric Search Grows in Popularity<br />Hanan Samet<br />Foundation of Multidimensional and<br />Metric Data Structures<br ...
The MUFIN Approach<br />MUFIN: MUlti-Feature Indexing Network<br />SEARCH<br />data & queries<br />index structure<br />in...
Metric Space an Abstraction of Similarity<br />Metric space:M = (D,d)<br />D– domain<br />distance function d(x,y)<br />x...
Peer-to-Peer Indexing<br />Native metric techniques: GHT*, VPT*<br />Transformation techniques: M-CAN, M-Chord<br />4.10.2...
Image search<br />Image base<br />similar?<br />4.10.2011<br />Searching Session NTK 2011<br />
Images and their Descriptors<br />Image level<br />B<br />Descriptor level<br />R<br />G<br />4.10.2011<br />Searching Ses...
100M<br />images + metadata + MPEG-7 VDs<br />http://cophir.isti.cnr.it/<br /><ul><li>Largest publicly available collectio...
Each image contains:
Five MPEG-7 VDs: Scalable Color, Color Structure, Color Layout, Edge Histogram, Homogeneous Texture
Other textual information: title, tags, comments, etc.
Photos have been crawled from the Flickr photo-sharing site.</li></ul>CoPhIR: Content-based PhotoImage Retrieval<br />4.10...
MUFIN<br />SEARCH<br />ENGINE<br />data & queries<br />index structure<br />infrastructure<br />6 x IBM server x3400<br />...
demos<br />http://mufin.fi.muni.cz/apps.html<br />4.10.2011<br />Searching Session NTK 2011<br />
Current Research Activities<br />Image Query Postprocessing<br />Sub-image Searching<br />Remote Biometrics<br />Event Det...
Query Postprocessing<br />The understanding of similarity is:<br />subjective<br />context-dependent<br />multi-modal<br /...
Query Postprocessing by Ranking<br />Initial search<br />Ranking<br />Two-phase query evaluation model<br />Search the who...
  Enables cooperation with user</li></ul>Disadvantages <br /><ul><li>  Only a subset of the whole dataset is used in the ...
Upcoming SlideShare
Loading in …5
×

Podobnostní hledání v netextových datech (Pavel Zezula)

1,507 views
1,445 views

Published on

Chcete vědět víc? Mnoho dalších prezentací, videí z konferencí, fotografií i jiných dokumentů je k dispozici v institucionálním repozitáři NTK: http://repozitar.techlib.cz

Would you like to know more? Find presentations, reports, conference videos, photos and much more in our institutional repository at: http://repozitar.techlib.cz/?ln=en

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,507
On SlideShare
0
From Embeds
0
Number of Embeds
769
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Podobnostní hledání v netextových datech (Pavel Zezula)

  1. 1. Similarity Search inNon-text Data<br />Pavel Zezula<br />Faculty of Informatics<br />Masaryk University, Brno<br />4.10.2011<br />Searching Session NTK 2011<br />
  2. 2. Real-Life MotivationThe social psychology view<br />Any event in the history of organism is, in a sense, unique.<br />Recognition, learning, and judgment presuppose an ability to categorize stimuli and classify situations by similarity.<br />Similarity (proximity, resemblance, communality, representativeness, psychologicaldistance, etc.) is fundamental to theories of perception, learning, judgment, etc.<br />Searching Session NTK 2011<br />4.10.2011<br />
  3. 3. Contemporary Networked MediaThe digital data view<br />Almost everything that we see, read, hear, write, measure, or observe can be digital.<br />Users autonomouslycontribute to production of global media and the growth is exponential.<br />Sites like Flickr, YouTube, Facebook host user contributed content for a variety of events.<br />The elements of networked media are related by numerous multi-facet links of similarity.<br />4.10.2011<br />Searching Session NTK 2011<br />
  4. 4. Examples with Similarity<br />Does the computer disk of a suspected criminal contain illegal multimedia material?<br />What are the stocks with similar price histories?<br />Which companies advertise their logos in the direct TV transmission of football match?<br />Is it the situation on the web getting close to any of the network attacks which resulted in significant damage in the past? <br />4.10.2011<br />Searching Session NTK 2011<br />
  5. 5. Challenge<br />Networked media is getting close to the human “fact-bases”.<br />Similaritydatamanagement is needed to connect, search, filter, merge, relate, rank, cluster, classify, identify, or categorize objects across various collections.<br />WHY?<br />It is the similarity which is in the world revealing.<br />4.10.2011<br />Searching Session NTK 2011<br />
  6. 6. Limitations:Data Types<br />We have<br />Attributes<br />Numbers, strings, etc.<br />Text (text-based)<br />Documents, annotations<br />We need<br />Multimedia<br />Image, video, audio<br />Security <br />Biometrics<br />Medicine<br />EKG, EEG, EMG, EMR, CT, etc.<br />Scientific data<br />Biology, chemistry, physics, life sciences, economics<br />Others<br />Motion, emotion, events, etc.<br />4.10.2011<br />Searching Session NTK 2011<br />
  7. 7. Limitations:Models of Similarity<br />We have<br />Simple geometric models, typically vector spaces<br />We need<br />More complex model<br />Non metric models<br />Asymmetric similarity<br />Subjective similarity<br />Context aware similarity<br />Complex similarity<br />Etc.<br />4.10.2011<br />Searching Session NTK 2011<br />
  8. 8. Limitations:Queries<br />We have<br />Simple query<br />Nearest neighbor<br />Range<br />We need<br />More query types<br />Reverse NN, distinct NN, similarity join<br />Other similarity-based operations<br />Filtering, classification, event detection, clustering, etc.<br />Similarity algebra<br />May become the basis of a “Similarity Data Management System”<br />4.10.2011<br />Searching Session NTK 2011<br />
  9. 9. Limitations:Implementation Strategies<br />We have<br />Centralized or parallel processing<br />We need<br />Scalable and distributed architectures<br />MapReduce like approaches<br />P2P architectures<br />Cloud computing<br />Self-organized architectures<br />Etc.<br />4.10.2011<br />Searching Session NTK 2011<br />
  10. 10. 4.10.2011<br />Searching Session NTK 2011<br />Search Strategy Evolution<br />Scalability<br /><ul><li>data volume - exponential
  11. 11. number of users (queries)
  12. 12. variety of data types
  13. 13. multi-lingual, -feature –modal queries</li></ul>well established<br />cutting-edge<br />research<br />high<br />Determinism<br />exact match ► similarity<br />precise ► approximate<br />same answer ► good answer; recommendation<br />fixed query ► personalized; context aware<br />fixed infrastr. ► dynamic mapping; mobile dev.<br />peer-to-peer<br />centralized<br />parallel<br />self-organized<br />distributed<br />grade<br />low<br />
  14. 14. Word Cloud of Applications<br />4.10.2011<br />Searching Session NTK 2011<br />
  15. 15. Metric Search Grows in Popularity<br />Hanan Samet<br />Foundation of Multidimensional and<br />Metric Data Structures<br />Morgan Kaufmann, 2006<br />P. Zezula, G. Amato, V. Dohnal, and M. Batko<br />Similarity Search: The Metric Space Approach<br />Springer, 2006<br />4.10.2011<br />Searching Session NTK 2011<br />
  16. 16. The MUFIN Approach<br />MUFIN: MUlti-Feature Indexing Network<br />SEARCH<br />data & queries<br />index structure<br />infrastructure<br />Scalability<br />P2P structure<br />Extensibility<br />metric space<br />Tuning of performance<br />Internet / GRID / LAN<br />network independence<br />4.10.2011<br />Searching Session NTK 2011<br />
  17. 17. Metric Space an Abstraction of Similarity<br />Metric space:M = (D,d)<br />D– domain<br />distance function d(x,y)<br />x,y,z  D<br />d(x,y) > 0 - non-negativity<br />d(x,y) = 0 x = y - identity<br />d(x,y) = d(y,x) - symmetry<br />d(x,y)≤ d(x,z)+ d(z,y) - triangle inequality<br />4.10.2011<br />Searching Session NTK 2011<br />
  18. 18. Peer-to-Peer Indexing<br />Native metric techniques: GHT*, VPT*<br />Transformation techniques: M-CAN, M-Chord<br />4.10.2011<br />Searching Session NTK 2011<br />
  19. 19. Image search<br />Image base<br />similar?<br />4.10.2011<br />Searching Session NTK 2011<br />
  20. 20. Images and their Descriptors<br />Image level<br />B<br />Descriptor level<br />R<br />G<br />4.10.2011<br />Searching Session NTK 2011<br />
  21. 21. 100M<br />images + metadata + MPEG-7 VDs<br />http://cophir.isti.cnr.it/<br /><ul><li>Largest publicly available collection of high-quality images metadata: 106 million images
  22. 22. Each image contains:
  23. 23. Five MPEG-7 VDs: Scalable Color, Color Structure, Color Layout, Edge Histogram, Homogeneous Texture
  24. 24. Other textual information: title, tags, comments, etc.
  25. 25. Photos have been crawled from the Flickr photo-sharing site.</li></ul>CoPhIR: Content-based PhotoImage Retrieval<br />4.10.2011<br />Searching Session NTK 2011<br />
  26. 26. MUFIN<br />SEARCH<br />ENGINE<br />data & queries<br />index structure<br />infrastructure<br />6 x IBM server x3400<br />Scalability<br />M-Chord + M-Tree<br />Extensibility<br />COPHIR<br />color structure<br />scalable color<br />color layout<br />edge histogram<br />homogeneous texture<br />Image Search Demohttp://mufin.fi.muni.cz/imgsearch/<br />4.10.2011<br />Searching Session NTK 2011<br />
  27. 27. demos<br />http://mufin.fi.muni.cz/apps.html<br />4.10.2011<br />Searching Session NTK 2011<br />
  28. 28. Current Research Activities<br />Image Query Postprocessing<br />Sub-image Searching<br />Remote Biometrics<br />Event Detection in Video<br />Signal Processing<br />4.10.2011<br />Searching Session NTK 2011<br />
  29. 29. Query Postprocessing<br />The understanding of similarity is:<br />subjective<br />context-dependent<br />multi-modal<br />Semantic gap<br />Overcoming semantic gap by combining aspects<br />semantics-learning<br />result postprocessing<br />relevance feedback & iterative search<br />Our objectives<br />Large general data collections with various quality of metadata<br />Online searching response times<br />4.10.2011<br />Searching Session NTK 2011<br />
  30. 30. Query Postprocessing by Ranking<br />Initial search<br />Ranking<br />Two-phase query evaluation model<br />Search the whole collection by some aspects => candidate set<br />Rank the candidate set – sort by other aspects<br /> Advantages<br /><ul><li> Fast, enables to combine more similarity measures
  31. 31. Enables cooperation with user</li></ul>Disadvantages <br /><ul><li> Only a subset of the whole dataset is used in the ranking phase</li></ul>4.10.2011<br />Searching Session NTK 2011<br />
  32. 32. Sub-image Searching<br />Retrieves all images containing the query image<br />Based on local image descriptors<br />Scale Invariant Feature Transform (SIFT):<br />Descriptor – content of a small neighborhood<br />Locator – coordinates of the neighborhood<br />Scale – importance of the descriptor<br />Image  a set of features, descriptors<br />Task: Find matching pairs (similar features)<br />4.10.2011<br />Searching Session NTK 2011<br />Query Answer:<br />
  33. 33. Remote Biometrics: Motivation<br />Most biometrics require the subject’s cooperation<br />Fingerprint, iris, palmprint, handwriting, voice recognition<br />Challenge – recognizing people at a distance<br />Capture devices do not require a close contact with the subject (e.g., surveillance cameras)<br />It can be applied unobtrusively<br />Face and gait recognition at a distance<br />Problems – camera view, lighting, pose<br />Applications – surveillance, security<br />4.10.2011<br />Searching Session NTK 2011<br />
  34. 34. Remote Biometrics: Approaches<br />Detection, normalization, extraction, recognition<br />Face recognition<br />Methods:<br />Appearance-based – analyze the face as a whole<br />Model-based – compare individual features (e.g., eyes, mouth)<br />MUFIN face recognition demo: http://mufin.fi.muni.cz/faces-feret/<br />Gait recognition<br />Less likely to be obscured, low resolution suffices<br />Methods are based on shape or dynamics of the person:<br />Appearance-based – analyze person’s silhouettes<br />Model-based – compare features (e.g., trajectory, angular velocity)<br />4.10.2011<br />Searching Session NTK 2011<br />
  35. 35. Event Detection in Video<br />Video<br />continuous data<br />several aspects<br />image, sound, text, motion, temporal<br />Event<br />defined aspects occurring in given time interval<br />definition of a sample aspect by example or value<br />definition is imprecise – looking for “similar” aspects<br />combination of aspects<br />aggregation function<br />Current approaches<br />annotation-based, learning-based (classifiers)<br />specific domains<br />Example<br />TV news (by image) AND about IRAQ (by text) AND burning vehicles (by image) AND time interval < 1 minute (by temporal)<br />4.10.2011<br />Searching Session NTK 2011<br />
  36. 36. Signal Processing<br />Vast amount of signals produced:<br />Biomedicine data – ECG, CT<br />Biometric data – personal identification<br />Audio data – audio similarity, recognition<br />Sub-image searching<br />Financial time series – analysis, forecasting<br />Time series streams<br />Demand for<br />a graceful handling of this data <br />flexible reactions to new application needs<br />4.10.2011<br />Searching Session NTK 2011<br />
  37. 37. Flexible Subsequence Matching<br />Generic engine for rapid development of subsequence matching applications<br />can be used for any class of one-dimensional signals<br />Implementation of various subsequence matching approaches<br />Demo web application<br />User Application<br />Subsequence Matching<br />Layer<br />4.10.2011<br />Searching Session NTK 2011<br />
  38. 38. Demo application<br />4.10.2011<br />Searching Session NTK 2011<br />
  39. 39. Face Retrieval Application<br />10,000 images with people<br />14,000 faces<br />Face detection – MPEG7<br />4.10.2011<br />Searching Session NTK 2011<br />

×