LIRE presentation at the ACM Multimedia Open Source Software Competition 2013

  • 1,543 views
Uploaded on

 

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,543
On Slideshare
0
From Embeds
0
Number of Embeds
32

Actions

Shares
Downloads
12
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Mathias Lux This work is licensed under a Creative Commons Attribution 3.0 Unported License.
  • 2. What is LIRE? • Library for CBIR • Easy access & instant “success” • Few loc to index & search
  • 3. It’s based on Lucene • Java text retrieval framework – based on inverted lists • Top level Apache project • Extends to Solr
  • 4. Modular Feature Architecture LireFeature as the basic Interface • Extraction, • Distance function, • Serialization (byte[] based) • toString(), field name, …
  • 5. Fast Access & Linear Search • Efficient coding of serialization – transformation to byte[] – run length coding for sparse vectors • Custom Lucene codec – Lucene field compression – update to DocValues in v1.0
  • 6. Search with sub Linear T ime Complexity • Hashing based approach for global features – Locality sensitive hashing • bit sampling – Proximity based hashing • nearest neighbors as “buckets”, • cp. work of G. Amato • Local features supported – SIFT, SURF, k-means, VLAD
  • 7. Tools • Parallel Indexing – consumer-producer based – up to the capabilities of the VM / HDD • Intermediate byte based data format – small footprint, efficient, relative paths
  • 8. Extending LIRE • Implement a global feature – extraction, distance function, serialization • Lire takes care of the rest – Parallel indexing, hashing, search
  • 9. Using Parts of LIRE Take what you need … • Feature implementations – cp. work of Xinchao Li et al. at Mediaeval 2013 • Image processing – Canny Edge Detector, SWT (coming soon), • Tools & code base – FastMap, Suffix Tree Clustering, …
  • 10. UCID Data Set MAP precision 10 ER CEDD 0,431 0,420 0,553 CEDD Color Correlogram 0,586 0,480 0,370 Color Correlogram Color Layout 0,277 0,285 0,679 Color Layout Edge Histogram 0,180 0,202 0,813 Edge Histogram FCTH 0,447 0,415 0,531 FCTH JCD 0,470 0,435 0,508 JCD Joint Histogram 0,348 0,313 0,603 Joint Histogram LBP Opponent Joined 0,266 0,267 0,729 LBP Opponent Joined Local Binary Patterns (LBP) 0,228 0,221 0,714 Local Binary Patterns (LBP) Opponent Histogram 0,319 0,309 0,649 Opponent Histogram PHOG 0,232 0,235 0,725 PHOG RGB Color Histogram 0,403 0,358 0,550 RGB Color Histogram Rotation Invariant LBP 0,165 0,174 0,813 Rotation Invariant LBP Scalable Color 0,172 0,183 0,840 Scalable Color SPCEDD 0,575 0,487 0,366 SPCEDD SPLBP 0,264 0,251 0,683 SPLBP Surf BoVW 0,348 0,313 0,634 Surf BoVW VLAD-SURF 0,370 0,356 0,603 VLAD-SURF
  • 11. SIMPLICity Data Set MAP precision 10 ER CEDD 0,513 0,706 0,193 Color Correlogram 0,498 0,740 0,159 Color Layout 0,439 0,612 0,303 Edge Histogram 0,333 0,500 0,401 FCTH 0,499 0,703 0,207 JCD 0,520 0,730 0,183 JCD Joint Histogram 0,449 0,689 0,197 Joint Histogram LBP Opponent Joined 0,418 0,569 0,347 LBP Opponent Joined Local Binary Patterns (LBP) 0,358 0,587 0,295 Local Binary Patterns (LBP) OpponentHistogram 0,450 0,635 0,270 OpponentHistogram PHOG 0,365 0,547 0,355 PHOG RGB Color Histogram 0,450 0,704 0,191 RGB Color Histogram Rotation Invariant LBP 0,338 0,520 0,375 Rotation Invariant LBP Scalable Color 0,305 0,470 0,464 Scalable Color SPCEDD 0,599 0,772 0,144 SPCEDD SPLBP 0,395 0,556 0,348 SPLBP SURF BoVW 0,338 0,464 0,475 SURF BoVW VLAD-SURF 0,365 0,518 0,407 VLAD-SURF CEDD Color Correlogram Color Layout Edge Histogram FCTH
  • 12. Hashing - BitSampling 1,000 0,900 JCD 0,800 CEDD 0,700 FCTH 0,600 ACC 0,500 PHOG 0,400 OPH 0,300 ColHist 0,200 ColLay 0,100 EH SPCEDD 0,000 0 500 1000 1500 2000 2500 100k images from flickr, 50 results cp. to linear search 3000
  • 13. Hashing - Proximity 1,000 JCD 0,900 CEDD 0,800 0,700 FCTH 0,600 ACC 0,500 PHOG 0,400 OPHIST 0,300 ColHist 0,200 Collay 0,100 EH 0,000 SPCEDD 0 500 1000 1500 2000 2500 100k images from flickr, 50 results cp. to linear search 3000
  • 14. Apache Solr Integration • Motivation: – Use a search and retrieval server with all its tools • Objectives: – indexing & management – efficient content based image search – content based ranking of results
  • 15. Solr Plugin • Custom Request Handler – Uses Solr’s request and response framework – Allows for content based retrieval • Custom ValueSourceFunction – Added to text based search queries – Allows for ranking based on the distance function
  • 16. Solr Plugin • Custom type of index field – DocValue based binary field – transmission base64 encoded • Custom Indexer – XML documents to be uploaded to Solr
  • 17. SOLR Plugin • http://demo-itec.uni- klu.ac.at/liredemo/wipo.html • Local demo
  • 18. Future Work • DocValues based indexing – make linear search faster • Proximity hashing – metric spaces approach – more accurate • Release version 1.0 – adding docs & features freeze
  • 19. Acknowledgements I’d like to thank Anna-Maria Pasterk, Arthur Li, Arthur Pitman, Bastian Hösch, Benjamin Sznajder, Christian Penz, Christine Keim, Christoph Kofler, Dan Hanley, Daniel Pötzinger, Fabrizio Falchi, Franz Graf, Giuseppe Amato, Glenn Macstravic, James Charters, Janine Lachner, Katharina Tomanec, Lukas Esterle, Manuel Oraze, Marian Kogler, Marko Keuschnig, Michael Riegler, Rodrigo Carvalho Rezende, Roman Divotkey, Roman Kern, Savvas Chatzichristofis and Sandeep Gupta.
  • 20. Lecture Book
  • 21. T hanks for listening … • Mathias Lux • mlux@itec.uni-klu.ac.at