Downloaded 15 times
Uploaded byJonathon Hare
OpenIMAJ and ImageTerrier: Java Libraries and Tools for Scalable Multimedia Analysis and Indexing of Images
OpenIMAJ and ImageTerrier are Java libraries designed for scalable multimedia analysis, including image, audio, and video processing. They provide tools for efficient indexing and searching of multimedia content using methods such as SIFT, BOVW, and Hadoop for scalability. The libraries support rapid application development and are extensible, enabling researchers and developers to implement and modify algorithms easily.
More Related Content
Similar to OpenIMAJ and ImageTerrier: Java Libraries and Tools for Scalable Multimedia Analysis and Indexing of Images
OpenIMAJ and ImageTerrier: Java Libraries and Tools for Scalable Multimedia Analysis and Indexing of Images
- 1. ImageTerrier www.imageterrier.org OpenIMAJIntelligent Multimedia AnalysisIn Java www.openimaj.org OpenIMAJ and ImageTerrier Java Libraries and Tools for Scalable Multimedia Analysis and Indexing of Images Jonathon Hare, Sina Samangooei & David Dupplaw The University of Southampton
- 2. What is OpenIMAJ? Scalable Clustering Scalable KNN OpenIMAJIntelligentMultimedia Analysis In Java Image Analysis Libraries Command-line Tools Hadoop Map-Reduce Implementations Hardware Machine Learning Video Web Audio Analysis AnalysisProcessing ProcessingProcessing Analysis Rendering Kinect Audio & Video Capture Exotic Hardware
- 3. Libraries Command-line Tools Hadoop Map-ReduceImplementations Efficient, Compressed Inverted Index Query Engine Term Metadata Integration Based on Terrier Constructable from Images Distance Metrics scale Re-Ranking affine orientation Compressed Extensible OpenIMAJIntelligent Multimedia Analysis In Java Scalable e.g SIFT BoVW (x, y, scale, etc)
- 4. What is ImageTerrier? Efficientinverted index construction for image/ video features (and potentially audio features too) Augmented inverted index, capable of storing almost any kind of meta information per term Easily extensible Provided implementations contain things like (x,y,scale) for SIFT BoWV Indexing is scalable across machines using Hadoop Complete OpenIMAJ integration Included tools can build an index straight from a set of images using common techniques like SIFT BoVW or read features extracted from other tools. This can even be done on Hadoop! Easy to define searching and re-ranking strategies Various distances, weightings and constraints (such as Affine & Homographic) are provided Tested with corpuses in excess of 10M images
- 5. A Little history... IAMMediaEngine c.2005 MapSnap c. 2006 JCVMLTK c. 2009 OpenIMAJ & ImageTerrier May 2011 MoVE c. 2010 Pure-Java SIFT & Matching Extensible Java (& JNI) Content- Based Retrieval System Refactoring of older code, addition of face detection and machine learning components Initial attempts at scalable BoWV indexing using terrier, shape analysis, etc Many new features added. Improvements to ImageTerrier, face pipeline, demos, video capture and hardware, ... PhotoCopain c. 2005 Image classifiers SMS-NLP c. 2010 Machine learning, clustering Massive code cleanup and documentation effort. First Open Source Release
- 6. It’s pure Java... (withthe exception of thin interfaces for hardware) Isn’t it slow?
- 7. Why another setof libraries? Scalability + Portability. OpenIMAJ projects work on: Android mobile devices Hadoop clusters ...without recompilation! Easy to use and extend Implement many important Multimedia algorithms While allowing easy adaption and extensions. Software engineering principles adopted clear separations of concerns maintainable, understandable We simplify data representations: Images are 2D-arrays of pixels; Videos are iterable streams of Images Audio is an array of samples
- 8. Both OpenIMAJ and ImageTerrierhave been designed from the ground-up to be easily extensible. We’ve made it easy to modify and understand algorithms and techniques by separating different components.
- 10.
- 11.
- 12.
- 13.
- 14. Rapid Application Development “Makeit easy to use” This philosophy lets researchers build complex applications quickly Example: Southampton Goggles synchronised: 6 cameras, GPS device, compass feature extraction and index construction mobile integration …all programmed over 2 days! Find out more at: http://bit.ly/SOTONGOG
- 16. Student projects Our studentsuse OpenIMAJ (…for 2 years now!) OpenIMAJ lets them: Get something basic working quickly Understand the algorithms they are using Implement new algorithms in a structured way Contribute to a living, open source project To support our students we wrote a comprehensive OpenIMAJ tutorial.
- 17. Research applications Through combinationof OpenIMAJ and ImageTerrier various multimedia research applications can be addressed - We have done so for over 6 years! 1)Guess The Building - part of the LiveMemories project Estimate geolocation and semantic information for an un- annotated image of a landmark 2)Stock Photo Finder - part of the Living Knowledge project Find the source of a web image in a stock photo database
- 18. Any Questions? Try theOpenIMAJ tutorial for yourself: http://openimaj.org/tutorial.pdf