2. Context
ColaDI Project
– Platform for Project Collaboration in Industrial
Design
Period: March 2010 / November 2011
(EnContRA – until February 2011)
3. Objectives
General Framework for (Content-Based)
Retrieval Approaches and Applications
– Features:
• Indexing
• Features Extraction
• Searching / Retrieval Algorithms
• Extensible Query Processing
4. Advantages for Development
1. Modularity
2. Easy to use – Low learning curve
3. Fast development of new approaches
– Examples:
• A new descriptor
• A new searching / retrieval algorithm
• A new indexing structure
• Etc.
11. Objective
Query
QBE
Extracts:
Scalable Color
12. Pieces to Assembly
1. Image Descriptor: Scalable Color
2. Indexing Structure: In Memory Simple Index
3. Searching Algorithm: Simple Searcher
13. Pieces to Assembly
1. Image Descriptor: Scalable Color
2. Indexing Structure: In Memory Simple Index
3. Searching Algorithm: Simple Searcher
14. Choosing a Feature to be extract
Scalable Color Descriptor (Extractor)
DescriptorExtractor extractor = new
ScalableColorDescriptor<IndexedObject>();
Extractor Descriptor
15. Pieces to Assembly
1. Image Descriptor: Scalable Color
2. Indexing Structure: In Memory Simple Index
3. Searching Algorithm: Simple Searcher
16. Choosing an Indexing Structure
In Memory Simple Index
AbstractIndex index = new SimpleIndex();
Descriptor Descriptor
17. Pieces to Assembly
1. Image Descriptor: Scalable Color
2. Indexing Structure: In Memory Simple Index
3. Searching Algorithm: Simple Searcher
18. Searching
Linear Search (for now!)
Searcher searcher = new SimpleSearcher<IndexedObject>();
Descriptor Descriptor
19. Assembling all the components
Searcher searcher = new SimpleSearcher<IndexedObject>();
Setting Main
searcher.setDescriptorExtractor(extractor);
Properties
searcher.setIndex(index);
searcher.setObjectStorage(new
Not required SimpleObjectStorage(IndexedObject.class));
(recommended)
searcher.setResultProvider(new DefaultResultProvider());
20. Indexing the Dataset
File [] pictures = getFilePictures(dataset);
for (File pic : pictures) {
BufferedImage image = ImageIO.read(pic);
searcher.insert(new IndexedObject(image));
}
Extracts descriptors and indexes them!
21. Performing a Search – Similar
//Load the image query
BufferedImage image = readQuery();
//Perform the search using similar
ResultSet<IndexedObject> results =
searcher.similar(new IndexedObject(image), 20);
//Print the Results
printResults(results);
22. Performing a Search – Query API
CriteriaBuilderImpl cb = new CriteriaBuilderImpl();
Path<IndexedObject> modelPath = new
Query Path<IndexedObject>(IndexedObject.class);
Building
Similar similar = cb.similar(modelPath, new IndexedObject(image));
CriteriaQuery query = cb.createQuery().where(similar).limit(20);
Searching ResultSet<StringObject> results = searcher.search(query);
24. Objectives
Create a Drawing Retrieval Application, by
employing Query By Example (or Sketch)
– Queries:
• 2D Drawings (e.g., SVG files)
• Pictures
25. Input Model
Drawing Model
public class DrawingModel implements IEntity<Long> {
…
private Drawing drawing;
private BufferedImage image;
…
@Indexed
public BufferedImage getImage();
@Indexed
public Drawing getDrawing();
}
31. BTree for Indexing
Parameters:
– the name of the index
– the type of objects to be indexed (class)
BTreeIndex exampleIndex = new BTreeIndex(“btreeName", Object.class);
33. NBTree Searcher
Two flavors:
– Regular (original)
AbstractSearcher searcher = new NBTreeSearcher();
– Parallel To speed the search
AbstractSearcher searcher = new ParallelNBTreeSearcher();
34. Picture – Composed Searching
AbstractSearcher imageSearcher = new ImageSearchEngine();
imageSearcher.setQueryProcessor(new QueryProcessorDefaultParallelImpl());
imageSearcher.setIndexedObjectFactory(new ImageIndexedObjectFactory());
//Creating a combined searcher, with the selected descriptor
for (Map.Entry<String, DescriptorExtractor> entry : availableDescriptors) {
AbstractSearcher entrySearcher = new ParallelNBTreeSearcher();
entrySearcher.setQueryProcessor(new QueryProcessorDefaultParallelImpl());
entrySearcher.setIndex(new BTreeIndex("image." + entry.descriptorName, objClass);
entrySearcher.setDescriptorExtractor(entry.extractor);
imageSearcher.setSearcher("image." + entry.descriptorName, entrySearcher);
}
searcher.setSearcher("image", imageSearcher);
35. Drawing Searcher
AbstractSearcher vectSearcher = new ParallelNBTreeSearcher();
vectSearcher.setQueryProcessor( new QueryProcessorDefaultParallelImpl());
vectorialSearcher.setIndex(new BTreeIndex(“vectIndex",
TopogeoDescriptor.class));
TopogeoDescriptorExtractor topogeoExtractor =
new TopogeoDescriptorExtractor();
vectorialSearcher.setDescriptorExtractor(topogeoExtractor);
searcher.setSearcher("drawing", vectSearcher);
36. Performing a Search – Individual
CriteriaQuery<DrawingModel> query =
cb.createQuery(DrawingModel.class);
Query Path<DrawingModel> modelPath = query.from(DrawingModel.class);
Building Path drawingPath = modelPath.get(“drawing”);
Similar similar = cb.similar(drawingPath, new IndexedObject(drawing));
CriteriaQuery query = cb.createQuery().where(similar).limit(20);
Searching ResultSet<StringObject> results = searcher.search(query);
37. Performing a Search – Combined
Similar simD = cb.similar(drawingPath, new IndexedObject(drawing));
Similar simP = cb.similar(picturePath, new IndexedObject(image));
Query And andPredicate = cb.and(simD, simP);
Building
CriteriaQuery query = cb.createQuery()
.where(andPredicate ).limit(20);
Searching ResultSet<StringObject> results = searcher.search(query);
38. Custom IndexedObject Factory
public class ImageIndexedObjectFactory extends AnnotatedIndexedObjectFactory{
…
protected List<IndexedObject> createObjects(List<IndexedField>
indexedFields) {
//create indexedObjects for the DrawingModel instances
…
}
…
}
39. Custom Image Searcher
public class ImageSearchEngine implements AbstractSearcher<Long> {
…
protected List<IndexedObject> getIndexedObjects(Object o) throws
IndexingException {
//create different indexedObjects for the same image, and use
//them in different individual searchers
}
public ResultSet search(Query query) {
//create subqueries to perform search in the individual image
//searchers
}
…
}
