The document discusses content-based multimedia similarity (CBMS) search, which allows users to search for visually similar images and media. CBMS works by automatically extracting features from images, calculating similarity scores between images, and returning a ranked list of the most similar results. The document outlines how CBMS can be used for applications like identifying objects, finding reused content, and open-ended discovery. It also discusses using CBMS for interactive learning by allowing users to search for educational material based on visual similarity.

1. Off the beaten track Using content-based multimedia similarity search for learning Suzanne Little Knowledge Media Institute The Open University 1 st November 2011

3. What can CBMS be used for?

4. How does CBMS work?

5. CBMS for interactive learning

7. Known object identification

8. General Search ?

10. Similarity

12. Calculate the “similarity” score between your query and the images you are searching

13. Return a ranked list of results

15. Find instances of re-use

16. Discover and/or exposit (undirected search)

17. Spot&Search

18. Spot&Search Scott Forrest : E=MC squared "Between finished surface texture and raw quarried stone. Between hard materials and soft concepts. Between text and context." More information

22. What could I use? I don't understand? How are other people using the concept in different contexts?

23. SocialLearn+Media

24. SocialLearn+Media

26. What can CBMS be used for?

27. How does CBMS work?

28. CBMS for interactive learning

30. Speed

31. Memory

32. Precision Q

35. Local features – invariant to changes in scale and transformation (e.g., colour, rotation) Lowe2004 - http://www.cs.ubc.ca/~lowe/keypoints/

36. Feature Space X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X SIFT has 128 dimensions X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

37. Results Rank images by frequency of occurrence in the neighbourhoods of a query's keypoints Each nn is a keypoint of an image in the index, I. Q {x 1 , x 2 ,…, x n } SIFT Nearest Neighbour Index (FLANN) x 1 ->{nn 1 , nn 2 , ..., nn K } x 2 ->{nn 1 , nn 2 , ..., nn K } x n ->{nn 1 , nn 2 , ..., nn K } … I

42. Precision at 4: 0.77

44. Users like to know why

45. Confidence

47. I've got a screenshot – where does it come from? What is it's context?

48. I've got to prepare a course on Rome. I need some illustrations.

49. I don't understand this slide – find alternatives or discussions How can multimedia content-based search help?

51. Users like to know why (they're more forgiving!)

52. Tradeoff – speed vs accuracy

54. More sophisticated nearest neighbour analysis

59. to interact with material

60. to identify interesting stuff How can multimedia content-based search help?

62. Paul Hogan for iPhone development