An innovative ensemble approach to image categorization was proposed that uses a hybrid of k-nearest neighbors (kNN) and support vector machines (SVM). The approach finds the k closest neighbors to a query sample and trains a local SVM on those neighbors. This preserves distance relationships while avoiding some issues with high variance that kNN faces. Empirical tests on challenging image datasets showed the hybrid method outperformed leading learning-based parametric image classification approaches.

1. Assignment 2 Application Case 6.5 Efficient Image Recognition and Categorization with kNN
Image recogniaion is an emerying dats mining appli-. As a technological discipline, computer
vision cation field involved in processing, analyzing, and secks to develop computer systems that
are capacatcgorizing visual objects such as pictures. In the ble of "secing" and reacting to their
environment. process of recognition (or catcgorization), images Examples of applications of
computer vision inctude are fint transformed into a multidimensional fea- systems for peocess
automation (industrial robots), ture spoce and then, using muchine-leuming navigation
(autonomous vehicles), monhoring/ sechniques, ane categorised into a finite number of detecting
(visual surveillance, searching and sorting classes Application areas of imuge recogribion and
visuals (indexing databuses of images and image categorization range from agricuture to
horncland sequences), engaging (oomprier-human interacsecurity, personalized marketing to
environmental tion), and inspection (manufacturing processes). protection. Image recognation is
an inteyral part of While the fichd of visual rocognikion and casegory to be done to neach humas
level perfomance. Current Ancether groop of reseanhers (Boiman et al, appecuadses ase capable
of dealing with only a limined 2008) argued that two peactice commonly used mumber of
categories (100 or so categories) and ane in image dessification methods (namely SVM and
computationally expensive. Mary machine leaming ANN:type modeh daven appeoaches and kN
type tectaigues (inctuding ANN, SVM, and LNO) ate used non parametric approuches) have led
to lest thanto develop conpuer sparms for visul nexprition desired perfonnance outcomes. They
also claim that and categorization, Though commendible foults have a hybrid method can
improve the performance of been oteined, generally speaking none of these tools image
reoognition and catcgatization. They propose in their curnent fom is capable of developing
spsems a trivial Naive Bayes iEN-bosed cassilier, which that can oompete with humans. employs
hN distances in the space of the local In a research project, sevenal reseanchers from imuge
descriptoes (and not in the spoce of imagee). the Computer Science Division of the Electrical
They claim that, although the modified kVN method Engineering and Computer Science
Depurtment at the is extremely simple, efficient, and requires no karnUniversily of Callfornia,
Berkeley, used an innovative ing/trining phuse, its performance ranks among the ensemble
approach to image cateporization (Zhang top leading learning baved parametne inuge clasd et al,
2000. They combdered visual category ree- fless. Empirkal comparbons of their method were
ogniition in the frainework of meavuring similari- shown on sevenal duallenging image
categonization ties, of pereptewl distances, to develop examples chtobuses (Caltech-101, Catect-
256, and Craz-01). of categories. Their recognition and categorizalion In addition to image
fecognitive and categoapproach was quite Aexible, permitting recogni- riazion, iNN is
successfully applied to complex tion bused on color, teature, and particularly shape. dassiceation
problems, such as content reuleval While nearest neighbor classifiers (Le., kNV are rat.
Gandwriting deretion, video content analysis, body aral in this setting, they suffered from the
problem and sign languge, where communication is done of high variance (in blas-variance
decompoition) using body or hand gestures), gene exprewion (this in the case of limited
sampling. Alernutively, one is ancaher area where kNN tends to perform betcould chocse to use
support vector machines but ter than other state of the ant tectiniques, in fact, a they also involve
time consuming optimization and combination of KNVSVM b one of the most popolar
compatations, They proposed a hytrid of these two tectiniques used here), and protein-so protein
inter methods, which deals naturally with the multiclass action and 3D structure peediction
(graph-based eNS setting. has reasonable computatiomal coinglexity is ofien used for interaction
structure prediction). bosh in training and at run time, and yieks excellent results in pactice. The

2. basic ides was to find Qursnoss ron Discussion dose neighbors to a query sample and train a
local sepport vector machine that preserves the distance 1. Why is image
recognition/dassification a worthy function on the collection of neighbon. Their method can be
appliced to lunge, malti- 2. How can iNN be effectively used for image rec: class data sets where
it outperforms nearest neighognition/dassitication applications? dient when the problem becomes
intractable. A wide KNN Divertinuse Mrana Nedibor. Clawillation for