This is my presentation on accelerating k Nearest Neighbors text classification using an FPGA. I presented this paper at the EIT 2015 conference in Naperville, IL.

1. k-NN Text Classification using an FPGA-Based Sparse
Matrix Vector Multiplication Accelerator
Kevin R. Townsend, Song Sun, Tyler Johnson, Osama G. Attia,
Phillip H. Jones, and Joseph Zambreno
Reconfigurable Computing Laboratory
Iowa State University
EIT’15
Townsend et al. (RCL@ISU) kNN Text Classification EIT’15 1 / 11

2. Outline
1 What is k-NN text Classification?
2 Example
3 Mapping to an Accelerator
4 Results
Townsend et al. (RCL@ISU) kNN Text Classification EIT’15 2 / 11

3. What is k-NN text Classification?
k-NN Text Classification
autumn
leaves
butterfly
D1
D2
D5
D3,
D6
D4 class a
class b
Text classification is the
machine learning task to
classify documents.
Examples include spam filters,
classifying books in library
catelogs, and determining the
sub topic a conference paper is.
The problem can be simplified by converting documents into vectors,
also known as term-document vectors.
Each dimension in the model represents a word.
Each vector has a classification.
To classify a test document the document is converted into a vector
then the k nearest training vectors ‘vote’ to determine the
classification of the test document.
Townsend et al. (RCL@ISU) kNN Text Classification EIT’15 3 / 11

4. Example
Dataset
name class text
Training
D1 a
Autumn was it when we first met
Autumn is it what I can’t forget
Autumn have made me alive
D2 a
Grinning pumpkins, falling leaves,
Dancing scarecrows, twirling breeze,
Color, color everywhere,
Autumn dreams are in the air!
Autumn is a woman growing old
D3 b
butterfly, butterfly
fly in the sky
butterfly, butterfly
flies so high
D4 b
Hoping to catch your eye
Circling around you, oh my
Butterfly, butterfly, come into the light
Oh, what a beautiful sight
Testing
D5 a
Its autumn again
Leaves whisper the sound of our past
In loss they pay a descent
To the ground we fall
D6 b
Butterfly; butterfly fly away,
teach me how to be as free as free can be.
Butterfly; butterfly I see you there
Each document (poem)
belongs to either class a
(poem about autumn)
or class b (poem about
butterflies)
In order to test the
algorithm there needs to
be a training set and a
testing set.
Townsend et al. (RCL@ISU) kNN Text Classification EIT’15 4 / 11

5. Example
Converting into vectors
D1
D2
D3
D4
D5
D6
A
training
B
testing
class
name
a
a
b
b
a
b
autumn
met
alive
leaves
color
growing
butterfly
fly
sky
flies
high
hoping
sight
whisper
fall
teach
free
3 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
2 0 0 1 2 1 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 4 1 1 1 1 0 0 0 0 0 0
0 0 0 0 0 0 2 0 0 0 0 1 1 0 0 0 0
1 0 0 1 0 0 0 0 0 0 0 0 0 1 1 0 0
0 0 0 0 0 0 4 1 0 0 0 0 0 0 0 1 2
By converting the documents in term-document vectors, 2 sparse
matrices are created: the training matrix A and the testing matrix B.
Now we can find the distance between any 2 vectors. We use dot
products to determine distance.
Townsend et al. (RCL@ISU) kNN Text Classification EIT’15 5 / 11

6. Example
Distances and Sorting
Finding the
distance between
every test
document to
every train
document
equates to matrix
matrix
multiplication.
D1
D2
D3
D4
training
D5
D6
testing
3 0
3 0
0 17
0 8
D5 D6
D1,a,3
D2,a,3
D3,b,0
D4,b,0
D3,b,17
D4,b,8
D1,a,0
D2,a,0
k
sum a=6
b=0
b=25
a=0
We sort the values in each column while keeping track of the
documents.
We discard everything except the k = 2 largest dot products (smallest
distances).
Then add the values by class. The class with the largest sum is the
classification of the test document.
Townsend et al. (RCL@ISU) kNN Text Classification EIT’15 6 / 11

7. Mapping to an Accelerator
Profiling
words
documents(index)
0 261,976
year
33,652 1979
112,359 1989
213,221 1999
328,692 2009
374,989 2014
We need a larger dataset to test
performance.
Profiling reveals that SpMV takes 90%
of the runtime.
Percentofruntime
Na¨ıve
Parallel
0%
25%
50%
75%
100%
Other
Partial
Sorting
SpMV
Townsend et al. (RCL@ISU) kNN Text Classification EIT’15 7 / 11

8. Mapping to an Accelerator
Dataflow with Accelerator
Host
Training
Documents
Rainbow Matrix
R3
FormatterR3
Formatted Matrix
R3
Formatted Matrix Coprocessor
We have developed a
FPGA-based SpMV
accelerator called R3.
For the training phase
the matrix is converted
into a new format.
Host Coprocessor
Testing Documents
Rainbow
Testing Matrix
y Vector
Partial sort
Indices and values of
k nearest documents
Classify
R3 Formatted
Training Matrix
Zeroing
0 Vector
Vector filler
x Vector
R3 SpMV
y Vector
Townsend et al. (RCL@ISU) kNN Text Classification EIT’15 8 / 11

9. Results
Results
Runtime(relativetona¨ıve)
0%
20%
40%
60%
80%
100%
Na¨ıve
Parallel
FPGA
Units in
seconds
0.16
0.014
0.0097
Townsend et al. (RCL@ISU) kNN Text Classification EIT’15 9 / 11

10. Results
New profile
Percentofruntime
Na¨ıve Parallel FPGA
0%
25%
50%
75%
100%
Other
Partial
Sorting
PCIe Com-
munication
SpMV
SpMV still takes
the majority of
the runtime, so
the introduction
of PCIe time is
not a high
priority.
Townsend et al. (RCL@ISU) kNN Text Classification EIT’15 10 / 11

11. Results
Future Work
Currenly we perform sparse matrix sparse matrix multiplication as a
series of sparse matrix (dense) vector multiplication operations. We
could use bitmaps to reduce the memory bandwidth. (SpMV is
memory bound.)
Integration into existing programs like Rainbow.
Townsend et al. (RCL@ISU) kNN Text Classification EIT’15 11 / 11