A short introduction for new programmers to the concepts behind a general, workhorse machine learning algorithm, k-Nearest Neighbor. Talk given at Pair Columbus, 23 May 2015

1. Machine Learning
in Ruby
An Introduction

2. kNN Algorithm
Who am I?
Brad Arner
Head of Product
website: http://www.bradfordarner.com
email: brad@capstory.me
twitter: @bradfordarner
github: github.com/arnerjohn

3. The Goal

4. Make Machine Learning
Less Scary

5. The Caveat

6. Machine Learning is a
Huge Topic

7. I will talk about a general,
workhorse algorithm

8. k Nearest Neighbor
(kNN)

9. Fill in missing information

10. kNN Algorithm
What is the value of a home?
$250K
$275K
$280K
$240K
$235K
$215K
$240K
$195K
$300K
$210K
$250K
$????
1650 sq ft 2100 sq ft 1950 sq ft 1700 sq ft 2100 sq ft 1800 sq ft
2100 sq ft1700 sq ft2100 sq ft 1650 sq ft 1800 sq ft1950 sq ft

11. kNN Algorithm
What is in a name?
k =>
N =>
N =>
Number of Elements
Nearest / Distance
Neighbors / More Exist

12. kNN Algorithm
What is in a name?
k =>
N =>
N =>
Number of Elements
Nearest / Distance
Neighbors / More Exist
k = 3
Square Feet
Value
Rooms

13. kNN Algorithm
What is in a name?
k =>
N =>
N =>
Number of Elements
Nearest / Distance
Neighbors / More Exist
k = 5
Square Feet
Value
Rooms

14. kNN Algorithm
What is in a name?
k =>
N =>
N =>
Number of Elements
Nearest / Distance
Neighbors / More Exist
Square Feet
Value
Rooms

15. kNN Algorithm
What is in a name?
k =>
N =>
N =>
Number of Elements
Nearest / Distance
Neighbors / More Exist
Value: $200K
Value: $220K
Value: $210K
Value: $195K
Value: $215K
Estimated Value: $208K

16. The Code

17. github.com/arnerjohn/machine_learning_in_ruby
eventually @
http://www.machinelearninginruby.com

18. The Code
General Organization
class Property
attr_accessor :rooms, :area, :type
def initialize(options = {})
options = {
rooms: 1,
area: 500,
type: false
}.merge(options)
@rooms = options[:rooms]
@area = options[:area]
@type = options[:type]
@neighbors = []
@distance = nil
@guess = nil
end
.
.
.
end
class Orchestrator
def initialize(k)
@property_list = []
@k = k
@rooms = { max: 0, min: 10000, range: 0 }
@area = { max: 0, min: 10000, range: 0 }
end
.
.
.
end

19. The Code
Program Execution
property_list = Orchestrator.new(3)
property_list.load_training_data
property_list.scale_features
property_list.add( Property.new({ rooms: 2, area:
1550, type: false }) )
property_list.add( Property.new({ rooms: 4, area:
1800, type: false }) )
property_list.determine_unknowns

20. The Code
Load Data
property_list = Orchestrator.new(3)
property_list.load_training_data
property_list.scale_features
property_list.add( Property.new({ rooms: 2, area:
1550, type: false }) )
property_list.add( Property.new({ rooms: 4, area:
1800, type: false }) )
property_list.determine_unknowns
def load_training_data
file = CSV.read(“data.csv”, { headers: true })
file.each do |line|
property = Property.new({rooms:
line[“rooms”].to_i, area: line[“area”].to_i, type:
line[“type”]})
add(property)
end
end

21. The Code
Load Data
property_list = Orchestrator.new(3)
property_list.load_training_data
property_list.scale_features
property_list.add( Property.new({ rooms: 2, area:
1550, type: false }) )
property_list.add( Property.new({ rooms: 4, area:
1800, type: false }) )
property_list.determine_unknowns
def scale_features
rooms_array = self.filter_knowns.map do |p|
property.rooms
end
area_array = self.filter_knowns.map do |p|
property.area
end
@rooms[:min] = rooms_array.min
@rooms[:max] = rooms_array.max
@rooms[:range] = rooms_array.max -
rooms_array.min
@area[:min] = area_array.min
@area[:max] = area_array.max
@area[:range] = area_array.max -
area_array.min
end

22. The Code
Find Neighbors
property_list = Orchestrator.new(3)
property_list.load_training_data
property_list.scale_features
property_list.add( Property.new({ rooms: 2, area:
1550, type: false }) )
property_list.add( Property.new({ rooms: 4, area:
1800, type: false }) )
property_list.determine_unknowns
def filter_unknowns
property_list.select do |property|
property.type == false
end
end
def determine_unknowns
self.filter_unknowns.each do |property|
property.neighbors = self.filter_knowns
property.calculate_neighbor_distances(self.ro
oms[:range], self.area[:range])
property.guess_type(self.k)
end
end

23. The Code
Calculate Distances
property_list = Orchestrator.new(3)
property_list.load_training_data
property_list.scale_features
property_list.add( Property.new({ rooms: 2, area:
1550, type: false }) )
property_list.add( Property.new({ rooms: 4, area:
1800, type: false }) )
property_list.determine_unknowns
def calculate_neighbor_distances(room_range,
area_range)
@neighbors.each do |neighbor|
rooms_delta = neighbor.rooms - self.rooms
area_delta = neighbor.area - self.area
rooms_delta = rooms_delta / room_range.to_f
area_delta = area_delta / area_range.to_f
neighbor.distance =
Math.sqrt(rooms_delta*rooms_delta +
area_delta*area_delta)
end
end

24. The Code
Guess Type
property_list = Orchestrator.new(3)
property_list.load_training_data
property_list.scale_features
property_list.add( Property.new({ rooms: 2, area:
1550, type: false }) )
property_list.add( Property.new({ rooms: 4, area:
1800, type: false }) )
property_list.determine_unknowns
def guess_type(k)
guess_hash =
gen_guess_hash(self.sort_neigbors_by_distance.
take(k))
@guess = assign_guess(guess_hash)
msg = %Q{
Property attrs => rooms: #{ @rooms }, area:
#{ @area }
The property type is guessed to be: #{ @guess
}
}
puts msg
return @guess
end

25. The Code
Guess Type
property_list = Orchestrator.new(3)
property_list.load_training_data
property_list.scale_features
property_list.add( Property.new({ rooms: 2, area:
1550, type: false }) )
property_list.add( Property.new({ rooms: 4, area:
1800, type: false }) )
property_list.determine_unknowns
def gen_guess_hash(properties)
guess_hash = Hash.new(0)
properties.each do |property|
guess_hash[property.type] += 1
end
return guess_hash
end
def assign_guess(guess_hash)
highest = 0
guess = ""
guess_hash.each do |key, value|
if value > highest
highest = value
guess = key
end
end
return guess
end

26. Questions?

27. Thank You!