You Might Also Like: Implementing User Recommendations in Rails

You
May
Also
Be
Interested
In:
Implemen'ng
User
Recommenda'ons
in
Rails

Ma#hew
Deiters
theagiledeveloper.com
Thursday, June 10, 2010


@mdeiters



@mdeiters

How
do
they
do
that?



@mdeiters

think
about
our
data
in

new
ways



@mdeiters

Understand
the
tools
to
do

something
interesBng



@mdeiters

Use
those
tools
in
your

Rails
applicaBon



@mdeiters

"Everything
should
be
made
as
simple
as

possible,
but
not
simpler."

-‐
Albert
Einstein



@mdeiters

CreaBve
RevoluBon
the
Golden
Age
of
Adver;sing



@mdeiters

Their
vision
of
consuming
life
had
liIle
to
do
with

actual
experience
of
American
consumers



@mdeiters

An
Academic
approach

“Adver;sers
followed
the
wishes
of
conserva;ve
corporate
clients,
who
wanted
the
safe,
“scien;fc”
adver;sing



@mdeiters

Technology
Changed
&
Approach
Changed



@mdeiters

In
the
late
40’s,
only
.05%
of
Americans
had
a
TV

End
of
50’s,
sBll
only
about
50%

5
years
later
in
1962,
90%
of
Americans
had
a
TV



@mdeiters

Audience
SegmentaBon
Adver;sing
was
comfortable
“Male
dominated”
“academic”
approach



@mdeiters

SimilariBes
in
Web

Development
Today



@mdeiters

NoSQL
Dynamic
Languages



@mdeiters

We’ve
been
doing
CRUD
for
30
years
boring
for
us
and
the
user



@mdeiters

Our
CreaBve
RevoluBon
How
can
we
change
our
approach



@mdeiters

RecommendaBons
=
Money

Amazon
is
reportedly
making
~25%
of

25% sales
on
personalized
suggesBons

Net
sales
are
at
$7.13
billion
in
the
ﬁrst
quarter

Jan-‐Mar
2010
-‐
roughly
2
billion
in
3
months

75%



@mdeiters

RecommendaBons
=
Traﬃc
#1
&
#2
–
StumbleUpon's
rank
among
social
media
traﬃc
sources
in
the
US.

118%
-‐
the
growth
in
acBve
users
since
2009.

Almost
10
Million
registered
users

400
–
average
number
of
Bmes
a
user
stumbles
per
month.



@mdeiters

Discovering
the
relaBonships
in
your
data

Modeling
the
relaBonships

Using
graphs
in
your
Rails
App



@mdeiters

President Obama ”The greatest thing about Facebook, is that you can
quote something and totally make up the source.” - George
Washington
Dick Cheney

Precious
John Adams lol...so true



@mdeiters

http://www.hackdiary.com/2010/02/10/algorithmic-recruitment-with-github/


@mdeiters

Social
Networking

Content

Website
AnalyBcs

PredicBve
Analysis



@mdeiters

RelaBonal
Databases
!=
works
SQL
is
set
based



@mdeiters

Lady Gaga
Kevin Bacon



@mdeiters

Example:
Finding
2
degrees
away
for
a
user
in
SQL

-‐
100
people
executes
in
0.01
sec

-‐
1000
people
executes
in
0.1
sec

-‐
4000
people
executes
in
17.78
sec

-‐
60K
over
an
hour



@mdeiters

SQL
Smell
2
degrees more
then
2
degrees
SELECT b FROM ( WITH RECURSIVE transitive_closure(a, b, distance, path_string) AS
WITH RECURSIVE transitive_closure(a, b, distance, path_string) AS ( SELECT a, b, 1 AS distance,
( SELECT a, b, 1 AS distance, a || '.' || b || '.' AS path_string a || '.' || b || '.' AS path_string,
FROM edges2 b AS direct_connection
WHERE a = 1 -- set the starting node FROM edges2
WHERE a = 1 -- set the starting node
UNION ALL
UNION ALL
SELECT tc.a, e.b, tc.distance + 1,
tc.path_string || e.b || '.' AS path_string SELECT tc.a, e.b, tc.distance + 1,
FROM edges2 AS e tc.path_string || e.b || '.' AS path_string,
JOIN transitive_closure AS tc ON e.a = tc.b tc.direct_connection
WHERE tc.path_string NOT LIKE '%' || e.b || '.%' FROM edges2 AS e
AND tc.distance = 0 JOIN transitive_closure AS tc ON e.a = tc.b
) WHERE tc.path_string NOT LIKE '%' || e.b || '.%'
SELECT b FROM transitive_closure AND tc.distance < 3
UNION ALL )
(WITH RECURSIVE transitive_closure(a, b, distance, path_string) AS SELECT * FROM transitive_closure
( SELECT a, b, 1 AS distance, a || '.' || b || '.' AS path_string --WHERE b=3 -- set the target node
FROM edges2 ORDER BY a,b,distance
WHERE a = 4 -- set the target node

UNION ALL

SELECT tc.a, e.b, tc.distance + 1,
tc.path_string || e.b || '.' AS path_string
FROM edges2 AS e
JOIN transitive_closure AS tc ON e.a = tc.b
WHERE tc.path_string NOT LIKE '%' || e.b || '.%'
AND tc.distance = 0
)
SELECT b FROM transitive_closure
)) AS immediate_connections
GROUP BY b
HAVING COUNT(b) > 1;

Courtsey of http://techportal.ibuildings.com/2009/09/07/graphs-in-the-database-sql-meets-social-networks/


@mdeiters

How
is
Luke
connected
to
Princess
Leah?

First Last
1 Luke Skywalker
2 Darth Vader
3 Princess Leah



@mdeiters

Graphs
Rela;onships
are
a
First
Class
Ci;zen
just
like
the
data

First Last
1 Luke Skywalker

Father

2 Darth Vader

Father

3 Princess Leah



@mdeiters

Rows
Nodes
First Last
1 Luke Skywalker
Node
Father
Point
2 Darth Vader
Actor
Father
Vertex
3 Princess Leah



@mdeiters

Edges
First Last
1 Luke Skywalker
Edge
Father
Rela;onship
2 Darth Vader
Arc
Father
Link
3 Princess Leah



@mdeiters

Less
complex,
100%
natural

Luke

Darth
Princess



@mdeiters

In
Memory
Ruby
Graph
Great
for
small
staBc
datasets
or
ad
hoc
querying



@mdeiters

RGL

Rails App

RGL

or

BackroundRB

Rails App
RGL



@mdeiters

# http://github.com/fmeyer/rgl

require 'rubygems'
require 'rgl/adjacency'
require 'rgl/dot'

graph = RGL::DirectedAdjacencyGraph.new
graph.add_edge 'mary', 'john'
graph.add_edge 'mary', 'henery'
graph.add_edge 'john', 'frank'
graph.add_edge 'frank', 'henery'

# Use DOT to visualize this graph:
# http://graphviz.org/Download_macos.php
graph.write_to_graphic_file('jpg')
`open graph.jpg`



@mdeiters

20,000
nodes
and
1
million
edges
=
300
MB



@mdeiters

Persistence
&
Dynamic
Data



@mdeiters

Neo4j



@mdeiters

Neo4j.rb

Rails App

Neo4j
Neo4j.rb
Lucene



@mdeiters

class Person
include Neo4j::NodeMixin
property :name
has_n :friends
end

Neo4j.start
Neo4j::Transaction.run do
andreas = Person.new :name => 'andreas'
john = Person.new :name => 'John', :age => 30
andreas.friends << john
end



@mdeiters

Neo4jr-‐social
More
REST,
Less
Java

Rails App RDBMS

Neo4jr-social

Neo4j

Solr
Lucene

Memcache



@mdeiters

Neo4jr-‐social
The
SOLR
of
Graphs
(uses
Neo4jr-‐simple)

#> sudo gem install neo4jr-social

#> start-neo4jr-social



@mdeiters

No
JRuby
Required

Packaged
as
self-‐contained
jeIy
webserver

Deployable
WAR

Focused
on
SNA

Basic
Built
in
Querying

Extensible
~/.neo4jr-‐social



@mdeiters

Facebook’s
Friend
SuggesBons
Demo
Johnathan



@mdeiters

LinkedIn’s
Degrees
of
SeparaBon
Demo
Johnathan



@mdeiters

http://wiki.github.com/tinkerpop/gremlin/



@mdeiters

SQL Graphs
(Neo4j)
Assembly
Required Works
out
of
the
Box


AllSimplePaths
&
ShortestPath
(Who
do
we
follow
on
Twiber
in
common)


Dijkstra
(Who
do
we
follow
on
Twiber
in
common
-‐
on
steroids)

score: 20 score: 5

score: 5

score: 5
score: 5

score: 5

score: 10

score: 10


Closeness
Centrality
(Who
has
the
most
followers
on
Twiber)


Betweenness
Centrality
(Who
has
more
inﬂuen;al
people
following
them
on
Twiber)

http://www.stoweboyd.com/message/its-betweenness-that-matters-not-your-eigenvalue-the-dark-ma.html

Eigenvector
Centrality
(PageRank)



@mdeiters

Performance



@mdeiters

Sparse
vs
Dense


Metadata

Person Project
started_on => 'Jan 1 2010'
finished_on => 'Jan 7 2010'
hours => 40
type => 'worked_on'


Metadata

Person Project
started_on => 1265639038
finished_on => 1266243897
hours => 40
type => 'worked_on_reorg'


Breadth or Depth



@mdeiters

What
Next...



@mdeiters

Inspired
to
think
about

your
data
in
new
ways



@mdeiters

Graphs
Understand
the
tools
to
do
something

interes;ng
with
it



@mdeiters

Use
these
tools
in
your

Rails
applicaBon


NoSQL

AugmentaBon
Solr

RDBMS

Application

Neo4j
Lucene

Memcache



@mdeiters

CreaBve
RevoluBon



@mdeiters

QuesBons?



@mdeiters

Links
•h#p://gist.github.com/431000
•h#p://github.com/andreasronge/neo4j
•h#p://github.com/mdeiters/neo4jr-‐simple
•h#p://github.com/mdeiters/neo4jr-‐social
•h#p://www.hackdiary.com/2010/02/10/algorithmic-‐recruitment-‐with-‐github/
•h#p://techportal.ibuildings.com/2009/09/07/graphs-‐in-‐the-‐database-‐sql-‐meets-‐social-‐networks/


You Might Also Like: Implementing User Recommendations in Rails

Recommended

Recommended

More Related Content

Similar to You Might Also Like: Implementing User Recommendations in Rails

Similar to You Might Also Like: Implementing User Recommendations in Rails (8)

Recently uploaded

Recently uploaded (20)

You Might Also Like: Implementing User Recommendations in Rails