2017-11 Recommendation Engine @ RaiseMe

THE RECOMMENDATION
ENGINE
EDMOND CHEUNG &TERRY CHAY
(PART OF THE AWESOME COLLEGE DISCOVERY CREW)
RAISEME OFFSITE
2017-11-01T15:25-15:55 PDT

CLARKE’S THREE LAWS (NOT IN ORDER)

WHY MAKE A
RECOMMENDATION ENGINE?

IMPROVE
STUDENT-
COLLEGE
MATCHING
STUDENTS ARE MORE ACTIVE IF THEY
FIND SCHOOLS THEY CARE ABOUT.
BUSINESS METRICS ARE BASED ON
APPLICATION RATE, NOT FOLLOW
RATE.

MAKE
PRODUCT
FLOW
INTUITIVE
GETTING MORE/BETTER
DATA WILL IMPROVETHE
MATCHING

“WHEN A DISTINGUISHED BUT ELDERLY
SCIENTIST STATESTHAT SOMETHING IS
POSSIBLE, HE IS ALMOST CERTAINLY
RIGHT.WHEN HE STATES THAT SOMETHING
IS IMPOSSIBLE, HE IS VERY PROBABLY
WRONG.”
— ARTHUR C. CLARK, “HAZARDS OF PROPHECY:THE FAILURE
OF IMAGINATION,” PROFILES OF THE FUTURE (1962)

I THINK IT SHOULD BE
POSSIBLE TO RECOMMEND
COLLEGES TO STUDENTS. IT'LL BE
LIKE COLLEGE SEARCH BUT
WITHOUT ANY INPUT FROM
THE USER.
OUR
DISTINGUISHED,
BUT ELDERLYCTO

…AND I TAKE EXCEPTION TO
"ELDERLY." I DO CROSSFIT SO I'M
QUITE SPRY.
OUR
DISTINGUISHED,
BUT ELDERLYCTO

…AND I TAKE EXCEPTION TO
"ELDERLY." I DO CROSSFIT SO I'M
QUITE SPRY.
OUR
DISTINGUISHED,
BUT ELDERLYCTO
IN FACT, YOU SHOULD TRY
CROSSFIT TOO,TERRY. IT'D BE
IMPOSSIBLE FOR YOU NOT TO
LOVE IT LIKE I DO!
“VERY
PROBABLY
WRONG”

ALMOST CERTAINLY RIGHT
AMAZON
“CUSTOMERS ALSO
BOUGHT”
AMAZON
“FREQUENTLY BOUGHT
TOGETHER”
AMAZON
“RECOMMENDATION TO
YOU…”

ALMOST CERTAINLY RIGHT
NETFLIX
“BECAUSE YOU
WATCHED ORANGE IS
THE NEW BLACK”
NETFLIX
“POPULAR ON NETFLIX”
NETFLIX
“% MATCH”

RECOMMENDATIONS EVERYWHERE!
TWITTER
“WHO TO FOLLOW”
“TRENDING NOW”,
“MOMENTS”
SPOTIFY
“RECOMMENDED
SONGS”
“RELATED ARTISTS”, “YOUR
DAILY MIXES”, RADIO
OKCUPID
“DOUBLETAKE”
“BROWSE MATCHES”, % MATCH

REQUEST PIPELINE
GET
BUCKETS
/COLLEGES/RECOMMENDATIONS
FRONT-END ASKS API FOR LIST OF
BUCKETS
GET RECOMMENDATIONS
FOR BUCKET
/V1/COLLEGES/DISCOVER/WEST
RUBY API IS GIVEN BUCKET
NAME (E.G.WEST) AND USES
"EXTRA" DATA TO CALL GO
API FOR RECOMMENDATIONS
LIST OF
BUCKETS
/V1/COLLEGES/DISCOVER
LIKE "MOVIES WITH A STRONG
FEMALE LEAD” BUT ARE THINGS
LIKE "BECAUSE YOU LIVE IN THE
WEST” (STORING "EXTRA"
DATA ON SERVER)
RENDER
RESULTS IN
BUCKET
/COLLEGES/RECOMMENDATIONS
SIMILAR TO HOW COLLEGE
SEARCH RENDERS SEARCH
RESULTS.
JAVASCRIPT
RUBY

THE RECOMMENDATION
ENGINE
THIS IS HOW THE REC ENGINE GENERATES
RECOMMENDATIONS FOR A BUCKET "WEST"
ROR API FOR BUCKET
“WEST”
NAME
POST EXTRA DATA TO GO
API
HTTPS://PROD-
RECOMMEND.RAISE.ME/API/V1/
DISCOVER/
CURL -I -H "CONTENT-TYPE: APPLICATION/JSON" -X
POST -D '{"CONTENT":{"STATES":["CA", "AK", "AZ",
"CO", "HI", "ID", "MT", "NM", "NV", "OR", "UT",
"WA", "WY"]}, "RANKING":"STUDENT-STUDENT",
"SCHOOL_ID":"535432DA385C8D490D000001",
"STATE":"CA",
"USERID":"5992010DE87EB427B1ECE416",
"ZIPCODE":"94121"}' HTTPS://PROD-
RECOMMEND.RAISE.ME/API/V1/DISCOVER/
EXTRACTS "RANKING"
AND UPLOADS RELEVANT
DATA TO PYTHON
COLLABORATIVE
FILTER (OR QUERIES
PYTHON PRE-COMPUTED
POPULARITY TABLE)
RANKING: "STUDENT-STUDENT",
STUDENTID:"5992010DE87EB427B1ECE416"
LOOK UP EXTRA DATA
{"CONTENT"=>{"STATES"=>["CA",
"AK", "AZ", "CO", "HI", "ID",
"MT", "NM", "NV", "OR", "UT",
"WA", "WY"]}, "RANKING"=>"STUDENT-
STUDENT",
"SCHOOL_ID"=>"535432DA385C8D490D00
0001", "STATE"=>"CA",
"USERID"=>"5992010DE87EB427B1ECE41
6", "ZIPCODE"=>"94121"}
JAVASCRIPT
RUBY
GOLANG
PYTHON

THE RECOMMENDATION
ENGINE
PYTHON MAGIC
HAPPENS
EXEC /USR/BIN/PYTHON34
STUDENTRANKS.PY
5992010DE87EB427B1ECE416
VIEWS_PLUS_FOLLOWING
IN THIS CASE IT'S A COLLABORATIVE
FILTER SO IT QUERIES THE PYTHON
MODEL AND GETS BACK A RANKED LIST
OF COLLEGE IDS
RUN RESULT THROUGH
CONTENT FILTER RDS
SELECT T1.ID, X.RANK FROM COLLEGE_CONTENT_FILTER
AS T1 INNER JOIN ( STUFF FROM PYTHON ) AS X (ID,
RANK) ON X.ID = T1.ID WHERE X.ID IS NOT NULL
AND SCHOOL_STATE IN ("CA", "AK", "AZ", "CO", "HI",
"ID", "MT", "NM", "NV", "OR", "UT", "WA", "WY"]) ORDER
BY X.RANK
RUBY RECEIVES AND
DOES EXTRA
PROCESSING
E.G. MAPS
"5498FFD56E670ECD4E00010C
" TO "UNIVERSITY OF SAN
DIEGO", ADDS ICON AND
FOLLOW STATUS, ETC.
RETURN
RECOMMENDATIONS TO
JAVASCRIPT FOR
RENDERING
NAME (E.G.WEST
JAVASCRIPT
RUBY
GOLANG
PYTHON

RUBY ON RAILS
ASIDE:WHY THREE LANGUAGES (+ JAVASCRIPT)?
RUBY PYTHON
DJANGO
JAVASCRIPT
RUBY
GOLANG
PYTHON

…FOUR YEARS AGO
IN MY ONLY SLIGHTLY LESS
DISTINGUISHED, BUT STILL SPRY
YOOT, I WAS A LITERAL ROCKSTAR SO
I TOTALLY PREFER PYTHON ON A
PLANE!! 🎸
(IN AN ALTERNATE GIT TIMELINE BRANCH)

DJANGO REINHARDT
FRENCH JAZZ GUITARIST (1910-1953)
JAZZ IS NOT ROCK AND ROLL!!

MONTY PYTHON (1969)
WHAT?! AND PYTHON IS
NAMED AFTER MONTY PYTHON AND
NOT THE SNAKE?
WHAT HAS MONTY
PYTHON EVER DONE FOR US?
BUFFY COULD TOTALLY RAMMED A
HOLY HAND GRENADE OF
ANTIOCH DOWN THE SPANISH
INQUISITION’S THROAT AND
THEN DRIVEN A STAKE IN THEM
JUST TO MAKE SURE!
I CHOOSE RUBY ON RAILS

…AND
THAT IS
WHY I HAD
TO SPEND
TWO WEEKS
LEARNING
GOLANG

“ANY SUFFICIENTLY ADVANCED
TECHNOLOGY IS
INDISTINGUISHABLE FROM
MAGIC.”

CLARKE'S THIRD
LAW
RECOMMENDO
COLLEGIA!
ANY SUFFICIENTLY ADVANCED
TECHNOLOGY IS INDISTINGUISHABLE FROM
MAGIC.
—ARTHUR C. CLARK, “HAZARDS OF PROPHECY:
THE FAILURE OF IMAGINATION,” PROFILES OF THE
FUTURE(1973)

RECOMMENDATIONS
➤ Student-student
• Recommend colleges based on similar
students
➤ College-college
• Recommend colleges based on similar
colleges
➤ State-college
• Recommend colleges based on state
➤ High school-college
• Recommend colleges based on high
school
➤ ZIP-college
• Recommend colleges based on ZIP
code
Collaborative filtering
Popularity tables

RECOMMENDATIONS
Popularity tables

College
W
College
X
College
Y
College
Z
Student
A
1 2 3 4
Student
B
1 ? 3 4
Student
C
4 3 2 1
Student
D
? 3 2 1
Student
E
? 1 ? 1
➤ Based on college page views
➤ Find two smaller matrices that
approximate this large matrix

X College Factors
StudentFactors

College
W
College
X
College
Y
College
Z
Student
A
1 2 3 4
Student
B
1 ? 3 4
Student
C
4 3 2 1
Student
D
? 3 2 1
Student
E
? 1 ? 1
~2
~4
~1 ~2
➤ Recommend colleges with the
highest predicted values

Popularity tables
College
W
College
X
College
Y
College
Z
State
Student
A
1 2 3 4 CA
Student
B
1 7 8 4 NY
Student
C
4 3 2 1 WA
Student
D
3 3 2 1 CA
Student
E
12 1 3 1 NY
Student
F
8 5 9 3 CA

Popularity tables
College
W
College
X
College
Y
College
Z
State
Student
A
1 2 3 4 CA
Student
D
3 3 2 1 CA
Student
F
8 5 9 3 CA
Student
B
1 ? 3 4 NY
Student
E
? 1 ? 1 NY
Student
C
4 3 2 1 WA

Popularity tables
College
W
College
X
College
Y
College
Z
State
Student
A, D, F
12 10 14 8 CA
Student
B, E
1 1 3 5 NY
Student
C
4 3 2 1 WA
➤ CA recommendations:
1. College Y
2. College W
3. College X
4. College Z

THINGS LEFT TO DO
•ADD SEGMENT TRACKING (FOR FEEDBACK)
•REPLACE COLLEGE-LISTING PAGE SUGGESTIONS WITH COLLEGE-COLLEGE
RECOMMENDATION ENGINE
•PORT OVER NEW MAJORS RANKING
•IMPROVE AND ITERATE ENGINE (HAVE CONTENT FILTER INTERACT WITH
RANKINGS?)

“THE ONLY WAY OF DISCOVERING
THE LIMITS OF THE POSSIBLE IS
TO VENTURE A LITTLE WAY PAST
THEM INTO THE IMPOSSIBLE.”

CLARKE'S SECOND
LAW
THE ONLY WAY OF DISCOVERING THE LIMITS
OF THE POSSIBLE IS TO VENTURE A LITTLE
WAY PAST THEM INTO THE IMPOSSIBLE.
—ARTHUR C. CLARK, “HAZARDS OF PROPHECY:
THE FAILURE OF IMAGINATION,” PROFILES OF THE
FUTURE (1962)

HELP
IMPROVE IT!
• ON PRODUCTION RIGHT
NOW
• IN TOP SECRET URL
• BEHIND FEATURE FLAG
(ONLY @RAISE.ME OR RAISE
HIGH STUDENTS) THAT WILL
SELF DESTRUCT WHEN IT GOES
LIVE
• HAS IMPOSTER MODE SO
YOU CAN "FACE/OFF" AS
OTHER USERS

HELP
IMPROVE IT!
• DEFAULT COLLEGE-COLLEGE
POPULARITY RANKING BASED ON
SIMILARITY SCORING (BM25
DISTANCE) INSTEAD OF VIEWS/
FOLLOWS
• "MAJORS" BUCKET USES IT’S OWN
POPULARITY TABLE INSTEAD
OF BEING A CONTENT FILTER
• THE CONTENT FILTER FOR
"LOW COST" BUCKET LOWERED
FROM $50,000/YEAR

HTTPS://WWW.RAISE.ME/COLLEGES/
RECOMMENDATIONS
HELP IMPROVE IT!

MORE THAN
COLLEGE
RECOMMENDATIONS

MORE THAN
COLLEGE
RECOMMENDATIONS
• COLLABORATIVE FILTER
• POPULARITY TABLE
• CONTENT FILTER
• (OR COMBINATION
THEREOF)
(ANYTHING REALLY!)

2017-11 Recommendation Engine @ RaiseMe

Recommended

Recommended

More Related Content

More from terry chay

More from terry chay (8)

Recently uploaded

Recently uploaded (20)

2017-11 Recommendation Engine @ RaiseMe