The document discusses recommendations for improving recommendations on the Zillow real estate website. It recommends: 1) Adding an onboarding wizard to collect user preferences and encourage ratings of properties. 2) Displaying a progress bar to motivate users to provide more ratings to improve recommendations. 3) Allowing users to provide 5-point ratings of properties to collect explicit preference data. 4) Adding a "Recommended" tab displaying properties predicted to match a user's preferences based on combined collaborative and content-based filtering models. 5) Only sending property emails when high-probability matches are found to improve conversion rates.

1. Recommend-ify

2. We’ll start from the
homepage before we
login.

3. And now we’re
logged in

4. This would be a great
opportunity to start off the
session with a
recommendation, or at least
an acknowledgement that
the user has returned
(The fold)

5. (The fold)
This would be a great
opportunity to start off the
session with a
recommendation, or at least
an acknowledgement that
the user has returned
This could be a place to
show off new
recommendations to kick
off the users’ new session,
but unfortunately it’s below
the fold

6. Currently Zillow is totally
search-focused, which may
be a reflection of historical
user behavior. Let’s continue
our last search and see
where how that goes…

7. My last search is
automatically restored.
Hi San Diego!

8. The Newest results are
displayed by default which is
a reasonable starting point,
but far from optimal without
any personalization

9. Switching to “Featured”
brings up better looking
photos, and at first glance
seems to be a better starting
point

10. But, how are the featured
listings determined? All the
top “Featured” listings have
special offers. Do
sellers/realtors pay for this
ranking? Is this in any way
actually helping me find
what I want and making my
experience better?
Switching to “Featured”
brings up better looking
photos, and at first glance
seems to be a better starting
point

11. Ok, Let’s take a look at this one…

12. Ok, Let’s take a look at this one…
There’s a TON of great info here
that could be used in a
personalized recommender.
Price, Zestimate, Special Offers,
description, facts and features.
Scrolling further down there’s 7
more sections of different data
about the property, all of which
is super valuable to build a
content-based profile of each
user.

13. Ok, Let’s take a look at this one…
There’s a TON of great info here
that could be used in a
personalized recommender.
Price, Zestimate, Special Offers,
description, facts and features.
Scrolling further down there’s 7
more sections of different data
about the property, all of which
is super valuable to build a
content-based profile of each
user.

14. We’ve got all the property
data we could ever hope
for. It’s an embarrassment
of riches. That’s why
Zillow is so great for
searching… but that’s not
enough to build a REALLY
effective recommender.

15. We need:
PREFERENCE DATA!
We’ve got all the property
data we could ever hope
for. It’s an embarrassment
of riches. That’s why
Zillow is so great for
searching… but that’s not
enough to build a REALLY
effective recommender.

16. We’ve got all the property
data we could ever hope
for. It’s an embarrassment
of riches. That’s why
Zillow is so great for
searching… but that’s not
enough to build a REALLY
effective recommender.
We need:
PREFERENCE DATA!
There’s only one mechanism
for users to explicitly tell
Zillow what they want – the
“Save” and “Hide” buttons

17. We’ve got all the property
data we could ever hope
for. It’s an embarrassment
of riches. That’s why
Zillow is so great for
searching… but that’s not
enough to build a REALLY
effective recommender.
We need:
PREFERENCE DATA!
There’s only one mechanism
for users to explicitly tell
Zillow what they want – the
“Save” and “Hide” buttons
These buttons aren’t
featured prominently, and at
no point has the UI
encouraged me to use them.

18. We’ve got all the property
data we could ever hope
for. It’s an embarrassment
of riches. That’s why
Zillow is so great for
searching… but that’s not
enough to build a REALLY
effective recommender.
We need:
PREFERENCE DATA!
There’s only one mechanism
for users to explicitly tell
Zillow what they want – the
“Save” and “Hide” buttons
These buttons aren’t
featured prominently, and at
no point has the UI
encouraged me to use them.
Furthermore, there’s no way
to differentiate between “I
LOVE THIS HOUSE” and
“Ehhh, I’ll look again later”

19. Aside from explicit preference
information, we also have several
mechanisms to measure implicit
user preference…

20. Aside from explicit preference
information, we also have several
mechanisms to measure implicit
user preference…
How many times have they
looked at this property’s details?
How much time have they spent
on this property? How many
times has this property shown
up in different searches they
conducted?

21. Aside from explicit preference
information, we also have several
mechanisms to measure implicit
user preference…
Did they look at all the pictures?
Did they look at them more than
once? Is this more than they
usually look?
How many times have they
looked at this property’s details?
How much time have they spent
on this property? How many
times has this property shown
up in different searches they
conducted?

22. Aside from explicit preference
information, we also have several
mechanisms to measure implicit
user preference…
Did they look at all the pictures?
Did they look at them more than
once? Is this more than they
usually look?
How many times have they
looked at this property’s details?
How much time have they spent
on this property? How many
times has this property shown
up in different searches they
conducted?
How far down did they scroll? Is
there a particular section they
always expand?

23. Aside from explicit preference
information, we also have several
mechanisms to measure implicit
user preference…
Did they look at all the pictures?
Did they look at them more than
once? Is this more than they
usually look?
How many times have they
looked at this property’s details?
How much time have they spent
on this property? How many
times has this property shown
up in different searches they
conducted?
How far down did they scroll? Is
there a particular section they
always expand?
And the most important of all –
Did they fill in their info and click
CONTACT AGENT?

24. Email is another type of
interaction that can be very
helpful for the user, as well as a
mechanism to obtain more
implicit preference data.

25. Email is another type of
interaction that can be very
helpful for the user, as well as a
mechanism to obtain more
implicit preference data.
A few days after my last search I
get an email showing me some
new properties that match my
search filter.

26. Here’s one other type of
interaction that can be very
helpful for the user, as well as a
mechanism to obtain more
implicit preference data.
A few days after my last search I
get an email showing me some
new properties that match my
search filter.
Any property I click helps to
refine my interests which is
great. On the other hand, if the
properties they show aren’t
good fits I’m likely to
unsubscribe and close this data
stream for this user.

27. Here’s one other type of
interaction that can be very
helpful for the user, as well as a
mechanism to obtain more
implicit preference data.
A few days after my last search I
get an email showing me some
new properties that match my
search filter.
Any property I click helps to
refine my interests which is
great. On the other hand, if the
properties they show aren’t
good fits I’m likely to
unsubscribe and close this data
stream for this user.
The fact that they’re showing me
rentals when my saved search
was properties for sale isn’t a
good indication that these are
personalized at all.

28. Recommender Gameplan
Step 1:
Add an optional wizard as part of the onboarding process. Think
Netflix:
- First they allow users to explicitly define which categories they
like and don’t like.
- Then, they actively encourage users to provide ratings which
are extremely useful for both content-based and collaborative
filtering recommenders.

29. Recommender Gameplan
Step 2:
Add a “Progress Bar” or some visual indication to the user of how
many properties they’ve rated, similar to the “Profile completed”
indicator on LinkedIn:
- Encourage users to rate lots of properties
- Explain the benefit of doing so clearly
- Research the number of ratings required for a reasonably
accurate preference model

30. Recommender Gameplan
Step 3:
Add a 5-point rating scale to each property:
- This is the most useful explicit preference data that can be
obtained, and is much better than just “save” or “hide”

31. Recommender Gameplan
Step 4:
Add a new “Recommended” tab to the map view:
- Display properties that match the current search criteria/map
view, ordered by predicted preference
- Create a content-based recommender for each property that
predicts interest in the property based on viewer characteristics.
- Create a collaborative-filtering/nearest neighbor model to find
similar users
- Combine both models for extra accuracy

32. Recommender Gameplan
Step 5:
Only send when high-probability properties are discovered:
- Stop sending new properties in daily emails that (partially?)
match the most recent search criteria
- Should increase the conversion rate and decrease the
unsubscribe rate