Ryosuke Hattori, Kazushi Okamoto, Atsushi Shibata: Visualizing the Importance of Floor-Plan Image Features in Rent-Prediction Models, Joint 11th International Conference on Soft Computing and Intelligent Systems and 21st International Symposium on Advanced Intelligent Systems (SCIS-ISIS2020), 2020.12

1. / 17
SCIS 2020
2020.12.5
Visualizing the Importance of Floor-Plan Image Features
in Rent-Prediction Models
1
Ryosuke Hattori¹ Kazushi Okamoto¹ Atushi Shibata²
¹ Graduate School of Informatics and Engineering，The University of Electro Communications
² Graduate School of Industrial Technology，Advanced Institute of Industrial Technology

2. / 17
SCIS 2020
2020.12.5
Features of properties
• almost properties with different attributes
• property attributes
• age, story, building structure, and so on
• effect on prices
Rent prediction model with images
• inside image, outside image, and street view image
• conversion an image to a vector by using feature extractor from
images
• improvement of the prediction accuracy
Introduction
2
Eman Ahmed, and Mohamed Moustafa, House price estimation from
visual and textual features, arXiv:1609.08399, 2016．
Stephen Law, Brooks Paige, and Chris Russell, Take a Look Around:
Using Street View and Satellite Images to Estimate House Prices, ACM
Transactions on Intelligent Systems and Technology, Vol. 10, No. 5, pp.
54:1–54:19, 2019.
[Eman+, 2016] [Stephen+, 2019]

3. / 17
SCIS 2020
2020.12.5
Explanatory of Rent Prediction Model with Images
3
Real-estate agents
or room owners
Understand the importances
of the property attributes
by the regression coef
fi
cients
Can not understand the
importances of the image features
by the regression coef
fi
cients
ŷi = ↵1x1 + ↵2x2 + ↵3x3 + · · · + ↵nxn +
<latexit sha1_base64="vQx08F43bZHjET/G6F0shHJHj2A=">AAADG3ichVFNSxxBFKwZYzSbGFdzCSSHJYtBEKR3FRQhIOYSSA5+ZFVYZenptNo4OzPMzC7ZLPkD+QMeJIcEJIT8DC8h5OrBnyAeDeSSQ2pmhxFdTB7MdPXrqlfvdTuBa6JYiFPLHrg1eHto+E7h7r2R+6PFsfH1yG+FSteU7/rhpiMj7RpP12ITu3ozCLVsOq7ecPafJ+cbbR1Gxvdex51Abzflrmd2jJIxU36xji3sQSJGCR00YLiW8IxZCRdBetZAFxW8x9scTZFznVHNGdUbGDM5YyZjHOENfHpHxFN9fC/ne3lFB5p82SiWxbRIo9QPKhkoI4tlv/iF4sRMoYUmi3gso2gnaR6hzrEErWNs004iJDLpuaZ1gdoWWZoMyew+/7vc1bOsx31SM0rVii4uv5DKEibEifgqLsR38U2ciT831uqmNZJeOlydnlYHjdEPD9d+/1fVTJ9w71L1z55j7GA+7dWw9yDNJFOonr797uBibWF1ovtUfBbn7P+TOBXHnMBr/1JHK3r1EAU+QOX6dfeD9ep0hXhltry4lD3FMB7hCSZ533NYxAssowZlPbaWrJfWK/ujfWz/sH/2qLaVaR7gStgnfwFUmKzq</latexit>
<latexit sha1_base64="vQx08F43bZHjET/G6F0shHJHj2A=">AAADG3ichVFNSxxBFKwZYzSbGFdzCSSHJYtBEKR3FRQhIOYSSA5+ZFVYZenptNo4OzPMzC7ZLPkD+QMeJIcEJIT8DC8h5OrBnyAeDeSSQ2pmhxFdTB7MdPXrqlfvdTuBa6JYiFPLHrg1eHto+E7h7r2R+6PFsfH1yG+FSteU7/rhpiMj7RpP12ITu3ozCLVsOq7ecPafJ+cbbR1Gxvdex51Abzflrmd2jJIxU36xji3sQSJGCR00YLiW8IxZCRdBetZAFxW8x9scTZFznVHNGdUbGDM5YyZjHOENfHpHxFN9fC/ne3lFB5p82SiWxbRIo9QPKhkoI4tlv/iF4sRMoYUmi3gso2gnaR6hzrEErWNs004iJDLpuaZ1gdoWWZoMyew+/7vc1bOsx31SM0rVii4uv5DKEibEifgqLsR38U2ciT831uqmNZJeOlydnlYHjdEPD9d+/1fVTJ9w71L1z55j7GA+7dWw9yDNJFOonr797uBibWF1ovtUfBbn7P+TOBXHnMBr/1JHK3r1EAU+QOX6dfeD9ep0hXhltry4lD3FMB7hCSZ533NYxAssowZlPbaWrJfWK/ujfWz/sH/2qLaVaR7gStgnfwFUmKzq</latexit>
<latexit sha1_base64="Mhn41z7d/Q2n10aqYFv7sL5P/oA=">AAADG3ichVHLShxBFD3dxqiTh6PZBJLFkMEQEKRmXCiCIGYTiAsfGRUmMlRXSi3s6W66ewYnQ34gP5BFyMKASMhnuAkhWxd+grg04MZFTvc0LXEwudBdp26dc8+9VU7gmigW4tSyB+4M3h0aHincu//g4WhxbHw98luh0jXlu3646chIu8bTtdjErt4MQi2bjqs3nL2XyflGW4eR8b03cSfQW02545lto2TMlF+s4y12IRGjhA4aMFxLmGdWwkWQnjXQRQUfsJ+jSXJuMqo5o3oLYzpnTGeMQ7yDT++IeLKP7+V8L6/oQJMvG8WymBJplPpBJQNlZLHsF48oTswUWmiyiMcyinaS5hHqHEvQOsYW7SRCIpOea1oXqG2RpcmQzO7xv8NdPct63Cc1o1St6OLyC6ksYUKciG/iQvwQ38WZuLq1VjetkfTS4er0tDpojH58vHb5X1UzfcLda9U/e46xjdm0V8PegzSTTKF6+vb7Txdrc6sT3efiqzhn/wfiVBxzAq/9Wx2u6NXPKPABKjevux+sV6cqxCvV8sJi9hTDeIJneMH7nsECXmEZNSjrqbVovbaW7C/2sf3T/tWj2lameYS/wj75A1P4rOg=</latexit>
rent image feature
story
age
ŷ
<latexit sha1_base64="xhpVBugrKGqsl/IOdSNq+waH9Pk=">AAACbXichVHLSsNAFD2N7/qqiiAoUixVV2WqguJKdOPSttYWq5QkTjWYJiGZFmroD7gWXIiCgoj4GW78ARf9BHHhQsGNC2/SgGhR7zAzZ87cc+fMjGLpmiMYa4SktvaOzq7unnBvX//AYGRoeMsxK7bKs6qpm3ZekR2uawbPCk3oPG/ZXC4rOs8ph2vefq7KbUczjU1Rs/huWd43tJKmyoKobXfnQBZurV6PFiMxlmB+RFtBMgAxBLFhRm6wgz2YUFFBGRwGBGEdMhxqBSTBYBG3C5c4m5Dm73PUESZthbI4ZcjEHtK4T6tCwBq09mo6vlqlU3TqNimjiLNHdste2QO7Y0/s49darl/D81KjWWlquVUcPB7LvP+rKtMscPCl+tOzQAlLvleNvFs+491CbeqrR6evmeV03J1mV+yZ/F+yBrunGxjVN/U6xdNnCNMHJH8+dyvYmksk5xNzqYXYymrwFd0YxxRm6b0XsYJ1bCBL5xo4wTkuQi/SqDQhTTZTpVCgGcG3kGY+AbAsjgA=</latexit>
image feature
It is useful to understand which parts of the image are relevant
for a rent prediction

4. / 17
SCIS 2020
2020.12.5
Floor-Plan Standards in Japan
4
kitchen and
other rooms
kitchen square
[0 m2,4.5 m2) [4.5 m2, 8.0 m2)
more than equal to
8.0 m2
not
separated
Room
(R)
separated
Kitchen
(K)
Dinning Kitchen
(DK)
Living Dinning Kitchen
(LDK)
DK LDK
K
R

5. / 17
SCIS 2020
2020.12.5
Floor-Plan Images
Floor-plan standard is the same, but
fl
oor layout is different
In japan, there is a custom to look at
fl
oor-plan images
when searching for a desired rental property [Kiyota+，2017]
5
Ex. Different
fl
oor-plan images in the same property and
fl
oor-plan standard
Y. Kiyota, T. Yamasaki, H. Suwa, C. Shimizu:
Real estate and AI, Journal of the Japanese
Society for Arti
fi
cial Intelligence, vol.32, no.4,
pp.529-535, 2017

6. / 17
SCIS 2020
2020.12.5
Purpose
• to develop methods visualizing the importances of
fl
oor-plan
image features
Method
• build rent prediction models with
fl
oor-plan images
• regressor: Bayesian linear regression
• feature extractor
• Principal Component Analysis (PCA)
• Speed Up Robust Features (SURF) + Bag of Features (BoF)
6
Our Approach

7. / 17
SCIS 2020
2020.12.5
✓(f) = ✓T
f
Prediction Model with Floor-Plan Image
7
ln
<latexit sha1_base64="cip2YhSnDwuYWh3CEG2B1EDrAqU=">AAACZnichVFNSwJBGH7cvsxKrYiCLpIYnWS0oOgkdenoR36Aiuxuoy2uu8vuKpj0B4KueehUEBH9jC79gQ7+g6KjQZcOva4LUVK9w8w888z7vPPMjGSoimUz1vMIY+MTk1Pead/M7Jw/EJxfyFl605R5VtZV3SxIosVVReNZW7FVXjBMLjYkleel+v5gP9/ipqXo2qHdNni5IdY0parIok1UpqRqlWCYRZkToVEQc0EYbiT14C1KOIIOGU00wKHBJqxChEWtiBgYDOLK6BBnElKcfY5T+EjbpCxOGSKxdRprtCq6rEbrQU3LUct0ikrdJGUIEfbE7lifPbJ79sI+fq3VcWoMvLRploZablQCZyuZ939VDZptHH+p/vRso4odx6tC3g2HGdxCHupbJ91+Zjcd6ayza/ZK/q9Yjz3QDbTWm3yT4ulL+OgDYj+fexTk4tHYZjSe2gon9tyv8GIVa9ig995GAgdIIkvn1nCOC3Q9z4JfWBKWh6mCx9Us4lsIoU+/norO</latexit>
u
<latexit sha1_base64="x42OIFZO63bltL4H6gjza3ReZFg=">AAACdXichVHLSsNAFD2N7/po1I0gQrBaXOlEBMWV6Malr2qhlZKk0xrMi7ygBn/AH3ChGwUV8TPc+AMu/ARxqeDGhTdpQLSoN2TumTP33DkzozqG7vmMPWWEjs6u7p7evmz/wOBQThwe2fXswNV4UbMN2y2piscN3eJFX/cNXnJcrpiqwffUw7V4fS/krqfb1o7fdPi+qTQsva5rik9UVRSjimobNa9pUpKCY6kq5tksS0JqB3IK8khjwxZvUEENNjQEMMFhwSdsQIFHXxkyGBzi9hER5xLSk3WOY2RJG1AVpwqF2EMaGzQrp6xF87inl6g12sWg3yWlhGn2yG7ZK3tgd+yZffzaK0p6xF6alNWWljvV3MnY9vu/KpOyj4Mv1Z+efdSxlHjVybuTMPEptJY+PDp93V7emo4K7JK9kP8L9sTu6QRW+KZdbfKtM2TpAeSf190OdudnZcKbC/mV1fQpejGOSczQfS9iBevYQJH2DXGOK1xn3oUJYUootEqFTKoZxbcQ5j4BcSyQhw==</latexit>
<latexit sha1_base64="x42OIFZO63bltL4H6gjza3ReZFg=">AAACdXichVHLSsNAFD2N7/po1I0gQrBaXOlEBMWV6Malr2qhlZKk0xrMi7ygBn/AH3ChGwUV8TPc+AMu/ARxqeDGhTdpQLSoN2TumTP33DkzozqG7vmMPWWEjs6u7p7evmz/wOBQThwe2fXswNV4UbMN2y2piscN3eJFX/cNXnJcrpiqwffUw7V4fS/krqfb1o7fdPi+qTQsva5rik9UVRSjimobNa9pUpKCY6kq5tksS0JqB3IK8khjwxZvUEENNjQEMMFhwSdsQIFHXxkyGBzi9hER5xLSk3WOY2RJG1AVpwqF2EMaGzQrp6xF87inl6g12sWg3yWlhGn2yG7ZK3tgd+yZffzaK0p6xF6alNWWljvV3MnY9vu/KpOyj4Mv1Z+efdSxlHjVybuTMPEptJY+PDp93V7emo4K7JK9kP8L9sTu6QRW+KZdbfKtM2TpAeSf190OdudnZcKbC/mV1fQpejGOSczQfS9iBevYQJH2DXGOK1xn3oUJYUootEqFTKoZxbcQ5j4BcSyQhw==</latexit>
<latexit sha1_base64="x42OIFZO63bltL4H6gjza3ReZFg=">AAACdXichVHLSsNAFD2N7/po1I0gQrBaXOlEBMWV6Malr2qhlZKk0xrMi7ygBn/AH3ChGwUV8TPc+AMu/ARxqeDGhTdpQLSoN2TumTP33DkzozqG7vmMPWWEjs6u7p7evmz/wOBQThwe2fXswNV4UbMN2y2piscN3eJFX/cNXnJcrpiqwffUw7V4fS/krqfb1o7fdPi+qTQsva5rik9UVRSjimobNa9pUpKCY6kq5tksS0JqB3IK8khjwxZvUEENNjQEMMFhwSdsQIFHXxkyGBzi9hER5xLSk3WOY2RJG1AVpwqF2EMaGzQrp6xF87inl6g12sWg3yWlhGn2yG7ZK3tgd+yZffzaK0p6xF6alNWWljvV3MnY9vu/KpOyj4Mv1Z+efdSxlHjVybuTMPEptJY+PDp93V7emo4K7JK9kP8L9sTu6QRW+KZdbfKtM2TpAeSf190OdudnZcKbC/mV1fQpejGOSczQfS9iBevYQJH2DXGOK1xn3oUJYUootEqFTKoZxbcQ5j4BcSyQhw==</latexit>
<latexit sha1_base64="x42OIFZO63bltL4H6gjza3ReZFg=">AAACdXichVHLSsNAFD2N7/po1I0gQrBaXOlEBMWV6Malr2qhlZKk0xrMi7ygBn/AH3ChGwUV8TPc+AMu/ARxqeDGhTdpQLSoN2TumTP33DkzozqG7vmMPWWEjs6u7p7evmz/wOBQThwe2fXswNV4UbMN2y2piscN3eJFX/cNXnJcrpiqwffUw7V4fS/krqfb1o7fdPi+qTQsva5rik9UVRSjimobNa9pUpKCY6kq5tksS0JqB3IK8khjwxZvUEENNjQEMMFhwSdsQIFHXxkyGBzi9hER5xLSk3WOY2RJG1AVpwqF2EMaGzQrp6xF87inl6g12sWg3yWlhGn2yG7ZK3tgd+yZffzaK0p6xF6alNWWljvV3MnY9vu/KpOyj4Mv1Z+efdSxlHjVybuTMPEptJY+PDp93V7emo4K7JK9kP8L9sTu6QRW+KZdbfKtM2TpAeSf190OdudnZcKbC/mV1fQpejGOSczQfS9iBevYQJH2DXGOK1xn3oUJYUootEqFTKoZxbcQ5j4BcSyQhw==</latexit>
x = [u, ✓(v)]
<latexit sha1_base64="tbvrKtiF/xOXDGMbwaD2COlchy8=">AAACqXichVFNSxtBGH6y9SPGr9hehF6WhpQUJUxUUApCsD30mKjRYBLC7jqawf1idxIal/0D/QMeemqhlLY/w4tHLz2k/6B4jNBLD32zWZAYrO8w8z7zvO/zzjszumsKXzLWSyhPJianppMzqdm5+YXF9NLTA99pewavGI7peFVd87kpbF6RQpq86npcs3STH+pnbwbxww73fOHY+7Lr8oalndriRBiaJKqZfhvUdcc89rsWOfV9qG6rtRGqHa6qdbclmmpQly0utTA3Eu+ErxrNdIblWWTqOCjEIIPYSk76K+o4hgMDbVjgsCEJm9Dg06ihAAaXuAYC4jxCIopzhEiRtk1ZnDI0Ys9oPaVdLWZt2g9q+pHaoFNMmh4pVWTZT/aN9dkV+8F+s78P1gqiGoNeuuT1oZa7zcUPy3t/HlVZ5CVad6r/9ixxgq2oV0G9uxEzuIUx1HfOL/p7r3ezwUv2md1Q/59Yj13SDezOrfGlzHc/IkUfULj/3OPgYC1fWM+vlTcyxZ34K5J4jhfI0Xtvooh3KKFC537HNXr4pawoZaWqHA1TlUSseYYRU4x/JIqlEA==</latexit>
特
徴
量
属性
ベクトル
賃料
Image
Features
Property
Variables
Log
Rent
f
w
P(w|Y, X) = N(w|µ̂, ˆ
⌃)
Bayesian linear regression:
of posterior distribution:
ŷ
w
P(ŷ|x, w) = N(ŷ|wT
x, )

8. / 17
SCIS 2020
2020.12.5
Feature Extraction of Floor-Plan Image by PCA
✓(f) = ✓T
f
f
µ̂f is coef
fi
cients for the
fl
oor-plan image
8
✓
✓ is a feature extractor
is a transformation matrix
✓µ̂f
Overlay: Visualized
feature importance
×
f

9. / 17
SCIS 2020
2020.12.5
Feature Extraction of Floor-Plan Image by BoF
d1
d2
d3
K-means
Floor-plan image
9
Visualized
feature importance
SURF
v1
v2
v3
v4
v1 v2 v3 v4
v1 v2 v3 v4
v1 v2 v3 v4
✓(Di) =
1
|Di|
X
d2Di
✓(d)
v1 v2 v3 v4
w4
w3
w1 w2
w5
Coef
fi
cient of the Floor-Plan Image
(d1, d2, · · · , dj)

10. / 17
SCIS 2020
2020.12.5
Experiment Environment
LIFULL HOME ’S dataset (as of September 2015)
• rental property data (70 variables, 5.33 million)
• 120 x 120 pixel image data (83 million
fi
les)
• used data
• Tokyo
• 100,000 properties
• 29 property variables
•
fl
oor-plan image
10

11. / 17
SCIS 2020
2020.12.5 11
Dataset (1/2)
building structure (13 variables)
exclusive area num. of rooms
num. of ground
fl
oors
num. of
underground
fl
oors
fl
oor num.
age of building property type purpose of use building structure
orientation of
property
keeper days of occupancy
num. of vacant
properties
the others: one-hot vector
: continuous variables

12. / 17
SCIS 2020
2020.12.5 12
Dataset (2/2)
location/access (15 variables)
city, ward, town city plan rail station 1 rail station 2 bus. times 1
bus. times 2
dist. to general
hospital
walking dist. to
station 1
or bus stop
walking dist. to
station 2
or bus stop
dist. to
convenience store
dist. to super
market
dist. to junior high
school
riding dist. to
station 1
riding dist. to
station 2
dist. to elementary
school
the others: one-hot vector
: categorical each 80m
grayscale
Floor-plan images

13. / 17
SCIS 2020
2020.12.5
Outline of Experiment
13
Evaluation metric: Mean Absolute Percentage Error (MAPE)
Training data
(80%)
Test data
(20%)
K
R
DK
LDK
Using as training data to predict
test data
5-fold cross validation
• Prediction accuracy
• Visualizing the importance
of
fl
oor-plan image features
13
5,000
properties
determine dimension of image features
from [500,1000,2000]
ȳi = E[P(ŷi|x, w)]
MAPE =
100
N
N
X
i=1
yi ȳi
yi

14. / 17
SCIS 2020
2020.12.5
Prediction Accuracy for Test Data
K R DK LDK
w/o FPI
(Floor-Plan Image)
11.84 10.51 7.700 12.37
w/ FPI
PCA
500
dimension
12.10 10.80 7.806 12.71
BoF
2000
dimension
11.68 10.27 7.660 12.09
MAPE [%]
14
Lower MAPE means better prediction accuracy

15. / 17
SCIS 2020
2020.12.5 15
Visualizing the Importance of Floor-Plan Image
Features in Floor-Plan Standard LDK
BoF
PCA

16. / 17
SCIS 2020
2020.12.5 16
Visualizing the Importance of Floor-Plan Image
Features
PCA for
fl
oor-plan standard K
PCA for
fl
oor-plan standard DK BoF for
fl
oor-plan standard DK
BoF for
fl
oor-plan standard K

17. / 17
SCIS 2020
2020.12.5
Purpose: to develop methods visualizing the importances of
fl
oor-plan image features
Method
• build rent prediction models with
fl
oor-plan images
• regressor: Bayesian linear regression
• feature extractor: PCA and BoF
Result
• BoF achieves a higher accuracy than PCA
• the wall and on orientation symbol are important
fl
oor-plan image features
• visualization by BoF is superior in visibility of which parts of a
fl
oor-plan image are
important
Future work: to trend analysis of importance of
fl
oor-plan image features
17
Conclusion
for each
fl
oor-plan standard