PerVehicle 2019 workshop paper

1. TOWA R D A B ETTER IPA EXPER IEN C E FOR A C ON N EC TED
VEH IC LE B Y MEA N S OF U SA GE PR ED IC TION
O s a m u M a s u t a n i , S h u n g o N e m o t o , Yu s u k e H i d e s h i m a
A C M S , N i s s a n M o t o r C o . , L t d .
Per Vehicle 2019
11/03/2019

2. CONNECTED VEHICLE IN NISSAN - ACMS
2
_ End to end connected experience are going to be built
_ Key factor : smart device integration, intelligent assistance
Connected
Experience
Connected Assistance
Car life assistance
Emergency support
Intelligent Navigation
Door to door navigation
Fleet management
Traveler Assistance
Smart phone apps
Shared travel
Sharing mobility
Car sharing
Multi modal traffic
Security
Driver surveillance
Vehicle security
Vehicle management
Remote maintenance
PAYD/PHYD insurance
Remote control
Remote console
Remote charge management
ACMS - solutions
* ACMS – Alliance Connected Vehicle and Mobility Service

3. 3
Off-board On-board
80’s
2019
Ideas of on/off board voice activated vehicle

4. ALEXA SKILL FOR CONNECTED VEHICLE
4
Amazon Echo
Leaf
Skill
Connected
Vehicle Server
Connected EV
“Leaf”

5. SUMMARY
5
_ Enhancement on IPA (Intelligent Personal Agent)
experience for connected vehicle
_ Problem
Voice operation is sometimes annoying.
Wake word, application word, command name…
_ Solution
Vehicle usage prediction
Enables proactive reminder from IPA which
ask probable actions.
Alexa open
vehicle skill
Charge
battery
Turn on air
conditioner
IPA Connected Vehicle
May I charge
vehicle now ?
Yes
May I turn on air
conditioner ?
Usage Prediction
Please
Ok
Ok
User
Proposal
How usage prediction refine user experience on IPA enabled
connected experience ?

6. B A C K G R O U N D
MARKET TREND
6
_ Voice activated IPA (aka VPA) is widely used
Rapid adoption than smart phone
Over 45 million smart speakers are sold in 2018 in US *
Over 20% of US population *
_ Connected vehicle is ready to buy anywhere
Connected vehicle will soon hit majority threshold
Over 30 million connected cars shipped in 2018 **
Will reach 75% of cars in 2020 **
_ IPA x Connected vehicle becomes more popular
* eMarketer : https://www.emarketer.com/content/hey-alexa-whos-using-smart-speakers
** Business insider : https://www.businessinsider.com/connected-cars-will-reach-a-75-share-of-the-total-car-market-by-2020-2015-2

7. B A C K G R O U N D
COMMON PROBLEM
7
_ IPA usage is not continue for long time
70% of users will not continue to use the IPAs [7]
_ Major reason to avoid of IPA use
difficulty of access (ex. non-intuitive word selection)
social embarrassment (ex. situation which voice might disturb others)
[7] L. ; Zhao, X. ; Lu, Y. Hu, L. ; Zhao, X. ; Lu, and " A Hu, “A Proposed Theoretical Model of Discontinuous Usage of Voice-Activated
Intelligent Personal Assistants (IPAs),” in Twenty-Second Pacific Asia Conference on Information Systems 2018 Proceedings, 2018, p. 245.

8. SOLUTION : USAGE PREDICTION
8
_ Usage prediction can realize
proactive IPA activation
Can reduce conversational turns
_ Drive preparation
Turning on air conditioner
Charge battery
Navigation setting
Contents sync
_ 2 scenarios
Prediction to preparation
Preparation to prediction
Prediction to preparation scenario
(prediction as a recommendation)
Driving
IPA operation
Usage prediction
Shall I turn on air
conditioner ?
Driving
Usage prediction
Preparation to prediction scenario
(improve prediction accuracy)
Turn on air
conditioner

9. PROBLEM DEFINITION
9
_ Source data : time series of activity
From vehicle
Time stamp
Driving, charging, parking
Location
From IPA
Time stamp
Intent
_ Target output
Predicted activity
Occurrence of driving status in a certain time
Estimated time to departure
Short time horizon / long time horizon
Hour 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Driving
Charging
Location H H H H H H H H O O O O O O O O E E E H H H H H
Hourly sequence of activities
Hour 0 1 2 3 4 5 6 7
Driving
Charging
Location H H H H H H H H
Hour 0 1 2 3 4 5 6 7
Driving
Charging
Location H H H H H H H H
Short term prediction
Long term prediction
Hour 0 1 2 3 4 5 6 7
Driving
Charging
Location H H H H H H H H
Short term prediction w/ IPA
Long term prediction w/ IPA
Prediction to preparation scenario Preparation to prediction scenario
Hour 0 1 2 3 4 5 6 7
Driving
Charging
Location H H H H H H H H
IPA

10. ALEXA INTENT USAGE
10
_ It is often used in the morning and holiday
_ A/C intent is majority of morning utterance
_ A/C intent is often used in winter and summer
_ Charging is often used in the night and morning
dailyhourly monthly
A/C control
Charging

11. B A S I C A N A LY S I S
RETENTION RATE OF IPA USAGE
11
_ Much user quit using Alexa Leaf Skill
45% retention rate (thresh = 3 month absent)
62% retention rate (thresh = 6 month absent)
45% retention
thresh=90days
62% retention
thresh=180days

12. B A S I C A N A LY S I S
TYPICAL USAGE PATTERN
12
_ Sometimes there is strong correlation between A/C and driving.
_ Nightly charges tend to lead driving next day.

13. B A S I C A N A LY S I S
TIME TO DRIVE
13
_ When the user will start driving after last action ?
Time gap from A/C by Alexa to Drive
53% (25/47) of users actually drove 1 hour after Alexa operation
Average time to drive is 878 sec
Time gap from charge by Alexa to Drive
93% (981/1051) of users actually drove in 24 hour
Average time to drive is 5.9 hours
time to drive (sec)
time to drive (hour)

14. PROPOSED METHOD
14
_ User segmentation
Find cluster according to usage pattern
_ Prediction Model
Dynamic Bayesian network
Predict next states with current or former states
Temporal and non-temporal node
Can involve hidden node (Hidden Markov Model)
User segmentation
Vehicle usage
data
IPA usage data
Usage Prediction
Drive preparation

15. USER SEGMENTATION
15
_ Identify user segments in diverse usage types
_ Clustering method
Week x hour matrix as a usage pattern
LDA (Latent Dirichlet Allocation) is applied to
reduce dimension
Clustering with K-Means to identify usage
pattern clusters
hour
weekday
LDA
Latent vector
Clustering
Clustered
Charge
Office
Home
Moving

16. PREDICTION MODEL
16
_ Dynamic Bayesian Networks
_ Variables
Status of vehicle : driving / charging
Hour in a day, day in a week
Alexa operation (intent)
_ Time span
1 hour for short term prediction
4 hours for long term prediction
𝑆𝑡 ∈ 𝑑𝑟𝑖𝑣𝑖𝑛𝑔, 𝑐ℎ𝑎𝑟𝑔𝑖𝑛𝑔, 𝑝𝑎𝑟𝑘𝑖𝑛𝑔 : 𝑠𝑡𝑎𝑡𝑢𝑠 𝑜𝑓 𝑣𝑒ℎ𝑖𝑐𝑙𝑒
ℎ 𝑡 ∈ 1 … 24 : ℎ𝑜𝑢𝑟 𝑖𝑛 𝑎 𝑑𝑎𝑦
𝑤𝑡 ∈ 1 … 7 : 𝑑𝑎𝑦 𝑖𝑛 𝑎 𝑤𝑒𝑒𝑘
𝑔𝑡 ∈ ℎ𝑜𝑚𝑒, 𝑜𝑓𝑓𝑖𝑐𝑒, 𝑜𝑡ℎ𝑒𝑟 : 𝑙𝑜𝑐𝑎𝑡𝑖𝑜𝑛
𝐼𝑡 ∈ 𝑠𝑡𝑎𝑟𝑡 𝑎𝑐, 𝑐ℎ𝑎𝑟𝑔𝑒 𝑜𝑡ℎ𝑒𝑟 ∶ intent of Alexa at time t
ቍP 𝑋1
𝑡
, … , 𝑋 𝑛
𝑡
= ෑ
𝑖=1
𝑛
𝑋𝑖
𝑡
|𝑃𝑎 𝑋𝑖
𝑡
, 𝑃𝑎(𝑋𝑖
𝑡−1
)

17. E VA L U AT I O N
SHORT TERM PREDICTION WITH A/C
17
_ Overall accuracy is around 85%-90%
Improved by Alexa data
HMM gives better prediction
Recall is relatively low.
HMM
DBN
w/o Alexa
w/ Alexa
w/o Alexa
w/ Alexa

18. E VA L U AT I O N
LONG TERM PREDICTION
18
_ Overall accuracy is around 70%
Precision is improved with charge evidence
User segment information also improves recall
w/o Charge
w/ Charge
w/o Charge w/ user segment
w/ Charge w/ user segment
Global model
Segmented model

19. E VA L U AT I O N
TIME DURATION PREDICTION
19
_ Time to drive prediction
Bayesian network is used as regression
model
_ Shot term
From A/C Alexa usage to actual drive time
Result shows certain predictability.
_ Long term
From charge start to actual drive time
Result shows more improvement is
necessary.
RMSE MAE 𝑅2
497 375 0.54
Long term
Short term
RMSE MAE 𝑅2
4.07 2.95 0.31
)P(𝑇𝑡 |𝑇𝑡−1 , ℎ 𝑡 , 𝑤𝑡

20. CONCLUSION & FUTURE WORKS
20
_ Feasibility of usage prediction
Vehicle usage prediction by DBN can be predicted in short term and long term.
Evidence of IPA operation significantly improve prediction accuracy.
User segmentation is necessary prior to usage prediction.
_ Future works
Additional features like battery level for better prediction performance
Application to traveler assistance (destination prediction)
Application for power management (SoC prediction)