This document summarizes the design and evaluation of two demonstrators for personalized online advertisements. The first demonstrator focuses on offline adaptation of ads based on movie context, user profile, social media posts, and preferences. An evaluation with 200 participants found that transparency and user control improved perceived quality, intention, understanding, and attitude toward ads. The second demonstrator aims to perform online adaptation by recognizing objects in videos and retrieving personalized ads to be integrated in real-time. It outlines the required technical capabilities and plans to leverage external APIs and partner technologies from the PARIS project.

1. PARIS Demonstrators
Yucheng Jin, Joris Klerkx, Erik Duval
Augment group
HCI @ KU Leuven
16 November 2015

2. Outline
• 2nd demonstrator
o Design and implementation
o Evaluation
o Limitations
• Final demonstrator
o Design
o Related work
o Technical requirements
o Implementation plan

3. 2nd demonstrator -
offline adaptation

4. Design
This proof-of-content demonstrator focuses on the retrieval of
relevant advertisements and off-line adaptation of the
advertisements .
The context of personalized ads is online movie.
The adaptation depends on four aspects:
• The types of movie
• Basic user profile on Facebook (Age, gender)
• Posts on Facebook wall (personality)
• Advertisement preferences

6. Advertiser: value presented by user data
User: value realized through personalization
What should we do toward such a trade-off?
Trade-off

7. Research hypothesis
We hypothesize that quality and effectiveness of personalized ads
can be increased by empowering users to explore and steer the
selection process.
To verify our assumption, we investigate the impact of Transparency
(T) and User control (UC) on four key aspects:
• Quality: interest match, context match, attractiveness and
annoyance?
• Behavioral intention: willingness to click, purchase, and use?
• Understanding: understand why and how a particular ad is
selected?
• Attitude: satisfaction, confidence and trust of ads?

8. https://www.facebook.com/business/products/ads
Transparency and user control of advertisement on Facebook

9. https://www.facebook.com/business/products/ads

10. Research methodology
Iterative design and rapid prototyping
Design
ImplementationEvaluate
Prototype design.
Refine the features of prototype
Implement the prototypeEvaluate the prototype with
users in diverse settings.

11. Workflow of demonstrator

12. http://paris-ad.evennode.com/paris/
Transparency
User control

13. Implementation
An web app of Facebook
RESTful API for accessing to user data and advertisement data
GET http://paris-ad.evennode.com/paris/api/ads?ageLevel=2
GET http://paris-ad.evennode.com/paris/api/user?id=564133123727385
PUT http://paris-ad.evennode.com/paris/api/user?id=564133123727385
data:{ gender : ”male”}
All data in JSON format

14. Schema of user data Schema of ad data
Filtering by

15. Evaluation
We conducted a between-subjects study on Amazon Mechanical Turk
(MTurk) where we recruited 200 subjects who have above 80% lifetime
approval rate for HITs.
Compensation was $1 for each study and average study completion time
was around 11 minutes.
We created four experimental conditions:
• Condition 1 (C1): (No-T & No-UC) base condition.
• Condition 2 (C2): (T & No-UC).
• Condition 3 (C3): (No-T & UC).
• Condition 4 (C4): (T & UC)

16. We used the user-centric evaluation framework of recommender
system and tailored the questionnaire to evaluate four aspects of
targeted advertisement: Quality, Behavioral intention, Understanding
and Attitude.
As a result, we created four post-study questionnaires QueA, QueB,
QueC and QueD to assess the effect of T and UC in different conditions.
~80% subjects noticed online targeted ads.
~10% subjects configured targeted ads.
Pu, Pearl, Li Chen, and Rong Hu. "A user-centric evaluation framework for recommender
systems." Proceedings of the fifth ACM conference on Recommender systems. ACM, 2011.

17. Common statements shown in four questionnaires.
Specific statements and optional questions regarding users’
perception of T (QueB), UC (QueC) and T & UC (QueD).
5-point Likert scale, Strongly agree - Strongly disagree

18. Evaluation steps
1. Introduce web app to subjects
2. Log in to the app with their Facebook accounts.
3. Play movie trailer and show ads.
4. During the trailer, subjects could rate the ads and configure ads.
5. After watching the trailer, subjects were asked to complete the
questionnaire.
What is the result ?

19. Common statements

20. Kruskal-Wallis Dunn post hoc
STM1
(Interest match)
(H=14.49, df=3, p=.002) C1 (Median: 3) and C4
(Median: 4), (p=.001)
STM2
(Willing. to click)
(H=11.42, df=3, p=.010) C1 (Median: 3) and C4
(Median: 4) (p=.014).
STM3
(Willing. to see)
(H=11.74, df=3, p=.008) C1 (Median: 3) and C4
(Median: 4), (p=.018)
C1 (Median: 3) and C2
(Median: 4), (p=.03)
STM2
(Understanding)
(H=13.68, df=3, p=.003) C1 (Median: 3) and C4
(Median: 4), (p=.009)
C1 (Median: 3) and C3
(Median: 4), (p=.010).
Statistical analysis result

21. Specific statements

22. Configuration Quality Behavioral
intention
Understanding Attitude
No-T & No-UC
T & No-UC ★
No-T & UC ★
T & UC ★ ★ ★

23. Submitted as a full paper to IUI 2016 (Rank A)
Deliverables 9.3 and 9.4

24. Limitations
• 70 elements of seven ad categories. Not a real data set of
ads.
• The algorithm for selecting appropriate ads is not validated.
• More advanced adaptive features based on vision
technology are not implemented.
• Not building directly on PARIS technology (yet).

25. Final demonstrator –
online adaptation

26. Design
This demonstrator will online analyze, adapt, and integrate content
and advertisements.
The same objects and attributes as used in the first two demonstrators
will be targeted, but now the linguistic or/and visual processing should
be very efficient timewise to ensure effective interaction.
A database of templates is queried and the advertisement template
is created in real-time adapted to the preferences of its user.

27. Scenario
Furniture advertisements

28. https://thetake.com/
Related work

30. Technical requirements
1
2
3
4
Object recognition
Ad retrieval
Advertisement adaptation
$349Buy
Link to webshops

31. Second screen (mobile)
apps with TV

32. 1. Object recognition
We need to recognize objects that appeared in a frame when the user
pauses the video.
• (The time when a particular object appears in a movie)
• The position where the recognized object appear in a key frame.
(Recognized objects should be labelled such as a rectangles)
• The description (query terms to webshops) of recognized object
(what is it? (chair, table), color, brand, etc.)
Discussed with VISICS

33. 2. Advertisement retrieval
• A valid data set of ads, each ad contains meaningful annotation such
as brand, color, category
o VLERICK, LIIR
• A defined user model for advertising (age, gender, personality …)
o CWI -> API
• An model for selecting advertisement for a targeted user
o CWI, LIIR
• A set of valid adaptive rules for showing personalized ads
o VLERICK

34. 3. Advertisement adaptation
• Modify a particular object according to the user profile.
o the object color
o the object orientation
o the object position
o the background of object
-> VISICS
• Maybe show these adaptations by using AR

35. 4. Link to webshops
• Issue query terms to webshops to find related furniture to the
identified object in the video (e.g., “EKTORP, Chair, Idemo red”)
https://developer.sears.com
http://docs.72lux.com/product-api-v1.html
Amazon Product Advertising API
Etc.
• Exact matching difficult
• Not too many products of furniture in these shopping APIs

36. Implementation plan
Iterative design and rapid prototyping
• Time:
~ January 2016 (1st version)
~ February 2016 (Evaluation for the 1st version)
~ March 2016 (2nd version with integrating other PARIS technology)
• Performance:
Online adaptation, (almost in) real time.
Might build on external APIs for some modules to speed up development
until PARIS technology is ready

37. Technical support PARIS partner Proposed date
Obj. recognition VISICS D4.4 Software for robust
recognition of object
classes in video (M30)
Data set of ads VLERICK, LIIR
User modelling for ads CWI D6.1 Software for inferring
demographic profile (M18)
D6.3 Software for learning
product preferences from
user generated content
(M24)
Model for selecting ads CWI, LIIR D7.1 Software for ad
selection model (M45)
Adaptive rules VLERICK D8.1 Report on the design
of personalized
advertisements (M24)
Obj. replacement VISICS D8.2 Software for object
replacement in images
and video (M42)

38. Thank you for your
attention.
Yucheng Jin
yucheng.jin@cs.kuleuven.be
Questions?