Peter Brusilovsky presents research on user control in adaptive information access systems. The document discusses three types of adaptive systems - adaptive hypermedia, adaptive search, and recommender systems. It explores how each system currently handles user control and collaboration with AI, and proposes methods to improve user control and transparency. These include allowing users to control personalization parameters, fusion of multiple ranking sources, and visualizing recommendation results to better understand the reasoning process. The goal is to develop systems where AI provides information and users make informed decisions, with the human firmly in control.

1. Two Brains are Better than One:
User Control in Adaptive
Information Access
Peter Brusilovsky
PAWS Lab
School of Computing and Information
University of Pittsburgh

2. AI or Humans + AI?
3

3. Two Brains are Better than One!
4
Image
credit:
https://towardsdatascience.com

4. Adaptive Information Access
Adaptive
Hypermedia
Adaptive
IR
Recommender
Systems
Text/Link Navigation Search Recommendation
Metadata-based
mechanism
Keyword-based
mechanism
Community-
based mechanism
Adaptation Mechanisms
Types of information access

5. User / AI control in 3 types of IA
Adaptive HM Adaptive Search Recommendation
User driven
How to add AI?
How to control it?
Some user
control (query)
How to improve?
Fully AI Driven
Add control
Collaborate

6. ADAPTIVE HYPERMEDIA:
SUPPORTING HUMAN NAVIGATION
Adding AI to user-controlled information access environment
8

7. Adaptive Hypermedia Technologies
AI provides information,
human makes an informed decision
AI is fully present
Human is in control

8. Navigation vs. Adaptive Sequencing
10
Human makes navigation decision AI makes navigation decision

9. Adaptive Navigation Support: Goals
• Guidance: Help me to find what I need!
– Local guidance (“next best”)
– Global guidance (“ultimate goal”)
• Orientation: Where am I?
– Local orientation support (local area)
– Global orientation support (whole hyperspace)

10. Adaptive Navigation Support
AI provides information,
human makes an informed decision
AI is fully present
Human is in control

11. WebWatcher: Direct Guidance
Joahims,
T.,
Freitag,
D.,
and
Mitchell,
T.
(1997)
WebWatcher:
A
tour
guide
for
the
World
Wide
Web.
In:
Proceedings
of
15th
International
Joint
Conference
on
Artificial
Intelligence,
IJCAI'97,
Nagoya,
Japan,
pp.
770-775.

12. ELM-ART: Adaptive Annotation (1996)
Weber,
G.
and
Brusilovsky,
P.
(2001)
ELM-ART:
An
adaptive
versatile
system
for
Web-based
instruction.
International
Journal
of
Artificial
Intelligence
in
Education
12
(4),
351-384.

13. NavEx: Adaptive Annotation
Brusilovsky,
P.
and
Yudelson,
M.
(2008)
From
WebEx
to
NavEx:
Interactive
Access
to
Annotated
Program
Examples.
Proceedings
of
the
IEEE
96
(6),
990-999.

14. Annotation: Knowledge vs Goals
ScentTrails
QuizGuide

15. ANS vs Search/Recommendations
• Presentation
– In-context guidance vs. generated ranked list
• Power
– Multipe personalization engines in ANS, relevance
engine in IR
– ANS can display simultaneously several aspects of
importance/interest/relevance
– Ranking used in recommendation approaches can
express only one dimension

16. More Control! Open Learner Model
Weber, G. and Brusilovsky, P. (2001) ELM-ART: An adaptive versatile system for Web-based instruction. International Journal of
Artificial Intelligence in Education 12 (4), 351-384.

17. Open Learner Models
Bull, S., Brusilovsky, P., and Guerra, J. (2018) Which Learning Visualisations to Offer Students? In: V. Pammer-Schindler, M. Pérez-Sanagustín, H.
Drachsler, R. Elferink and M. Scheffel (eds.) Proceedings of 13th European Conference on Technology Enhanced Learning, EC-TEL 2018, Leeds, UK,
September 3–5, 2018, Springer, pp. 524–530.

18. Adaptive Annotation Can:
• Reduce navigation efforts
• Reduce repetitive visits to learning content
pages
• Encourage non-sequential navigation
• Increase learning outcome
• For those who is ready to follow and advice
• Make system more attractive for students
• Students stay much longer without any reward

19. Adaptive Presentation

20. Adaptive Presentation: Goals
• Provide the different content for users
with different knowledge, goals,
background
• Provide additional material for some
categories of users
– comparisons
– extra explanations
– details
• Remove irrelevant or already known
content

21. AP: NL Generation in PEBA-II
Milosavljevic, M. (1997) Augmenting the user's knowledge via comparison. In: A. Jameson, C. Paris and C. Tasso (eds.)
Proceedings of 6th International Conference on User Modeling, UM97, Chia Laguna, Sardinia, Italy, June 2-5, 1997,
SpringerWienNewYork, pp. 119-130.

22. Adaptive Comparison in PEBA-II

23. Canned Text: Scaling
Tsandilas,
T.
and
M.
C.
schraefel
(2004).
"Usable
adaptive
hypermedia
systems."
New
Review
in
Hypermedia
and
Multimedia
10(1):
5.

24. User Control: Scrutable
Adaptive Presentation in SASY
Czarkowski, M. and Kay, J. (2002) A scrutable adaptive hypertext. In: P. De Bra, P. Brusilovsky and R. Conejo (eds.)
Proceedings of Second International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH'2002),
Málaga, Spain, May 29-31, 2002, pp. 384-387.

25. Human-AI Collaboration: Stretchtext

26. More User Control: Open Model
Tsandilas, T. and M. C. schraefel (2004). "Usable adaptive hypermedia systems." New Review in Hypermedia and
Multimedia 10(1): 5.

27. Adaptive presentation: Evaluation
• Decrease reading comprehension time
• Increase learning outcome
• No effect for navigation overhead - time,
number of nodes visited, number of
operations

28. USER-CONTROLLED
ADAPTIVE SEARCH AND
RECOMMENDATION
Adding user-control to AI-driven information access in
personalized search and recommender systems
30

29. Search vs. Recommendation
• Adaptive Search
– Results are generated on the basis of user
query and user profile
– Some user control / collaboration is
embedded (query!) – the system has an idea
what the user wants now
• Personalized Recommendation
– Results are generated on the basis of user
profile alone - no control, no collaboration

30. 32

31. • Compromise between several sources of relevance
– Items might be relevant for to the user profile or query
for different reasons
• Single-source: different parts/aspects of the profile
• Hybrid: different sources of information or approaches
• Hard to get universally perfect ranking
– A recommendation approach is tuned to an
overall/generic situation, but users could consult
recommendation for different needs
– Some profile aspects, sources, approaches are less
relevant in the current context, but some are more
33
While Single Ranked List is A Problem?

32. What are Possible Solutions?
• Control (Keep the ranked list, better engage users)
– Change user profile
– Change parameters (how personalization is produced)
• Visualize and Explore (Go beyond the ranked list)
– Present items visually
– Make the ranking/relevance process more transparent
– Allow users to change presentation parameters, play
with the results, better understand the process, isolate
most relevant results
34

33. What Can Be Controlled?
35
Profile Generation Presentation
User Model Ranking
Source Fusion
EXPLORE!

34. Simple Ranking Control
Allow the user to control how the ranking list is produced to adapt
personalization for the current context as well as better explore
recommendation results
37

35. How Ranking is Generated?
• [your profile] + [your query] +
personalization engine = ranked list
• Control fusion – profile vs query
• Control query (current needs)
• Control profile (generic preferences)
• Control the engine
38

36. Should we use profile or query?
• AI approach: Everything is done by AI
– use ML to classify queries into those where profile
is good and those where it is not
– White, R. W., Bennett, P., and Dumais, S. (2010) Predicting
Short-Term Interests Using Activity-Based Search Context.
In: Proceedings of the 19th ACM conference on Information
and knowledge management (CIKM 2010), Toronto, Canada,
October 2010 ACM, pp. 1009-1018.
• Human-AI collaboration approach:
– User decides whether to use personalization, AI
does the job
– TaskSieve

37. TaskSieve: Controllable Personalized Search
Ahn, Jae-wook, Peter Brusilovsky, Daqing He, Jonathan Grady, and Qi Li. 2008. "Personalized Web Exploration with Task Models." In the
17th international conference on World Wide Web, WWW '08, 1-10. Beijing, China: ACM.

38. TaskSieve Controllable Ranking
• Combine query relevance and task relevance
– Alpha * Task_Model_Score + (1-alpha) * Search
Score
– Alpha : user control (0.0, 0.5, or 1.0)
• Results
– Better than regular adaptive search
– Better then non adaptive baseline even in cases
when profile was excluded
– Users were really good in deciding when to engage
the profile and how
41

39. O'Donovan, John, Barry Smyth, Brynjar Gretarsson, Svetlin Bostandjiev, and Tobias Höllerer. 2008. "PeerChooser: visual interactive recommendation."
In Proceedings of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, 1085-88. Florence, Italy: ACM.
PeerChooser: Control the Engine
42

40. YourNews: Control the Profile
Ahn, J.-w., Brusilovsky, P., Grady, J., He, D., and Syn, S. Y. (2007) Open user profiles for adaptive news systems: help or harm? In: 16th
international conference on World Wide Web, WWW '07, Banff, Canada, May 8-12, 2007, ACM, pp. 11-20
Open user
model

41. Concept-Level Open User Model
(SciNet)
44
Glowacka, Dorota, Tuukka Ruotsalo, Ksenia Konuyshkova, Kumaripaba Athukorala, Samuel Kaski, and Giulio Jacucci. 2013.
"Directing Exploratory Search: Reinforcement Learning from User Interactions with Keywords." In international conference on
Intelligent user interfaces, IUI '2013, 117-27. Santa Monica, USA: ACM Press.

42. Movie Tuner: Control Current Interests
45
Vig,
J.,
Sen,
S.,
and
Riedl,
J.
(2012)
The
Tag
Genome:
Encoding
Community
Knowledge
to
Support
Novel
Interaction.
ACM
Transactions
on
Interactive
Intelligent
Systems
2
(3),
Article
13.

43. uRank: Fine Control of Interests
di Sciascio, C., Sabol, V., and Veas, E. E. (2016) Rank As You Go: User-Driven Exploration of Search Results.
In: Proceedings of the 21st International Conference on Intelligent User Interfaces (IUI '16), Sonoma, California, pp. 118-129.

44. Control and Transparency:
Two Sides of the Same Coin
Explain Visualize
Explore
Control
48
Transparency
Controllability
No full transparency
without controllability
Control is challenging
without transparency

45. TasteWeights: Profile and Mechanism
Control + Transparency
49
Knijnenburg, Bart P., Svetlin Bostandjiev, John O'Donovan, and Alfred Kobsa. 2012. "Inspectability and Control in Social Recommenders." In 6th ACM
Conference on Recommender System, 43-50. Dublin, Ireland.

46. Multiple Sources of Relevance
• Conference Navigator System for conference support (2010+)
• Classic content-based relevance prospects (search)
– Items that has a specific keyword
• Social relevance prospects (browsing)
– Items bookmarked by a socially connected user
• Tag relevance prospects (browsing)
– Items tagged by a specific tag
• Personal relevance prospects (recommendation)
– Several different recommender engines
– Each engine offer one relevance prospect
50
Brusilovsky, P., Oh, J. S., López, C., Parra, D., and Jeng, W. (2017) Linking information and people in a social system for academic
conferences. New Review of Hypermedia and Multimedia 23 (2), 81-111.

47. SetFusion: User-Controlled Fusion
• Using set relevance visualization in
the familiar Venn diagram form
– One recommendation source = one set
• Allow controlled ranking
fusion
• Combine ranking with
annotation showing source(s)
of recommendation
55
Parra, D. and Brusilovsky, P. (2015) User-controllable personalization: A case study
with SetFusion. International Journal of Human-Computer Studies 78, 43–67.

49. Set Fusion: Brief Results
• SetFusion provides strong engaging effect
– Number of engaged users, bookmarked talks,
explored talks doubled
– The effect is larger in UMAP “natural” settings
• SetFusion allows more efficient work
– Increases yield of bookmarks in relation to
overhead actions
• But only 3 dimensions of relevance with Venn!
• How to control for more than 3 dimensions?
57

50. RelevanceTuner: Control+Visualization
in a Hybrid Social Recommender
Tsai, Chun-Hua and Peter Brusilovsky (2018) Beyond the Ranked List: User-Driven Exploration and Diversification of Social
Recommendation. In 23rd International Conference on Intelligent User Interfaces, 239--50. Tokyo, Japan: ACM.

51. Transparent Profile-Result Match
Millecamp, M., Htun, N. N., Conati, C., and Verbert, K. (2020) What's in a User? Towards Personalising
Transparency for Music Recommender Interfaces. In: Proceedings of Proceedings of the 28th ACM Conference on
User Modeling, Adaptation and Personalization, Genoa, Italy, July 14–17, 2020, pp. 173-182.

52. Transparent Control Sliders
Kleemann, T. and Ziegler, J. (2020) Distribution sliders: visualizing data distributions in range selection sliders.
In: Proceedings of the Conference on Mensch und Computer (MuC '20), pp. 67–78.

53. TasteWeights
61
Bostandjiev,
S.,
O'Donovan,
J.
and
Höllerer,
T.
TasteWeights:
a
visual
interactive
hybrid
recommender
system.
In
Proceedings
of
the
sixth
ACM
conference
on
Recommender
systems
(RecSys
'12).
ACM,
New
York,
NY,
USA
(2012),
35-42.

54. Grapevine: Collaborative Profile
Construction
62
Rahdari,
B.,
Brusilovsky,
P.,
and
Babichenko,
D.
(2020)
Personalizing
Information
Exploration
with
an
Open
User
Model.
In:
Proceedings
of
31st
ACM
Conference
on
Hypertext
and
Social
Media,
July
13-15,
2020,
ACM,
pp.
167-176.

55. Personalization – Exploration Loop
Adapt-Discover-Control-Adapt-Discover-
63

56. Beyond the Ranking List:
Visualize + Explore
Present recommendations visually helping users to understand
how relevance mechanism work
64

57. Experiments with Visual
Exploration
• Adaptive Vibe (2006-2015)
– With Jae-Wook Ahn
• Relevance Explorer (2013-2016)
– With Katrien Verbert and Denis Parra
• Intersection Explorer (2017-2019)
– With Katrien Verbert, Karsten Seipp, Chen He, Denis
Parra, Bruno Cardoso, Gayane Sedrakyan, Francisco
Gutiérrez
• ScatterViz (2018)
– With Chun Hua Tsai
65

58. Adaptive VIBE: Exploring and
Controlling Adaptive Search
67
https://www.youtube.com/watch?v=Yt1fMEFlLVA&index=2&list=PLyCV9FE42dl7JG_i7m_kvwuYRpfwwJ4iY
Ahn,
Jaewook,
and
Peter
Brusilovsky.
2013.
'Adaptive
visualization
for
exploratory
information
retrieval',
Information
Processing
and
Management,
49:
1139–64.

60. VIBE based query-profile fusion
User Profile Terms
Query Terms
Documents
Mixing user profile and query terms as VIBE POI

61. • User profile is added on the same playfield
as user query
• Topology is adaptive
• Mediate between profile (green POI) and
query (red POI) terms
• Browse documents free with control on
profile and query terms
Adaptive topology in VIBE

62. Adaptive VIBE with Concepts
74

63. Some Study Results
• A sequence of user studies
– Search vs. VIBE vs. VIBE+NE
• Search -> VIBE -> VIBE+NE offers:
– Better visual separation of relevant documents (system)
– Supports better opening relevant documents (user)
• VIBE+NE supports more meanigful interaction
– No degradation found even with active visual UM
manipulation
– While over performance retained or increased
Ahn, J., Brusilovsky, P., and Han, S. (2015) Personalized Search: Reconsidering the Value of Open User Models. In:
Proceedings of Proceedings of the 20th International Conference on Intelligent User Interfaces, Atlanta, Georgia, USA, March 29-
April 1, 2015, ACM, pp. 202-212

64. ScatterViz: Diversity-Focused
Exploration of Hybrid Recommendations
Tsai, Chun-Hua, and Peter Brusilovsky. 2018. "Beyond the Ranked List: User-Driven Exploration and Diversification of Social
Recommendation." In 23rd International Conference on Intelligent User Interfaces, 239--50. Tokyo, Japan: ACM.
Relevance sources as selectable axes

65. Relevance Explorer
• Context: multiple dimensions of relevance
– social - users, content - tags, recommender engines
• Using set relevance visualization
– One dimension of relevance = one set
• Agent metaphor to mix user- tag- and
engine-based relevance
– Users, tags, and recommender systems are shown as
agents collecting relevant talks
– Multiple-relevance match -> stronger evidence
78

66. TalkExplorer
• Recommendation engines are shown as agents in parallel to users and tags
• Uses Aduna clustermap library: http://www.aduna-software.com/
79

67. Interrelations agents and users
83

68. Evaluation
• Setup
– supervised user study
– 21 participants at UMAP 2012 and ACM Hypertext 2012 conferences
• Results
– The more aspects of relevance are fused, the more effective it is for
getting to relevant items. Especially effective are fusions across
relevance dimensions
– The more relevance prospects are merged, the better is the yield, the
easier is to find good items
– Dimensions of relevance are not equal
– ADUNA approach is challenging for beyond fusion of 3 aspects 84
Verbert, K., Parra-Santander, D., and Brusilovsky, P. (2016) Agents Vs. Users: Visual Recommendation of Research Talks
with Multiple Dimension of Relevance. ACM Transactions on Interactive Intelligent Systems 6 (2), Article No. 11

69. Intersection Explorer
• Based on ideas of
SetFusion and Talk
Explorer
• New approach for
scalable multi-set
visualization
85
Cardoso, Bruno, Gayane Sedrakyan, Francisco Gutiérrez, Denis Parra, Peter Brusilovsky, and Katrien Verbert. 2019. 'IntersectionExplorer, a multi-
perspective approach for exploring recommendations', International Journal of Human-Computer Studies, 121: 73-92.

70. Intersection Explorer (2017)
86

71. Questions?
87

72. Readings
• Ahn, Jae-wook, Peter Brusilovsky, Jonathan Grady, Daqing He, and Sue Yeon Syn (2007) Open user profiles
for adaptive news systems: help or harm? In the 16th international conference on World Wide Web, WWW '07, 11-20.
• Ahn, Jae-wook, Peter Brusilovsky, Daqing He, Jonathan Grady, and Qi Li.( 2008.) Personalized Web
Exploration with Task Models."In the 17th international conference on World Wide Web, WWW '08, 1-10. Beijing, China:.
• Ahn, J. and Brusilovsky, P. (2013) Adaptive visualization for exploratory information retrieval. Information Processing
and Management 49 (5), 1139–1164.
• Ahn, J., Brusilovsky, P., and Han, S. (2015) Personalized Search: Reconsidering the Value of Open User Models. In:
Proceedings of Proceedings of the 20th International Conference on Intelligent User Interfaces, Atlanta, Georgia, USA,
March 29-April 1, 2015, ACM, pp. 202-212
• Verbert, K., Parra-Santander, D., and Brusilovsky, P. (2016) Agents Vs. Users: Visual Recommendation of
Research Talks with Multiple Dimension of Relevance. ACM Transactions on Interactive Intelligent Systems 6 (2), Article
No. 11
• Parra, D. and Brusilovsky, P. (2015) User-controllable personalization: A case study with SetFusion. International
Journal of Human-Computer Studies 78, 43–67.
• Cardoso, Bruno, Gayane Sedrakyan, Francisco Gutiérrez, Denis Parra, Peter Brusilovsky, and Katrien
Verbert (2019). IntersectionExplorer, a multi-perspective approach for exploring recommendations, International
Journal of Human-Computer Studies, 121: 73-92.
• Verbert, K., Parra-Santander, D., Brusilovsky, P., Cardoso, B., and Wongchokprasitti, C. (2017) Supporting
Conference Attendees with Visual Decision Making Interfaces. In: Companion of the 22nd International Conference on
Intelligent User Interfaces (IUI '17), Limassol, Cyprus, ACM.
• Tsai, Chun-Hua and Peter Brusilovsky (2018) Beyond the Ranked List: User-Driven Exploration and Diversification
of Social Recommendation. In 23rd International Conference on Intelligent User Interfaces, 239--50. Tokyo, Japan: ACM.
• Rahdari, B., Brusilovsky, P., and Babichenko, D. (2020) Personalizing Information Exploration with an Open User
Model. In: Proceedings of 31st ACM Conference on Hypertext and Social Media, July 13-15, 2020, ACM, pp. 167-176.
88