Exploring Web Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches International Conference on Multidisciplinary Information Sciences and Technologies (InSciT) October 25-28, 2006 Merida, SPAIN Sofia Stamou, Panagiotis Kapros, Dimitris Christodoulakis Computer Engineering and Informatics Department Patras University, Greece {stamou, kapros, dxri}@ceid.upatras.gr

Personalized Web searching is the process of adapting the information that is relevant to a search query to the particular needs / interests of a user or groups of users . Introduction Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006

Personalized Web searching is the process of adapting the information that is relevant to a search query to the particular needs / interests of a user or groups of users .

Web personalization Advantages Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006 Users obtain only relevant information Increased accuracy of search results Improved user search experience

Users obtain only relevant information

Increased accuracy of search results

Improved user search experience

Web personalization Challenges Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006 Automatically creating and maintaining accurate user profiles within large-scale Web Search Engines requires significant effort

Automatically creating and maintaining accurate user profiles within large-scale Web Search Engines requires significant effort

The main difficulties Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006 Build accurate profiles for the users’ interests Modify the profiles when necessary (e.g. when user interests change over time) Adjust search results to fit user interests Greatest Difficulty … Users’ reluctance to give information about their search interests!

Build accurate profiles for the users’ interests

Modify the profiles when necessary (e.g. when user interests change over time)

Adjust search results to fit user interests

Greatest Difficulty …

Users’ reluctance to give information about their search interests!

Common approaches to personalization Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006 Explicit user feedback : adopted my most commercial systems. Users define their interests. Implicit user feedback : monitor the users search behavior and infer their interests based on their past click history Content-based implicit user feedback : learn the user interests based on the semantic content of the previously visited pages

Explicit user feedback : adopted my most commercial systems. Users define their interests.

Implicit user feedback : monitor the users search behavior and infer their interests based on their past click history

Content-based implicit user feedback : learn the user interests based on the semantic content of the previously visited pages

Our approach We employ a subject hierarchy for annotating user issued queries with search topic intentions Identified user interests are considered in the search process as user profiles Retrieved results contain pages that satisfy the topical categories used to describe the user interests Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006

We employ a subject hierarchy for annotating user issued queries with search topic intentions

Identified user interests are considered in the search process as user profiles

Retrieved results contain pages that satisfy the topical categories used to describe the user interests

Outline… We classify web pages in the hierarchy’s topics We map the terms in the issued queries to the hierarchy’s topics Retrieved results contain pages that satisfy the topical categories used to describe the user queries Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006

We classify web pages in the hierarchy’s topics

We map the terms in the issued queries to the hierarchy’s topics

Retrieved results contain pages that satisfy the topical categories used to describe the user queries

Getting started on pages’ classification Download Web pages HTML parse, tokenize, POS-tag, lemmatize Generate shingles (Broder et al., 1997) Find thematic words inside the pages’ shingles, using the lexical chaining technique (Barzilay and Elhadad, 1997) Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006

Download Web pages

HTML parse, tokenize, POS-tag, lemmatize

Generate shingles (Broder et al., 1997)

Find thematic words inside the pages’ shingles, using the lexical chaining technique (Barzilay and Elhadad, 1997)

Reducing web pages’ to lexical chains Select a set of candidate terms from every shingle For each candidate term find an appropriate chain relying on the type of WordNet links that connect the candidate term to the terms already stored in existing lexical chains If it is found, insert the term in the chain and update accordingly Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006

Select a set of candidate terms from every shingle

For each candidate term find an appropriate chain relying on the type of WordNet links that connect the candidate term to the terms already stored in existing lexical chains

If it is found, insert the term in the chain and update accordingly

Categorizing pages in the hierarchy Map pages’ thematic words to the hierarchy’s concepts Following hypernymic links of the hierarchy’s matching nodes, reach to topic concepts Compute a Relatedness Score ( RScore ) of every page to each of the hierarchy’s matching topics Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006 Classify a page to the topic of max RScore(i,k) ≥ T

Map pages’ thematic words to the hierarchy’s concepts

Following hypernymic links of the hierarchy’s matching nodes, reach to topic concepts

Compute a Relatedness Score ( RScore ) of every page to each of the hierarchy’s matching topics

Classify a page to the topic of max RScore(i,k) ≥ T

Mapping query terms to the hierarchy Map all queries issued by a user to the hierarchy’s concepts Disambiguate query terms based on their semantic relations in WordNet Compute the Correlation Score between query terms following the formula: Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006 Select the WordNet senses that maximize the Correlation score for query terms disambiguation Pick the topical category of the WordNet matching nodes for describing the query topics

Map all queries issued by a user to the hierarchy’s concepts

Disambiguate query terms based on their semantic relations in WordNet

Compute the Correlation Score between query terms following the formula:

Select the WordNet senses that maximize the Correlation score for query terms disambiguation

Pick the topical category of the WordNet matching nodes for describing the query topics

Evaluation of personalization accuracy Experimental setup 17 experienced Web users 10 self-defined queries/user describing a specified topic of interest total set of 170 queries Issue self-selected queries twice: with and without activating the personalization search Review top 10 pages retrieved for each of the queries, in each of the search modes (plain and personalized) and evaluate relevance on a 5-point scale Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006

Experimental setup

17 experienced Web users

10 self-defined queries/user describing a specified topic of interest

total set of 170 queries

Issue self-selected queries twice: with and without activating the personalization search

Review top 10 pages retrieved for each of the queries, in each of the search modes (plain and personalized) and evaluate relevance on a 5-point scale

Personalization performance Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006

Conclusions Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006 Relevance of personalized results is analogous to the hierarchy’s richness Hierarchy-based personalization has a notable potential in improving the user search experience User profiles can be built automatically with the use of a subject hierarchy Dynamic profile construction and updates

Relevance of personalized results is analogous to the hierarchy’s richness

Hierarchy-based personalization has a notable potential in improving the user search experience

User profiles can be built automatically with the use of a subject hierarchy

Dynamic profile construction and updates

Thank You!! Exploring Pages’ Semantics and a Subject Hierarchy for Supporting Personalized Web Searches InSciT 2006