Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
A Research Plan to Study Impact of a
CollaborativeWeb SearchTool on
Novice's Query Behavior
Mareh Al-Sammarraie
M.S. Stude...
Collaborative Information Seeking
■ Collaborative information seeking (CIS) involves people seeking
information collaborat...
Related fields of CIS
3
Information Seeking
Collaboration
Collaboration
Information
Seeking (CIS)
Information
Retrieval
Collaborative Information Seeking (CIS) Examples
■ Collaboration is a useful and often necessary component of complex proj...
Studying CIS Practices
■ People collaborating on information-seeking tasks work in distinct
ways, they should be studied a...
CISTools
■ Collaborative search tools are expected to provide
– awareness features; that
■ archive group member query hist...
SearchTeam
7
Studying CISTools
■ CIS tools have minimal to none adoption rates.
■ Despite availability of CIS tools, people use general...
Motivating Scenario
■ Motivation for this research study is raised from researcher’s personal
experience as well as from i...
ResearchContext
■ Web information retrieval process can be divided into three stages: finding
documents, query formulation...
Research Questions and Hypotheses
Research Questions:
■ Are there any differences among collaborators who are using a trad...
Research Plan
CollaborativeTeam
Collaborative Search Systems
SearchTeam Google
Novice-ExpertTeam Group A Group B
Novice-No...
SearchTask:ASP MVC Problems
Problem Context:
■ Bookstore ManagementApp is a web application that automates all operations ...
Simple SearchTask: Web Form Data EntryValidation
■ Task 1: Client-side FormValidation
– Identify an appropriate web search...
Task 2: Server-side FormValidation
■ Entries for ‘New Release Coming Soon’ and ‘New Release: Last 30 Days’ fields
need to ...
Complex SearchTask: URL Routing
■ Task 3: Add New URL routing for Index Action
– Direct Get call from three additionalURLs...
Data Collection and Measures
■ We will collect data from transaction logs such as users’ queries (search terms), search
re...
Conclusion
■ Learning how users with different levels of expertise reformulate their queries in
collaborative searches can...
ThankYou!
19
Upcoming SlideShare
Loading in …5
×

A Research Plan to Study Impact of a Collaborative Web Search Tool on Novice's Query Behavior

177 views

Published on

In the past decade, research efforts dedicated to studying the process of collaborative web search have been on the rise. Yet, limited number of studies have examined the impact of collaborative information search process on novice’s query behaviors. Studying and analyzing factors that influence web search behaviors, specifically users’ patterns of queries when using collaborative search systems can help with making query suggestions for group users. Improvements in user query behaviors and system query suggestions help in reducing search time and increasing query success rates for novices. In this paper, we present an empirical study plan designed to investigate the influence of collaboration between experts and novices as well as use of a collaborative web search tool on novice’s query behavior. In this research-in-progress study, we intend to use SearchTeam as our collaborative search tool. The results of this study are expected to provide information that could help collaborative web search tool designers to find ways to improve the query suggestions feature for group users. Additionally, this study will test the hypothesis that – having domain experts working with non-experts using collaborative search systems would immensely increase the query success rates for non-expert users, and help them learn querying strategies over the course of time. If the above hypothesis is proven, then use of collaborative web search tools during training of interns would be highly recommended.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

A Research Plan to Study Impact of a Collaborative Web Search Tool on Novice's Query Behavior

  1. 1. A Research Plan to Study Impact of a CollaborativeWeb SearchTool on Novice's Query Behavior Mareh Al-Sammarraie M.S. Student Karthikeyan Umapathy Associate Professor, School of Computing, University of North Florida Jacksonville, FL http://karthikeyan.umapathy.com/ International Conference on CollaborationTechnologies and Systems (CTS 2016) Orlando, FL November 3, 2016 Work-in-Progress, SessionC16
  2. 2. Collaborative Information Seeking ■ Collaborative information seeking (CIS) involves people seeking information collaboratively for a common goal, where – the information seeking is intentional, – the collaboration explicitly defined, and – the goal mutually beneficial. ■ CIS focuses on how groups of people – understand search tasks, – formulate queries, – determine results relevance, – share results, – make sense of the information, and – use information collaboratively. 2
  3. 3. Related fields of CIS 3 Information Seeking Collaboration Collaboration Information Seeking (CIS) Information Retrieval
  4. 4. Collaborative Information Seeking (CIS) Examples ■ Collaboration is a useful and often necessary component of complex projects, and the settings for CIS are diverse. ■ In CIS context, people come together with intention, looking for and sharing information, and making sense out of shared findings to attain their goals. ■ Business/ Professional contexts – a team of engineers working together to create an infrastructure design, – researchers coauthoring a scholarly article, – a group of medical professionals diagnosing a patient, and – a team of programmers solving software development tasks. ■ Social/Family contexts – a family planning a two-week vacation, and – a couple organizing their wedding. 4
  5. 5. Studying CIS Practices ■ People collaborating on information-seeking tasks work in distinct ways, they should be studied and supported separately. ■ Predominant literature focuses on and considers information retrieval as an individual activity. ■ While there has been a growing interest in understanding and supporting CIS practices, we still lack clear understanding of people’s CIS behavior. ■ CIS activities are supported by collaborative search tools.Thus, investigating and gaining better understanding of these activities is important for designing and developing effective collaborative web search tools. 5
  6. 6. CISTools ■ Collaborative search tools are expected to provide – awareness features; that ■ archive group member query histories, ■ selected results, and ■ comments, – division of labor features; which include ■ chat systems, ■ ability to divide search tasks, and ■ selecting search results based on a group member's action. ■ Currently available, known and/or researched, CIS tools are: – SearchTeam (http://searchteam.com/) – Coagmento (http://www.coagmento.org/) – SearchTogether (http://research.microsoft.com/en- us/um/redmond/projects/searchtogether/) 6
  7. 7. SearchTeam 7
  8. 8. Studying CISTools ■ CIS tools have minimal to none adoption rates. ■ Despite availability of CIS tools, people use general purpose search tools like Google along with communication technologies (email, texting, IM, and social media) that are part of their everyday routine. ■ Conducting comparative study between CIS tool and Google integrated platform for CIS activities are important, as it could provide insights into CIS features needed for designing and developing effective collaborative web search tools. 8
  9. 9. Motivating Scenario ■ Motivation for this research study is raised from researcher’s personal experience as well as from interactions with local software companies. ■ When an intern software developer is recruited by a software company, an intern is assigned to an experienced software developer. ■ Intern is expected to learn how to troubleshoot software development problems by working along with an expert. ■ Interactions between the interns and the experts are essentially a CIS process. ■ In this context, the question of effectiveness of using collaborative search systems like SearchTeam as opposed to integrated Google platform was posed to researchers. ■ The main research objective for the empirical study is to assess effectiveness of SearchTeam tool in the context of novice-expert pairs troubleshooting software development problems. 9
  10. 10. ResearchContext ■ Web information retrieval process can be divided into three stages: finding documents, query formulations, and determining document relevance. ■ In this research study, we focus on the second stage, query formulations as it requires user action. ■ Research-in-progress paper proposes a research plan to examine: 1. The effect of users’ domain expertise on users’ query behavior in terms of basic query features, query reformulation behaviors, and query performance in the context of collaborative web search; and 2. The impact of using tools designed for collaborative information seeking (CIS) process in aiding novices to learn effective querying strategies from domain experts through collaborative problem solving. ■ We are interested in gaining an in-depth understanding of the impact of pairing novices with experts in the collaborative search process. 10
  11. 11. Research Questions and Hypotheses Research Questions: ■ Are there any differences among collaborators who are using a traditional search engine (Google) as opposed to using a collaborativeWeb search tool (SearchTeam) from perspectives of query features including query vocabulary richness, number of queries, classes/patterns of query reformulations, and query performance? We hypothesize that: ■ Non-experts collaboratively solving problems with an expert using SearchTeam would have better query behaviors than solving with other non-experts. ■ Collaborative teams that use SearchTeam would need fewer numbers of queries with more accurate terms for the difficult search tasks than teams that use Google. ■ Collaborative teams that use SearchTeam would tend to use new and specialized query reformulation more often than teams that use Google. ■ Collaborative teams that use SearchTeam would require less time to search for both simple and complex tasks with higher query performance rates than teams that use Google. ■ Domain experts collaborating with non-experts, using SearchTeam, will immensely increase the query success rates for novice users, and help them improve their querying strategies over the course of time. 11
  12. 12. Research Plan CollaborativeTeam Collaborative Search Systems SearchTeam Google Novice-ExpertTeam Group A Group B Novice-NoviceTeam Group C Group D 12 ■ Research Method: Controlled Lab Experiments. ■ Experimental Design: 2x2 between subjects design. ■ Novices participants will be junior or senior computing students. Plan to recruit 40 students. ■ Experts will be software developers with more than 5 years of experience. Plan to recruit 20 experts. ■ Participants will be divided into teams of two. Each team will be randomly assigned to use either SearchTeam or Google to complete search tasks collaboratively. ■ We plan to have 10 teams for each experimental group. * Google Integrated Platform: Google web search, Google Docs, and Hangout. H1 – (A vs. C) vs. (B vs. D) H2, H3, &H4 – A+C vs. B+D H5 – A vs. B
  13. 13. SearchTask:ASP MVC Problems Problem Context: ■ Bookstore ManagementApp is a web application that automates all operations of an online book store. Using this application, users will be able to search the online book catalog and place an order for a book.The store manager will be able to manage the book catalog, and review and fulfill book orders. Instructions given to participants: ■ The paired teams will be expected to work together remotely but at the same time to identify relevant solutions for the problems. ■ Team members will be placed in different lab rooms to simulate remotely-located collaboration to ensure participants use chat for communication and read each other’s search histories and saved information in the team’s workspace. ■ The paired teams will be expected to search for relevant solutions using appropriate query terms and select a result item as the solution for the given problem. ■ The teams will be encouraged to review each other’s query terms and engage in an iterative search process to reach consensus on which result item describes the most ideal solution. ■ While participants will receive source codes, they are not expected to implement those codes and solve the problems. 13
  14. 14. Simple SearchTask: Web Form Data EntryValidation ■ Task 1: Client-side FormValidation – Identify an appropriate web search result that describes the process for adding client-side form validation for text box fields. 14
  15. 15. Task 2: Server-side FormValidation ■ Entries for ‘New Release Coming Soon’ and ‘New Release: Last 30 Days’ fields need to be validated at the server-side. 15
  16. 16. Complex SearchTask: URL Routing ■ Task 3: Add New URL routing for Index Action – Direct Get call from three additionalURLs to trigger the ‘Index()’ action in the ‘Books’ controller. ■ Task 4: Restrict access to default URL – Restrict access to defaultURL to only authenticated users. 16
  17. 17. Data Collection and Measures ■ We will collect data from transaction logs such as users’ queries (search terms), search results selected as a solution, and users’ actions, such as procedure followed to come up with relevant results collaboratively. ■ Data gathered will be analyzed using independent sample t-Test to compare means of experimental groups. ■ We will analyze the data in three dimensions: – Basic query features ■ Total Number of Queries (TNQ) = summation of team member queries ■ Vocabulary Richness (QVR) = No. of unique queries /Total No. of queries – Query performance = Queries with items saved /Total no. of queries – Query reformulation patterns 17 Type Definition New If Q1 is the first issued query , and does not share any common terms with Q1+1 Generalization Q1 and Q1+1 share common terms , and Q1+1 contains fewer terms than Q1 Specialization Q1 and Q1+1 share common terms , and Q1+1 contains more terms than Q1 Reconstructio n Q1 and Q1+1 share common terms , and Q1+1 has the same length as Q1
  18. 18. Conclusion ■ Learning how users with different levels of expertise reformulate their queries in collaborative searches can help search systems to improve query suggestion feature for group users. ■ This paper contributes by presenting a research plan to investigate influence of CIS tools in CIS contexts. ■ Anticipated findings – Successful query rates in collaborative search teams, where at least one of the team members has sufficient knowledge about the domain are expected to be higher when collaborative web search tool is used. – Having domain experts in collaborative search team working with novices will help novices pick up new terms, querying strategies, and tricks to identify reliable sources. ■ Future studies – Focus on factors that impact effective usage of CIS tools and improve CIS process efficiencies. – Would usage of CIS tools help in transfer of domain knowledge from experts to novices? ■ Findings from empirical investigations can help designers determine importance of collaborative search system features and CIS process activities. 18
  19. 19. ThankYou! 19

×