Relevance and Speed of Answers:How Can MAS Answering SystemsDeal With That?Albert Trias i MansillaGirona12th july of 2011
Search Paradigms• Library Paradigm:– Search is based on Catalogues.– Search results on published content.– Trust is based ...
Village Paradigm3• C1: “The village paradigm (social search) hassome advantages in front of the library paradigm”– Example...
Agents and Q&A Automation4• Agents are relevant for Social Search:• Reactivity• Sociability• Proactivity• Autonomy• Agents...
Automated Q&A in AgentsSocial Network5• Approach• P2P Social Network.• Each user is represented by an agent.• Each agent h...
Automated Q&A in AgentsSocial Network6• Related Work:• MARS.• P2P Multi-Agent Referral System.• 6Search.• P2P web search (...
7Question Waves
8Question WavesAssumptions:• Model does not consider context.• Agents are homogeneous.• Agents are benevolent and cooperat...
9Question Waves“T1: Answers relevancy (in village paradigm) iscorrelated with answer time”• A question wave is an attempt ...
10Question WavesT=1T=1T=0.7T=0.7Example:ShellyBobDaleGordonLauraLeoT=0.20 1 5 6 11 40
11EvaluationSimulations:Get answers and sort by answer relevance, compare therankings using Spearman’s correlationAgents u...
12Results• Correlation between answers sorted by relevance, andsorted by the following heuristics:• Answer Distance (D)• T...
13Results• Heuristics ExampleT=1T=1T=0.7T=0.7ShellyBobDaleGordonLauraLeoT=0.20 1 5 6 11 40Laura Gordon BobD 1 2 2H 1 +1 6 ...
14EvaluationEv(a) T D H DT HT Tr TT TLM 𝝑𝝑mean .8,.7,.6 .14 .67 .17 .66 .14 .52 .9 .66mean .85,.8,.7 .10 .49 .16 .48 .17 ....
Evaluation– Using Question Waves behavior and underour assumptions, answer relevance iscorrelated with answering time.– Be...
16DiscussionAnswer velocity is affected by:• Answering time.• Expertise (Algorithms with faster convergence)• Effort (Exam...
Thank you very much for yourattention17
18Evaluation
19Evaluation
Village Paradigm• Proverbs:– “A teacher is better than 2 books”– “A library of books does not equal one good teacher”• Res...
Social Search21• Social search use social interactions (implicit orexplicit) to obtain results.• (Chi, 2009) Social Search...
22Content•Introduction•Social Search•Agents and Q&A Automation•Automated Q&A in Agents Social Network•Question Waves•Evalu...
23Introduction•Centralized Search Engines provide generallygood results, but they go down with atypicalsearches.•Interest ...
Automated Q&A in AgentsSocial Network24• Why?• Reuse previous pairs of questions and answers.• 30% of the time that a quer...
25Evaluationset of agents A={a0 , a1, …, ai}, connected in a p2p social networkMethod StepFor each Received AnswersIf Own ...
Ev(a) T D H DT HT Tr TT TLM 𝝑𝝑mean .8,.7,.6 .12 .51 .12 .49 .10 .38 .72 .66mean .85,.8,.7 .09 .37 .11 .34 .12 .41 .74 .68m...
Upcoming SlideShare
Loading in...5
×

ARlab RESEARCH | Social search

188

Published on

Relevance and Speed of Answers: How Can MAS Answering Systems Deal With That?

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
188
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

ARlab RESEARCH | Social search

  1. 1. Relevance and Speed of Answers:How Can MAS Answering SystemsDeal With That?Albert Trias i MansillaGirona12th july of 2011
  2. 2. Search Paradigms• Library Paradigm:– Search is based on Catalogues.– Search results on published content.– Trust is based on authority.• Village Paradigm:– People ask with natural language.– Answers are generated in real-time by anyone inthe community.– Trust is based on intimacy– “It’s not what you know, but who you know”[Nardi2000]2
  3. 3. Village Paradigm3• C1: “The village paradigm (social search) hassome advantages in front of the library paradigm”– Example: People can address new questions.• C2: “Village Paradigm can be automated”– Example: Recommender System.
  4. 4. Agents and Q&A Automation4• Agents are relevant for Social Search:• Reactivity• Sociability• Proactivity• Autonomy• Agents enable the construction of informationsystems from multiple heterogeneous sources[Dignum2006]• C3: “Agents are a natural approach for socialsearch automation”.
  5. 5. Automated Q&A in AgentsSocial Network5• Approach• P2P Social Network.• Each user is represented by an agent.• Each agent has a FAQ list.• Agents Can:• Send Questions (asker).• Forward Questions (mediator).• Answer Questions (answerer).
  6. 6. Automated Q&A in AgentsSocial Network6• Related Work:• MARS.• P2P Multi-Agent Referral System.• 6Search.• P2P web search (bookmark based)• BFS (Gnutella)• TTL• [Walter2008]• Recommender System, filtering with trust using BFS.• Trust Path (in base of trust transitivity)• [Mychlmir2007].• Query Routing in P2P• Ant Optimization techniques.
  7. 7. 7Question Waves
  8. 8. 8Question WavesAssumptions:• Model does not consider context.• Agents are homogeneous.• Agents are benevolent and cooperative.• Agents are always online.• Answering time is constant.• Static Social Network
  9. 9. 9Question Waves“T1: Answers relevancy (in village paradigm) iscorrelated with answer time”• A question wave is an attempt to find an answer to aquestion.• In every attempt, the same question is sent to a subsetof acquaintances.• The expectancy of finding appropriate answers decaysafter every attempt.• In P2P, to request a question to all possible peers is notfeasible because it can overload the system.• However, reducing the number of recipients too far canprovide the worst results.
  10. 10. 10Question WavesT=1T=1T=0.7T=0.7Example:ShellyBobDaleGordonLauraLeoT=0.20 1 5 6 11 40
  11. 11. 11EvaluationSimulations:Get answers and sort by answer relevance, compare therankings using Spearman’s correlationAgents use 4 waves: T={t1,t2,t3}1st : after 1 simulation step; Trust > t12nd : after 5 simulation steps; t1 >Trust > t23rd : after 20 simulation steps; t2> Trust > t34th: after 40 simulation steps; t3>Trust > 0
  12. 12. 12Results• Correlation between answers sorted by relevance, andsorted by the following heuristics:• Answer Distance (D)• Trust of the last sender (Tr).• Receiving Order (H).• Answer Distance and Trust (DT).• Receiving Order and Trust (HT).• Transitive Trust (TT).• Trust of the Last Mediator (TLM).
  13. 13. 13Results• Heuristics ExampleT=1T=1T=0.7T=0.7ShellyBobDaleGordonLauraLeoT=0.20 1 5 6 11 40Laura Gordon BobD 1 2 2H 1 +1 6 +2 11 +2Tr 1 0.7 (Dale) 0.7 (Dale)TT 1 1* 0.7 0.7 * 0.7TLM 1 1 0.7
  14. 14. 14EvaluationEv(a) T D H DT HT Tr TT TLM 𝝑𝝑mean .8,.7,.6 .14 .67 .17 .66 .14 .52 .9 .66mean .85,.8,.7 .10 .49 .16 .48 .17 .56 .91 .68mean .85,.75,.5 .11 .43 .16 .43 .19 .53 .91 .67mean .85,.7,.5 .12 .56 .16 .55 .16 .52 .9 .67max .8,.7,.6 .23 .7 .27 .69 .14 .53 .83 .72max .85,.8,.7 .13 .62 .2 .61 .2 .57 .87 .73max .85,.75,.5 .15 .6 .23 .59 .22 .58 .87 .72max .85,.7,.5 .19 .67 .24 .65 .16 .56 .85 .72Spearman’s Correlation
  15. 15. Evaluation– Using Question Waves behavior and underour assumptions, answer relevance iscorrelated with answering time.– Benefits:• Relevant: answers come ranked• Faster: reduce the burden of questions• Robust: agents search answers persistently.– Risks:• Different point of view as answer quality.• Trust is needed: Answering always the same is really fast.15
  16. 16. 16DiscussionAnswer velocity is affected by:• Answering time.• Expertise (Algorithms with faster convergence)• Effort (Example: numerical analysis, more iterations moreprecision)• State of answerer• Automated or “Manual” answer?• Implication: Most important tasks will be performed early.• Communication time.• Answering delay (time after receiving and before trying toanswer)• Can MAS use answering time to consider answer relevance?• Can MAS behavior be based on reciprocity?
  17. 17. Thank you very much for yourattention17
  18. 18. 18Evaluation
  19. 19. 19Evaluation
  20. 20. Village Paradigm• Proverbs:– “A teacher is better than 2 books”– “A library of books does not equal one good teacher”• Researchers:– Sometimes information only can be accessed asking the rightpeople [Yu2003].– “It’s not what you know, but who you know” [Nardi2000]20
  21. 21. Social Search21• Social search use social interactions (implicit orexplicit) to obtain results.• (Chi, 2009) Social Search Engines can beclassified in:– Social Feedback Systems. (Sorting results).• Immediate Answer.• Cannot adress new questions.– Social Answering Systems. (People answersquestions)• Can handle new questions.• Answer not immediate• Experts can get several times same question.
  22. 22. 22Content•Introduction•Social Search•Agents and Q&A Automation•Automated Q&A in Agents Social Network•Question Waves•Evaluation•Discussion and Future Work
  23. 23. 23Introduction•Centralized Search Engines provide generallygood results, but they go down with atypicalsearches.•Interest is Social Networking sites is growing.•Researchers and Companies show interest in the“village paradigm”.
  24. 24. Automated Q&A in AgentsSocial Network24• Why?• Reuse previous pairs of questions and answers.• 30% of the time that a query was performed, it had beencarried out before by the same user. [Smyth2005]• 70% of the time it was searched before by an acquaintance ofthe user. [Smyth2005]
  25. 25. 25Evaluationset of agents A={a0 , a1, …, ai}, connected in a p2p social networkMethod StepFor each Received AnswersIf Own QuestionUpdate result and TrustElseForward it and update trustIf I have a new Own questionSelect contacts in contact waves;Program messagesFor each received questionIf I received the same question beforeIgnore itElse If I am good enough for answering,Generate Answer Value; Send answerElseSelect contacts in contacts waves;Program messagesSend programmed messages
  26. 26. Ev(a) T D H DT HT Tr TT TLM 𝝑𝝑mean .8,.7,.6 .12 .51 .12 .49 .10 .38 .72 .66mean .85,.8,.7 .09 .37 .11 .34 .12 .41 .74 .68mean .85,.75,.5 .1 .32 .11 .30 .14 .39 .74 .67mean .85,.7,.5 .1 .42 .11 .39 .12 .38 .73 .67max .8,.7,.6 .2 .54 .19 .52 .11 .39 .64 .72max .85,.8,.7 .12 .47 .15 .44 .15 .41 .68 .73max .85,.75,.5 .13 .46 .16 .43 .16 .42 .69 .72max .85,.7,.5 .16 .52 .17 .48 .12 .41 .67 .7226EvaluationKendall’s Correlation
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×