AgentSchool 2013 at Dunedin, New Zealand. (Dec. 1, 2013)

Automated Negotiation
Takayuki Ito

Dept. of Computer Science / ...
Takayuki Ito
Associate Professor, Nagoya Institute of Technology

E-mail : ito@nitech.ac.jp

http:/
/www.itolab.nitech.ac....
Area : 

Computer Science, Artificial Intelligence, Multi-Agent Systems,
Automated Negotiations, Auction Theory, Mechanism ...
Today’s schedule
•

Part 1: Introduction to Automated Negotiation 


•

Part 2: Bargaining : Game Theoretic Approaches


•...
Part 1

Introduction to Negotiation
What is Negotiation?
•

Negotiation is a form of interaction in which a group of agents
with conflicting interests try to c...
What is Negotiation?
“Negotiation can be seem as a distributed search
through a space of potential agreements.” [Jennings
...
What is Negotiation?

issue (attribute) 2

“Negotiation can be seem as a distributed search
through a space of potential a...
bilateral negotiation
•

We focus on “bilateral negotiations” , that is,
negotiations involving two agents 


•

Multi-par...
Main ingredients of Negotiation
1. The negotiation object, which defines the set of
possible outcomes

2. The agents conduc...
Negotiation outcomes
•

There are many ways to define the outcomes.


•

Also called as agreements or deals.


•

Character...
Example 1
•

1.0 litter milk between Alice and Bob

• The issue is (dividing) milk, that is
single issue & continuous

• T...
Example 2
•

Parking slot 1 and 2 for Charles and Daniel

• The issue is 3 parking slots, that is single issue &
discrete
...
Example 3
•

Buying a house between Seller and Buyer

• The issues are price, design, and place (3 issues),
that is multip...
Preferences
•

Two different agents prefer different allocations of the
resources. 


•

Preference representation or “ord...
Utility function
•

One way to define preference relation for agent i is to
define a utility function
to assign real number ...
Example
•

There are many definitions of utility functions


•

Example:

•

A quasi-linear utility function


•

The utili...
Protocols
•

Given a set of the agents and their preferences/
utilities, we need a protocol. 


•

A protocol is rules of ...
Strategy
•

Given a set of agents, their preferences, and an
agreed protocol, the final ingredient is the agent’s
strategy
...
Pareto Optimality
•

At the Pareto optimal situation, without reducing
another agent’s utility, there is no agent who can
...
Example: Cake division
•

When dividing one cake, it is Pareto optimal if the entire cake is
completely divided and alloca...
Approaches
•

•

•

Bargaining: Game Theoretic Approaches : Part 2

• How game theory can be used to analyze negotiation.
...
Cooperative game or Noncooperative game
•

There are two ways to model bilateral negotiations : using
cooperative game or ...
Prisoner’s Dilemma:

In non-cooperative game
Bob
Silent
Alex

Confess

Silent

8,8

0 , 10

Confess

10 , 0

5,5
Prisoner’s Dilemma:

In cooperative game
Bob
Silent
Alex

Confess

Silent

8,8

0 , 10

Confess

10 , 0

5,5
Cooperative game based bargaining
•

Most of work on cooperative models of bargaining followed from the
seminal work of Na...
Nash solution
•

Definition: A bargaining problem is defined as a pair (S, d). A
bargaining solution is a function f that ma...
u1(x1)

u1(d1)
0

(s1,d2)

(d1,d2)
u2(d2)

S: feasible set
(d1,s2)

u2(x2)
u1(x1)

(s1,d2)

(u1(x)-u1(d))x(u2(x)-u2(d))

Nash solution
u1(d1)
0

(d1,d2)
u2(d2)

S: feasible set
(d1,s2)

u2(x2)
Nash solution
•

Nash proved that the solution that satisfies the five axioms below is
Nash solution and its unique.


•

Ax...
Bargaining based on non-cooperative game
•

Usually, non-cooperative model of bargaining
specifies a procedure of negotiati...
A brief overview of Rubinstein’s bargaining
[Rubinstein 1982] Perfect equilibrium in a bargaining model. Econometrica, 50(...
Alternating offers protocol
θ

•

This game is played over a series of
discrete time periods t = 1,2,3,...


•

The agents...
Characteristics of the alternating offer protocol

•

The utility is increasing in the player’s share and
decreasing in ti...
Equilibrium of the alternating offer protocol
•

If this game is played infinitely overtime, then
Rubinstein showed that th...
drawbacks
•

Rubinstein’s model does not take “deadlines” into account. 


•

There is nothing to prevent the agents from ...
Multi-issue Negotiations:

Heuristic approaches
Heuristic approaches
•

The heuristic approach is particularly useful when there are
multiple issues to negotiate, and find...
Monotonic concession

[Rosenschein & Zlotkin 94] Rules of Encounter

•

Players are not allowed to make offers which have ...
Time dependent concession
•

Suppose we have a buyer (the case of the seller is
symmetrical) which desires to buy a good f...
F
1
Conceder (β>1)

Linear (β=1)

Boluware (β<1)
Time

Ka
Tmax

T0

(deadline)

1/ β

' min(t , Tmax ) $
"
F (t ) = k a + ...
Time dependent concession
•

Hard-headed (β->0): No concessions, sticks to the initial offer
throughout (the opponent may ...
Multi-issue negotiations
•

Single issue negotiations

•

•

Example: seller and buyer for a bottle of wine negotiating ov...
Bidding Based Protocol
for Multiple Interdependent Issue
Negotiations
Takayuki Ito#*
!

Collaborative work with
Mark Klain...
Summary
Target : Multi-issue Negotiation Protocol!
Negotiation with multiple interdependent issues!
Non-linearity of agent...
[Preliminaries]




Modeling Non-linear Utilities
Bumpy non-linear utility space!
The utility is a summation of satisfied c...
[Preliminaries]




Non-linear Utilities
Non-linear utility space!
m issues with the domain of integers [0, X]!
Issues are...
Bidding-based Negotiation Protocol
Sampling --- Bid-generation --- Winner
determination!
An agent submits bids to an media...
Adjusting Samples
An agent adjusts samples based on simulated annealing
method.!
Multiple simulated annealing in the utili...
Bid-generation
A bid is defined as a set of contracts which can
offer the same utility around an adjusted
sampling point.!
...
Winner Determination
The mediator identifies the combinations of bids as
the final contract.!
The final contract is a consist...
Experiments
Setting!
Constraints satisfying many issues could have larger weights.!
The maximum value for a constraint: 10...
Experiments
Linear utility function case!
Comparison between the optimal result and the result
of Hill Climbing protocol (...
Experiments
Hill Climbing / Bidding-based method for non-linear utility
function!
HC mediator tends to converge to a local...
Definition of Optimality Rate
Optimum contract
Sum of all agents’ utility functions and
use the Simulated Annealing (SA) to...
Experiments
Hill Climbing / Bidding-based method for non-linear utility
function!
HC mediator can quickly obtain the final ...
Scalability

Optimality Rate

The impact of the scaling-up of the problem space!
90%+ optimality for up to 8 issues

1
0.9...
Optimality v.s. Sampling Rate
1.2
100
300

500

700

1.15

200
900

Optimality Rate

1.1
1.05
1
0.95
0.9
0.85
2

3

4

5

...
Sampling Rates v.s. Agreement
The Failure rate, % of negotiations that do not lead to an
agreement, is higher when there a...
Discussion
The number of bids is ...!
# of
!
issues

2

3

4

5

6

7

8

9

10

11

12

13

14

15

# of!
!
bids

54

200...
Summary
An bidding-based protocol for the negotiation with
multiple interdependent issues.!
Our bidding-based protocol out...
ANAC overview
•

This competition brings together researchers from the negotiation community and
provides a unique benchma...
ANAC 2010-2013
•

•

Competitions

• ANAC2010 Toronoto, Canada

ANAC2010 ANAC2011
Agents
7
18
• ANAC2011 Taipei, Taiwan

D...
GENIUS: Tournament Environment
•

A research tool for automated multi-issue negotiation


•

Negotiation tournaments in di...
Negotiation Domains
•

Agents negotiate based on negotiation domain which has multiple issues.
Utility spaces are not know...
Creating an Agent
•

Implementation itself is very easy if you know Java


•

extend negotiator.Agent class


•

override ...
ANAC2014
•

Coming soon!


•

http:/
/www.itolab.nitech.ac.jp/ANAC2014


•

Registration will start from February - March ...
Summary
•

Part 1: Introduction of Automated Negotiation 


•

Part 2: Bargaining : Game Theoretic Approaches


•

Part 3:...
References (1)
•

I. Rahwan and S. Fatima, “Negotiations”, Chapter 4 in the Book “Multiagent Systems: 2nd
edition” edited ...
References (2)
•

Guoming Lai, Katia Sycara, and Cuihong Li. A decentralized model for multi-attribute negotiations
with i...
Questions / Comments

•

E-mail : ito@nitech.ac.jp
Automated Negotiation
Automated Negotiation
Automated Negotiation
Upcoming SlideShare
Loading in …5
×

Automated Negotiation

1,108 views

Published on

This slide was used in the Agent School 2013 at Duniden, New Zealand.

Published in: Education, Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,108
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Automated Negotiation

  1. 1. AgentSchool 2013 at Dunedin, New Zealand. (Dec. 1, 2013) Automated Negotiation Takayuki Ito Dept. of Computer Science / School of Techno-Business Administration Nagoya Institute of Technology, Japan ito@nitech.ac.jp
  2. 2. Takayuki Ito Associate Professor, Nagoya Institute of Technology E-mail : ito@nitech.ac.jp http:/ /www.itolab.nitech.ac.jp/~ito/ 2000 Dr. of Engineering, Nagoya Institute of Technology 1999 Research fellow, Japan Society for the Promotion of Science (JSPS). 2000 Visiting Researcher, USC/ISI. 2001 Assoc. Professor, Japan Advanced Insti. of Sci. & Tech. (JAIST). 2003 Assoc. Professor, Dept. of CSE, Nagoya Institute of Technology. 2005 Visiting Scholar, Computer Science, Harvard University. 2005 Visiting Researcher, MIT Sloan School of Management. 2006-Now Assoc. Prof., School of Techno-Business Admin., Nagoya Institute of Technology. 2008 Visiting Scientist, Center for Collective Intelligence, MIT Sloan School of Management. 2010 JST PREST researcher (super challenging type) 2011 Japanese Cabinet Office’ NEXT fund Principal Investigator s 2013 AAMAS 2013 Program Chair
  3. 3. Area : Computer Science, Artificial Intelligence, Multi-Agent Systems, Automated Negotiations, Auction Theory, Mechanism Design, Smart City, Smart Grid, etc. Main achievements: •2013 JSPS Award •Prizes for Science and Technology, The Commendation for Science and Technology by the Minister of Education, Culture, Sports, Science and Technology, 2013. •The NEXT Funding Program from the Japanese Cabinet Office •The Young Scientists' Prize, The Commendation for Science and Technology by the Minister of Education, Culture, Sports, Science and Technology, 2007. •Nagao Special Researcher Award, IPSJ 2007 •Best Paper Award, The Fifth International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS2006, 1/553), 2006. •2005 Best Paper Award, Japan Society for Softoware Science and Technoglogy. •2004 IPA Exploratory Software Creation Project, Super Creator Award.
  4. 4. Today’s schedule • Part 1: Introduction to Automated Negotiation • Part 2: Bargaining : Game Theoretic Approaches • Part 3: Multi-issue Negotiation : Heuristic Approaches • Part 4: Automated Negotiating Agent Competition
  5. 5. Part 1 Introduction to Negotiation
  6. 6. What is Negotiation? • Negotiation is a form of interaction in which a group of agents with conflicting interests try to come to a mutually acceptable agreement over some outcome. • The outcome is typically represented in terms of the allocation of resources (commodities, services, time, money, CPU cycles, etc.) • Agents’ interests are conflicting in the sense that they cannot be simultaneously satisfied, either partially or fully (= trade-off) • Automated negotiation would be negotiation that is automated with some computation support, e.g., fully automated negotiation among computational agents, partially automated negotiation with a computational mediator with human negotiators, etc.
  7. 7. What is Negotiation? “Negotiation can be seem as a distributed search through a space of potential agreements.” [Jennings 2001] [Jennings 2001] N. R. Jennings, P. Faratin, A. R. Lomuscio, S. Parsons, C. Sierra and M. Wooldridge, Automated Negotiation: Prospects, Methods and Challenges, International Journal of Group Decision and Negotiation, 10(2):199-215, 2001
  8. 8. What is Negotiation? issue (attribute) 2 “Negotiation can be seem as a distributed search through a space of potential agreements.” [Jennings 2001] issue (attribute) 1 This negotiation space can be seen as 2 dimensional [Jennings 2001] N. R. Jennings, P. Faratin, A. R. Lomuscio, S. Parsons, C. Sierra and M. Wooldridge, Automated Negotiation: Prospects, Methods and Challenges, International Journal of Group Decision and Negotiation, 10(2):199-215, 2001
  9. 9. bilateral negotiation • We focus on “bilateral negotiations” , that is, negotiations involving two agents • Multi-party negotiations refers negotiations involving many of ag ents. In g en e ral, auctio ns an d m e chan isms can been seen as mu lti-party negotiations. Also, some researchers now focusing on heuristic-based one.
  10. 10. Main ingredients of Negotiation 1. The negotiation object, which defines the set of possible outcomes 2. The agents conducting the negotiation 3. The protocol according to which agents search for a specific agreement 4. The individual strategies that determine the agents’ behavior based on their preferences over the outcomes
  11. 11. Negotiation outcomes • There are many ways to define the outcomes. • Also called as agreements or deals. • Characteristics • Continuous or discrete • Single issue or multiple issues
  12. 12. Example 1 • 1.0 litter milk between Alice and Bob • The issue is (dividing) milk, that is single issue & continuous • The possible outcome can be represented as a number in interval [0,1.0]. • One possible outcome is 0.2l for Alice and 0.8l for Bob.
  13. 13. Example 2 • Parking slot 1 and 2 for Charles and Daniel • The issue is 3 parking slots, that is single issue & discrete • The possible outcome can be represented as assignment of the parking slot • One possible outcome is slot 1 for Charles and slot 2 for Bob
  14. 14. Example 3 • Buying a house between Seller and Buyer • The issues are price, design, and place (3 issues), that is multiple issue (& discrete and continuous) • The possible outcome can be represented as a tuple of values of the issues. • One possible outcome is ($150,000, modern, Dunedin)
  15. 15. Preferences • Two different agents prefer different allocations of the resources. • Preference representation or “ordinal utility” • Binary preference relation : • means outcome o1 is at least as good as outcome o2 for agent i • means and it is not a case that
  16. 16. Utility function • One way to define preference relation for agent i is to define a utility function to assign real number to each possible outcome (cardinal utility <-> ordinal utility). • The utility function have if • In multi-issue negotiation, it is possible to have a multiattribute utility function which maps a vector of attribute values to a real number. • A rational agent attempts to reach a deal that maximizes his/her utility. represents the relation . when we
  17. 17. Example • There are many definitions of utility functions • Example: • A quasi-linear utility function • The utility, ui, for an item, x, is defined as, vi, the value of it minus the cost, ci, to acquire it. • ui(x) = vi(x) - ci(x)
  18. 18. Protocols • Given a set of the agents and their preferences/ utilities, we need a protocol. • A protocol is rules of interaction for enabling the agents to search for an agreement • One-shot or repeated • There are many protocols proposed so far. • Example: Alternative-offer protocol (we will see this in the later section), auction, mediator, etc.
  19. 19. Strategy • Given a set of agents, their preferences, and an agreed protocol, the final ingredient is the agent’s strategy • The strategy may specify what offer to make next or what information to reveal (truthfully or not). • A rational agent’s strategy must aim to achieve the best possible outcome for him/her. • Game-theory is analyzing agents’ strategic behavior.
  20. 20. Pareto Optimality • At the Pareto optimal situation, without reducing another agent’s utility, there is no agent who can increase his/her utility. • An outcome d is Pareto efficient (Pareto optimal) if there is no outcome that is better for at least one agent and not worse for the other agent • There is no game outcome d’ for agents A and B s.t. 
 [ uA(d’) ≥ uA(d) and uB(d’) ≥ uB(d) ] 
 and [ uA(d’) > uA(d) or uB(d’) > uB(d) ]
  21. 21. Example: Cake division • When dividing one cake, it is Pareto optimal if the entire cake is completely divided and allocated to members, and there is no remaining pieces • Pareto optimal does not mean fairness Blue or yellow can increase his/ her cake without reducing opponent’s cake! Without reducing opponent’s utility, there is no agent who can increase his/her cake.
  22. 22. Approaches • • • Bargaining: Game Theoretic Approaches : Part 2 • How game theory can be used to analyze negotiation. • Cooperative game or non-cooperative game • Assumptions: • Rules of the game, preferences & beliefs of all players are common knowledge • A2: Full rationality on the part of all players (=unlimited computation) • Preferences encoded in a (limited) set of player types (utility functions) • Closed systems, predetermined interaction, small sized games Heuristic Approaches (AI approach): Part 3 • No common knowledge or perfect rationality assumptions needed • Agent behaviour is modeled directly • Suitable for open, dynamic environments • Space of possibilities is very large Argumentation Approaches : out of scope in this lecture • Based on formal logics of dialogue games
  23. 23. Cooperative game or Noncooperative game • There are two ways to model bilateral negotiations : using cooperative game or using non-cooperative game • • • In cooperative games, agreements are enforceable or binding, and it’s possible for the agents to negotiate outcomes that are mutually beneficial. In non-cooperative game, the agents are self-interested and thus they have incentive to deviate from an agreement to improve his/her utility Thus, a same game would have the different outcome between cooperative games and non-cooperative games
  24. 24. Prisoner’s Dilemma: In non-cooperative game Bob Silent Alex Confess Silent 8,8 0 , 10 Confess 10 , 0 5,5
  25. 25. Prisoner’s Dilemma: In cooperative game Bob Silent Alex Confess Silent 8,8 0 , 10 Confess 10 , 0 5,5
  26. 26. Cooperative game based bargaining • Most of work on cooperative models of bargaining followed from the seminal work of Nash [Nash1950, Nash1953] • [Nash1950] J.F. Nash, The bargaining problem, Econometrica, 18:155-162, 1950 • [Nash1953] J.F. Nash, Two-person cooperative game, Econometrica, 21:128-140, 1953 • Nash analyzed the bargaining problem and defined a solution/outcome for it using an axiomatic approach • Nash defined a solution without the details of negotiation process • The solution is called as “Nash solution” for bargaining/negotiation problems and it is widely used as one of the ideal solutions. • Assumption • The two agents are perfectly rational: each can accurately compare its preferences for the possible outcomes, they are equal in bargaining skill, and each has complete knowledge of the preference of the other.
  27. 27. Nash solution • Definition: A bargaining problem is defined as a pair (S, d). A bargaining solution is a function f that maps every barging problem (S, d) to an outcome in S, i.e., 
 f(S,d)-> S • S is bargaining set that is the set of all utility pairs result from an agreement. • d is the disagreement point where each agent i gets ui(d) even if there is no agreement • Definition : Nash solution is defined as follows: ! • Nash product : (u1(x)-u1(d)) x (u2(x)-u2(d))
  28. 28. u1(x1) u1(d1) 0 (s1,d2) (d1,d2) u2(d2) S: feasible set (d1,s2) u2(x2)
  29. 29. u1(x1) (s1,d2) (u1(x)-u1(d))x(u2(x)-u2(d)) Nash solution u1(d1) 0 (d1,d2) u2(d2) S: feasible set (d1,s2) u2(x2)
  30. 30. Nash solution • Nash proved that the solution that satisfies the five axioms below is Nash solution and its unique. • Axiom 1 (Individual Rationality) : Each agent can get at least disagreement point. f(S,d) >= d. • Axiom 2 (Symmetry) : The solution is independent form agent’s name, like A or B. • Axiom 3 (Pareto Optimality) • Axiom 4 (Invariance from Afine Transformation) : The solution should not change as a result of linear changes to the utility for either agent • Axiom 5 (Independence of Irrelevant Alternatives) : Eliminating feasible alternatives that are not chosen should not affect the solution. Namely,
  31. 31. Bargaining based on non-cooperative game • Usually, non-cooperative model of bargaining specifies a procedure of negotiation. • Most influential non-cooperative model is the wallowing Rubinstein’s work.
 [Rubinstein 1982] Perfect equilibrium in a bargaining model. Econometrica, 50(1):97-109, Jan 1982.
 [Rubinstein 1985] A bargaining model with incomplete information about time preference. Econometrica, 53:1151-1172, Jan 1985.
  32. 32. A brief overview of Rubinstein’s bargaining [Rubinstein 1982] Perfect equilibrium in a bargaining model. Econometrica, 50(1):97-109, Jan 1982.
 • There are two agents (players) and a unit of good, a pie, to be split between them (Issue is divisible). • If agent a gets a share of gets . • If agents cannot reach an agreement, they do not get anything. then agent b
  33. 33. Alternating offers protocol θ • This game is played over a series of discrete time periods t = 1,2,3,... • The agents take turns in making offers. • t = 1, player A proposes an offer (Xa,t=1). If player B accepts A’s offer, they reach an agreement. If not (reject), goto t = 2. θ s(θ) • • t = 2, player B proposes a counter offer (Xb,t=2). If player A accepts B’s offer, they reach an agreement. If not (reject), goto t = 3. t = 3, ... θ s(θ) M θ s(θ) M’ Concretely, • s(θ) offer offer offer offer offer
  34. 34. Characteristics of the alternating offer protocol • The utility is increasing in the player’s share and decreasing in time. • This decrease in utility with time is modeled with a discount factor, and . • If a and b receive a share of xa and xb respectively where xa + xb =1, then their utilities at time t are as follows:
  35. 35. Equilibrium of the alternating offer protocol • If this game is played infinitely overtime, then Rubinstein showed that there is a unique (subgame perfect) equilibrium outcome in which the players immediately reach an agreement on the following shares:
  36. 36. drawbacks • Rubinstein’s model does not take “deadlines” into account. • There is nothing to prevent the agents from haggling for as long as they wish. • A player’s bargaining power depends on the relative magnitude of the players’ respective costs of haggling. ! • A lot of works on this line have been done.
  37. 37. Multi-issue Negotiations: Heuristic approaches
  38. 38. Heuristic approaches • The heuristic approach is particularly useful when there are multiple issues to negotiate, and finding an equilibrium offer is computationally hard. • Of course there are Game theoretic approaches to multiissue negotiations (e.g. [Fatima2006]). However, here, heuristic approaches are more focusing on computational hardness, complex utilities, etc. • [Fatima2006] S.S.Fatima, M.Wooldridge, and N.R.Jennings, Multi-issue negotiation with deadlines. Journal of Artificial Intelligence Research, 27:381-417, 2006.
  39. 39. Monotonic concession [Rosenschein & Zlotkin 94] Rules of Encounter • Players are not allowed to make offers which have a lower utility for their opponent than their last offer. The minimum concession per round can be fixed above 0 
 => It guarantees to terminate. But, anyway, they have to concede. • Question: how to make concessions? • • • If I do not know the opponents preferences If there are multiple issues Note: In multi-issue negotiations with unknown opponent preferences, it is not always possible to make monotonic concessions
  40. 40. Time dependent concession • Suppose we have a buyer (the case of the seller is symmetrical) which desires to buy a good for an aspiration price Pmin and reservation price Pmax (highest he is willing to pay); deadline is a time Tmax • Price offered at time t will be: ! • P(t ) = Pmin + F (t )( Pmax − Pmin ) F(t) gives the fraction of the distance left between the first (best) offer and the reservation value 1/ β ' min(t , Tmax ) $ " F (t ) = k a + (1 − k a )% % " Tmax & #
  41. 41. F 1 Conceder (β>1) Linear (β=1) Boluware (β<1) Time Ka Tmax T0 (deadline) 1/ β ' min(t , Tmax ) $ " F (t ) = k a + (1 − k a )% % " Tmax & # ka is constant
  42. 42. Time dependent concession • Hard-headed (β->0): No concessions, sticks to the initial offer throughout (the opponent may concede, though) • Linear time-dependent concession (β=1): Concession is linear in the time remaining until the deadline • Boulware (β<1): Concedes very slowly; initial offer is maintained until just before the deadline • Conceder (β>1): Concedes to the reservation value very quickly ! • Tit-for-tat : Cooperating on the first move and then mirroring whatever the other player did in the preceding round
  43. 43. Multi-issue negotiations • Single issue negotiations • • Example: seller and buyer for a bottle of wine negotiating over a price Multi-issue negotiations • Example: seller and buyer negotiating for a house over multiple issues, price, place, style, architecture, etc. • Trade-off between issues : An agent can make concessions in one or more issues in order to extract concessions in other issues preferable to him/her • Example • Buyer concede about style instead of proposing nicer place. • Seller concede about price instead of proposing un-preferred place.
  44. 44. Bidding Based Protocol for Multiple Interdependent Issue Negotiations Takayuki Ito#* ! Collaborative work with Mark Klain*, Hiromitsu Hattori+, and Katsuhide Fujita# ! #Nagoya Institute of Technology, JAPAN ! *Sloan School of Management, Massachusetts Institute of Technology, USA ! +Kyoto University, JAPAN
  45. 45. Summary Target : Multi-issue Negotiation Protocol! Negotiation with multiple interdependent issues! Non-linearity of agent’s utility functions! Approach : Biding-based Negotiation Protocol! An agent bids conditions to obtain better utility as a bid! Intractability of bid-generation! Result : Outperform protocols applied in liner domains! Difficulty in scalability
  46. 46. [Preliminaries] 
 Modeling Non-linear Utilities Bumpy non-linear utility space! The utility is a summation of satisfied contracts’ values Many constraints are satisfied A few/no constraints are satisfied Existing protocols assuming linear utility functions are not effective. How to obtain a solution with high social welfare for non-linear utility function ?
  47. 47. [Preliminaries] 
 Non-linear Utilities Non-linear utility space! m issues with the domain of integers [0, X]! Issues are common for agents.! A contract is a vector of issue values s = (s1,...,sm).! Agent’s utility function! The function is represented 
 in terms of constraints.! A constraint represents an 
 acceptable region 
 and its value (utility).
  48. 48. Bidding-based Negotiation Protocol Sampling --- Bid-generation --- Winner determination! An agent submits bids to an mediator for high individual utility. An agent samples its utility space to find high-utility region. Trade-off between high-utility and the limit of # of samples. Samples do NOT always lie on optimal contracts.
 How to detect better contract regions ?
  49. 49. Adjusting Samples An agent adjusts samples based on simulated annealing method.! Multiple simulated annealing in the utility space! All random sampling points are initial solutions.! ! ! ! ! ! ! Each sampling point may move to each close optimal contract.
  50. 50. Bid-generation A bid is defined as a set of contracts which can offer the same utility around an adjusted sampling point.! An agent can submit a bid iff its utility is larger than the threshold. Collect all constraints
 satisfied by this point Find the intersection
 of the constraints Several contracts could be submitted as one bid.
  51. 51. Winner Determination The mediator identifies the combinations of bids as the final contract.! The final contract is a consistent bids with the highest social welfare.! Only one bid from each agent is included. 1. Find mutually consistent bids Specifying overlapping contract region Agent 1 The final contract 2. Select the best contracts Comparing the summed bid values Agent 2
  52. 52. Experiments Setting! Constraints satisfying many issues could have larger weights.! The maximum value for a constraint: 100 x # of issues
 e.g., the possible value for a binary constraint is 200.! Agents have the same issues and domain for each issue value.! Domain for issue value is [0,9]! Approximate search-based winner determination! The final contract is searched by the simulated annealing.! The annealer for sample adjustment does not run too long.! The purpose of the sample adjustment is to find the peak of the optimum nearest point.
  53. 53. Experiments Linear utility function case! Comparison between the optimal result and the result of Hill Climbing protocol (for each issue)! ! Issues ! 1 2 3 4 5 6 7 8 9 10 ! HC 0.973 0.991 0.998 0.989 0.986 0.987 0.986 0.996 0.988 0.991 ! Optimality with linear utility function (4 agents) The simple HC protocol can produce nearly optimal results even for a large space.! The mediator can find the best value for issue 1, then issue 2, ...
  54. 54. Experiments Hill Climbing / Bidding-based method for non-linear utility function! HC mediator tends to converge to a local optimum. ! AR mediator has more chances to find better contract because agents can generate bids covering multiple optima. 1.2 Optimality Rate 1 0.8 0.6 0.4 HC AR 0.2 2 3 4 5 6 7 Number of Issues 8 9 10
  55. 55. Definition of Optimality Rate Optimum contract Sum of all agents’ utility functions and use the Simulated Annealing (SA) to find the contract with the highest possible social welfare. ! Optimality Rate = (Optimum solution in the mechanism) / (Optimum contract)
  56. 56. Experiments Hill Climbing / Bidding-based method for non-linear utility function! HC mediator can quickly obtain the final contract. ! AR mediator takes more time, but the final contract is calculated within a practical time. 4500 AR 3500 CPU time [ms] 4000 HC 3000 2500 2000 1500 1000 500 0 2 3 4 5 6 7 Number of Issues 8 9 10
  57. 57. Scalability Optimality Rate The impact of the scaling-up of the problem space! 90%+ optimality for up to 8 issues 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 2 3 4 Number of Agents 5 10 9 8 7 6 5 4 3 Number of Issues 2
  58. 58. Optimality v.s. Sampling Rate 1.2 100 300 500 700 1.15 200 900 Optimality Rate 1.1 1.05 1 0.95 0.9 0.85 2 3 4 5 6 7 Number of Issues 8 9 10
  59. 59. Sampling Rates v.s. Agreement The Failure rate, % of negotiations that do not lead to an agreement, is higher when there are more sampling points Reason: ! When there are many sampling points, each agent has a better chance of finding good local optima in its utility space.! However, the num. of bids is limited for computation time.! This increases a risk of not finding an overlap between the bids
  60. 60. Discussion The number of bids is ...! # of ! issues 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # of! ! bids 54 200 461 758 1074 1341 1636 1746 1972 2086 2238 2326 2491 2648 The winner determination computation grows exponentially as (# of bids per agent)# of agents! If we use an exhaustive search method (with branch cutting),
 the problem size is limited to the small one.! The trade-off between the computation time and the optimality
  61. 61. Summary An bidding-based protocol for the negotiation with multiple interdependent issues.! Our bidding-based protocol outperforms existing protocols applied in linear problems.
  62. 62. ANAC overview • This competition brings together researchers from the negotiation community and provides a unique benchmark for evaluating practical negotiation strategies in multiissue domains. The three previous competitions have spawned novel research in AI in the field of autonomous agent design which are available to the wider research community. • The declared goals of the competition are: • to encourage the design of practical negotiation agents that can proficiently negotiate against unknown opponents and in a variety of circumstances, • to provide a benchmark for objectively evaluating different negotiation strategies, • to explore different learning and adaptation strategies and opponent models, and • to collect state-of-the-art negotiating agents and negotiation scenarios, and making them available to the wider research community.
  63. 63. ANAC 2010-2013 • • Competitions • ANAC2010 Toronoto, Canada ANAC2010 ANAC2011 Agents 7 18 • ANAC2011 Taipei, Taiwan Domains 3 8 • ANAC2012 Valencia, Spain Rules Discount1Factor ( • ANAC2013, Saint Paul, USA ! Organizers • Tim Baarslag, Delft University of Technology • Kobi Gal, Ben-Gurion University • Enrico Gerding, University of Southampton • Koen Hindriks, Delft University of Technology • Takayuki Ito, Nagoya Institute of Technology • Nicholas R. Jennings, University of Southampton • Catholijn Jonker, Delft University of Technology • Sarit Kraus, University of Maryland and Bar-Ilan University • Raz Lin, Bar-Ilan University • Valentin Robu, University of Southampton • Colin R. Williams, University of Southampton ANAC2012 ANAC2013 17 19 24 24 Reserva8on1Value Bid1History
  64. 64. GENIUS: Tournament Environment • A research tool for automated multi-issue negotiation • Negotiation tournaments in different scenarios • analytical toolbox • Simplifies and supports agent development • repositories of domains and agents • Education : teach students to design negotiation algorithms ! • Programming is all in Java. http://mmi.tudelft.nl/negotiation/index.php/Genius
  65. 65. Negotiation Domains • Agents negotiate based on negotiation domain which has multiple issues. Utility spaces are not known Example of Bid (fashion style) sweaters Pants classic  pants Shoes boots Accessories hat Utility(A) 1.0 Utility(B) Utility(AgentB) Shirts ? Utility(AgentA) • Large variety of domain characteristics possible, and easy to identify Laptop Grocery Energy Number of issues 3 issues 5 issues 8 issues Size 27 1600 390625 Opposition Weak Medium Strong
  66. 66. Creating an Agent • Implementation itself is very easy if you know Java • extend negotiator.Agent class • override the three methods: • ReceiveMessage() • init() • chooseAction() • Create a package, compile them, and load the main class. • See More details in userguide.pdf
  67. 67. ANAC2014 • Coming soon! • http:/ /www.itolab.nitech.ac.jp/ANAC2014 • Registration will start from February - March 2014.
  68. 68. Summary • Part 1: Introduction of Automated Negotiation • Part 2: Bargaining : Game Theoretic Approaches • Part 3: Multi-issue Negotiation : Heuristic Approaches • Part 4: Automated Negotiating Agent Competition
  69. 69. References (1) • I. Rahwan and S. Fatima, “Negotiations”, Chapter 4 in the Book “Multiagent Systems: 2nd edition” edited by G. Weiss, MIT Press, ISBN 978-0-262-01889-0, 2013. (Recommended) • N. R. Jennings, P. Faratin, A. R. Lomuscio, S. Parsons, C. Sierra and M. Wooldridge, Automated Negotiation: Prospects, Methods and Challenges, International Journal of Group Decision and Negotiation, 10(2):199-215, 2001 • J.F. Nash, The bargaining problem, Econometrica, 18:155-162, 1950 • J.F. Nash, Two-person cooperative game, Econometrica, 21:128-140, 1953 • A. Rubinstein, Perfect equilibrium in a bargaining model. Econometrica, 50(1):97-109, Jan 1982. • A. Rubinstein, A bargaining model with incomplete information about time preference. Econometrica, 53:1151-1172, Jan 1985. • Howard Raiffa, The art and science of negotiation, Harvard Univ. Press, 1982 • M.J. Osborne, A. Rubinstein, Bargaining and Markets, Academic Press, 1990. • J.S. Rosenschein, G. Zlotkin, Rules of encounter, MIT Press, 1994. • Roger B. Myerson, Game Theory: Analysis of Conflict, Harvard University Press, 1997. • Sarit Kraus, Strategic Negotiation in Multi-Agent Environments, MIT Press, 2001. • P. Faratin, C. Sierra, and N. R. Jennings. Negotiation decision functions for autonomous agents. Int. Journal of Robotics and Autonomous Systems, 24(3-4):159-182, 1998. • Catholijn Jonker and Valentin Robu. Automated multi-attribute negotiation with efficient use of incomplete preference information. In 3rd Int. Conf. on Autonomous Agents & Multi Agent Systems (AAMAS), New York, pages 1056-1063, 2004
  70. 70. References (2) • Guoming Lai, Katia Sycara, and Cuihong Li. A decentralized model for multi-attribute negotiations with incomplete information and general utility functions. In Proc. of RRS’06, Hakodate, Japan, 2006. • V. Robu, D.J.A. Somefun, and J. A. La Poutré. Modeling complex multi-issue negotiations using utility graphs. In Proc. of the 4th Int. Conf. on Autonomous Agents & Multi Agent Systems (AAMAS), Utrecht, 2005, • Tim Baarslag, Katsuhide Fujita, Enrico Gerding, Koen Hindriks, Takayuki ITO, Nick R. Jennings, Catholijn Jonker, Sarit Kraus, Raz Lin, Valentin Robu, Colin Williams, The First International Automated Negotiating Agents Competition, Artificial Intelligence Journal (AIJ), ELSEVIER, 2013. (About ANAC) • Katsuhide Fujita, Takayuki ITO, Mark Klein "Efficient issue-grouping approach for multiple interdependent issues negotiation between exaggerator agents", Decision Support Systems, 2013. • Miguel Angel Lopez Carmona, Iván Marsá Maestre, Mark Klein, Takayuki ITO "Addressing Stability Issues in Mediated Complex Contract Negotiations for Constraint-based, Non-monotonic Utility Spaces", Journal of Autonomous Agents and Multi-Agent Systems (JAAMAS), 3 December 2010. • Takayuki Ito, Mark Klein, Hiromitsu Hattori, "Multi-issue Negotiation Protocol for Agents: Exploring Nonlinear Utility Spaces", In the Proceedings of IJCAI2007, Hyderabad, India, January 6-12, pp. 1347- 1352, 2007.
  71. 71. Questions / Comments • E-mail : ito@nitech.ac.jp

×