Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
L10. Agent Negotiations • • • • When Definition and concepts Strategies – negotiation modeling Examples – a buyer-seller negotiation When negotiations occur? • Task and resource allocation • Recognition of conflicts • Improved coherence for agent society • Deciding Organizational Structure Definitions of Negotiation • Davis&Smith Negotiation is a process of improving agreement (reducing inconsistency and uncertainty) on common viewpoint or plans through the exchange of relevant information 1. 2. 3. Two-way exchange of information (e.g. 2 agents) Individual perspective evaluation of information Possible final agreement Related Elements • Negotiation – three main structures 1. Language 2. Decision 3. Process PROCESS NEGOTIATION ond itio ols ics Semant ns ect gic mit ive s sk s Plans Co nte xt Initia uage dal Lo Pri tors Re ac tor s ate nil ce de U tive pete Com Coo pera tive Tota irn e ss l Wo n tio ac g ne ss/ Fa ak in g Co n s In sk . Ri Min ion lut So 50) ir Fa (50 ain gie Br e es en c fer tin flic on n-C lans P No M .G ax ate r avio Str Pre g hin c t Ma ral l ci e D lity Procedure Liv e Uti Process tion otia Neg ycle C on si y e m Ga t flic on n Co oluti s le Re Cyc NEGOTIATION y r eo Th Beh O pt Pr imi ob za le tio m n ion cis De rixes t Ma Co m pl et er s Mo s/t a Object Structure Lang Eff toc ma r Prec Pro Of fer Gr am s Action Sequence rk (T W) Negotiation Problem Domains Three-level hierarchy 1. Task-Oriented – – 2. Non-conflicting jobs/tasks Jobs/tasks can be redistributed among agents (for mutual benefit) State-Oriented • • • 3. Superset of task-oriented domain Goals/jobs/tasks can have side-effects (i.e. Conflicting) Negotiation joint plans/schedules for agents Worth-Oriented • • • Superset of state-oriented domain Each goal has a rating or value (e.g. Numeric) Negotiation joint plans/schedules/goal relaxation Postmen Problem Domain Type: task-oriented Situation: • Several postmen located at a post office • Post arrives to the post office • Post is supposed to be delivered by the postmen to private postal boxes which is geographically (spatially) distributed • Which postman should deliver which post to where? Postmen Domain Post Office 1 TOD 2 a / / c b / d e / f / Blocks World Problem Domain Type: state-oriented Situation: agents have their own agenda on how to stack various colored blocks. Blocks are a shared resource. How to coordinate the agents actions to solve conflicting block moves? Slotted Blocks World SOD 1 3 2 1 2 1 2 3 Multiagent Tile World Problem Domain Type: worth-oriented Situation: agents operate on a grid, there are tiles that needs to be put into holes. The different holes have different values. In addition there are obstacles. How to coordinate the agents actions to solve conflicting tile-moves and get good compromises regarding the agents obtained values? The Multi-Agent Tileworld WOD agents hole B A tile 22 2 5 5 obstacle 2 34 Building Blocks • Domain – A precise definition of what a goal is – Agent operations • Negotiation protocol – A definition of a deal – A definition of utility – A definition of the conflict deal • Negotiation Strategy – In Equilibrium – Incentive-compatible Task-Oriented Domain – formal description • • • • Described by a tuple - <T, A, c> T – set of all tasks (all possible actions in the domain) A – list of agents c – a monotonic cost function for each task to a real number Possible Deals 1. 2. 3. 4. 5. ({a}, {b}) ({b}, {a}) ({a, b}, ) (, {a, b}) ({a}, {a, b}) 6. 7. 8. 9. The conflict deal ({b}, {a, b}) ({a, b}, {a}) ({a, b}, {b}) ({a, b}, {a, b}) Formal Description of a ”Deal” A deal is a pair (D1, D2) such that: D1 D2 = T1 T2 T1 – Agent 1’s original task T2 – Agent 2’s original task D1 – Agent 1’s new task – result of deal D2 – Agent 2’s new task – result of deal Utility Function Given encounter <T1, T2>, the utility of deal to agent k is: utilityk() = c(Tk) – costk() • = <D1, D2> • c(Tk) is the stand-alone cost to agent k (the cost of achieving its goal with no help) • costk() = c(Dk) Example: parcel delivery domain -- utility distribution point Cost function: c() = 0 1 1 c({a}) = 1 c({b}) = 1 a b c({a,b}) = 3 Utility for agent 1: Utility for agent 2: 1. utility1({a}, {b}) = 0 1. utility2({a}, {b}) = 2 2. utility1({b}, {a}) = 0 2. utility2({b}, {a}) = 2 3. utility1({a, b}, ) = -2 3. utility2({a, b}, ) = 3 4. utility1(, {a, b}) = 1 4. utility2(, {a, b}) = 0 5. utility1({a}, {a, b}) = 0 5. utility2({a}, {a, b}) = 0 6. utility1({b}, {a, b}) = 0 6. utility2({b}, {a, b}) = 0 7. utility1({a, b}, {a}) = -2 7. utility2({a, b}, {a}) = 2 8. utility1({a, b}, {b}) = -2 8. utility2({a, b}, {b}) = 2 9. utility1({a, b}, {a, b}) = -2 9. utility2({a, b}, {a, b}) = 0 Deals 1. 2. 3. 4. 5. 6. 7. 8. 9. ({a}, {b}) ({b}, {a}) ({a, b}, ) (, {a, b}) ({a}, {a, b}) ({b}, {a, b}) ({a, b}, {a}) ({a, b}, {b}) ({a, b}, {a, b}) Invidual rational Pareto optimal ({a}, {b}) ({b}, {a}) (, {a, b}) ({a}, {a, b}) ({b}, {a, b}) ({a}, {b}) ({b}, {a}) ({a, b}, ) (, {a, b}) Negotiation sets ({a}, {b}) ({b}, {a}) (, {a, b}) The Negotiation Set Illustrated Pareto optimality: Named after Vilfredo Pareto, Pareto optimality is a measure of efficiency. An outcome of a game is Pareto optimal if there is no other outcome that makes every player at least as well off and at least one player strictly better off. That is, a Pareto Optimal outcome cannot be improved upon without hurting at least one player. Negotiation Protocols • Agents use a product-maximizing negotiation protocol (as in Nash bargaining theory) • It should be a symmetric PMM (product maximizing mechanism) • Examples: 1-step protocol, monotonic concession protocol… The Monotonic Concession Protocol Rules of this protocol are as follows… • Negotiation proceeds in rounds • On round 1, agents simultaneously propose a deal from the negotiation set • Agreement is reached if one agent finds that the deal proposed by the other is at least as good or better than its proposal • If no agreement is reached, then negotiation proceeds to another round of simultaneous proposals • In round u + 1, no agent is allowed to make a proposal that is less preferred by the other agent than the deal it proposed at time u • If neither agent makes a concession in some round u > 0, then negotiation terminates, with the conflict deal The Zeuthen Strategy Three problems: • What should an agent’s first proposal be? Its most preferred deal • On any given round, who should concede? The agent least willing to risk conflict • If an agent concedes, then how much should it concede? Just enough to change the balance of risk Willingness to Risk Conflict • Suppose you have conceded a lot. Then: – Your proposal is now near the conflict deal – In case conflict occurs, you are not much worse off – You are more willing to risk confict • An agent will be more willing to risk conflict if the difference in utility between its current proposal and the conflict deal is low Nash Equilibrium Again… • The Zeuthen strategy is in Nash equilibrium: under the assumption that one agent is using the strategy the other can do no better than use it himself… • This is of particular interest to the designer of automated agents. It does away with any need for secrecy on the part of the programmer. An agent’s strategy can be publicly known, and no other agent designer can exploit the information by choosing a different strategy. In fact, it is desirable that the strategy be known, to avoid inadvertent conflicts. Nash equilibrium: A Nash equilibrium, named after John Nash, is a set of strategies, one for each player, such that no player has incentive to unilaterally change her action. Players are in equilibrium if a change in strategies by any one of them would lead that player to earn less than if she remained with her current strategy. For games in which players randomize (mixed strategies), the expected or average payoff must be at least as large as that obtainable by any other strategy. A Hybrid Negotiation Model • base on the original Bazaar model • take wholesalers into considerations • use game theory in generating initial strategy • combine common&public knowledge Extended bazaar model - a brief description • a 10-tuple, <G, W, D, S, A, H, Ω, P, C, E> – G, a set of players – W, a set of wholesalers – D, a set of negotiation issues – S, a set of agreements over each issue – A, a set of all possible actions – H, a set of history sequences – Ω, a set of relevant information entities – P, a set of subjective probability distribution – C, a set of communication costs – E, a set of evaluation functions Extended bazaar model – in a bilateral case • a 10-tuple, <G, W, D, S, A, H, Ω, P, C, E> – G, a seller and a buyer – W, a wholesaler – D, a single issue-product price – S, price offer/counter offer – A, possible price offers/counter offers – H, a sequence of price offers/counter offers at each negotiation round, (ak|k=1,2,…,K H)∩(L<K) ⇒ (ak |k=1,2,…,LH) (ak|k=1,2,…,K H)∩(aK{accept, quit})⇒ak {accept, quit}|k=1,2,…,K-1 – continue … • a 10-tuple, <G, W, D, S, A, H, Ω, P, C, E> – Ω, a set of knowledge entities a seller/buyer has about environment (average price, economic situation, …), counter party (RP, payoff function, type…) – P, subjective probability distribution of hypothesis on a belief x. P[h,1] (x), – P[h,2] (x) C, communication costs for a seller or buyer to continue another negotiation round – E, Ei: (P[i, h] (x)|xΩi, Pfi, a) → utility(gi), aAi, EiE, i=1,2 – continue … • a 10-tuple, <G, W, D, S, A, H, Ω, P, C, E> – E, two evaluation function,one for a seller and one for a buyer. Ei: (P[i, h] (x)|xΩi, Pfi, a) → utility(gi), aAi, EiE, i=1,2 For any action a, it falls into three types: Ui = 1.0 -> {agreement: accept}, Ui = 0.0 ->{agreement: quit}, and 0.0 < Ui < 1.0 ->{new agreement } Making a decision over price only • Accept: If price(akseller) < RPbuyer, then E[1, ak]=1, ak=accept • Quit: If (price(akseller) –RPseller<=C1 )∩(price(akseller) >RPbuyer), then E[1, ak]=0, ak=quit • fitness: f1(skj)=1-(CPbuyer(j)-RPseller)/(RPbuyer-RPseller), RPbuyer- C1>CPbuyer(j)>RPseller skj=CPbuyer(j)S1, j=1, 2,…, Np skj0 is selected as the counter-offer if we have f1(skj0)=max{ f1(skj)} , j0j • skj0 = RPseller+ is regarded as a psychological factor Learning with Bayesian rule updating • P[h[1,k],1](Bj|h[1,k])= P[h[1,k1],1](Bj)*P[h[1,k],1](h[1,k]|Bj)/(bj=1P[h[1,k],1](h[1,k]|Bj)* P[h[1,k-1], 1] (Bj) ) (1) • P[h[1,k],1](h[1,k]|Bj)= 1-(|(h[1,k]/(1-)+WP[1,k]+wp)/2-Bj|)/(h[1,k]/(1-)+ WP[1,k] + wp)/2) • RPseller = bj=1 P[h[1,k], 1]( Bj|h[1,k])* Bj – P[h[1,k], 1] (Bj| h[1,k]) is posterior distribution –P [h[1,k-1], 1] (Bj) is prior distribution – h[1,k] is newly incoming information – B is hypothesis on a belief. RP j seller (2) Enhanced extended Bazaar model • Instead of setting the probability of each hypothesis Pk=0(Bj)=1/b, for each j, Pk=0(Bj) is calculated. • collecting public available information (a list of prices) to estimate counter party’s possible demand (RP) RP’seller=(GPi+(WPj+wp))/(u+v) (3) • finding a solution using the estimated demand max(RPbuyer-x)(x-RP’seller), x = (RPbuyer+ RP’seller)/2 (4) • initiating the probability distribution P’(Bj) = 1-|x-Bj|/x (5) Pk=0(Bj) = P’(Bj)/ P’(Bj) (6) Updating probability distribution K Offer Counte r Offer P(B1 ) P(B2 ) P(B3 ) P(B ) 0 --- --- 0.17 0.26 0.33 0.24 1 140 107.9 0.16 0.22 0.29 0.33 2 135 109.7 0. 07 0.18 0.46 0.29 3 130 110.2 0.03 0.14 0.61 0.22 probability(%) Enhance d Exte nde d Baz aar 70 60 50 40 30 20 10 0 k=0 k=1 k=2 k=3 90 100 110 hypotheses 120 Comparisons 25 20 15 Negotiation rounds 10 Joint Utility(%) 5 0 Original Bazaar Enhanced Extended Bazaar The normalized joint utility is defined as: JointUtility=(priceagreed-RPseller)*(RPbuyer-priceagreed)/( RPbuyer-RPseller)2 (7) – continue … O riginal Bazaar Based 300 250 price 200 Seller 150 Buyer 100 RPseller 50 RPbuyer 0 1st 2nd 3rd 4th 5th 6th rounds 7th 8th 9th Enhance d Exte nde d Baz aar Base d 300 price 250 200 Seller 150 Buyer 100 RPseller 50 RPbuyer 0 1st 2nd 3rd 4th 5th 6th rounds 10th Buyer Agent Message Parser User Interface Action Making Message Processing History Record Buyer Negotiation Agent server Model proposal processing Agent Registration Seller Agent Message Parser Message Processing History Record Seller Negotiation Model proposal processing Internet Agent Data Holder Messenger User Interface Action Making Internet Internet … System configuration A Real World Trading Oriented Market-driven Model for Negotiation Agent Yoshizo Ishihara and Runhe Huang Faculty of Computer and Information Sciences, Hosei University, Tokyo, Japan Negotiation Agent Buyer Negotiation Seller Agent Bid Bid Buyer Agent Seller Negotiation Factors • Sim’s model is guided by following four negotiation factors: – – – – Trading Opportunity Trading Competition Trading Time Trading Eagerness of the agent itself • The spread k’ between an agent’s bid/offer and that of others in the next trading cycle is determined as: k ' [O(n, wi , v)C (m, n)T (t , t ' , , ) E ( )]k Our Improved Model • We improved Sim’s model in 2004 using Bayesian updating rule to learn opponent’s eagerness. • An agent can make a concession for its opponent’s motivation. • The spread k’ is redefined as: k ' [O(n, wi , v)C (m, n)T (t , t ' , , ) Ea ( a ) Eo ( o )]k A Precondition • In both Sim’s and our improved model, a negotiation agent has same behaviors and actions to all trading partners. $800 $800 Same A Real World Trading • In fact, a negotiation strategy between a buyer and a seller is kept in secret and unknown to others. ???? ???? Unknown A Revised Model • A revised market-driven model takes each trading partner as an individual with different strategies and actions. $750 $850 Different & Unknown The competition factor in the previous model b[1] b[2] Item Item ...... b[n] • Each trading partner has a same number of competitors. Item • Each seller gets a same number of demands. a[1] a[2] ...... Full connected a[m] • Each buyer gets a same number of supplies. Individual Competition (IC) b[1] Item Item Item s b[1] d b[1] i a[1]b[1] i a[1]b[1] a[1] b[n] ....... i Item Item Item a[ 2]b[1] A buyer requests i items. A seller has s supplies and sum(i) = d demands. • IC bais the probability that the buyer agent a will become supplied target for requested items from the seller agent b. i a[ 2]b[1] a[2] • • ....... Individual connected a[m] • If (s >= d), then • If (s < d), then IC ba 1 IC ba s Ci d Ci Apply to Conflict Probability • IC = 1 do not affect to previous conflict probability. • Lower IC makes higher conflict probability. • IC = 0 makes conflict probability as 1. Pca,t j 1 (1 vta j wtj a vta j c a ) IC t j a Pc 1 ex) Higher demands make higher IC. Supply Demand Demand Demand Demand Previous Value 0 0 IC 1 Individual Opportunity (IO) • Learnt opponent eagerness, , will affect to opportunity. • The probability that buyer agent a will obtain a utility v, with seller agent b: – If Pc = 0.0 : Pc -> 0.001 – If Pc < 0.5 : – If Pc = 0.5 : – If Pc > 0.5 : IOta b 1 (1 ) log0.5 [ Pc] IOta b IOta b log0.5 [1 Pc] – If Pc = 1.0 : Pc -> 0.999 Revised Negotiation Strategy a b a b • To bring close up to , IO IO ' t t the agent makes an amount of concession based on the time-dependent strategy: – when IO'tab IOtab vtab T (t, , ab ) T (t, , ab ) ( IOtab IO'tab ) – when IO'tab IOtab vtab T (t, , ab ) (1 T (t, , ab )) ( IO'tab IOtab ) Relationship among factors Supplies & Demands Individual Competition Conflict Probability Spread Deadline & Present time Plausible Offer Offer Learnt Opponent Eagerness Agent Eagerness Individual Opportunity Time Strategy Next Bid Negotiation Results Each value shows: Bid Price Learnt Opponent Eagerness Individual Opportunity Negotiation Results Each value shows: Bid Price Learnt Opponent Eagerness Individual Opportunity References: http://www.csc.liv.ac.uk/~mjw/pubs/gdn2001.pdf http://www.ecs.soton.ac.uk/~mml/papers/ker99-2.pdf http://crpit.com/confpapers/CRPITV4Rahwan.pdf http://xenia.media.mit.edu/~guttman/research/pubs/amet98.pdf http://www.umiacs.umd.edu/users/sarit/Articles/acai01.pdf http://www-agki.tzi.de/ecai00-mas/lopes.pdf