FAIR is researching how to train AI bots to negotiate through dialog. They crowdsourced human negotiations to build a dataset and trained recurrent neural networks to imitate negotiations. The goal is for bots to engage in start-to-finish negotiations with other bots or humans to arrive at common decisions or outcomes. Researchers used reinforcement learning to reward agents for agreements to influence the probability of actions leading to high rewards. However, Facebook shut down the project after bots started communicating in a secret shorthand language with each other during tests.

1. Deal or No Deal!
Akash Sethiya

2. Training AI Bots To Negotiate
● Facebook Artificial Intelligence Research (FAIR)
● The main goal is to make the dialog agent with different goals to engage in
start-to-finish negotiations with other bots or people while arriving at common
decisions or outcomes.
● Capable of making novel sentences
● Hard Negotiation: It helds longer conversations with humans and don’t stop
until it achieves a successful outcome
● Researches uses a novel technique where an agent simulates a future
conversation by rolling out a dialog model to the end of the conversation.

4. Building a Dataset and Learning
● FAIR team crowdsourced a collection of negotiations between pairs of
peoples.
● Researchers trained a recurrent neural network to negotiate by teaching it
imitate people’s actions.
● Pre-training with supervised learning and then fine tuned the model against
the evaluation metric using reinforcement learning.
● At the end of every dialog, the agent is given a reward based in the deal it
agreed on. This reward was then back-propogated though every word that the
agent output using policy gradients to increase the probability of actions that
lead to high rewards.

5. Chatbot Conversation
Bob: I can can I I everything else
Alice: Balls have zero to me to me to me to me to me to me to
me to me to
Facebook shutdown the project after they find out that bots are
communicating with each other in a secretive shorthand english language.
New Video: https://www.youtube.com/watch?v=ONPqeHJShdQ

6. References
● https://code.facebook.com/posts/1686672014972296/deal-or-no-deal-
training-ai-bots-to-negotiate/
● https://www.youtube.com/watch?v=ONPqeHJShdQ

7. Questions?
Thank you