This document discusses regret-based econometrics in repeated games. It introduces regret as minimizing the maximum distance to the best response over time. While initial regret models showed minor improvements, quantal regret forms a weighted average on the regret curve and had better performance in estimating players' unknown parameters from observed data in repeated auctions. The document also describes data gathering from simulations to analyze the reduction of regret over time.

1. Regret-Based Econometrics
in Repeated Games
Arash Pourdamghani
Sharif CE Algorithms Lab
Fall 2017

2. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Review
2
Game
Theory

3. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Auction
There are a goods for sell.
Auctioneers want to maximum their profit.
Buyer wants to minimum its payoff.
Each Buyer has a value for a good.
3

4. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Case study: Bing
4

5. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Question outline
Estimate players unknown(private) parameters.
5

6. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Econometrics
6
Observed
values
Model Non-
observer
parameters

7. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Econometrics
7
Observed
values
Model Non-
observer
parameters
Which model
should we use?

8. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Case study: Google
8

9. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Sponsored Search Auction
9
1- Probability of clicking on ad
 Which is based on the
1. Content of ad
2. Placement of ad
2- Payment rule is a usually general second-price
 Highest bidder pay second price, the next one third …..
𝑢𝑡𝑖𝑙𝑖𝑡𝑦𝑖 𝑏; vi = vi. Pi b − Ci(b)

10. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Assumption
10
Nash equilibrium
𝑢𝑡𝑖𝑙𝑖𝑡𝑦𝑖 𝑥, 𝑏−𝑖 ≤ 𝑢𝑡𝑖𝑙𝑖𝑡𝑦𝑖(𝑏)
Randomness
𝑃𝑖 𝑎𝑛𝑑 𝐶𝑖 𝑎𝑟𝑒 𝑑𝑖𝑓𝑓𝑟𝑒𝑛𝑡𝑖𝑎𝑏𝑙𝑒
Convergence
V and B are bounded
together!

11. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Importance of experiment
The Assumptions are wrong!
11

12. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Almost infinitely repeated games
Why not one-shot game ?
Why not finite?
Is it really infinite ?
12

13. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Sophisticated bidding tools
Learning Algorithms wins!
 Adopt opponents
 No prior knowledge
 Take advantage of playing badly!
13

14. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Regret
 Minimizing maximum distance to best response
14
𝑅 𝑥, 𝑇 =
1
𝑇
Σ 𝑡 𝑢𝑡𝑖𝑙𝑖𝑡𝑦 𝑥, 𝑎−𝑖
𝑡
−
1
𝑇
Σ 𝑡 𝑢𝑡𝑖𝑙𝑖𝑡𝑦 𝑎𝑖
𝑡
< 𝜖

15. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Approach
∀𝑥:
1
T
Σ 𝑡 𝑢𝑡𝑖𝑙𝑡𝑦𝑖 𝑥, 𝑏−𝑖
𝑡
, 𝑣𝑖 <
1
𝑇
Σ 𝑡 𝑢𝑡𝑖𝑙𝑖𝑡𝑦𝑖 𝑎 𝑡
, 𝑣𝑖 − 𝜀
15
∀𝑥: 𝑣𝑖. Δ𝑃 𝑖 − Δ𝐶 𝑖 < 𝜖

16. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Is this good enough?
Not Really, at least for humans !
Nisan & Noti showed that it only had a minor improvement in a
controlled experiment.
16

17. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Let’s start Over
Nash Eq:
𝑝𝑥 = 9𝑝 + 1 − 𝑝 10
𝑥 =
10
𝑝
− 1
𝑥 =
10
.68
− 1 ≈ 13.7
17

18. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Next step
Min-regret : 13
The answer is 10 !
18

19. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Quantal Regret
Regret curves seem to have shallower left side, so just finding
minimum is not the answer.
Therefor Quantal Regret form a weighted average on regret curve.
19

20. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Next step
Min-regret : 10.7
The answer is 10 !
20

21. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Data Gathering
5 players
Each time assigned a random value
25 minutes simulation
1 auction per second →
1500 auctions in a game
21

22. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Data Gathering
22

23. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Reduction of regert
23

24. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Quantal Regret Results
24

25. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Conclusion & future work
Nikpelov et al. have introduced a new model that covers the
concerns of learning agent, but Nisan & Noti showed that it wasn’t
powerful enough, therefore introduce Quantal that had better
performance.
25

26. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
References
Nekipelov, Denis, Vasilis Syrgkanis, and Eva Tardos. "Econometrics for learning
agents." Proceedings of the Sixteenth ACM Conference on Economics and
Computation. ACM, 2015.
Nisan, Noam, and Gali Noti. "A “Quantal Regret” Method for Structural
Econometrics in Repeated Games."Proceedings of the Eighteenth ACM
Conference on Economics and Computation. ACM, 2017.
Nisan, Noam, and Gali Noti. "An experimental evaluation of regret-based
econometrics." Proceedings of the 26th International Conference on World Wide
Web. International World Wide Web Conferences Steering Committee, 2017.
26

27. Algorithmic Game Theory Regret-Based Econometrics in Repeated Games
Thank You
27