Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

On the Dynamics of Machine Learning Algorithms and Behavioral Game Theory

Like this presentation? Why not share!

- CVPR2016読み会 Sparsifying Neural Netw... by Koichi Takahashi 3718 views
- Stochastic Variational Inference by Kaede Hayashi 6376 views
- 20170819 CV勉強会 CVPR 2017 by issaymk2 1982 views
- LCA and RMQ ~簡潔もあるよ！~ by Yuma Inoue 9680 views
- Greed is Good: 劣モジュラ関数最大化とその発展 by Yuichi Yoshida 7945 views
- プログラミングコンテストでのデータ構造 2 ～動的木編～ by Takuya Akiba 45970 views

Presentation Material used in guest lecturing at University of Tsukuba on September 17, 2016.

Target audience is part-time PhD student working at a machine learning, data mining, or agent-based simulation project.

No Downloads

Total views

7,579

On SlideShare

0

From Embeds

0

Number of Embeds

6,280

Shares

0

Downloads

32

Comments

0

Likes

1

No notes for slide

- 1. On the Dynamics of Machine Learning Algorithms and Behavioral Game Theory Towards Effective Decision Making in Multi-Agent Environment Graduate School of Systems and Information Engineering University of Tsukuba Sep 17, 2016 Rikiya Takahashi, Ph.D. (SmartNews, Inc.) rikiya.takahashi@smartnews.com
- 2. About Myself ● Rikiya TAKAHASHI ( 高橋 力矢 ) ● Engineer in SmartNews, Inc., from 2015 to current ● Research Staff Member in IBM Research – Tokyo, from 2004 to 2015 ● Ph.D in Engineering from University of Tsukuba, 2014 – Dissertation: "Stable Fitting of Nonparametric Models to Predict Complex Human Behaviors" – Supervisor: Prof. Setsuya Kurahashi ● M.Sc (2004) & B.Eng (2002) from The University of Tokyo ● Research Interests: machine learning, reinforcement learning, cognitive science, behavioral economics, complex systems ● Descriptive models about real human behavior ● Prescriptive decision making by exploiting such descriptive models
- 3. References Choice and Social Interaction Why did you purchase Windows 10 XXX Edition? Because the price and quality of that OS were good? Or because your friends were using it? Or both reasons? Are you interested in quantifying each factor for better decision making? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 4. References Decision Making under Social Uncertainty You can be either a player or a designer of the market. Players: consumers, ﬁrms competing with other brands Designers: politicians, platformer of auction or SNS In both scenarios you must optimize your decisions under uncertainty over other players’ decisions. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 5. 0 5 10 15 20 25 30 35 0 0.25 0.5 0.75 1 Elapsed Time [days] Retention What I was doing during PhD: Stable Fitting of Power Law Models ● Heavy-tail distributions / long-range dependence ● Bounded rationality: incomplete information, cognitive bias ● Positive feedback: richer-get-richer, increasing survival prob. Travel time in road network Ebbinghaus forgetting curve Asset returns in finance Pageview of a video in YouTube Power-law decay by cascading word- of-mouth (Crane & Sornette, 2008) Heavy-tail by price crash (stimultaneous shorting) http://finance.yahoo.com Heavy-tail by huge traffic congestion http://en.wikipedia.org/ wiki/Traffic_congestion Power-law decay by interaction among short-, mid-, & long- term memories
- 6. What I was doing during PhD: Power-Law Models = Multi-Scale Nonparametrics ● Global optimization in fitting nonparametric models ● Non-linear modeling by linearly mixing local or multi-scale basis functions ● Convex optimization of the mixing weights ● Domain-specific design of fixed basis functions ElapsedTime Retention Value ProbabilityDensity Heavy-tail distribution as scale-mixture of Gaussians Power-law decay as scale- mixture of exponential decays
- 7. Agenda ● Irrationality and Disequilibrium: essential phenomena making social science challenging ● Failed Forecasts by Rational Economic Models: Irrational Disequilibrium or Multiple Equilibria? ● Frontiers in Mathematical Modeling of Irrationality: a transitional-state perspective ● For on-going PhD students: how to exploit your research experiences into Jobs
- 8. Intertemporal Decision Making ● $100 today = $100 * (1+interest rate) in the future ● Objective must be time-consistent. ● Exponential discounting (constant interest rate) ● Unchanged preference order over time V (t0)=V (t)exp(−λ (t−t0)) where t> t0, λ > 0
- 9. Real Human Discounts by Power Law ● Hyperbolic discounting (Ainslie, 1974) ● Time-inconsistent preference order V (t0)= V (t) (1+ λ (t−t0))α where t> t0, λ > 0,α > 0
- 10. Irrationality of Hyperbolic Discounting ● Discrepancy between thought and action ● Long-term-oriented when the decision time is distant. ● But suddently become myopic as the time reaches. "About 1 month ago, I was thinking I would study hard (=long- term large utility) in the last 1 week before the exam, but I did play video games (=short-term small utility) this week..." Long-term Option B is more prioritized than Short-term Option A at t=0, but its order is reversed at t=2.
- 11. Irrationality of Hyperbolic Discounting ● Money pumps (Cubit and Sugden, 2001) ● We can steal money from hyperbolic discounter without risks, while cannot steal from exponential discounter. At time t=0, we borrow Option B and exchange it with the target's Option A and $2.5 (=$15-$12.5). Then we earn interests on this $2.5. At time t=3, we exchange our Option A with the target's Option B and $10 (=$30-$20). Then return this Option B. We get ($2.5 * (1 + interests) + $10 – borrowing cost) without risks.
- 12. Known Counterarguments ● Hyperbolic discounting is rather rational, when the interest rates in the future is uncertain. ● E.g., (Azfar, 1999; Farmer & Geanakoplos, 2009) ● Meaningful particularly in financial decision making ● Integral on gamma prior distribution for interest rate (multi-scale mixture of exponential discountings)
- 13. Power-Law and Disequilibrium ● Power-law or fat tails in asset-return distributions ● See (Cont, 2001) for stylized facts. ● Short-term momentum generates outlying returns ● Positive autocorrelation in rare events (Sornette, 2004). http://finance.yahoo.com http://www.proba.jussieu.fr/pageperso/ramacont/papers/empirical.pdf
- 14. What Causes Fat Tails? ● Hypothesis #1. Interplay among momentum traders ● E.g., Log-Periodic Power-Law (LPPL) model (Johansen+, 1999) as an extension of rational bubble model (Blanchard & Watson, 1982) Sell ? Sell ! Sell More!! $$: market price http://arxiv.org/pdf/1107.3171.pdf si=sign(K ∑j∈N (i) sj+ ε i) si∈{−1,+ 1} K: strength of interaction N(i): set of neighbors for investor i epsiloni : investor i's own indiosyncratic prediction
- 15. What Causes Fat Tails? ● Hypothesis #2. Over-confidence on stability ● Leverage in low-volatility period (Thurner+, 2012) – Once a downward price fluctuation occurs, resulting margin call causes rushes of selling into an already falling market, amplifying the downward price movement. http://finance.yahoo.com Low-Volatility Period with Leverage Sudden Price Drop with Margin Calls
- 16. What Causes Fat Tails? ● Implications are obtained by explicitly modeling and simulating the dynamics in trading. ● Physical modeling using stochastic processes ● Transitionary states and disequilibrium play crucial roles. ● Do not think that the system is always in equilibrium. https://www.amazon.co.jp/dp/B009IRP3GW M. Buchanan, “Forecast: What Physics, Meteorology, and the Natural Sciences Can Teach Us About Economics,” A&C Black, 2013
- 17. Regarding Irrationality as Disequilibrium ● Assume that human plays a game in his mind. ● Then irrationality is regarded as an outcome from state dynamics in mental processes. ● Rationality = choose the strategy in stationary state ● Irrationality = choose a strategy in transitional state ● Possibility to formalize many social phenomena universally via explicit state dynamics ● For better understanding: play p-beauty contest
- 18. References Understand Dynamics by (2/3)-beauty Contest What are the numbers chosen by these n players? Each player i ∈{1, . . ., n} chooses an integer Yi ∈[0, 100]. Winner(s): player(s) whose Yi is closest to 2 3 1 n n j=1Yj . SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 19. References Equilibrium of p-beauty Contest (Moulin, 1986) Nash Equilibrium when 0 ≤ p < 1: ∀i Yi = 0 1 Let C0 be a set of purely na¨ıve players, who choose from 0 to 100 at uniformly random. 2 Since E[ 1 |C0| i∈C0 Yi ] = 50, a slightly more strategic player in class C1 will choose round(50 × 2/3)=33. 3 Further strategic players in class C2 will choose round(33 × 2/3)=22. Players in class C3 will choose ... At convergence, every player should choose zero. However, do you believe such prediction? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 20. References A Result of p-beauty Contest by Real Humans Mean is apart from 0 (Camerer et al., 2004; Ho et al., 2006). Table: Average Choice in (2/3)-beauty Contests Subject Pool Group Size Sample Size Mean[Yi ] Caltech Board 73 73 49.4 80 year olds 33 33 37.0 High School Students 20-32 52 32.5 Economics PhDs 16 16 27.4 Portfolio Managers 26 26 24.3 Caltech Students 3 24 21.5 Game Theorists 27-54 136 19.1 SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 21. References Unreality of Nash Equilibrium Every player is homogeneous. All of them adopt the same thinking process. Every player has inﬁnite forecasting horizon. Can all real humans think so intelligently? Such unrealistic assumption leads vulnerability to perturbation. What if one player does not understand the game rule? What if one player intends to punish “rational” others? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 22. Analyzing More Complex Interactions ● Agent-based modeling ● Can be free from some assumptions: homogeneity, complete information, rationality, etc. ● True challenge: design of good agent models ● Often too many degrees of freedom in tuning J. M. Epstein, “Generative Social Science: Studies in Agent-Based Computational Modeling,“ Princeton University Press, 2012. S. F. Railsback and V. Grimm, “Agent-Based and Individual-Based Modeling: A Practical Introduction,“ Princeton University Press, 2011.
- 23. One Viewpoint for Good Agent Design ● Explicitly model human's bounded rationality. ● Irrationality is not the outcome of human's stupidity. ● Human does try optimization, but cannot reach the true optimum due to the lack of mental resources. – Finite memory about past events – Uncertainty over the future environment – Uncertainty over other agents' decisions ● Refer to Behavioral Game Theory ● Jewels in modeling bounded-rational agents
- 24. Short Summary ● Discussed irrationality in the real world. ● Observed that transitional states are often more realistic forecasts than equilibrium. ● Discussed direction for good agent models: hints for accurately modeling dynamics.
- 25. Agenda ● Irrationality and Disequilibrium: essential phenomena making social science challenging ● Failed Forecasts by Rational Economic Models: Irrational Disequilibrium or Multiple Equilibria? ● Frontiers in Mathematical Modeling of Irrationality: a transitional-state perspective ● For on-going PhD students: how to exploit your research experiences into Jobs
- 26. What is Rational / Irrational? ● Rationality = optimizing a consistent objective ● Irrationality = any behavior different from rationality ● Inconsistent optimization risks being manipulated by others. ● E.g., hyperbolic discounting: time-inconsistent preference order causes vulnerability of money pumps. ● Other forms of irrational decision making ● Choice from options whose coverage is manipulated by others
- 27. References Discrete Choice Modelling Goal: predict prob. of choosing an option from a choice set. Why solving this problem? For business: brand positioning among competitors For business: sales promotion (yet involving some abuse) To deeply understand how human makes decisions SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 28. References Random Utility Theory as Rational Model Each human is a maximizer of a probabilistic utility. i’s choice from Si = arg max j∈Si fi (vj ) mean utility + εij random noise Si : choice set for i, vj : vector of j’s attributes, fi : i’s mean utility function Assuming independence among every option’s attractiveness For both mean and noise: (e.g., logit (McFadden, 1980)) For only mean: (e.g., nested logit (Williams, 1977)) SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 29. References Context Eﬀects: Complexity of Human’s Choice An example of choosing PC (Kivetz et al., 2004) Each subject chooses 1 option from a choice set A B C D E CPU [MHz] 250 300 350 400 450 Mem. [MB] 192 160 128 96 64 Choice Set #subjects {A, B, C} 36:176:144 {B, C, D} 56:177:115 {C, D, E} 94:181:109 Can random utility theory still explain the preference reversals? B C or C B? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 30. References Similarity Eﬀect (Tversky, 1972) Top-share choice can change due to correlated utilities. E.g., one color from {Blue, Red} or {Violet, Blue, Red}? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 31. References Attraction Eﬀect (Huber et al., 1982) Introduction of an absolutely-inferior option A− (=decoy) causes irregular increase of option A’s attractiveness. Despite the natural guess that decoy never aﬀects the choice. If D A, then D A A− . If A D, then A is superior to both A− and D. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 32. References Compromise Eﬀect (Simonson, 1989) Moderate options within each chosen set are preferred. Diﬀerent from non-linear utility function involving diminishing returns (e.g., √ inexpensiveness+ √ quality). SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 33. Multiple Equilibria also Spoil Forecasts ● Pivotal mechanism (Clarke, 1971) to decide whether to start a public project ● Every player discloses a utility of the project outcome. ● If and only if sum(utilities) > 0, then project is started. ● Player i must pay tax amount abs(other players' utility sum), when sign of player i's utility and that of other players is opposite. ● For every player, honestly disclosing his true utility is optimal regardless other players' utilities. sum(utilities) = 1 decision = start disclosed utility -1 2 5 -3 -2 tax 2 0 0 4 3
- 34. Multiple Equilibria also Spoil Forecasts ● Failure of pivotal mechanism (Attiyeh+, 2000) ● Being rational is difficult because of too complex rules ● Even if rationality leading into an equilibrium exists, which equilibrium will be actually chosen? ● Each equilibrium has its own path from initial state. ● Identifying both of the path and finite time is hard. ● One promising way: converting transitional state in one game into an equilbrium of other game.
- 35. Short Summary ● Introduced more examples of irrational decision making by real humans. ● Irrationality spoils forecasting by standard economic models. ● Multiple equilibria further complicate the forecasting in addition to the irrational disequilibrium.
- 36. Agenda ● Irrationality and Disequilibrium: essential phenomena making social science challenging ● Failed Forecasts by Rational Economic Models: Irrational Disequilibrium or Multiple Equilibria? ● Frontiers in Mathematical Modeling of Irrationality: a transitional-state perspective ● For on-going PhD students: how to exploit your research experiences into Jobs
- 37. References Game with Heterogeneous Pay-Oﬀs Which numbers will be chosen by these 3 players? Each player i ∈{1, . . ., n} chooses an integer Yi ∈[0, 10]. Player #1’s pay-oﬀ: 39 + 12Y1 − (Y1+Y2)2 Player #2’s pay-oﬀ: 47 + 20Y2 − (Y2+Y3)2 Player #3’s pay-oﬀ: 6Y3 − (Y3− 1 2 (Y1 + Y2))2 SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 38. References An Idiot’s View of Game Theory If other players’ decisions Y i (Y1, . . . , Yi−1, Yi+1, . . . , Yn) are known, optimal decision Y ∗ i for player i is given by ∀i ∈{1, . . . , n} Y ∗ i |Y i = arg max Y ui (Y , Y i ). (1) ui : utility function of player i Game theory is merely solving a system of n equations by assuming ∀i Yi ≡ Y ∗ i in Eq. (1). Every player is assumed to be a utility maximizer. Variety of games just comes from the variable type of Yi . However, what if players are irrational or unpredictable? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 39. References Equilibrium of Linearly-Solvable Games Maximization of concave-quadratic equation = linear equality 0 = 12 − 2(Y ∗ 1 +Y2) 0 = 20 − 2(Y ∗ 2 +Y3) 0 = 6 − 2(Y ∗ 3 − 1 2 (Y1 + Y2)) ∀i Yi ≡Y ∗ i leads a matrix-vector relationship 2 2 0 0 2 2 −1 −1 2 Y ∗ 1 Y ∗ 2 Y ∗ 3 = 12 20 6 . (Y ∗ 1 , Y ∗ 2 , Y ∗ 3 ) = (2, 4, 6) with Pay-oﬀs = (27, 27, 27) SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 40. References Belief Learning: Iterative Solving of Game Equilibrium is tractable only for limited classes of utility functions, while in general is iteratively computed as t =0: Initialize each player’s decision by some value. t > 0: Compute the t-step optimum given the (t−1)-step decisions by others. ∀i ∈{1, . . . , n} Y (t) i |Y (t−1) i = arg max Y ui (Y , Y (t−1) i ) Belief learning: classes of algorithms to iteratively compute the equilibrium. (t + 1)-step looking-ahead player beats the t-step-only players, (t + 2)-step player beats... How about using Y (t) i at ﬁnite t, instead of the one at t →∞? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 41. How to Formalize Context Effects? ● What dynamics causes context effects? ● Hypothesis: a dynamical process to estimate utility function (Takahashi & Morimura, 2015). ● Irrational contextual effects are observed via regularized estimates of the utility function. ● Machine learning as a dynamical process ● Transitionary state in maximum-likelihood estimation ● Stationary state in Bayesian shrinkage estimation
- 42. References Gaussian Process Uncertainty Aversion (GPUA) A dual-personality model regarding utilities as samples in statistics (Takahashi and Morimura, 2015) Assumption 1: Utility function is partially disclosed to DMS. 1 UC computes the sample value of every option’s utility, and sends only these samples to DMS. 2 DMS statistically estimates the utility function. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 43. References GPUA: Mental Conﬂict as Bayesian Shrinkage Assumption 2: DMS does Bayesian shrinkage estimation. i ∈{1, . . . , n}: context, yi ∈{1, . . . , m[i]}: ﬁnal choice Xi (xi1 ∈RdX , . . . , xim[i]) : features of m[i] options Objective Data: values of random utilities vi (vi1, . . . , vim[i]) ∼N µi , σ2 Im[i] , vij = b+wφ φ (xij ) µi : Rm[i]: vec. of the true mean utility, σ2: noise level b: bias term, φ : RdX →Rdφ : mapping function. wφ: vec. of coeﬃcients Subjective Prior: choice-set-dependent Gaussian process µi ∼ N 0m[i], σ2 K(Xi ) s.t. K(Xi ) = (K(xij , xij ))∈Rm[i]×m[i] µi ∈Rm[i]: vec. of random utilities, K(·, ·): similarity between options Final choice: based on (Posterior mean u∗ i + i.i.d. noise) as u∗ i = K(Xi ) Im[i]+K(Xi ) −1 b1m[i]+Φi wφ , yi = arg max j (u∗ ij + εij ) where ∀j εij ∼ Gumbel. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 44. References GPUA: Irrationality by Bayesian Shrinkage Implication of (2): similarity-dependent discounting u∗ i = K(Xi ) Im[i]+K(Xi ) −1 shrinkage factor b1m[i]+Φi wφ vec. of utility samples . (2) Under RBF kernel K(x, x ) = exp(−γ x − x 2 ), an option dissimilar to others involves high uncertainty. Strongly shrunk into prior mean 0. Context eﬀects as Bayesian uncertainty aversion 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1 2 3 4 FinalEvaluation X1=(5-X2) DA - A {A,D} {A,A- ,D} 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1 2 3 4FinalEvaluation X1=(5-X2) DCBA {A,B,C} {B,C,D} SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 45. References GPUA: Convex Optimization using Posterior Mean Global ﬁtting of the parameters using data (Xi , yi )n i=1 Fix the mapping and similarity functions during updates. Shrinkage factor Hi K(Xi )(Im[i] + K(Xi ))−1 is constant! Obtaining a MAP estimate is convex w.r.t. (b, wφ). max b,wφ n i=1 ( bHi 1m[i]+Hi Φi wφ Context−speciﬁc Hi is multiplied. , yi ) − c 2 wφ 2 Exploiting the log-concavity of multinomial logit (u∗ i , yi ) log exp(u∗ iyi ) m[i] j =1 exp(u∗ ij ) SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 46. References GPUA: Experimental Settings Evaluates accuracy & log-likelihood for real choice data. Dataset #1: PC (n=1, 088, dX =2) Dataset #2: SP (n=972, dX =2) Subjects are asked of choosing a speaker. A B C D E Power [Watt] 50 75 100 125 150 Price [USD] 100 130 160 190 220 Choice Set #subjects {A, B, C} 45:135:145 {B, C, D} 58:137:111 {C, D, E} 95:155: 91 Dataset #3: SM (n=10, 719, dX =23) SwissMetro dataset (Antonini et al., 2007) Subjects are asked of choosing one transportation, either from {train, car, SwissMetro} or {train, SwissMetro}. Attribute of option: cost, travel time, headway, seat type, and type of transportation. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 47. References GPUA: Cross-Validation Performances High predictability in addition to the interpretable mechanism. For SP, successfully detected combination of compromise eﬀect & prioritization of power. 1st best for PC & SP. 2nd best for higher-dimensional SM: slightly worse than highly expressive nonparametric version of mixed multinomial logit (McFadden and Train, 2000). -1.1 -1 -0.9 -0.8 AverageLog-Likelihood Dataset PC SP SM LinLogit NpLogit LinMix NpMix GPUA 0.3 0.4 0.5 0.6 0.7 ClassificationAccuracy Dataset PC SP SM LinLogit NpLogit LinMix NpMix GPUA 2 3 4 100 150 200 Evaluation Price [USD] EDCBA Obj. Eval. {A,B,C} {B,C,D} {C,D,E} SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 48. Linking ML with Game Theory (GT) via Shrinkage Principle Optimization without shrinkage Optimization with shrinkage ML GT Maximum-Likelihood estimation Bayesian estimation Transitional State or Quantal Response Equilibrium Nash Equilibrium Optimal for training data, but less generalization capability to test data Optimal for given game but less predictable to real- world decisions Shrinkage towards uniform probabilities causes suboptimality for the given game, but more predictable to real-world decisions Shrinkage towards prior causes suboptimality for training data, but more generalization capability to test data
- 49. References Quantitative Handling of Irrationality Iterative equilibrium computation lightens two natural ways. Early stopping at step k: Level-k thinking or Cognitive Hierarchy Theory (Camerer et al., 2004) Humans cannot predict the inﬁnite future. Using non-stationary transitional state Randomization of utility via noise εit: Quantal Response Equilibrium (McKelvey and Palfrey, 1995) ∀i ∈{1, . . . , n} Y (t) i |Y (t−1) i = arg max Y fi (Y , Y (t−1) i ) + εit Both methods essentially work as regularization of rationality. Shrinkage into initial values or uniform choice probabilities Aﬃnity to Bayesian regularization in ML SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 50. References Logit Quantal Response Equilibrium (LQRE) A special form of QRE associated with RUT. If εit obeys the standard Gumbel distribution and Y (t) i |Y (t−1) i = arg max Y ∈S fi (Y , Y (t−1) i ) + εit/βi , then choice probability becomes closed-form as P(Y (t) i = y|Y (t−1) i ) = exp βi fi (y, Y (t−1) i ) y ∈S exp βi fi (y , Y (t−1) i ) . βi is called the degree of irrationality of player i. βi →0: uniform choice probability (na¨ıve) βi →∞: Nash equilibrium (deterministic & rational) SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 51. Early Stopping and Regularization ML as a Dynamical System to find the optimal parameters GT as a Dynamical System to find the equilibrium Parameter #1 Parameter #2 Exact Maximum-likelihood estimate (e.g., OLS) Exact Bayesian estimate shrunk towards zero (e.g., Ridge regression) 0 t=10 t=20 t=30 t=50 An early-stopping estimate t=0 t=1 t →∞ t=2 ... mean = 50 mean = 34 mean = 15 mean = 0 Nash Equilibrium Level-2 Transitional State
- 52. References Towards Useful Decision Making by using QRE Economists discuss when utility functions {fi }n i=1 are known. QRE is analytically-intractable but can be simulated. E.g., ad-auction for irrational bidders (Rong et al., 2015) ML scientists should estimate unknown utility functions! Extension of statistical marketing research methods through rich functional approximation techniques in ML SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 53. References Multi-Agent Extension of RUT in Marketing RUT in marketing research has already been data-oriented. Estimating utility functions from real data DCM such as Logit model (McFadden, 1980) Identical opt. objective to multinomial logistic regression Conjoint analysis (Green and Srinivasan, 1978) Special case of DCM by showing only 2 options Related with learning to rank problem: see (Chapelle and Harchaoui, 2005) Adding other-player-dependent terms into existing marketing research models yields a simulation model to compute QRE. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 54. References Possible Formalisms & Algorithmic Studies Multi-agent generalization of DCM or learning to rank Simulation-based ﬁtting (e.g., Approximate Bayesian Computation (Tavar´e et al., 1997)) Functional approximations (e.g., Gradient Boosting Decision Trees (Friedman, 2001), Deep Neural Network) with partially-observable other players’ decisions SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 55. References A Future Forecast: Rise of Deep Belief Learning Belief Learning (BL) vs Reinforcement Learning (RL) BL: explicitly guessing other players’ thinking processes RL: choosing optimal actions purely from experiences Other players’ decision functions are implicitly parts of the environment While predictive accuracies would be similar, BL provides more white boxes than RL in terms of thinking processes AlphaGo is a successful application of Deep RL (e.g., (Mnih et al., 2013; van Hasselt et al., 2016)). What will be killer applications of Deep BL? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 56. Other Approaches for Irrationality ● Use quantum theory instead of probability ● Quantum Cognition – (Burza+, 2009; Mogiliansky+, 2009; de Barros & Suppes, 2009; Busemeyer & Burza, 2012) ● Key mechanism: double standard in quantum theory – During (unobserved) thinking: integrated over complex state space – In (observed) decision: classical probability by taking the absolute magnitude of state https://www.amazon.co.jp/Quantum-Models-Cognition-Decision-Busemeyer/dp/1107419883/
- 57. Short Summary ● Introduced recent advances on mathematical modeling of human's irrationality, for more accurate forecasts. ● Handled irrationality as transional states in both Machine Learning and Game Theory. ● Importing mathematical techniques from both ML and GT communities will serve better social decision making with more accurate forecasts.
- 58. Agenda ● Irrationality and Disequilibrium: essential phenomena making social science challenging ● Failed Forecasts by Rational Economic Models: Irrational Disequilibrium or Multiple Equilibria? ● Frontiers in Mathematical Modeling of Irrationality: a transitional-state perspective ● For on-going PhD students: how to exploit your research experiences into Jobs
- 59. WARNING The following pages exhibit the author's personal opinions on how to make a good research direction and/or identify a good area of business. Effectiveness of these ideas has not been scientifically proved. Read them at your own risk.
- 60. How PhD changed my life ● Before obtaining PhD ● Job: Research Staff Member in a large B2B enterprise ● Research Topic: Required to be sticked with one coherent research direction ● Seeking for problems that are solvable via my Machine Learning (ML) disciplines ● After obtaining PhD ● Job: Engineer in a small B2C startup ● Research Topic: Freedom to target more ambitious topics in broader area ● Integrating ML disciplines with multi-agent perspective obtained during schooling
- 61. Hope and Actuality in PhD Course ● What I was intending ● Exploit ML for automatically designing agents. ● Or learn the essence in manually designing agents, through seminar discussions. ● What actually occurred ● Still difficult to know how to design agents! – Why this paper's agent model is designed like this? ● Effective viewpoints on the design of agents came after finishing PhD.
- 62. Interplay between Research and Job ● Paid Job requires real-world decision making. ● Skin in the game: you cannot use models or approaches that you do not rely on. ● In order to be confident on your approach, make focus & apply Occam's razor strongly. ● Avoid using models #1 & #2 & #3 ... Combintion makes difficult of root-cause finding in failure. ● Define your unique optimization problem, which is directly solvable by one essential approach. – Also one-principle-based paper is easily publishable.
- 63. Interplay between Research and Job ● A case in job: how to create network externality? ● The key factor in successful platform business (e.g., Operating Systems, Social Network Services) ● You must have a good mechanism to incentivize users to use your platform. ● Do the existing mechanisms really incentivize users? ● Are they quantitative to enable real operations? ● Freeing from unrealistic assumptions and practicality requirement are natual sources of research ideas.
- 64. Some Tactics under Competitions ● Development of the truly universal approach = Red Ocean fought by the World's Top Talents ● Identify the minimum requirement. You create an approach at least universal in your area. ● Make an approach that competitors dislike to use. ● Such approach often causes disruptive innovation. ● Do not confuse simplicity with naïvety
- 65. Necessity of Ample Surveys ● Avoid reinventing wheels. Most industrial problems have already been partially solved. ● Respect & steal other players' ideas by reading. ● Remember that some prior work is written over- confidently; prior authors do not know conditions that spoil their approaches in your new problem. ● Key for success: good strategy to search for relevant papers and books
- 66. Encouraging Bottom-Up Learning ● Check the neighboring disciplines from yours; be in Optimum Stimulation Level (Berlyne, 1960) ● Your brain is strongly stimulated by insights in slightly distant areas from your expertise. ● Deep understanding on the very slight difference between two areas often clarifies the white space in your area. Machine Learnng StatisticsBiostatistics Econometrics Psychometrics Cognitive Science Neuroscience Behavioral Economics Behavioral Game Theory
- 67. Uncertainty is Your Friend ● Most people hate uncertainty, but you must love it. ● Further one tactic: beat the irrationality of your competitors! ● The more uncertain parts your research or business contains, the more competitors will be fooled by too much complexity. ● You: solve the entire problem by one critial solution. ● Competitors: solve each of the sub-problems by its specific method, and trapped by poor sub-optima. ● Optimism in face of uncertainty
- 68. Uncertainty is Your Friend ● Care the difference between risks and uncertainty. ● Risks: volatility calibrated from existing data ● Uncertainty: cannot be quantified from data ● Donald Rumsfeld's unknown unknowns. ● You do not have take high risks. But you should take high uncertainties. ● In big-data era, competitors rush into the areas with ample datasets, and become professed with risks. ● By contrast, the human's nature of hating uncertainty would remain, and it will be a source of your success.
- 69. References References I Ainslie, G. W. (1974). Impulse control in pigeons. Journal of the Experimental Analysis of Behavior, 21(3):485–489. Antonini, G., Gioia, C., Frejinger, E., and Th´emans, M. (2007). Swissmetro: description of the data. http://biogeme.epﬂ.ch/swissmetro/examples.html. Attiyeh, G., Franciosi, R., and Isaac, R. M. (2000). Experiments with the pivot process for providing public goods. Public Choice, 102:95–114. Azfar, O. (1999). Rationalizing hyperbolic discounting. Journal of Economic Behavior and Organization, 38(2):245–252. Blanchard, O. J. and Watson, M. W. (1982). Bubbles, rational expectations and ﬁnancial markets. Crises in the Economic and Financial Structure, pages 295–316. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 70. References References II Bruza, P., Kitto, K., Nelson, D., and McEvoy, C. (2009). Is there something quantum-like about the human mental lexicon? Journal of Mathematical Psychology, 53(5):362–377. Camerer, C. F., Ho, T. H., and Chong, J. (2004). A cognitive hierarchy model of games. Quarterly Journal of Economics, 119:861–898. Chapelle, O. and Harchaoui, Z. (2005). A machine learning approach to conjoint analysis. In Advances in Neural Information Processing Systems 17, pages 257–264. MIT Press, Cambridge, MA, USA. Clarke, E. H. (1971). Multipart pricing of public goods. Public Choice, 2:19–33. Cont, R. (2001). Empirical properties of asset returns: stylized facts and statistical issues. Quantitative Finance, 1(2):223–236. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 71. References References III Cubitt, R. P. and Sugden, R. (2001). On money pumps. Games and Economic Behavior, 37(1):121–160. de Barros, J. A. and Suppes, P. (2009). Quantum mechanics, interference, and the brain. Journal of Mathematical Psychology, 53(5):306–313. Dotson, J. P., Lenk, P., Brazell, J., Otter, T., Maceachern, S. N., and Allenby, G. M. (2009). A probit model with structured covariance for similarity eﬀects and source of volume calculations. http://ssrn.com/abstract=1396232. Farmer, J. D. and Geanakoplos, J. (2009). Hyperbolic discounting is rational: Valuing the far future with uncertain discount rates. Cowles Foundation Discussion Paper No. 1719. Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5):1189–1232. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 72. References References IV Gonz´alez-Vallejo, C. (2002). Making trade-oﬀs: A probabilistic and context-sensitive model of choice behavior. Psychological Review, 109:137–154. Green, P. and Srinivasan, V. (1978). Conjoint analysis in consumer research: Issues and outlook. Journal of Consumer Research, 5:103–123. Ho, T. H., Lim, N., and Camerer, C. F. (2006). Modeling the psychology of consumer and ﬁrm behavior with behavioral economics. Journal of Marketing Research, 43(3):307–331. Huber, J., Payne, J. W., and Puto, C. (1982). Adding asymmetrically dominated alternatives: Violations of regularity and the similarity hypothesis. Journal of Consumer Research, 9:90–98. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 73. References References V Johansen, A., Ledoit, O., and Sornette, D. (2000). Crashes as critical points. International Journal of Theoretical and Applied Finance, 3:219–255. Johansen, A. and Sornette, D. (1999). Critical crashes. Risk, 12(1):91–94. Johansen, A., Sornette, D., and Ledoit, O. (1999). Predicting ﬁnancial crashes using discrete scale invariance. Journal of Risk, 1(4):5–32. Kivetz, R., Netzer, O., and Srinivasan, V. S. (2004). Alternative models for capturing the compromise eﬀect. Journal of Marketing Research, 41(3):237–257. McFadden, D. and Train, K. (2000). Mixed MNL models for discrete response. Journal of Applied Econometrics, 15:447 –470. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 74. References References VI McFadden, D. L. (1980). Econometric models of probabilistic choice among products. Journal of Business, 53(3):13–29. McKelvey, R. and Palfrey, T. (1995). Quantal response equilibria for normal form games. Games and Economic Behavior, 10:6–38. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. In NIPS Deep Learning Workshop. Mogiliansky, A. L., Zamir, S., and Zwirn, H. (2009). Type indeterminacy: A model of the KT (kahneman tversky)-man. Journal of Mathematical Psychology, 53(5):349–361. Moulin, H. (1986). Game Theory for the Social Sciences. NYU Press, second edition edition. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 75. References References VII Roe, R. M., Busemeyer, J. R., and Townsend, J. T. (2001). Multialternative decision ﬁeld theory: A dynamic connectionist model of decision making. Psychological Review, 108:370–392. Rong, J., Qin, T., and An, B. (2015). Computing quantal response equilibrium for sponsored search auctions. In Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2015), pages 1803–1804, Richland, SC. International Foundation for Autonomous Agents and Multiagent Systems. Shenoy, P. and Yu, A. J. (2013). A rational account of contextual eﬀects in preference choice: What makes for a bargain? In Proceedings of the Cognitive Science Society Conference. Simonson, I. (1989). Choice based on reasons: The case of attraction and compromise eﬀects. Journal of Consumer Research, 16:158–174. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 76. References References VIII Sornette, D. (2004). Why Stock Markets Crash: Critical Events in Complex Financial Systems. Princeton University Press. Takahashi, R. and Morimura, T. (2015). Predicting preference reversals via gaussian process uncertainty aversion. In Proceedings of the 18th International Conference on Artiﬁcial Intelligence and Statistics (AISTATS 2015), pages 958–967. Tavar´e, S., Balding, D. J., Griﬃths, R. C., and Donnelly, P. (1997). Inferring coalescence times from DNA sequence data. Genetics, 145(2):505–518. Thurner, S., Farmer, J. D., and Geanakoplos, J. (2012). Leverage causes fat tails and clustered volatility. Quantitative Finance, 12(5):695–707. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 77. References References IX Trueblood, J. S. (2014). The multiattribute linear ballistic accumulator model of context eﬀects in multialternative choice. Psychological Review, 121(2):179– 205. Tversky, A. (1972). Elimination by aspects: A theory of choice. Psychological Review, 79:281–299. Usher, M. and McClelland, J. L. (2004). Loss aversion and inhibition in dynamical models of multialternative choice. Psychological Review, 111:757– 769. van Hasselt, H., Guez, A., and Silver, D. (2016). Deep reinforcement learning with double Q-learning. In Proceedings of the Thirtieth AAAI Conference on Artiﬁcial Intelligence (AAAI-16). Wen, C.-H. and Koppelman, F. (2001). The generalized nested logit model. Transportation Research Part B, 35:627–641. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 78. References References X Williams, H. (1977). On the formulation of travel demand models and economic evaluation measures of user beneﬁt. Environment and Planning A, 9(3):285–344. Yai, T. (1997). Multinomial probit with structured covariance for route choice behavior. Transportation Research Part B: Methodological, 31(3):195–207. Yue, Y. and Guestrin, C. (2011). Linear submodular bandits and their application to diversiﬁed retrieval. In Shawe-taylor, J., Zemel, R., Bartlett, P., Pereira, F., and Weinberger, K., editors, Advances in Neural Information Processing Systems 24, pages 2483–2491. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 79. References References XI Zhang, S. and Yu, A. J. (2013). Forgetful Bayes and myopic planning: Human learning and decision-making in a bandit setting. In Burges, C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K., editors, Advances in Neural Information Processing Systems 26, pages 2607–2615. Curran Associates, Inc. Ziegler, C.-N., McNee, S. M., Konstan, J. A., and Lausen, G. (2005). Improving recommendation lists through topic diversiﬁcation. In Proceedings of the 14th international conference on World Wide Web (WWW 2005), pages 22–32. ACM. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
- 80. THANK YOU FOR ATTENDING!

No public clipboards found for this slide

Special Offer to SlideShare Readers

The SlideShare family just got bigger. You now have unlimited* access to books, audiobooks, magazines, and more from Scribd.

Cancel anytime.
Be the first to comment