Here are a few suggestions on how to improve the Zermelo algorithm when it is too slow:
1. Add a depth limit. Stop recursion when a maximum search depth is reached. Return a heuristic evaluation instead of continuing search.
2. Use alpha-beta pruning. Track the best value found (alpha) and prune branches that cannot improve on it.
3. Iterative deepening. Run successive searches with increasing depth limits to get progressively better approximations.
4. Move ordering. Evaluate better moves earlier in the search tree. This prunes bad moves earlier.
5. Transposition tables. Store previously computed move evaluations to avoid re-expanding the same position.
6. Parallelize the
The document discusses tools for artificial intelligence and their applications. It describes Olivier Teytaud's work at Tao, a research group in Paris focused on reservoir computing, optimal decision making under uncertainty, optimization, and machine learning. It then provides examples of applications for these tools in electricity generation, urban rivals, pokemons, minesweeper, and solving unsolved situations in the game of Go. Olivier suggests that breakthroughs in games can help open doors to applying these algorithms to more important real-world problems by building trust in the approaches.
Towards new solutions for scientific computing: the case of JuliaMaurizio Tomasi
The year 2018 marks the consolidation of Julia (https://julialang.org/), a programming language designed for scientific computing, as the first stable version (1.0) has been released, in August 2018. Among its main features, expressiveness and high execution speeds are the most prominent: the performance of Julia code is similar to
statically compiled languages, yet Julia provides a nice interactive shell and fully supports Jupyter; moreover, it can transparently call external codes written in C, Fortran, and even Python and R without the need of wrappers. The usage of Julia in the astronomical community is growing, and a GitHub oganization named JuliaAstro takes care of coordinating the development of packages. In this ADASS talk we present the features and shortcomings of this language, and discuss its application in astronomy and astrophysics.
Big Data Analytics for connected home: a few usecases, some important messages and a little example. Presentation given at CEA Cadarache - Cité des Nouvelles Energies at the strategic comittee of ARCSIS (http://www.arcsis.org/missions.html)
Deep Water - GPU Deep Learning for H2O - Arno CandelSri Ambati
1) The document discusses the history and development of artificial intelligence, machine learning, and deep learning from the 1950s to present.
2) It introduces H2O.ai's Deep Water platform for deep learning which leverages popular open source tools like TensorFlow, MXNet, and Caffe to enable large-scale deep learning on CPUs and GPUs.
3) Deep Water allows users to easily train, compare, and deploy deep learning models through H2O's APIs in R, Python, and Flow for big data use cases like image, video, text, and time series analysis.
- The document discusses games with simultaneous actions and hidden information. It presents games as directed graphs with actions, players, observations, rewards, and loops.
- Games with simultaneous actions and short-term hidden information can be represented as games with hidden information by removing intermediate turns.
- Questions about the existence of a sure-win strategy for one player (the "UD" question) are only relevant for games with full observability, not matrix games.
Noisy Optimization combining Bandits and Evolutionary AlgorithmsOlivier Teytaud
@inproceedings{rolet:inria-00437140,
hal_id = {inria-00437140},
url = {http://hal.inria.fr/inria-00437140},
title = {{Bandit-based Estimation of Distribution Algorithms for Noisy Optimization: Rigorous Runtime Analysis}},
author = {Rolet, Philippe and Teytaud, Olivier},
abstract = {{We show complexity bounds for noisy optimization, in frame- works in which noise is stronger than in previously published papers[19]. We also propose an algorithm based on bandits (variants of [16]) that reaches the bound within logarithmic factors. We emphasize the differ- ences with empirical derived published algorithms.}},
keywords = {noisy optimization evolutionary algorithms bandits},
language = {Anglais},
affiliation = {Laboratoire de Recherche en Informatique - LRI , TAO - INRIA Futurs , TAO - INRIA Saclay - Ile de France},
booktitle = {{Lion4}},
address = {Venice, Italie},
audience = {internationale },
year = {2010},
pdf = {http://hal.inria.fr/inria-00437140/PDF/lion4long.pdf},
}
@inproceedings{coulom:hal-00517157,
hal_id = {hal-00517157},
url = {http://hal.archives-ouvertes.fr/hal-00517157},
title = {{Handling Expensive Optimization with Large Noise}},
author = {Coulom, R{\'e}mi and Rolet, Philippe and Sokolovska, Nataliya and Teytaud, Olivier},
abstract = {{This paper exhibits lower and upper bounds on runtimes for expensive noisy optimization problems. Runtimes are expressed in terms of number of fitness evaluations. Fitnesses considered are monotonic transformations of the {\em sphere} function. The analysis focuses on the common case of fitness functions quadratic in the distance to the optimum in the neighborhood of this optimum---it is nonetheless also valid for any monotonic polynomial of degree p>2. Upper bounds are derived via a bandit-based estimation of distribution algorithm that relies on Bernstein races called R-EDA. It is known that the algorithm is consistent even in non-differentiable cases. Here we show that: (i) if the variance of the noise decreases to 0 around the optimum, it can perform optimally for quadratic transformations of the norm to the optimum, (ii) otherwise, it provides a slower convergence rate than the one exhibited empirically by an algorithm called Quadratic Logistic Regression based on surrogate models---although QLR requires a probabilistic prior on the fitness class.}},
keywords = {Noisy optimization, Bernstein races},
language = {Anglais},
affiliation = {SEQUEL - INRIA Lille - Nord Europe , TAO - INRIA Saclay - Ile de France , Laboratoire de Recherche en Informatique - LRI},
booktitle = {{Foundations of Genetic Algorithms (FOGA 2011)}},
pages = {TBA},
address = {Autriche},
editor = {ACM },
audience = {internationale },
year = {2011},
month = Jan,
pdf = {http://hal.archives-ouvertes.fr/hal-00517157/PDF/foga10noise.pdf},
}
Ilab Metis: we optimize power systems and we are not afraid of direct policy ...Olivier Teytaud
Ilab METIS is a collaboration between TAO, a machine learning and optimization team within INRIA, and Artelys, an SME focused on optimization. They work on optimizing energy policies through simulations of power systems while taking into account uncertainties and stochastic variables. Their methodologies use a hybrid of reinforcement learning, mathematical programming, and direct policy search to optimize investments and operational decisions for power grids over multiple timescales while handling constraints. They have applied their approaches to problems involving interconnection planning, demand balancing, and renewable integration on scales from cities to entire continents.
- The document discusses energy management in France and potential areas of research collaboration between France and Taiwan.
- Key areas discussed include optimizing long-term investment policies for electricity generation using tools like reinforcement learning and stochastic programming to account for uncertainties.
- Specific questions mentioned are around optimal connections between Europe and Africa, impacts of subsidizing solar power or switching off nuclear plants, and benefits of demand reduction contracts.
- The researcher proposes combining methods like direct policy search and Monte Carlo tree search to better optimize long-term planning while accounting for short-term effects. Plans are discussed to test new ideas, share data and codes, and potentially organize joint work between the two regions.
The document discusses tools for artificial intelligence and their applications. It describes Olivier Teytaud's work at Tao, a research group in Paris focused on reservoir computing, optimal decision making under uncertainty, optimization, and machine learning. It then provides examples of applications for these tools in electricity generation, urban rivals, pokemons, minesweeper, and solving unsolved situations in the game of Go. Olivier suggests that breakthroughs in games can help open doors to applying these algorithms to more important real-world problems by building trust in the approaches.
Towards new solutions for scientific computing: the case of JuliaMaurizio Tomasi
The year 2018 marks the consolidation of Julia (https://julialang.org/), a programming language designed for scientific computing, as the first stable version (1.0) has been released, in August 2018. Among its main features, expressiveness and high execution speeds are the most prominent: the performance of Julia code is similar to
statically compiled languages, yet Julia provides a nice interactive shell and fully supports Jupyter; moreover, it can transparently call external codes written in C, Fortran, and even Python and R without the need of wrappers. The usage of Julia in the astronomical community is growing, and a GitHub oganization named JuliaAstro takes care of coordinating the development of packages. In this ADASS talk we present the features and shortcomings of this language, and discuss its application in astronomy and astrophysics.
Big Data Analytics for connected home: a few usecases, some important messages and a little example. Presentation given at CEA Cadarache - Cité des Nouvelles Energies at the strategic comittee of ARCSIS (http://www.arcsis.org/missions.html)
Deep Water - GPU Deep Learning for H2O - Arno CandelSri Ambati
1) The document discusses the history and development of artificial intelligence, machine learning, and deep learning from the 1950s to present.
2) It introduces H2O.ai's Deep Water platform for deep learning which leverages popular open source tools like TensorFlow, MXNet, and Caffe to enable large-scale deep learning on CPUs and GPUs.
3) Deep Water allows users to easily train, compare, and deploy deep learning models through H2O's APIs in R, Python, and Flow for big data use cases like image, video, text, and time series analysis.
- The document discusses games with simultaneous actions and hidden information. It presents games as directed graphs with actions, players, observations, rewards, and loops.
- Games with simultaneous actions and short-term hidden information can be represented as games with hidden information by removing intermediate turns.
- Questions about the existence of a sure-win strategy for one player (the "UD" question) are only relevant for games with full observability, not matrix games.
Noisy Optimization combining Bandits and Evolutionary AlgorithmsOlivier Teytaud
@inproceedings{rolet:inria-00437140,
hal_id = {inria-00437140},
url = {http://hal.inria.fr/inria-00437140},
title = {{Bandit-based Estimation of Distribution Algorithms for Noisy Optimization: Rigorous Runtime Analysis}},
author = {Rolet, Philippe and Teytaud, Olivier},
abstract = {{We show complexity bounds for noisy optimization, in frame- works in which noise is stronger than in previously published papers[19]. We also propose an algorithm based on bandits (variants of [16]) that reaches the bound within logarithmic factors. We emphasize the differ- ences with empirical derived published algorithms.}},
keywords = {noisy optimization evolutionary algorithms bandits},
language = {Anglais},
affiliation = {Laboratoire de Recherche en Informatique - LRI , TAO - INRIA Futurs , TAO - INRIA Saclay - Ile de France},
booktitle = {{Lion4}},
address = {Venice, Italie},
audience = {internationale },
year = {2010},
pdf = {http://hal.inria.fr/inria-00437140/PDF/lion4long.pdf},
}
@inproceedings{coulom:hal-00517157,
hal_id = {hal-00517157},
url = {http://hal.archives-ouvertes.fr/hal-00517157},
title = {{Handling Expensive Optimization with Large Noise}},
author = {Coulom, R{\'e}mi and Rolet, Philippe and Sokolovska, Nataliya and Teytaud, Olivier},
abstract = {{This paper exhibits lower and upper bounds on runtimes for expensive noisy optimization problems. Runtimes are expressed in terms of number of fitness evaluations. Fitnesses considered are monotonic transformations of the {\em sphere} function. The analysis focuses on the common case of fitness functions quadratic in the distance to the optimum in the neighborhood of this optimum---it is nonetheless also valid for any monotonic polynomial of degree p>2. Upper bounds are derived via a bandit-based estimation of distribution algorithm that relies on Bernstein races called R-EDA. It is known that the algorithm is consistent even in non-differentiable cases. Here we show that: (i) if the variance of the noise decreases to 0 around the optimum, it can perform optimally for quadratic transformations of the norm to the optimum, (ii) otherwise, it provides a slower convergence rate than the one exhibited empirically by an algorithm called Quadratic Logistic Regression based on surrogate models---although QLR requires a probabilistic prior on the fitness class.}},
keywords = {Noisy optimization, Bernstein races},
language = {Anglais},
affiliation = {SEQUEL - INRIA Lille - Nord Europe , TAO - INRIA Saclay - Ile de France , Laboratoire de Recherche en Informatique - LRI},
booktitle = {{Foundations of Genetic Algorithms (FOGA 2011)}},
pages = {TBA},
address = {Autriche},
editor = {ACM },
audience = {internationale },
year = {2011},
month = Jan,
pdf = {http://hal.archives-ouvertes.fr/hal-00517157/PDF/foga10noise.pdf},
}
Ilab Metis: we optimize power systems and we are not afraid of direct policy ...Olivier Teytaud
Ilab METIS is a collaboration between TAO, a machine learning and optimization team within INRIA, and Artelys, an SME focused on optimization. They work on optimizing energy policies through simulations of power systems while taking into account uncertainties and stochastic variables. Their methodologies use a hybrid of reinforcement learning, mathematical programming, and direct policy search to optimize investments and operational decisions for power grids over multiple timescales while handling constraints. They have applied their approaches to problems involving interconnection planning, demand balancing, and renewable integration on scales from cities to entire continents.
- The document discusses energy management in France and potential areas of research collaboration between France and Taiwan.
- Key areas discussed include optimizing long-term investment policies for electricity generation using tools like reinforcement learning and stochastic programming to account for uncertainties.
- Specific questions mentioned are around optimal connections between Europe and Africa, impacts of subsidizing solar power or switching off nuclear plants, and benefits of demand reduction contracts.
- The researcher proposes combining methods like direct policy search and Monte Carlo tree search to better optimize long-term planning while accounting for short-term effects. Plans are discussed to test new ideas, share data and codes, and potentially organize joint work between the two regions.
Tools for Discrete Time Control; Application to Power SystemsOlivier Teytaud
3 main algorithms from the state of the art:
- Model Predictive Control
- Stochastic Dynamic Programming
- Direct Policy Search
==> and our proposal, a modified Direct Policy Search
termed Direct Value Search
Hydroelectricity uses water to produce electricity and has advantages for electricity storage. It provides daily, yearly, and negative electricity production by pumping water to higher reservoirs. However, expanding hydroelectricity is challenging due to its large infrastructure requirements and local environmental impacts. New technologies may improve energy storage capabilities and grid stability in the future, but developing large-scale annual storage remains difficult given constraints. Hydroelectricity will continue playing an important role in energy systems alongside other renewable technologies and efficiency strategies.
Choosing between several options in uncertain environmentsOlivier Teytaud
The document discusses bandit problems with strategic choices and small budgets. It defines bandit problems, strategic bandit problems, and compares the two. It presents algorithms for exploring options and making recommendations in both one-player and two-player settings. Experimental results on a Go positioning problem and an online card game show that TEXP3 outperforms other algorithms in two-player settings. The document concludes with discussions on extensions to structured bandits and using strategic bandits to model investment choices.
Artificial Intelligence and Optimization with ParallelismOlivier Teytaud
This document discusses parallelism in artificial intelligence and evolutionary computation. It explains that comparison-based optimization algorithms, which include many evolutionary algorithms, can be naturally parallelized by speculatively running multiple branches in parallel with a branching factor of 3 or more. This allows theoretical logarithmic speedups to be achieved in practice through simple parallelization tricks.
This document provides an overview of distributed decision making in partially observable dynamic games and multiobjective policy optimization. It discusses applying these techniques to optimization problems in games like chess and Go, as well as industrial applications like managing groups of power plants involving renewable energy, nuclear power, coal, hydroelectric power, and interactions with electricity consumers and networks. The goal is to optimize strategies using parallel computing and test these approaches on games and energy systems.
A simple tutorial on Monte-Carlo Tree Search
Contains a description of dynamic programming and alpha-beta search, then MCTS. Special cases for simultaneous actions are discussed.
I should add comments so that it can be used without preliminary knowledge of MCTS, if there is at least one request for doing so I'll do it.
@article{gelly:hal-00695370,
hal_id = {hal-00695370},
url = {http://hal.inria.fr/hal-00695370},
title = {{The Grand Challenge of Computer Go: Monte Carlo Tree Search and Extensions}},
author = {Gelly, Sylvain and Kocsis, Levente and Schoenauer, Marc and Sebag, Mich{\`e}le and Silver, David and Szepesvari, Csaba and Teytaud, Olivier},
abstract = {{The ancient oriental game of Go has long been considered a grand challenge for artificial intelligence. For decades, com- puter Go has defied the classical methods in game tree search that worked so successfully for chess and checkers. How- ever, recent play in computer Go has been transformed by a new paradigm for tree search based on Monte-Carlo meth- ods. Programs based on Monte-Carlo tree search now play at human-master levels and are beginning to challenge top professional players. In this paper we describe the leading algorithms for Monte-Carlo tree search and explain how they have advanced the state of the art in computer Go.}},
language = {Anglais},
affiliation = {TAO - INRIA Saclay - Ile de France , Laboratoire de Recherche en Informatique - LRI , LPDS , Microsoft Research - Inria Joint Centre - MSR - INRIA , University of Alberta, Canada , Department of Computing Science},
publisher = {ACM},
pages = {106-113},
journal = {Communication of the ACM},
volume = {55},
number = {3 },
audience = {internationale },
year = {2012},
pdf = {http://hal.inria.fr/hal-00695370/PDF/CACM-MCTS.pdf},
}
Don't believe what is written in these slides.
These statements are just provocative statements, most of them found on internet, here for discussion and for brain storming.
Theory of games, with a short reminder of computational complexity and an independent appendix on human complexity and the game of Go
@article{david:hal-00710073,
hal_id = {hal-00710073},
url = {http://hal.inria.fr/hal-00710073},
title = {{The Frontier of Decidability in Partially Observable Recursive Games}},
author = {David, Auger and Teytaud, Olivier},
abstract = {{The classical decision problem associated with a game is whether a given player has a winning strategy, i.e. some strategy that leads almost surely to a victory, regardless of the other players' strategies. While this problem is relevant for deterministic fully observable games, for a partially observable game the requirement of winning with probability 1 is too strong. In fact, as shown in this paper, a game might be decidable for the simple criterion of almost sure victory, whereas optimal play (even in an approximate sense) is not computable. We therefore propose another criterion, the decidability of which is equivalent to the computability of approximately optimal play. Then, we show that (i) this criterion is undecidable in the general case, even with deterministic games (no random part in the game), (ii) that it is in the jump 0', and that, even in the stochastic case, (iii) it becomes decidable if we add the requirement that the game halts almost surely whatever maybe the strategies of the players.}},
language = {Anglais},
affiliation = {Laboratoire de Recherche en Informatique - LRI , TAO - INRIA Saclay - Ile de France},
booktitle = {{Special Issue on "Frontier between Decidability and Undecidability"}},
publisher = {World Scinet},
journal = {International Journal on Foundations of Computer Science (IJFCS)},
volume = {Accepted},
note = {revised 2011, accepted 2011, in press },
audience = {internationale },
year = {2012},
}
The document discusses the computational complexity of partially observable games. Some key points:
1. Two-player unobservable games are EXPSPACE-complete, as strategies are just sequences of actions with no observability.
2. Encoding a Turing machine as a game shows the hardness of the unobservable case. The tape configurations can be represented in a game state of size logarithmic in the tape size.
3. Two-player partially observable games or one-player partially observable games against randomness are 2EXPTIME-complete, even more complex than the unobservable case.
This document discusses blind Go, a variant of the game where players do not look at the board and must memorize positions. It explores strategies for blind Go, such as playing unusual moves that are harder for the opponent to remember. Experiments found that providing an empty board as a visual aid helped players. When playing against professionals in blind 9x9 Go, the computer won 2 of 3 games. In a 19x19 game against a top human player, the computer won through an unexpected, unusual move where the human made a rare mistake due to not seeing the board. Further research is needed, but playing unconventional moves seems beneficial in blind Go.
This document discusses how to save money by using open source software instead of proprietary software like Microsoft Office. It recommends downloading and using OpenOffice or LibreOffice instead, as they are free alternatives that work very well. It also recommends installing a free open source operating system like Linux, as this can save a lot of money on software costs over time. Open source is discussed as an economic model where the marginal cost of sharing and distributing code is very low, enabling new business models to earn money through services, support or customization rather than just software licenses. A variety of important open source software projects are listed across different domains like operating systems, office suites, web servers and more.
Introduction to the TAO Uct Sig, a team working on computational intelligence...Olivier Teytaud
The document discusses research from Tao-Uctsig, a special interest group within Tao focused on artificial intelligence. Tao has 11 permanent staff and around 22 PhD students/postdocs working across various fields of mathematics, computer science, and sciences. The SIG works on problems where computers make decisions, particularly challenges where humans are currently better than computers. Their work involves games, important applications, and previously included mathematics. Specific applications discussed include controlling a robot arm, analyzing strategy in Pokemon and Urban Rivals, solving Minesweeper puzzles, and Go. Industrial applications include helping optimize France's major electricity industry.
Neural Networks and Deep Learning for PhysicistsHéloïse Nonne
Introduction to neural networks and deep learning. Seminar given by Héloïse Nonne on February 19th, 2015 at CINaM (Centre Interdisciplinaire de Nanosciences de Marseille) at Aix-Marseille University
Big Data Business InnovationsBased on Infinity and Permanence
1. The Computer Newspaper (e.g. Craigslist)
2. Infinite Spreadsheet (IS) Decision Center
(Big Data are the inputs and the outputs of IS.)
3. Universal Permanent Number (UPN)
4. UPN Self-Checkout
5. UPN Machine Translation
6. UPN Search
7. UPN DNA Sequencing
8. Universal Permanent Device
9. Self-generating Neural Network (deep AI)
10. Self-manufactured General-Purpose Robot
(Adept, DAI unsuccessful competing w/ humans)
- Stochastic dynamic programming (SDP) and stochastic dual dynamic programming (SDDP) are algorithms for solving sequential decision making problems under uncertainty.
- They represent the value function and controller as piecewise linear functions that can be encoded in linear programming formulations. This allows solving problems with up to around 100,000 decision variables per time step.
- However, solving the full problem using SDP/SDDP can be computationally expensive due to the "curse of dimensionality" as the number of states increases. The methods also require linear programming approximations and convex value functions.
This document summarizes Jim Gray's 1998 Turing Lecture which discusses remaining challenges in information technology research. It identifies the need for long-term, university-led research projects supported by government funding. Specific challenges mentioned include making parallel programming easier, improving the scalability of databases and transaction processing systems, and advancing the state of artificial intelligence to pass the Turing Test within the next 50 years. The document outlines properties of effective long-term research goals and provides examples like devising an architecture that scales indefinitely.
Deep Water - Bringing Tensorflow, Caffe, Mxnet to H2OSri Ambati
Arno Candel introduces Deep Water, which brings Tensorflow, Caffe, Mxnet to H2O. It also brings support for GPUs, image classification, NLP and much more to H2O.
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
The early 21c has brought the power of the computer into the hands of the general population, and though these computers consume small amounts of energy they are so numerous that their Energy Efficiency will soon become a major issue. This presentation looks at modern Computing, the ways that Energy Efficiency is currently being enhanced, and the principles behind this.
This document summarizes a presentation on data science and big data for actuaries given by Arthur Charpentier. It discusses the history of data collection and analysis. It provides an overview of big data, including definitions of volume, variety and velocity. It also covers topics like unsupervised learning techniques including principal component analysis and cluster analysis. Computational issues for large-scale data analysis using techniques like parallelization are also summarized.
Tools for Discrete Time Control; Application to Power SystemsOlivier Teytaud
3 main algorithms from the state of the art:
- Model Predictive Control
- Stochastic Dynamic Programming
- Direct Policy Search
==> and our proposal, a modified Direct Policy Search
termed Direct Value Search
Hydroelectricity uses water to produce electricity and has advantages for electricity storage. It provides daily, yearly, and negative electricity production by pumping water to higher reservoirs. However, expanding hydroelectricity is challenging due to its large infrastructure requirements and local environmental impacts. New technologies may improve energy storage capabilities and grid stability in the future, but developing large-scale annual storage remains difficult given constraints. Hydroelectricity will continue playing an important role in energy systems alongside other renewable technologies and efficiency strategies.
Choosing between several options in uncertain environmentsOlivier Teytaud
The document discusses bandit problems with strategic choices and small budgets. It defines bandit problems, strategic bandit problems, and compares the two. It presents algorithms for exploring options and making recommendations in both one-player and two-player settings. Experimental results on a Go positioning problem and an online card game show that TEXP3 outperforms other algorithms in two-player settings. The document concludes with discussions on extensions to structured bandits and using strategic bandits to model investment choices.
Artificial Intelligence and Optimization with ParallelismOlivier Teytaud
This document discusses parallelism in artificial intelligence and evolutionary computation. It explains that comparison-based optimization algorithms, which include many evolutionary algorithms, can be naturally parallelized by speculatively running multiple branches in parallel with a branching factor of 3 or more. This allows theoretical logarithmic speedups to be achieved in practice through simple parallelization tricks.
This document provides an overview of distributed decision making in partially observable dynamic games and multiobjective policy optimization. It discusses applying these techniques to optimization problems in games like chess and Go, as well as industrial applications like managing groups of power plants involving renewable energy, nuclear power, coal, hydroelectric power, and interactions with electricity consumers and networks. The goal is to optimize strategies using parallel computing and test these approaches on games and energy systems.
A simple tutorial on Monte-Carlo Tree Search
Contains a description of dynamic programming and alpha-beta search, then MCTS. Special cases for simultaneous actions are discussed.
I should add comments so that it can be used without preliminary knowledge of MCTS, if there is at least one request for doing so I'll do it.
@article{gelly:hal-00695370,
hal_id = {hal-00695370},
url = {http://hal.inria.fr/hal-00695370},
title = {{The Grand Challenge of Computer Go: Monte Carlo Tree Search and Extensions}},
author = {Gelly, Sylvain and Kocsis, Levente and Schoenauer, Marc and Sebag, Mich{\`e}le and Silver, David and Szepesvari, Csaba and Teytaud, Olivier},
abstract = {{The ancient oriental game of Go has long been considered a grand challenge for artificial intelligence. For decades, com- puter Go has defied the classical methods in game tree search that worked so successfully for chess and checkers. How- ever, recent play in computer Go has been transformed by a new paradigm for tree search based on Monte-Carlo meth- ods. Programs based on Monte-Carlo tree search now play at human-master levels and are beginning to challenge top professional players. In this paper we describe the leading algorithms for Monte-Carlo tree search and explain how they have advanced the state of the art in computer Go.}},
language = {Anglais},
affiliation = {TAO - INRIA Saclay - Ile de France , Laboratoire de Recherche en Informatique - LRI , LPDS , Microsoft Research - Inria Joint Centre - MSR - INRIA , University of Alberta, Canada , Department of Computing Science},
publisher = {ACM},
pages = {106-113},
journal = {Communication of the ACM},
volume = {55},
number = {3 },
audience = {internationale },
year = {2012},
pdf = {http://hal.inria.fr/hal-00695370/PDF/CACM-MCTS.pdf},
}
Don't believe what is written in these slides.
These statements are just provocative statements, most of them found on internet, here for discussion and for brain storming.
Theory of games, with a short reminder of computational complexity and an independent appendix on human complexity and the game of Go
@article{david:hal-00710073,
hal_id = {hal-00710073},
url = {http://hal.inria.fr/hal-00710073},
title = {{The Frontier of Decidability in Partially Observable Recursive Games}},
author = {David, Auger and Teytaud, Olivier},
abstract = {{The classical decision problem associated with a game is whether a given player has a winning strategy, i.e. some strategy that leads almost surely to a victory, regardless of the other players' strategies. While this problem is relevant for deterministic fully observable games, for a partially observable game the requirement of winning with probability 1 is too strong. In fact, as shown in this paper, a game might be decidable for the simple criterion of almost sure victory, whereas optimal play (even in an approximate sense) is not computable. We therefore propose another criterion, the decidability of which is equivalent to the computability of approximately optimal play. Then, we show that (i) this criterion is undecidable in the general case, even with deterministic games (no random part in the game), (ii) that it is in the jump 0', and that, even in the stochastic case, (iii) it becomes decidable if we add the requirement that the game halts almost surely whatever maybe the strategies of the players.}},
language = {Anglais},
affiliation = {Laboratoire de Recherche en Informatique - LRI , TAO - INRIA Saclay - Ile de France},
booktitle = {{Special Issue on "Frontier between Decidability and Undecidability"}},
publisher = {World Scinet},
journal = {International Journal on Foundations of Computer Science (IJFCS)},
volume = {Accepted},
note = {revised 2011, accepted 2011, in press },
audience = {internationale },
year = {2012},
}
The document discusses the computational complexity of partially observable games. Some key points:
1. Two-player unobservable games are EXPSPACE-complete, as strategies are just sequences of actions with no observability.
2. Encoding a Turing machine as a game shows the hardness of the unobservable case. The tape configurations can be represented in a game state of size logarithmic in the tape size.
3. Two-player partially observable games or one-player partially observable games against randomness are 2EXPTIME-complete, even more complex than the unobservable case.
This document discusses blind Go, a variant of the game where players do not look at the board and must memorize positions. It explores strategies for blind Go, such as playing unusual moves that are harder for the opponent to remember. Experiments found that providing an empty board as a visual aid helped players. When playing against professionals in blind 9x9 Go, the computer won 2 of 3 games. In a 19x19 game against a top human player, the computer won through an unexpected, unusual move where the human made a rare mistake due to not seeing the board. Further research is needed, but playing unconventional moves seems beneficial in blind Go.
This document discusses how to save money by using open source software instead of proprietary software like Microsoft Office. It recommends downloading and using OpenOffice or LibreOffice instead, as they are free alternatives that work very well. It also recommends installing a free open source operating system like Linux, as this can save a lot of money on software costs over time. Open source is discussed as an economic model where the marginal cost of sharing and distributing code is very low, enabling new business models to earn money through services, support or customization rather than just software licenses. A variety of important open source software projects are listed across different domains like operating systems, office suites, web servers and more.
Introduction to the TAO Uct Sig, a team working on computational intelligence...Olivier Teytaud
The document discusses research from Tao-Uctsig, a special interest group within Tao focused on artificial intelligence. Tao has 11 permanent staff and around 22 PhD students/postdocs working across various fields of mathematics, computer science, and sciences. The SIG works on problems where computers make decisions, particularly challenges where humans are currently better than computers. Their work involves games, important applications, and previously included mathematics. Specific applications discussed include controlling a robot arm, analyzing strategy in Pokemon and Urban Rivals, solving Minesweeper puzzles, and Go. Industrial applications include helping optimize France's major electricity industry.
Neural Networks and Deep Learning for PhysicistsHéloïse Nonne
Introduction to neural networks and deep learning. Seminar given by Héloïse Nonne on February 19th, 2015 at CINaM (Centre Interdisciplinaire de Nanosciences de Marseille) at Aix-Marseille University
Big Data Business InnovationsBased on Infinity and Permanence
1. The Computer Newspaper (e.g. Craigslist)
2. Infinite Spreadsheet (IS) Decision Center
(Big Data are the inputs and the outputs of IS.)
3. Universal Permanent Number (UPN)
4. UPN Self-Checkout
5. UPN Machine Translation
6. UPN Search
7. UPN DNA Sequencing
8. Universal Permanent Device
9. Self-generating Neural Network (deep AI)
10. Self-manufactured General-Purpose Robot
(Adept, DAI unsuccessful competing w/ humans)
- Stochastic dynamic programming (SDP) and stochastic dual dynamic programming (SDDP) are algorithms for solving sequential decision making problems under uncertainty.
- They represent the value function and controller as piecewise linear functions that can be encoded in linear programming formulations. This allows solving problems with up to around 100,000 decision variables per time step.
- However, solving the full problem using SDP/SDDP can be computationally expensive due to the "curse of dimensionality" as the number of states increases. The methods also require linear programming approximations and convex value functions.
This document summarizes Jim Gray's 1998 Turing Lecture which discusses remaining challenges in information technology research. It identifies the need for long-term, university-led research projects supported by government funding. Specific challenges mentioned include making parallel programming easier, improving the scalability of databases and transaction processing systems, and advancing the state of artificial intelligence to pass the Turing Test within the next 50 years. The document outlines properties of effective long-term research goals and provides examples like devising an architecture that scales indefinitely.
Deep Water - Bringing Tensorflow, Caffe, Mxnet to H2OSri Ambati
Arno Candel introduces Deep Water, which brings Tensorflow, Caffe, Mxnet to H2O. It also brings support for GPUs, image classification, NLP and much more to H2O.
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
The early 21c has brought the power of the computer into the hands of the general population, and though these computers consume small amounts of energy they are so numerous that their Energy Efficiency will soon become a major issue. This presentation looks at modern Computing, the ways that Energy Efficiency is currently being enhanced, and the principles behind this.
This document summarizes a presentation on data science and big data for actuaries given by Arthur Charpentier. It discusses the history of data collection and analysis. It provides an overview of big data, including definitions of volume, variety and velocity. It also covers topics like unsupervised learning techniques including principal component analysis and cluster analysis. Computational issues for large-scale data analysis using techniques like parallelization are also summarized.
Artificial Intelligence for Undergrads is a textbook by J. Berengueres that introduces key concepts in artificial intelligence. It covers topics like spell checking algorithms, machine translation, game playing, and Monte Carlo tree search. The book also discusses early pioneers in AI like Marco Dorigo and his work on ant colony optimization algorithms. It aims to explain complex AI concepts in a simple way for undergraduate students new to the field.
The document discusses a senior design project from 1970 where the author created a limited assembler for the GE-235 computer. It then provides the author's background and experience working with early computer systems from the 1960s through today. Key lessons discussed include understanding user needs, allowing for growth and change using Moore's Law, managing expectations through the Gartner Hype Cycle, and an iterative design process of testing, fixing, and retesting.
This document discusses future computing technologies and challenges. It describes how current computing relies on silicon chips that will soon hit physical limits. Alternative technologies like quantum, photonic and neuromorphic computing are presented as possibilities to overcome these limits. A new university, SIT, is proposed to conduct research on these new computing paradigms through interdisciplinary partnerships. SIT aims to become a top research university and prepare students for leadership roles in technology companies through new advanced degree programs.
- Ilab METIS is a collaboration between Inria-Tao, a research team focused on optimization and machine learning problems, and Artelys, an SME focused on power systems modeling.
- They develop black-box planning tools for power systems that aim to minimize model error by using direct policy search techniques on high-fidelity simulations.
- These tools are applied to problems like optimizing investments in new power plants, transmission lines, and other infrastructure for power grids under uncertainty.
A Sucess Culture for the UK Electronics Systems CommunityIan Phillips
The ESCO Report identified an erstwhile invisible Electronic Systems community in the UK. These are businesses and individuals who's (technical) roles are in the life-cycle of Electronic Systems provision. These 21c enterprises are the evolutionary product of the old 20c electronic companies that used to be visible here ... and whose demise is erroneously seen as the UKs exit from this domain. The 850k individuals; employed in 35,000 ES Enterprises and embedded into other consumer and professional businesses; collectively contribute around 5.4% to UK-GDP. This is a very different 'shaped' business, but is a story of successful adoption to the global business of creating the Electronic Systems we would miss today and on which we will depend tomorrow. This is an opportunity for the UK ... but also for the UK's Academic community within it.
Capabilities: The Bridge Between R-&-D - 21may14Ian Phillips
Research can seem very isolated from Product Development. This work illustrates the role of Research in establishing Capabilities; Capabilities which will subsequently be used in Product Development. Thus showing Research to be important in the ecology of a healthy business.
This document discusses the history and evolution of computing technologies from ancient mechanical devices to modern electronic systems. It begins with early mechanical computers from ancient Greece and progresses through developments like Babbage's Difference Engine, early electronic computers, and integrated circuits. The document notes that modern computing platforms involve contributions from global teams and enterprises, with most technologies being reused to improve productivity. It argues that businesses must focus on competitive products rather than technology for its own sake, and that designer productivity has become the main driver of technology advancement as complexity increases exponentially with Moore's Law.
Big Data & Machine Learning - TDC2013 São Paulo - 12/0713Mathieu DESPRIEE
Machine learning and big data technologies enable new types of data analysis. Hadoop is an open-source framework that allows distributed storage and processing of large datasets across clusters of computers. It includes tools for working with structured and unstructured data to power applications in areas like recommendations, customer churn prediction, and more.
The document provides an overview of topics covered in a course on secure and intelligent Internet of Things, with a focus on wearables and motion sensing. It includes announcements about the course, a discussion of readings, and a recap of concepts like IoT devices, power-performance tradeoffs, the Kinect pose estimation pipeline, and motion sensors. The document summarizes how accelerometers and gyroscopes work and how sensor fusion can provide a better estimate of orientation than single sensors. It also discusses Apple's patent on short-range connectivity and considerations for evaluating smart watches.
This document summarizes a talk on data science for software engineering. It discusses how data science involves various fields like statistics, machine learning, and data mining. It notes that while "big data" is often discussed, software engineering data is typically small and sparse. Domain knowledge is important for data mining to avoid misinterpreting data. Data science with software engineering data requires understanding organizations and their willingness to share data given privacy concerns. The document outlines sharing data, models, and methods for learning across different organizations and discusses techniques for balancing privacy and utility when sharing data.
Zero to a Billion in 4.86 Years (A Whirlwind History of Operating Systems)David Evans
The document discusses the history of operating systems from early recursive machines and programming languages to modern operating systems like Android. It traces the development of key concepts like processes, virtual memory, and how open source operating systems enabled operating systems to spread widely. It notes that while the first Android activation was in 2008, it took only 4.86 years to reach 1 billion activations by September 2013, showing how operating systems can now spread extremely rapidly across many affordable computing devices.
Similar to Tools for artificial intelligence: EXP3, Zermelo algorithm, Alpha-Beta, and some applications (20)
In the realm of cybersecurity, offensive security practices act as a critical shield. By simulating real-world attacks in a controlled environment, these techniques expose vulnerabilities before malicious actors can exploit them. This proactive approach allows manufacturers to identify and fix weaknesses, significantly enhancing system security.
This presentation delves into the development of a system designed to mimic Galileo's Open Service signal using software-defined radio (SDR) technology. We'll begin with a foundational overview of both Global Navigation Satellite Systems (GNSS) and the intricacies of digital signal processing.
The presentation culminates in a live demonstration. We'll showcase the manipulation of Galileo's Open Service pilot signal, simulating an attack on various software and hardware systems. This practical demonstration serves to highlight the potential consequences of unaddressed vulnerabilities, emphasizing the importance of offensive security practices in safeguarding critical infrastructure.
Dandelion Hashtable: beyond billion requests per second on a commodity serverAntonios Katsarakis
This slide deck presents DLHT, a concurrent in-memory hashtable. Despite efforts to optimize hashtables, that go as far as sacrificing core functionality, state-of-the-art designs still incur multiple memory accesses per request and block request processing in three cases. First, most hashtables block while waiting for data to be retrieved from memory. Second, open-addressing designs, which represent the current state-of-the-art, either cannot free index slots on deletes or must block all requests to do so. Third, index resizes block every request until all objects are copied to the new index. Defying folklore wisdom, DLHT forgoes open-addressing and adopts a fully-featured and memory-aware closed-addressing design based on bounded cache-line-chaining. This design offers lock-free index operations and deletes that free slots instantly, (2) completes most requests with a single memory access, (3) utilizes software prefetching to hide memory latencies, and (4) employs a novel non-blocking and parallel resizing. In a commodity server and a memory-resident workload, DLHT surpasses 1.6B requests per second and provides 3.5x (12x) the throughput of the state-of-the-art closed-addressing (open-addressing) resizable hashtable on Gets (Deletes).
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
How information systems are built or acquired puts information, which is what they should be about, in a secondary place. Our language adapted accordingly, and we no longer talk about information systems but applications. Applications evolved in a way to break data into diverse fragments, tightly coupled with applications and expensive to integrate. The result is technical debt, which is re-paid by taking even bigger "loans", resulting in an ever-increasing technical debt. Software engineering and procurement practices work in sync with market forces to maintain this trend. This talk demonstrates how natural this situation is. The question is: can something be done to reverse the trend?
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Taking AI to the Next Level in Manufacturing.pdfssuserfac0301
Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as:
1. How quickly AI is being implemented in manufacturing.
2. Which barriers stand in the way of AI adoption.
3. How data quality and governance form the backbone of AI.
4. Organizational processes and structures that may inhibit effective AI adoption.
6. Ideas and approaches to help build your organization's AI strategy.
"Choosing proper type of scaling", Olena SyrotaFwdays
Imagine an IoT processing system that is already quite mature and production-ready and for which client coverage is growing and scaling and performance aspects are life and death questions. The system has Redis, MongoDB, and stream processing based on ksqldb. In this talk, firstly, we will analyze scaling approaches and then select the proper ones for our system.
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor IvaniukFwdays
At this talk we will discuss DDoS protection tools and best practices, discuss network architectures and what AWS has to offer. Also, we will look into one of the largest DDoS attacks on Ukrainian infrastructure that happened in February 2022. We'll see, what techniques helped to keep the web resources available for Ukrainians and how AWS improved DDoS protection for all customers based on Ukraine experience
Conversational agents, or chatbots, are increasingly used to access all sorts of services using natural language. While open-domain chatbots - like ChatGPT - can converse on any topic, task-oriented chatbots - the focus of this paper - are designed for specific tasks, like booking a flight, obtaining customer support, or setting an appointment. Like any other software, task-oriented chatbots need to be properly tested, usually by defining and executing test scenarios (i.e., sequences of user-chatbot interactions). However, there is currently a lack of methods to quantify the completeness and strength of such test scenarios, which can lead to low-quality tests, and hence to buggy chatbots.
To fill this gap, we propose adapting mutation testing (MuT) for task-oriented chatbots. To this end, we introduce a set of mutation operators that emulate faults in chatbot designs, an architecture that enables MuT on chatbots built using heterogeneous technologies, and a practical realisation as an Eclipse plugin. Moreover, we evaluate the applicability, effectiveness and efficiency of our approach on open-source chatbots, with promising results.
What is an RPA CoE? Session 1 – CoE VisionDianaGray10
In the first session, we will review the organization's vision and how this has an impact on the COE Structure.
Topics covered:
• The role of a steering committee
• How do the organization’s priorities determine CoE Structure?
Speaker:
Chris Bolin, Senior Intelligent Automation Architect Anika Systems
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsDianaGray10
Join us to learn how UiPath Apps can directly and easily interact with prebuilt connectors via Integration Service--including Salesforce, ServiceNow, Open GenAI, and more.
The best part is you can achieve this without building a custom workflow! Say goodbye to the hassle of using separate automations to call APIs. By seamlessly integrating within App Studio, you can now easily streamline your workflow, while gaining direct access to our Connector Catalog of popular applications.
We’ll discuss and demo the benefits of UiPath Apps and connectors including:
Creating a compelling user experience for any software, without the limitations of APIs.
Accelerating the app creation process, saving time and effort
Enjoying high-performance CRUD (create, read, update, delete) operations, for
seamless data management.
Speakers:
Russell Alfeche, Technology Leader, RPA at qBotic and UiPath MVP
Charlie Greenberg, host
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Tools for artificial intelligence: EXP3, Zermelo algorithm, Alpha-Beta, and some applications
1. SOME TOOLS
FOR ARTIFICIAL INTELLIGENCE
Olivier Teytaud --- olivier.teytaud@gmail.com
NUTN, Tainan, 2011
2. Tao (Inria, Cnrs, Lri, Paris-Sud)
People:
Permanent staff: 11
~15 ph.D. Students
In Université Paris-Sud
Largest campus in France
Faculty of sciences: mathematics, computer science,
physics, chemistry, biology, earth and space
sciences ==> 12000 students
Inria affiliation:
Around 50 years old
Devoted to research in comp. science
3. Tao (Inria, Cnrs, Lri, Paris-Sud)
Reservoir computing
Optimal decision making under uncertainty
Optimization
Autonomic computer
Machine learning
4. Communication not always so easy:
Many of you speak Chinese + Taiwanese.
So English = third language.
I am French.
English = second language.
I work mainly in mathematical aspects
of computer science, more than computer science.
Difficulties might also be an enrichment.
Feel free to interrupt me as much as useful.
NUTN, Tainan, 2011
5. Communication not always so easy:
Many of you speak Chinese + Taiwanese.
So English = third language.
I am French.
English = second language.
I work mainly in mathematical aspects
of computer science, more than computer science.
Difficulties might also be an enrichment.
Feel free to interrupt me as much as useful.
NUTN, Tainan, 2011
6. Vita in a nutshell:
1) First research: mathematical logic
2) I had fun, but I wanted to be “directly” useful. I switched
to Statistics.
3) I had fun, but I wanted to be “more directly” useful. Switched
to Operational Research, in industry.
- Many applications.
- My favorite: electricity generation.
4) Now (40 dangerously approaching), Artificial Intelligence:
- Mathematics.
- Challenges (in particular games).
- Applications.
7. Vita in a nutshell:
1) First research: mathematical logic
2) I had fun, but I wanted to be “directly” useful. I switched
to Statistics.
3) I had fun, but I wanted to be “more directly” useful. Switched
to Operational Research, in industry.
- Many applications.
- My favorite: electricity generation.
4) Now (40 dangerously approaching), Artificial Intelligence:
- Mathematics.
- Challenges (in particular games).
- Applications.
8. Vita in a nutshell:
1) First research: mathematical logic
2) I had fun, but I wanted to be “directly” useful. I switched
to Statistics.
3) I had fun, but I wanted to be “more directly” useful.
Switched to Operation Research, in industry.
- Many applications.
- My favorite: electricity generation.
4) Now (40 dangerously approaching), Artificial Intelligence:
- Mathematics. Goes back to military
- Challenges application around world war II,
(in particular games).
- Applications. UK resisted to Hitler thanks
when
to optimized radars.
Now essentially civil applications.
9. Vita in a nutshell:
1) First research: mathematical logic
2) I had fun, but I wanted to be “directly” useful. I switched
to Statistics.
3) I had fun, but I wanted to be “more directly” useful. Switched
to Operational Research, in industry.
- Many applications.
- My favorite: electricity generation.
4) Now (40 years old soon...), Artificial Intelligence:
- Mathematics.
- Beautiful challenges (in particular games).
- Applications.
10. Outline of what I'll discuss:
1) Some concepts:
- simplified problems
- toolboxes for these problems
2) Principle:
- reducing real problems to groups of artificial problems
- small problems might be considered as artificial
and useless when considered alone.
- but when you solve a clearly stated small problem, usually
you can find an application for this solution.
- we will see applications as well.
==> For the moment let's see “big” applications
3) I'll also show some works on which contributors are welcome.
13. ELECTRICITY GENERATION
The case of France
Data:
- climate model (stochastic)
- model of electricity demand (stochastic)
- model of power plants
Each day we receive:
- electricity consumption
- weather information
- info on faults
Each day, we decide how to distribute the production
among the power plants. (also: schedule long-term
investiments)
14. Data:
- climate model (stochastic)
- model of electricity demand (stochastic)
- model of power plants (PP): nuclear PP (NPP), thermal PP (TPP),
Hydroelectric PP (HPP)...
Each day we receive:
- electricity consumption
- weather information
- info on faults
Each day, we decide how to distribute the production among the power
plants.
Daily information
DATA
(climate, Electric
PROGRAM STRATEGY
plants, system
economy)
Decisions
15. One of the most important industrial problem you can imagine:
how to produce energy ?
France has specific elements:
- heavily nuclearized (most nuclearized country in the world)
- often cooled by rivers (do not work in case of droughts ==> hard
to predict)
- we must schedule maintenance
- we must take long-term decisions (building new NPP ? Removing ?)
- also hydroelectricity:
- should we use water now ?
- should we keep it for winter (in France, high consumption is in
winter)
Daily information
DATA
(climate, Electric
PROGRAM STRATEGY
plants, system
economy)
Decisions
16. Problem 1: Taiwan is very different from France :-)
Almost no nuclear power plant ? Cooled by sea ?
Electrically connected to other countries ? (France might
be connected to Africa)
Sun sufficient for massive photo-voltaic units ?
Wind much stronger than in France - can be used ?
Other questions ?
Electriciy consumption dominated by air conditioning ?
Maybe electric cars in the future ?
Climate maybe more regular ? Problem easier than
in France ?
==> I don't know
==> I'd like to work on it (energy is an important
concern, in Taiwan as well – lack of independence ?)
==> Need Chinese-reading persons
==> Other (Taiwan-independent) concern: tackling partial
observation in energy generation problem
17. GOOD NEWS: we had a
GAME OF GO lot of progress with
**generic** algorithms
(with Nutn) (algorithms which can be
used for many things).
The revolution in Go which
occurred in 2007-2009 is a
major breakthrough in
Artificial Intelligence.
We'll see that in details.
I am a little bit tired of the
game of Go, because I
have no recent progress,
and recent progress in the
community comes from Go
expertise, which is only
useful for Go...
18. Problem 2: Solving unsolved situations in Go
Now computers are much stronger than in the past.
However, they still
misunderstand some
trivial situations
(in particular,
liberty races).
You have an idea ?
Tell me :-)
We have a solver in
France (not for playing Go;
aimed at provably solving),
that we would
like to test on various
situations. We do not
play Go. If you are 5kyu
or better, you can
contribute.
20. URBAN RIVALS
- Choose 4 cards, your opponent chooses 4
Cards
- Each player gets 12 “Pilz” (i.e. strength points)
- Each player gets health points.
- Each turn:
- each player chooses a card
- each player uses pilz
(each used pilz is
lost forever, but
it gives strength)
- read cards, apply rules
==> no more health point ?
==> you're dead.
21. Urban Rivals
==> Partial information
because you don't observe your opponent's decisions
==> There are “on the shell” algorithms and programs
for full information games,
but not for partial information games.
==> We used a (provable) combination of MCTS and EXP3
==> Immediately human level performance
==> suggests that maths can help
==> still possible works:
- automatic choice of cards ?
- reducing comp. cost ?
22. POKEMONS
皮卡丘
Second most lucrative video
game.
Meta-gaming: choosing your deck.
23. POKEMONS: Problem 3
Second most lucrative video
game.
Meta-gaming:
choosing your deck.
In-gaming: playing with your set of
cards.
24. Problem 4: Solving MineSweeper.
Find an optimal
move ?
Looks like a trivial boring problem.
Certainly not indeed.
Many papers with the same approach
(so-called CSP technique)
We could outperform these algorithms thanks to
a probabilistic approach.
But my approach only works on small board (or huge
computational cost) ==> we want to extend.
Quite similar to electricity generation (yes, I believe in this)
25. Game applications can be considered as childish.
Shouldn't we focus on more important things ?
However:
- If you have a breakthrough in an important game,
people will trust you. Doors will be opened when you
will propose new algorithms for real-world applications.
- Testing ideas on a nuclear power plant is more dangerous
than testing ideas on a game of Go.
- It's easier to compare approaches in games than in
electricity generation.
28. ONE FUNDAMENTAL TOOL: ZERMELO
Consider the following game:
- there are 5 sticks;
- in turn, each player removes 1 or 2 sticks;
- the player which removes the last stick looses.
Example:
Player I: IIIII
Player II: III
Player I: I ==> looses!
How should I play ?
29. ONE FUNDAMENTAL TOOL: ZERMELO
Zermelo proposed a solution (for full-information games).
Born in 1871.
1900-1905: major contributions in logic.
1913: major contribution to games in 1913.
1931: Optimized navigation (from games to applications).
Resigned in 1935 (he did not like Hitler).
Died in 1953.
31. ZERMELO: I HAVE
THE OPTIMAL STRATEGY!
5
LOSS!
4 3
WIN!
WIN! 3 2 2 1
LOSS!
WIN! WIN!
1 2
WIN! LOSS!
32. ZERMELO: not limited to win/loss games.
Can work on games with continuous rewards.
New rule: if the game contains 4, reward is multiplied by 2.
YELLOW NODES: 5 BLUE NODES:
LABEL = MINIMUM 2 LABEL = MAXIMUM
OF CHILDREN's LABELS OF CHILDREN's LABELS
0
4 3
2
2 3 2 2 1
0
2 1
1 2
2 0
33. ZERMELO: C CODE
struct gameState
{
int *descriptionOfState;
int numberOfLegalMoves;
int * legalMoves;
int turn; // 1 if player 1 plays, -1 otherwise
int result; // final reward, if numberOfLegalMoves=0
};
struct gameState next(struct gameState s,int move) { RULES };
double zermeloValue(struct gameState s)
{
int i;double value;
double maxValue=-MAXDOUBLE;
if (s.numberOfLegalMoves==0) return(s.turn * s.result);
for (i=0;i<s.numberOfLegalMoves;i++)
{
value=s.turn*zermeloValue(next(s,s.legalMoves[i]));
if (value>maxValue) maxValue=value;
}
return s.turn*maxValue; //we return value for player 1
}
34. ZERMELO: C CODE
struct gameState
{
int *descriptionOfState;
int numberOfLegalMoves;
Int * legalMoves;
int turn; // 1 if player 1 plays, -1 otherwise
int result; // final reward, if numberOfLegalMoves=0
};
struct gameState next(struct gameState s,int move) { RULES };
double zermeloValue(struct gameState s)
{
int i;double value;
double maxValue=-MAXDOUBLE;
if (s.numberOfLegalMoves==0) return(s.turn * s.result);
for (i=0;i<s.numberOfLegalMoves;i++)
{
value=s.turn*zermeloValue(next(s,s.legalMoves[i]));
if (value>maxValue) maxValue=value;
}
return s.turn*maxValue; //we return value for player 1
}
35. Last week: Zermelo algorithm.
What is Zermelo ?
= Simplest algorithm for solving 1Player
or 2Player games.
= Recursive algorithm
= Conveniently (but slowly) implemented with “struct”
This week
= a bit more on Zermelo algorithm
= C development: “static” random variables
Future weeks
Still some C implementation (or other languages ? as you wish)
Still some (not always easy) algorithms
Models of applications
I hope I can convince you that
operational research / artificial intelligence
are useful and fun.
36. Zermelo again.
What does the “zermeloValue()” function returns ?
===> The reward in case of perfect play.
===> A perfect strategy.
===> Gods can run Zermelo algorithms: perfect play.
==> humans have no time for this.
==> Can we design a new version in case
it is too slow ?
37. Let's see a pseudo-code, instead of a code.
double zermeloValue(struct gameState s)
{
if (s is end of game) then return score.
else
{
If (play 1 plays) then
return max(zermeloValue(children))
Else
return min(zermeloValue(children))
}
}
39. ZERMELO: C CODE FOR THE DEPTH
double zermeloValue(struct gameState s)
{
static int depth=0;
int i;double value;
double maxValue=-MAXDOUBLE;
if (s.numberOfLegalMoves==0) return(s.turn * s.result);
depth++;
for (i=0;i<s.numberOfLegalMoves;i++)
{
value=s.turn*zermeloValue(next(s,s.legalMoves[i]));
if (value>maxValue) maxValue=value;
}
depth--;
return s.turn*maxValue; //we return value for player 1
}
43. We will not go But, what should
below this depth. zermeloFunction return ?
44. double zermeloValue(struct gameState s)
{
static int depth=0; Should we return
int i;double value; a random number ?
double maxValue=-MAXDOUBLE;
if (s.numberOfLegalMoves==0) return(s.turn * s.result);
if (depth>5) return drand48();
depth++;
for (i=0;i<s.numberOfLegalMoves;i++)
{
value=s.turn*zermeloValue(next(s,s.legalMoves[i]));
if (value>maxValue) maxValue=value;
}
depth--;
return s.turn*maxValue; //we return value for player 1
}
45. double zermeloValue(struct gameState s)
{
static int depth=0;
int i;double value;
double maxValue=-MAXDOUBLE;
if (s.numberOfLegalMoves==0) return(s.turn * s.result);
if (depth>5) return heuristicValue(s);
depth++;
for (i=0;i<s.numberOfLegalMoves;i++)
{ A function written
by some expert of
value=s.turn*zermeloValue(next(s,s.legalMoves[i]));
if (value>maxValue) maxValue=value; game.
the
}
depth--;
return s.turn*maxValue; //we return value for player 1
}
46. SHANNON and games
This idea is a main contribution
by Shannon (for European chess).
Shannon 1916-2001
Noble prize (not Nobel!)
Works in:
- Logic
- Games (also: artificial
mouse for mazes)
- Financial analysis
50. ALPHA-BETA
PRINCIPLE OF ALPHA-BETA:
In zermeloFunction, considering a opponent node, if I know:
- THAT AT PREVIOUS DEPTH,
I CAN REACH SCORE ALPHA=6,
- THAT IN CURRENT STATE
MY OPPONENT CAN ENSURE SCORE BETA<6,
I CAN STOP STUDYING THIS BRANCH.
==> THIS IS A “ALPHA-CUTOFF“
==> OTHER PLAYER:
“BETA-CUTOFF“ (just exchange players)
52. EXAMPLE OF GAME (we can
discuss why it is a good game)
- Randomly generate a 4x4 matrix with 0 and 1 (K=4).
0011
1001
0111
1000
- Player one removes top part or bottom part
0111
1000
- Player two removes left part or right part
01
10
- Player one removes top part of bottom part
01
- Player two removes left part or right part
0 ==> Player one wins if 1, player two wins if 0!
53. POSSIBLE HOME WORK
1) ZERMELO: can you implement it on a simple game ?
2) MINIMAX: can you add a heuristic function ?
Which heuristic function ?
Experiments: plot a graph:
X(depth) = computation time of minimax
(divided by Zermelo's computation time)
Y(depth) = win rate against Zermelo
3) ALPHA-BETA
Can you modify it ==> alpha-beta pruning ?
Plot a graph for various sizes:
X = number of visited nodes
Y = average winning rate of alpha-beta vs minimax
Or
X = depth
Y = average winning rate of a-b vs a-b with depth -1
54. APPLICATION OF ZERMELO
WE HAVE SEEN THE 5-STICKS GAME.
CAN WE FIND A REALLY USEFUL APPLICATION ?
55. APPLICATION OF ZERMELO
WE HAVE SEEN THE 5-STICKS GAME.
CAN WE FIND A REALLY USEFUL APPLICATION ?
I have:
- water
56. APPLICATION OF ZERMELO
WE HAVE SEEN THE 5-STICKS GAME.
CAN WE FIND A REALLY USEFUL APPLICATION ?
I have:
- water
- plants (which need water during summer's
heat wave)
57. APPLICATION OF ZERMELO
WE HAVE SEEN THE 5-STICKS GAME.
CAN WE FIND A REALLY USEFUL APPLICATION ?
I have:
- water
- plants (which need water during summer's
heat wave)
Actions = giving water to plants, or not.
58. APPLICATION OF ZERMELO
I have:
- water
- plants (which need water during summer's
heat wave)
Each day, I choose an action.
State = { date +water level in stock
+ water level in plants }
Reward = quality / quantity of production.
60. IMPORTANT REMARK:
- Maybe this does not look serious.
- But heat waves are a serious problem.
- Here the problem is simplified, but the concepts
for the real application are the same.
- Applying this just requires a computer and
datas/models about plants/water resources.
==> if you can apply Zermelo variants
correctly, you can help for a better world.
62. s.turn == 0: action is
randomly chosen.
double zermeloValue(struct gameState s)
{ This is Zermelo, adapted to
int i;double value; static int depth=0; stochastic games.
If (s.turn==0) References:
{ value=0; - Massé
double total=0; - Bellman
for (i=0;i<s.numberOfLegalMoves;i++)
value+=zermeloValue(next(s,s.legalMoves[i]));
return value/s.numberOfLegalMoves;
}
double maxValue=-MAXDOUBLE;
if (s.numberOfLegalMoves==0) return(s.turn * s.result);
if (depth>5) return heuristicValue(s);
depth++;
for (i=0;i<s.numberOfLegalMoves;i++)
{ value=s.turn*zermeloValue(next(s,s.legalMoves[i]));
if (value>maxValue) maxValue=value; }
depth--;
return s.turn*maxValue; //we return value for player 1
}
63. ONE MORE TOOL: MATRIX GAMES
The problem:
Solving Matrix Games.
A solution:
EXP3.
64. What is a (0-sum) Matrix Game ?
Example:
1 0 0
M= 0 1 1
1 0 1
- You choose (privately) a row (i is 1, 2 or 3).
- In same time, I choose (privately) a column (j=1, 2 or 3).
- My reward: M(i,j)
- Your reward: -M(i,j)
I want a 1, you want a 0.
Given M, how should I play ?
65. What is a (0-sum) Matrix Game ?
Example: rock-paper-scissor
Rock Paper Scissor
Rock 0 -1 1
M= Paper 1 0 -1
Scissor -1 1 0
- You choose (privately) a row (i is 1, 2 or 3).
- In same time, I choose (privately) a column (j=1, 2 or 3).
- My reward: M(i,j)
- Your reward: -M(i,j)
I want a 1, you want a 0.
Given M, how should I play ?
66. Given M, how should I play ?
Nash (diagnosed with paranoid schizophrenia)
got a Nobel prize for his work around that.
Principle of a Nash equilibrium:
- pure strategy = “fixed” strategy
(e.g. “play scissor”)
- mixed strategy = randomized strategy
(e.g. “play scissor with probability ½
and play rock with probability ½”
- choose the mixed strategy such that
“The worst possible score against
any opponent strategy is maximum”
==> “Nash” strategy
==> EXP3: algorithm for finding Nash strategies.
67. IMPORTANT FACTS ON GAMES:
- Turn-based, full-information games,
solvers exist:
- Too slow for chess, Go.
- Ok for 8x8 checkers.
==> Zermelo
==> variants: Minimax, Alpha-beta, play
reasonably well many games
- Matrix games:
- Nash strategies = wort-case optimal
- Nash strategies = randomized strategies
68. A BETTER EXAMPLE ? POKEMON.
Each player chooses 2 pokemons among
the 3 possible ones (real life: 3 or 4
among hundreds).
70. A BETTER EXAMPLE ? POKEMON.
Three possibilities (the same as choosing
a row in a 3x3 matrix
game):
Player 2
Player 1 Check who
wins (by some
full-observation
game-solver).
71. A BETTER EXAMPLE ? POKEMON.
Three possibilities (the same as choosing
a row in a 3x3 matrix
game):
Player 2
Player 1
P1 P2 P2
P2 P1 P1
P1 P2 P1
72. A BETTER EXAMPLE ? POKEMON.
Three possibilities (the same as choosing
a row in a 3x3 matrix
game):
Player 2
Player 1
1 0 0
0 1 1
1 0 1
73. EXP3 principle for Nash equilibrium of KxK matrix M:
- choose a number N of iterations
- S1=null vector
- S2=null vector
- at each iteration t=1, ..., t=N:
{
- compute p1 as a function of S1 // we will see how
- compute p2 as a function of S2 // we will see how
- randomly draw i according to probability distribution p1
- randomly draw j according to probability distribution p2
- define r=M(i,j) in the matrix
- S1(i)+= r / p1(i)
- S2(j)+=(1-r) / p2(j)
- Player1Nash(i)+= (1/N);
- Player2Nash(j)+= (1/N);
}
74. EXP3 principle for Nash equilibrium of KxK matrix M:
- choose a number N of iterations
- S1=null vector
- S2=null vector
- at each iteration t=1, ..., t=N:
{
- compute p1 as a function of S1 // we will see how
- compute p2 as a function of S2 // we will see how
- randomly draw i according to probability distribution p1
- randomly draw j according to probability distribution p2
- define r=M(i,j) in the matrix
- S1(i)+= r / p1(i)
- S2(j)+=(1-r) / p2(j)
- Player1Nash(i)+= (1/N);
- Player2Nash(j)+= (1/N);
}
76. Q&A: (my questions, and also yours)
Q: Who cares about matrix games ?
A: Useful for many things. Unfortunately, it's usually
a building block inside more complex algorithms.
We will see examples, but later.
Q: Is a Nash strategy optimal ?
A: It depends for what... It is optimal in a worst case sense
(i.e. against a very strong opponent).
Not necessarily very good against a weak opponent.