- The document discusses games with simultaneous actions and hidden information. It presents games as directed graphs with actions, players, observations, rewards, and loops.
- Games with simultaneous actions and short-term hidden information can be represented as games with hidden information by removing intermediate turns.
- Questions about the existence of a sure-win strategy for one player (the "UD" question) are only relevant for games with full observability, not matrix games.
This document outlines the course content for a Higher Computing course, which is divided into 3 main units: Computer Systems (40 hours), Software Development (40 hours), and Artificial Intelligence (40 hours). The Computer Systems unit covers topics like data representation, computer structure, networking, and computer software across 5 sections. Specific lessons in the Data Representation section discuss how numbers, text, and images are stored in binary and how storage capacities are measured. Graphics representation and compression techniques are also introduced. Students will complete assessments including end of unit tests, coursework tasks, and a written exam.
Choosing between several options in uncertain environmentsOlivier Teytaud
The document discusses bandit problems with strategic choices and small budgets. It defines bandit problems, strategic bandit problems, and compares the two. It presents algorithms for exploring options and making recommendations in both one-player and two-player settings. Experimental results on a Go positioning problem and an online card game show that TEXP3 outperforms other algorithms in two-player settings. The document concludes with discussions on extensions to structured bandits and using strategic bandits to model investment choices.
This document discusses how to save money by using open source software instead of proprietary software like Microsoft Office. It recommends downloading and using OpenOffice or LibreOffice instead, as they are free alternatives that work very well. It also recommends installing a free open source operating system like Linux, as this can save a lot of money on software costs over time. Open source is discussed as an economic model where the marginal cost of sharing and distributing code is very low, enabling new business models to earn money through services, support or customization rather than just software licenses. A variety of important open source software projects are listed across different domains like operating systems, office suites, web servers and more.
Ilab Metis: we optimize power systems and we are not afraid of direct policy ...Olivier Teytaud
Ilab METIS is a collaboration between TAO, a machine learning and optimization team within INRIA, and Artelys, an SME focused on optimization. They work on optimizing energy policies through simulations of power systems while taking into account uncertainties and stochastic variables. Their methodologies use a hybrid of reinforcement learning, mathematical programming, and direct policy search to optimize investments and operational decisions for power grids over multiple timescales while handling constraints. They have applied their approaches to problems involving interconnection planning, demand balancing, and renewable integration on scales from cities to entire continents.
The document discusses the computational complexity of partially observable games. Some key points:
1. Two-player unobservable games are EXPSPACE-complete, as strategies are just sequences of actions with no observability.
2. Encoding a Turing machine as a game shows the hardness of the unobservable case. The tape configurations can be represented in a game state of size logarithmic in the tape size.
3. Two-player partially observable games or one-player partially observable games against randomness are 2EXPTIME-complete, even more complex than the unobservable case.
Noisy Optimization combining Bandits and Evolutionary AlgorithmsOlivier Teytaud
@inproceedings{rolet:inria-00437140,
hal_id = {inria-00437140},
url = {http://hal.inria.fr/inria-00437140},
title = {{Bandit-based Estimation of Distribution Algorithms for Noisy Optimization: Rigorous Runtime Analysis}},
author = {Rolet, Philippe and Teytaud, Olivier},
abstract = {{We show complexity bounds for noisy optimization, in frame- works in which noise is stronger than in previously published papers[19]. We also propose an algorithm based on bandits (variants of [16]) that reaches the bound within logarithmic factors. We emphasize the differ- ences with empirical derived published algorithms.}},
keywords = {noisy optimization evolutionary algorithms bandits},
language = {Anglais},
affiliation = {Laboratoire de Recherche en Informatique - LRI , TAO - INRIA Futurs , TAO - INRIA Saclay - Ile de France},
booktitle = {{Lion4}},
address = {Venice, Italie},
audience = {internationale },
year = {2010},
pdf = {http://hal.inria.fr/inria-00437140/PDF/lion4long.pdf},
}
@inproceedings{coulom:hal-00517157,
hal_id = {hal-00517157},
url = {http://hal.archives-ouvertes.fr/hal-00517157},
title = {{Handling Expensive Optimization with Large Noise}},
author = {Coulom, R{\'e}mi and Rolet, Philippe and Sokolovska, Nataliya and Teytaud, Olivier},
abstract = {{This paper exhibits lower and upper bounds on runtimes for expensive noisy optimization problems. Runtimes are expressed in terms of number of fitness evaluations. Fitnesses considered are monotonic transformations of the {\em sphere} function. The analysis focuses on the common case of fitness functions quadratic in the distance to the optimum in the neighborhood of this optimum---it is nonetheless also valid for any monotonic polynomial of degree p>2. Upper bounds are derived via a bandit-based estimation of distribution algorithm that relies on Bernstein races called R-EDA. It is known that the algorithm is consistent even in non-differentiable cases. Here we show that: (i) if the variance of the noise decreases to 0 around the optimum, it can perform optimally for quadratic transformations of the norm to the optimum, (ii) otherwise, it provides a slower convergence rate than the one exhibited empirically by an algorithm called Quadratic Logistic Regression based on surrogate models---although QLR requires a probabilistic prior on the fitness class.}},
keywords = {Noisy optimization, Bernstein races},
language = {Anglais},
affiliation = {SEQUEL - INRIA Lille - Nord Europe , TAO - INRIA Saclay - Ile de France , Laboratoire de Recherche en Informatique - LRI},
booktitle = {{Foundations of Genetic Algorithms (FOGA 2011)}},
pages = {TBA},
address = {Autriche},
editor = {ACM },
audience = {internationale },
year = {2011},
month = Jan,
pdf = {http://hal.archives-ouvertes.fr/hal-00517157/PDF/foga10noise.pdf},
}
This document discusses blind Go, a variant of the game where players do not look at the board and must memorize positions. It explores strategies for blind Go, such as playing unusual moves that are harder for the opponent to remember. Experiments found that providing an empty board as a visual aid helped players. When playing against professionals in blind 9x9 Go, the computer won 2 of 3 games. In a 19x19 game against a top human player, the computer won through an unexpected, unusual move where the human made a rare mistake due to not seeing the board. Further research is needed, but playing unconventional moves seems beneficial in blind Go.
Hydroelectricity uses water to produce electricity and has advantages for electricity storage. It provides daily, yearly, and negative electricity production by pumping water to higher reservoirs. However, expanding hydroelectricity is challenging due to its large infrastructure requirements and local environmental impacts. New technologies may improve energy storage capabilities and grid stability in the future, but developing large-scale annual storage remains difficult given constraints. Hydroelectricity will continue playing an important role in energy systems alongside other renewable technologies and efficiency strategies.
This document outlines the course content for a Higher Computing course, which is divided into 3 main units: Computer Systems (40 hours), Software Development (40 hours), and Artificial Intelligence (40 hours). The Computer Systems unit covers topics like data representation, computer structure, networking, and computer software across 5 sections. Specific lessons in the Data Representation section discuss how numbers, text, and images are stored in binary and how storage capacities are measured. Graphics representation and compression techniques are also introduced. Students will complete assessments including end of unit tests, coursework tasks, and a written exam.
Choosing between several options in uncertain environmentsOlivier Teytaud
The document discusses bandit problems with strategic choices and small budgets. It defines bandit problems, strategic bandit problems, and compares the two. It presents algorithms for exploring options and making recommendations in both one-player and two-player settings. Experimental results on a Go positioning problem and an online card game show that TEXP3 outperforms other algorithms in two-player settings. The document concludes with discussions on extensions to structured bandits and using strategic bandits to model investment choices.
This document discusses how to save money by using open source software instead of proprietary software like Microsoft Office. It recommends downloading and using OpenOffice or LibreOffice instead, as they are free alternatives that work very well. It also recommends installing a free open source operating system like Linux, as this can save a lot of money on software costs over time. Open source is discussed as an economic model where the marginal cost of sharing and distributing code is very low, enabling new business models to earn money through services, support or customization rather than just software licenses. A variety of important open source software projects are listed across different domains like operating systems, office suites, web servers and more.
Ilab Metis: we optimize power systems and we are not afraid of direct policy ...Olivier Teytaud
Ilab METIS is a collaboration between TAO, a machine learning and optimization team within INRIA, and Artelys, an SME focused on optimization. They work on optimizing energy policies through simulations of power systems while taking into account uncertainties and stochastic variables. Their methodologies use a hybrid of reinforcement learning, mathematical programming, and direct policy search to optimize investments and operational decisions for power grids over multiple timescales while handling constraints. They have applied their approaches to problems involving interconnection planning, demand balancing, and renewable integration on scales from cities to entire continents.
The document discusses the computational complexity of partially observable games. Some key points:
1. Two-player unobservable games are EXPSPACE-complete, as strategies are just sequences of actions with no observability.
2. Encoding a Turing machine as a game shows the hardness of the unobservable case. The tape configurations can be represented in a game state of size logarithmic in the tape size.
3. Two-player partially observable games or one-player partially observable games against randomness are 2EXPTIME-complete, even more complex than the unobservable case.
Noisy Optimization combining Bandits and Evolutionary AlgorithmsOlivier Teytaud
@inproceedings{rolet:inria-00437140,
hal_id = {inria-00437140},
url = {http://hal.inria.fr/inria-00437140},
title = {{Bandit-based Estimation of Distribution Algorithms for Noisy Optimization: Rigorous Runtime Analysis}},
author = {Rolet, Philippe and Teytaud, Olivier},
abstract = {{We show complexity bounds for noisy optimization, in frame- works in which noise is stronger than in previously published papers[19]. We also propose an algorithm based on bandits (variants of [16]) that reaches the bound within logarithmic factors. We emphasize the differ- ences with empirical derived published algorithms.}},
keywords = {noisy optimization evolutionary algorithms bandits},
language = {Anglais},
affiliation = {Laboratoire de Recherche en Informatique - LRI , TAO - INRIA Futurs , TAO - INRIA Saclay - Ile de France},
booktitle = {{Lion4}},
address = {Venice, Italie},
audience = {internationale },
year = {2010},
pdf = {http://hal.inria.fr/inria-00437140/PDF/lion4long.pdf},
}
@inproceedings{coulom:hal-00517157,
hal_id = {hal-00517157},
url = {http://hal.archives-ouvertes.fr/hal-00517157},
title = {{Handling Expensive Optimization with Large Noise}},
author = {Coulom, R{\'e}mi and Rolet, Philippe and Sokolovska, Nataliya and Teytaud, Olivier},
abstract = {{This paper exhibits lower and upper bounds on runtimes for expensive noisy optimization problems. Runtimes are expressed in terms of number of fitness evaluations. Fitnesses considered are monotonic transformations of the {\em sphere} function. The analysis focuses on the common case of fitness functions quadratic in the distance to the optimum in the neighborhood of this optimum---it is nonetheless also valid for any monotonic polynomial of degree p>2. Upper bounds are derived via a bandit-based estimation of distribution algorithm that relies on Bernstein races called R-EDA. It is known that the algorithm is consistent even in non-differentiable cases. Here we show that: (i) if the variance of the noise decreases to 0 around the optimum, it can perform optimally for quadratic transformations of the norm to the optimum, (ii) otherwise, it provides a slower convergence rate than the one exhibited empirically by an algorithm called Quadratic Logistic Regression based on surrogate models---although QLR requires a probabilistic prior on the fitness class.}},
keywords = {Noisy optimization, Bernstein races},
language = {Anglais},
affiliation = {SEQUEL - INRIA Lille - Nord Europe , TAO - INRIA Saclay - Ile de France , Laboratoire de Recherche en Informatique - LRI},
booktitle = {{Foundations of Genetic Algorithms (FOGA 2011)}},
pages = {TBA},
address = {Autriche},
editor = {ACM },
audience = {internationale },
year = {2011},
month = Jan,
pdf = {http://hal.archives-ouvertes.fr/hal-00517157/PDF/foga10noise.pdf},
}
This document discusses blind Go, a variant of the game where players do not look at the board and must memorize positions. It explores strategies for blind Go, such as playing unusual moves that are harder for the opponent to remember. Experiments found that providing an empty board as a visual aid helped players. When playing against professionals in blind 9x9 Go, the computer won 2 of 3 games. In a 19x19 game against a top human player, the computer won through an unexpected, unusual move where the human made a rare mistake due to not seeing the board. Further research is needed, but playing unconventional moves seems beneficial in blind Go.
Hydroelectricity uses water to produce electricity and has advantages for electricity storage. It provides daily, yearly, and negative electricity production by pumping water to higher reservoirs. However, expanding hydroelectricity is challenging due to its large infrastructure requirements and local environmental impacts. New technologies may improve energy storage capabilities and grid stability in the future, but developing large-scale annual storage remains difficult given constraints. Hydroelectricity will continue playing an important role in energy systems alongside other renewable technologies and efficiency strategies.
Tools for Discrete Time Control; Application to Power SystemsOlivier Teytaud
3 main algorithms from the state of the art:
- Model Predictive Control
- Stochastic Dynamic Programming
- Direct Policy Search
==> and our proposal, a modified Direct Policy Search
termed Direct Value Search
- The document discusses energy management in France and potential areas of research collaboration between France and Taiwan.
- Key areas discussed include optimizing long-term investment policies for electricity generation using tools like reinforcement learning and stochastic programming to account for uncertainties.
- Specific questions mentioned are around optimal connections between Europe and Africa, impacts of subsidizing solar power or switching off nuclear plants, and benefits of demand reduction contracts.
- The researcher proposes combining methods like direct policy search and Monte Carlo tree search to better optimize long-term planning while accounting for short-term effects. Plans are discussed to test new ideas, share data and codes, and potentially organize joint work between the two regions.
Artificial Intelligence and Optimization with ParallelismOlivier Teytaud
This document discusses parallelism in artificial intelligence and evolutionary computation. It explains that comparison-based optimization algorithms, which include many evolutionary algorithms, can be naturally parallelized by speculatively running multiple branches in parallel with a branching factor of 3 or more. This allows theoretical logarithmic speedups to be achieved in practice through simple parallelization tricks.
A simple tutorial on Monte-Carlo Tree Search
Contains a description of dynamic programming and alpha-beta search, then MCTS. Special cases for simultaneous actions are discussed.
I should add comments so that it can be used without preliminary knowledge of MCTS, if there is at least one request for doing so I'll do it.
@article{gelly:hal-00695370,
hal_id = {hal-00695370},
url = {http://hal.inria.fr/hal-00695370},
title = {{The Grand Challenge of Computer Go: Monte Carlo Tree Search and Extensions}},
author = {Gelly, Sylvain and Kocsis, Levente and Schoenauer, Marc and Sebag, Mich{\`e}le and Silver, David and Szepesvari, Csaba and Teytaud, Olivier},
abstract = {{The ancient oriental game of Go has long been considered a grand challenge for artificial intelligence. For decades, com- puter Go has defied the classical methods in game tree search that worked so successfully for chess and checkers. How- ever, recent play in computer Go has been transformed by a new paradigm for tree search based on Monte-Carlo meth- ods. Programs based on Monte-Carlo tree search now play at human-master levels and are beginning to challenge top professional players. In this paper we describe the leading algorithms for Monte-Carlo tree search and explain how they have advanced the state of the art in computer Go.}},
language = {Anglais},
affiliation = {TAO - INRIA Saclay - Ile de France , Laboratoire de Recherche en Informatique - LRI , LPDS , Microsoft Research - Inria Joint Centre - MSR - INRIA , University of Alberta, Canada , Department of Computing Science},
publisher = {ACM},
pages = {106-113},
journal = {Communication of the ACM},
volume = {55},
number = {3 },
audience = {internationale },
year = {2012},
pdf = {http://hal.inria.fr/hal-00695370/PDF/CACM-MCTS.pdf},
}
Don't believe what is written in these slides.
These statements are just provocative statements, most of them found on internet, here for discussion and for brain storming.
Theory of games, with a short reminder of computational complexity and an independent appendix on human complexity and the game of Go
@article{david:hal-00710073,
hal_id = {hal-00710073},
url = {http://hal.inria.fr/hal-00710073},
title = {{The Frontier of Decidability in Partially Observable Recursive Games}},
author = {David, Auger and Teytaud, Olivier},
abstract = {{The classical decision problem associated with a game is whether a given player has a winning strategy, i.e. some strategy that leads almost surely to a victory, regardless of the other players' strategies. While this problem is relevant for deterministic fully observable games, for a partially observable game the requirement of winning with probability 1 is too strong. In fact, as shown in this paper, a game might be decidable for the simple criterion of almost sure victory, whereas optimal play (even in an approximate sense) is not computable. We therefore propose another criterion, the decidability of which is equivalent to the computability of approximately optimal play. Then, we show that (i) this criterion is undecidable in the general case, even with deterministic games (no random part in the game), (ii) that it is in the jump 0', and that, even in the stochastic case, (iii) it becomes decidable if we add the requirement that the game halts almost surely whatever maybe the strategies of the players.}},
language = {Anglais},
affiliation = {Laboratoire de Recherche en Informatique - LRI , TAO - INRIA Saclay - Ile de France},
booktitle = {{Special Issue on "Frontier between Decidability and Undecidability"}},
publisher = {World Scinet},
journal = {International Journal on Foundations of Computer Science (IJFCS)},
volume = {Accepted},
note = {revised 2011, accepted 2011, in press },
audience = {internationale },
year = {2012},
}
This document provides an overview of distributed decision making in partially observable dynamic games and multiobjective policy optimization. It discusses applying these techniques to optimization problems in games like chess and Go, as well as industrial applications like managing groups of power plants involving renewable energy, nuclear power, coal, hydroelectric power, and interactions with electricity consumers and networks. The goal is to optimize strategies using parallel computing and test these approaches on games and energy systems.
Tools for artificial intelligence: EXP3, Zermelo algorithm, Alpha-Beta, and s...Olivier Teytaud
Here are a few suggestions on how to improve the Zermelo algorithm when it is too slow:
1. Add a depth limit. Stop recursion when a maximum search depth is reached. Return a heuristic evaluation instead of continuing search.
2. Use alpha-beta pruning. Track the best value found (alpha) and prune branches that cannot improve on it.
3. Iterative deepening. Run successive searches with increasing depth limits to get progressively better approximations.
4. Move ordering. Evaluate better moves earlier in the search tree. This prunes bad moves earlier.
5. Transposition tables. Store previously computed move evaluations to avoid re-expanding the same position.
6. Parallelize the
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
In the realm of cybersecurity, offensive security practices act as a critical shield. By simulating real-world attacks in a controlled environment, these techniques expose vulnerabilities before malicious actors can exploit them. This proactive approach allows manufacturers to identify and fix weaknesses, significantly enhancing system security.
This presentation delves into the development of a system designed to mimic Galileo's Open Service signal using software-defined radio (SDR) technology. We'll begin with a foundational overview of both Global Navigation Satellite Systems (GNSS) and the intricacies of digital signal processing.
The presentation culminates in a live demonstration. We'll showcase the manipulation of Galileo's Open Service pilot signal, simulating an attack on various software and hardware systems. This practical demonstration serves to highlight the potential consequences of unaddressed vulnerabilities, emphasizing the importance of offensive security practices in safeguarding critical infrastructure.
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyScyllaDB
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Dandelion Hashtable: beyond billion requests per second on a commodity serverAntonios Katsarakis
This slide deck presents DLHT, a concurrent in-memory hashtable. Despite efforts to optimize hashtables, that go as far as sacrificing core functionality, state-of-the-art designs still incur multiple memory accesses per request and block request processing in three cases. First, most hashtables block while waiting for data to be retrieved from memory. Second, open-addressing designs, which represent the current state-of-the-art, either cannot free index slots on deletes or must block all requests to do so. Third, index resizes block every request until all objects are copied to the new index. Defying folklore wisdom, DLHT forgoes open-addressing and adopts a fully-featured and memory-aware closed-addressing design based on bounded cache-line-chaining. This design offers lock-free index operations and deletes that free slots instantly, (2) completes most requests with a single memory access, (3) utilizes software prefetching to hide memory latencies, and (4) employs a novel non-blocking and parallel resizing. In a commodity server and a memory-resident workload, DLHT surpasses 1.6B requests per second and provides 3.5x (12x) the throughput of the state-of-the-art closed-addressing (open-addressing) resizable hashtable on Gets (Deletes).
Skybuffer SAM4U tool for SAP license adoptionTatiana Kojar
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...Jason Yip
The typical problem in product engineering is not bad strategy, so much as “no strategy”. This leads to confusion, lack of motivation, and incoherent action. The next time you look for a strategy and find an empty space, instead of waiting for it to be filled, I will show you how to fill it in yourself. If you’re wrong, it forces a correction. If you’re right, it helps create focus. I’ll share how I’ve approached this in the past, both what works and lessons for what didn’t work so well.
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsDianaGray10
Join us to learn how UiPath Apps can directly and easily interact with prebuilt connectors via Integration Service--including Salesforce, ServiceNow, Open GenAI, and more.
The best part is you can achieve this without building a custom workflow! Say goodbye to the hassle of using separate automations to call APIs. By seamlessly integrating within App Studio, you can now easily streamline your workflow, while gaining direct access to our Connector Catalog of popular applications.
We’ll discuss and demo the benefits of UiPath Apps and connectors including:
Creating a compelling user experience for any software, without the limitations of APIs.
Accelerating the app creation process, saving time and effort
Enjoying high-performance CRUD (create, read, update, delete) operations, for
seamless data management.
Speakers:
Russell Alfeche, Technology Leader, RPA at qBotic and UiPath MVP
Charlie Greenberg, host
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
Tools for Discrete Time Control; Application to Power SystemsOlivier Teytaud
3 main algorithms from the state of the art:
- Model Predictive Control
- Stochastic Dynamic Programming
- Direct Policy Search
==> and our proposal, a modified Direct Policy Search
termed Direct Value Search
- The document discusses energy management in France and potential areas of research collaboration between France and Taiwan.
- Key areas discussed include optimizing long-term investment policies for electricity generation using tools like reinforcement learning and stochastic programming to account for uncertainties.
- Specific questions mentioned are around optimal connections between Europe and Africa, impacts of subsidizing solar power or switching off nuclear plants, and benefits of demand reduction contracts.
- The researcher proposes combining methods like direct policy search and Monte Carlo tree search to better optimize long-term planning while accounting for short-term effects. Plans are discussed to test new ideas, share data and codes, and potentially organize joint work between the two regions.
Artificial Intelligence and Optimization with ParallelismOlivier Teytaud
This document discusses parallelism in artificial intelligence and evolutionary computation. It explains that comparison-based optimization algorithms, which include many evolutionary algorithms, can be naturally parallelized by speculatively running multiple branches in parallel with a branching factor of 3 or more. This allows theoretical logarithmic speedups to be achieved in practice through simple parallelization tricks.
A simple tutorial on Monte-Carlo Tree Search
Contains a description of dynamic programming and alpha-beta search, then MCTS. Special cases for simultaneous actions are discussed.
I should add comments so that it can be used without preliminary knowledge of MCTS, if there is at least one request for doing so I'll do it.
@article{gelly:hal-00695370,
hal_id = {hal-00695370},
url = {http://hal.inria.fr/hal-00695370},
title = {{The Grand Challenge of Computer Go: Monte Carlo Tree Search and Extensions}},
author = {Gelly, Sylvain and Kocsis, Levente and Schoenauer, Marc and Sebag, Mich{\`e}le and Silver, David and Szepesvari, Csaba and Teytaud, Olivier},
abstract = {{The ancient oriental game of Go has long been considered a grand challenge for artificial intelligence. For decades, com- puter Go has defied the classical methods in game tree search that worked so successfully for chess and checkers. How- ever, recent play in computer Go has been transformed by a new paradigm for tree search based on Monte-Carlo meth- ods. Programs based on Monte-Carlo tree search now play at human-master levels and are beginning to challenge top professional players. In this paper we describe the leading algorithms for Monte-Carlo tree search and explain how they have advanced the state of the art in computer Go.}},
language = {Anglais},
affiliation = {TAO - INRIA Saclay - Ile de France , Laboratoire de Recherche en Informatique - LRI , LPDS , Microsoft Research - Inria Joint Centre - MSR - INRIA , University of Alberta, Canada , Department of Computing Science},
publisher = {ACM},
pages = {106-113},
journal = {Communication of the ACM},
volume = {55},
number = {3 },
audience = {internationale },
year = {2012},
pdf = {http://hal.inria.fr/hal-00695370/PDF/CACM-MCTS.pdf},
}
Don't believe what is written in these slides.
These statements are just provocative statements, most of them found on internet, here for discussion and for brain storming.
Theory of games, with a short reminder of computational complexity and an independent appendix on human complexity and the game of Go
@article{david:hal-00710073,
hal_id = {hal-00710073},
url = {http://hal.inria.fr/hal-00710073},
title = {{The Frontier of Decidability in Partially Observable Recursive Games}},
author = {David, Auger and Teytaud, Olivier},
abstract = {{The classical decision problem associated with a game is whether a given player has a winning strategy, i.e. some strategy that leads almost surely to a victory, regardless of the other players' strategies. While this problem is relevant for deterministic fully observable games, for a partially observable game the requirement of winning with probability 1 is too strong. In fact, as shown in this paper, a game might be decidable for the simple criterion of almost sure victory, whereas optimal play (even in an approximate sense) is not computable. We therefore propose another criterion, the decidability of which is equivalent to the computability of approximately optimal play. Then, we show that (i) this criterion is undecidable in the general case, even with deterministic games (no random part in the game), (ii) that it is in the jump 0', and that, even in the stochastic case, (iii) it becomes decidable if we add the requirement that the game halts almost surely whatever maybe the strategies of the players.}},
language = {Anglais},
affiliation = {Laboratoire de Recherche en Informatique - LRI , TAO - INRIA Saclay - Ile de France},
booktitle = {{Special Issue on "Frontier between Decidability and Undecidability"}},
publisher = {World Scinet},
journal = {International Journal on Foundations of Computer Science (IJFCS)},
volume = {Accepted},
note = {revised 2011, accepted 2011, in press },
audience = {internationale },
year = {2012},
}
This document provides an overview of distributed decision making in partially observable dynamic games and multiobjective policy optimization. It discusses applying these techniques to optimization problems in games like chess and Go, as well as industrial applications like managing groups of power plants involving renewable energy, nuclear power, coal, hydroelectric power, and interactions with electricity consumers and networks. The goal is to optimize strategies using parallel computing and test these approaches on games and energy systems.
Tools for artificial intelligence: EXP3, Zermelo algorithm, Alpha-Beta, and s...Olivier Teytaud
Here are a few suggestions on how to improve the Zermelo algorithm when it is too slow:
1. Add a depth limit. Stop recursion when a maximum search depth is reached. Return a heuristic evaluation instead of continuing search.
2. Use alpha-beta pruning. Track the best value found (alpha) and prune branches that cannot improve on it.
3. Iterative deepening. Run successive searches with increasing depth limits to get progressively better approximations.
4. Move ordering. Evaluate better moves earlier in the search tree. This prunes bad moves earlier.
5. Transposition tables. Store previously computed move evaluations to avoid re-expanding the same position.
6. Parallelize the
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
In the realm of cybersecurity, offensive security practices act as a critical shield. By simulating real-world attacks in a controlled environment, these techniques expose vulnerabilities before malicious actors can exploit them. This proactive approach allows manufacturers to identify and fix weaknesses, significantly enhancing system security.
This presentation delves into the development of a system designed to mimic Galileo's Open Service signal using software-defined radio (SDR) technology. We'll begin with a foundational overview of both Global Navigation Satellite Systems (GNSS) and the intricacies of digital signal processing.
The presentation culminates in a live demonstration. We'll showcase the manipulation of Galileo's Open Service pilot signal, simulating an attack on various software and hardware systems. This practical demonstration serves to highlight the potential consequences of unaddressed vulnerabilities, emphasizing the importance of offensive security practices in safeguarding critical infrastructure.
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyScyllaDB
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Dandelion Hashtable: beyond billion requests per second on a commodity serverAntonios Katsarakis
This slide deck presents DLHT, a concurrent in-memory hashtable. Despite efforts to optimize hashtables, that go as far as sacrificing core functionality, state-of-the-art designs still incur multiple memory accesses per request and block request processing in three cases. First, most hashtables block while waiting for data to be retrieved from memory. Second, open-addressing designs, which represent the current state-of-the-art, either cannot free index slots on deletes or must block all requests to do so. Third, index resizes block every request until all objects are copied to the new index. Defying folklore wisdom, DLHT forgoes open-addressing and adopts a fully-featured and memory-aware closed-addressing design based on bounded cache-line-chaining. This design offers lock-free index operations and deletes that free slots instantly, (2) completes most requests with a single memory access, (3) utilizes software prefetching to hide memory latencies, and (4) employs a novel non-blocking and parallel resizing. In a commodity server and a memory-resident workload, DLHT surpasses 1.6B requests per second and provides 3.5x (12x) the throughput of the state-of-the-art closed-addressing (open-addressing) resizable hashtable on Gets (Deletes).
Skybuffer SAM4U tool for SAP license adoptionTatiana Kojar
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...Jason Yip
The typical problem in product engineering is not bad strategy, so much as “no strategy”. This leads to confusion, lack of motivation, and incoherent action. The next time you look for a strategy and find an empty space, instead of waiting for it to be filled, I will show you how to fill it in yourself. If you’re wrong, it forces a correction. If you’re right, it helps create focus. I’ll share how I’ve approached this in the past, both what works and lessons for what didn’t work so well.
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsDianaGray10
Join us to learn how UiPath Apps can directly and easily interact with prebuilt connectors via Integration Service--including Salesforce, ServiceNow, Open GenAI, and more.
The best part is you can achieve this without building a custom workflow! Say goodbye to the hassle of using separate automations to call APIs. By seamlessly integrating within App Studio, you can now easily streamline your workflow, while gaining direct access to our Connector Catalog of popular applications.
We’ll discuss and demo the benefits of UiPath Apps and connectors including:
Creating a compelling user experience for any software, without the limitations of APIs.
Accelerating the app creation process, saving time and effort
Enjoying high-performance CRUD (create, read, update, delete) operations, for
seamless data management.
Speakers:
Russell Alfeche, Technology Leader, RPA at qBotic and UiPath MVP
Charlie Greenberg, host
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/temporal-event-neural-networks-a-more-efficient-alternative-to-the-transformer-a-presentation-from-brainchip/
Chris Jones, Director of Product Management at BrainChip , presents the “Temporal Event Neural Networks: A More Efficient Alternative to the Transformer” tutorial at the May 2024 Embedded Vision Summit.
The expansion of AI services necessitates enhanced computational capabilities on edge devices. Temporal Event Neural Networks (TENNs), developed by BrainChip, represent a novel and highly efficient state-space network. TENNs demonstrate exceptional proficiency in handling multi-dimensional streaming data, facilitating advancements in object detection, action recognition, speech enhancement and language model/sequence generation. Through the utilization of polynomial-based continuous convolutions, TENNs streamline models, expedite training processes and significantly diminish memory requirements, achieving notable reductions of up to 50x in parameters and 5,000x in energy consumption compared to prevailing methodologies like transformers.
Integration with BrainChip’s Akida neuromorphic hardware IP further enhances TENNs’ capabilities, enabling the realization of highly capable, portable and passively cooled edge devices. This presentation delves into the technical innovations underlying TENNs, presents real-world benchmarks, and elucidates how this cutting-edge approach is positioned to revolutionize edge AI across diverse applications.
Main news related to the CCS TSI 2023 (2023/1695)Jakub Marek
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Discover top-tier mobile app development services, offering innovative solutions for iOS and Android. Enhance your business with custom, user-friendly mobile applications.
What is an RPA CoE? Session 1 – CoE VisionDianaGray10
In the first session, we will review the organization's vision and how this has an impact on the COE Structure.
Topics covered:
• The role of a steering committee
• How do the organization’s priorities determine CoE Structure?
Speaker:
Chris Bolin, Senior Intelligent Automation Architect Anika Systems
1. Monte-Carlo Tree Search
Games with partial
observation
Olivier.Teytaud@inria.fr + David Auger
+Hervé Fournier + Fabien Teytaud + Sébastien Flory
+ JY Audibert+ S. Bubeck + R. Munos + ...
Includes Inria, Cnrs, Univ. Paris-Sud, LRI, CMAP,
Taiwan universities, Lille, Paris, Boostr...
TAO, Inria-Saclay IDF, Cnrs 8623,
Lri, Univ. Paris-Sud,
Digiteo Labs, Pascal
Network of Excellence.
Grenoble
June 2011
Games with simultaneous actions 1 Grenoble, June 19th, 2011.
2. Monte-Carlo Tree Search
1. Games (a bit of formalism)
2. Hidden information <==> SA
3. Decidability / complexity
4. Real implementation
==> appli to UrbanRivals
Games with simultaneous actions 2 Grenoble, June 19th, 2011.
5. A game is a directed graph with actions
and players
1 White
Black
2
3
White 12
43
White Black
Black
Black
Black
Games with simultaneous actions Grenoble, June 19th, 2011. 5
6. A game is a directed graph with actions
and players and observations
Bob
Bear Bee
Bee 1 White
Black
2
3
White 12
43
White Black
Black
Black
Black
Games with simultaneous actions Grenoble, June 19th, 2011. 6
7. A game is a directed graph with actions
and players and observations and rewards
Bob
Bear Bee
Bee 1 White
Black
2
3
+1
0
White 12
43 Rewards
White Black on leafs
Black only!
Black
Black
Games with simultaneous actions Grenoble, June 19th, 2011. 7
8. A game is a directed graph +actions
+players +observations +rewards +loops
Bob
Bear Bee
Bee 1 White
Black
2
3
+1
0
White 12
43
White Black
Black
Black
Black
Games with simultaneous actions Grenoble, June 19th, 2011. 8
9. Monte-Carlo Tree Search
1. Games (a bit of formalism)
2. Hidden information <==> SA
3. Decidability / complexity
4. Real implementation
Games with simultaneous actions 9 Grenoble, June 19th, 2011.
10. A game is a directed graph +actions
+players +observations +rewards +loops
Consider games as follows:
Bob
Bear Bee
Bee 1
Black
Turn 1 White
Turn 2
2
…
3
+1
0
Turn K: all information is revealed.
Turn K+1 White 12
Turn K+2
… White 43
Black
Turn 2K: all information is revealed
Black
…
… Black
TurnBlack all information is revealed
NK:
Games with simultaneous actions Grenoble, June 19th, 2011. 10
11. A game is a directed graph +actions
Rewrite it as follows:
+players +observations +rewards +loops
Bob
Turn 1: player 1 chooses Bee Bear
Bee 1
Black
(privately) his strategy until turn K
White
Turn 2: player 2 chooses
2
(privately) his strategy until turn K +1
3
Intermediate turns removed! 0
Turn K: all information is revealed.
White 12
Turn K+1
Turn White K+2 43
Black
… Black
Turn 2K: all information is revealed
… Black
… Black
Games with simultaneous actions all information 2011. revealed
Turn NK: Grenoble, June 19th, is 11
12. A game is a directed graph +actions
Rewrite it as follows:
+players +observations +rewards +loops
Bob
Turn 1: player 1 chooses Bee Bear
Bee 1
Black
(privately) his strategy until turn K
White
Equivalent
Turn 2: player 2 chooses to
2
(privately) his strategy until turn K +1 simultaneous
3 actions
Intermediate turns removed! 0
Turn K: all information is revealed.
White 12
Turn K+1
Turn White K+2 43
Black
… Black
Turn 2K: all information is revealed
… Black
… Black
Games with simultaneous actions all information 2011. revealed
Turn NK: Grenoble, June 19th, is 12
13. A game is a directed graph +actions
+players +observations +rewards +loops
Bob
Bear Bee
Bee 1 White
Black
Now it's a game with simultaneous information
2
and no hidden information.
3
+1
0
Simultaneous actions
White 12
White
= (almost) Black
43
short term hidden information.
Black
Black
Black
Games with simultaneous actions Grenoble, June 19th, 2011. 13
14. Monte-Carlo Tree Search
1. Games (a bit of formalism)
2. Hidden information <== SA
(and sometimes <==>)
3. Decidability / complexity
4. Real implementation
Games with simultaneous actions 14 Grenoble, June 19th, 2011.
15. Compact representation ?
Succinct representation (in short, without tedious details):
- graph of size N represented in size O(log N) ;
- usually not better in terms of complexity;
- keep this in mind when considering complexity.
Games with simultaneous actions 15 Grenoble, June 19th, 2011.
16. Complexity question ?
Instance = position.
Question = Is there a strategy
which wins whatever
are the decisions
of the opponent ?
= natural question if full observability.
Answering this question then allows perfect
play.
Games with simultaneous actions 16 Grenoble, June 19th, 2011.
17. Complexity question ? (UD)
Instance = position.
Question = Is there a strategy
which wins whatever
are the decisions
of the opponent ?
= natural question if full observability.
Answering this question then allows perfect
play.
Games with simultaneous actions 17 Grenoble, June 19th, 2011.
18. Complexity question for matrix
game ?
100000
Good for column-player !
010000
001000 ==> but no sure win.
000100 ==> the “UD” question is not
000010 relevant here!
000001
Games with simultaneous actions 18 Grenoble, June 19th, 2011.
19. Complexity question for
Joint work with
phantom-games ? F. Teytaud
This is phantom-go.
Good for black: wins
with proba 1-1/(8!)
Here,
there's no move
which ensures a win.
But some moves are
much better than
others!
Games with simultaneous actions 19 Grenoble, June 19th, 2011.
20. It becomes complicated
Isn't it possible to
consider
a better question ?
Games with simultaneous actions Grenoble, June 19th, 2011. 20
21. Complexity (2P, no random)
X= proba(winning) that we look for
Unbounded Exponential Polynomial
horizon horizon horizon
Full
Observability EXP EXP PSPACE
No obs EXPSPACE NEXP
(X=100%) (Hasslum et al, 2000)
Partially 2EXP EXPSPACE
Observable (Rintanen) (Mundhenk)
(X=100%)
Simult. Actions ? EXPSPACE ? <<<= EXP <<<= EXP
No obs undecidable Teytaud,
Auger, IJFCS
Partiallywith simultaneous actions
Games undecidable 21 (accepted)
Grenoble, June 19th, 2011.
Observable
22. State of the art
EXPTIME-complete in the general
Grenoble, June 19th, 2011.
fully-observable case
Games with simultaneous actions 22
23. EXPTIME-complete fully
observable games
- Chess (for some nxn generalization)
- Go (with no superko)
- Draughts (international or english)
- Chinese checkers
- Shogi
Games with simultaneous actions 23 Grenoble, June 19th, 2011.
24. PSPACE-complete fully
observable games
- Amazons
- Hex
- Go-moku
- Connect-6
- Qubic
- Reversi
- Tic-Tac-Toe
Many games with filling of each cell once and only once
Games with simultaneous actions 24 Grenoble, June 19th, 2011.
25. EXPSPACE-complete
unobservable games (Hasslun & Jonnsson)
The two-player unobservable case is
EXPSPACE-complete
(games in succinct form).
Games with simultaneous actions 25 Grenoble, June 19th, 2011.
26. E X P S P Atwo-player unobservable case is
The C E - c o m p l e t e
unobservable games (Hasslun & Jonnsson)
EXPSPACE-complete
(games in succinct form).
PROOF:
(I) First note that strategies are just sequences of actions
(no observability!) + UD=>opponent can see the state!
(II) It is in EXPSPACE=NEXPSPACE, because of the
following algorithm:
(a) Non-deterministically choose the sequence of
Actions
(b) Check the result against all possible strategies
(III) We have to check the hardness only.
Games with simultaneous actions 26 Grenoble, June 19th, 2011.
27. E X P S P Atwo-player unobservable case is
The C E - c o m p l e t e
unobservable games (Hasslun & Jonnsson)
EXPSPACE-complete
(games in succinct form).
PROOF:
(I) First note that strategies are just sequences of actions
(no observability!) + UD=>opponent can see the state!
(II) It is in EXPSPACE=NEXPSPACE, because of the
following algorithm:
(a) Non-deterministically choose the sequence of
actions
(b) Check the result against all possible strategies
(III) We have to check the hardness only.
Games with simultaneous actions 27 Grenoble, June 19th, 2011.
28. E X P S P Atwo-player unobservable case is
The C E - c o m p l e t e
unobservable games (Hasslun & Jonnsson)
EXPSPACE-complete
(games in succinct form).
PROOF:
(I) First note that strategies are just sequences of actions
(no observability!)
(II) It is in EXPSPACE=NEXPSPACE, because of the
following algorithm:
(a) Non-deterministically choose the sequence of
actions
(b) Check the result against all possible strategies
(III) We have to check the hardness only.
Games with simultaneous actions 28 Grenoble, June 19th, 2011.
29. E X P S P Atwo-player unobservable case is
The C E - c o m p l e t e
unobservable games (Hasslun & Jonnsson)
EXPSPACE-complete
(games in succinct form).
PROOF of the hardness:
Reduction to: is my TM with exponential tape
going to halt ?
Consider a TM with tape of size N=2^n.
We must find a game
- with size n ( n= log2(N) )
- such that the first player has a winning
strategy iff the TM halts.
Games with simultaneous actions 29 Grenoble, June 19th, 2011.
30. EXPSPACE-complete
uEncoding ravTuring machine with Ha stape & J osizes oN)
n o b s e a b l e g a m e s ( a s l u n of n n s n
as a game with state O(log(N))
Player 1 chooses the sequence of
configurations of the tape (N=4):
x(0,1),x(0,2),x(0,3),x(0,4) ==> initial state
x(1,1),x(1,2),x(1,3),x(1,4)
x(2,1),x(2,2),x(2,3),x(2,4)
x(3,1),x(3,2),x(3,3),x(3,4)
.....................................
Games with simultaneous actions 30 Grenoble, June 19th, 2011.
31. EXPSPACE-complete
uEncoding ravTuring machine with Ha stape & J osizes oN)
n o b s e a b l e g a m e s ( a s l u n of n n s n
as a game with state O(log(N))
Player 1 chooses the sequence of
configurations of the tape (N=4):
x(0,1),x(0,2),x(0,3),x(0,4) ==> initial state
x(1,1),x(1,2),x(1,3),x(1,4)
x(2,1),x(2,2),x(2,3),x(2,4)
x(3,1),x(3,2),x(3,3),x(3,4)
.....................................
x(N,1), x(N,2), x(N,3), x(N,4)
Grenoble, June 19th, 2011.
Wins by
Games with simultaneous actions 31
final state !
32. EXPSPACE-complete
uEncoding ravTuring machine with Ha stape & J osizes oN)
n o b s e a b l e g a m e s ( a s l u n of n n s n
as a game with state O(log(N))
Player 1 chooses the sequence of
configurations of the tape (N=4):
x(0,1),x(0,2),x(0,3),x(0,4) ==> initial state
x(1,1),x(1,2),x(1,3),x(1,4)
x(2,1),x(2,2),x(2,3),x(2,4)Except if P2 finds an
x(3,1),x(3,2),x(3,3),x(3,4) illegal transition!
..................................... ==> P2 can check the
x(N,1), x(N,2), x(N,3), x(N,4)
consistency of one 3-uple per line
Wins by
Games with simultaneous actions 32 ==> requests space log(N)
Grenoble, June 19th, 2011.
final state ! ( = position of the 3-uple)
33. EXPSPACE-complete PO games
The one-player PO case is
EXPSPACE-complete
(games in succinct form).
Games with simultaneous actions 33 Grenoble, June 19th, 2011.
34. 2EXPTIME-complete PO games
The two-player PO case is
2EXP-complete
(games in succinct form).
Games with simultaneous actions 34 Grenoble, June 19th, 2011.
35. Undecidable games (B. Hearn)
The three-player PO case is
undecidable. (two players against one,
not allowed to communicate)
Games with simultaneous actions 35 Grenoble, June 19th, 2011.
36. Complexity (2P, no random)
Unbounded Exponential Polynomial
horizon horizon horizon
Full
Observability EXP EXP PSPACE
No obs EXPSPACE NEXP
(X=100%) (Hasslum et al, 2000)
Partially 2EXP EXPSPACE
Observable (Rintanen 97)
(X=100%) Reduction to 1P + random
(Madani et al)
Simult. Actions ? EXPSPACE ? <<<= EXP <<<= EXP
No obs undecidable
Partiallywith simultaneous actions
Games undecidable 36 Grenoble, June 19th, 2011.
Observable
37. Another formalization
c
==> much more satisfactory
Games with simultaneous actions 37 Grenoble, June 19th, 2011.
38. Madani et al.
c
1 player + random = undecidable.
Games with simultaneous actions 38 Grenoble, June 19th, 2011.
39. Madani et al.
1 player + random = undecidable.
We extend to two players with no random.
Problem: rewrite random nodes, thanks to additional
player.
Games with simultaneous actions 39 Grenoble, June 19th, 2011.
40. A random node to be rewritten
Games with simultaneous actions 40 Grenoble, June 19th, 2011.
41. A random node to be rewritten
Games with simultaneous actions 41 Grenoble, June 19th, 2011.
42. A random node to be rewritten
Rewritten as follows:
Player 1 chooses a in [[0,N-1]]
Player 2 chooses b in [[0,N-1]]
c=(a+b) modulo N
Go to tc
Each player can force the game to be equivalent to
the initial one (by playing uniformly)
==> the proba of winning for player 1 (in case of perfect play)
is the same as for the initial game
==> undecidability!
Games with simultaneous actions 42 Grenoble, June 19th, 2011.
43. Important remark
Existence of a strategy for winning with
proba > 0.5
==> also undecidable for the
restriction to games in which the proba
is >0.6 or <0.4
==> not just a subtle
precision trouble.
Games with simultaneous actions 43 Grenoble, June 19th, 2011.
44. Monte-Carlo Tree Search
1. Games (a bit of formalism)
2. Hidden information <==> SA
3. Decidability / complexity
4. Real implementation
Games with simultaneous actions 44 Grenoble, June 19th, 2011.
45. Real implementation for
simultaneous action ?
MCTS principle
But with EXP3 in nodes.
Games with simultaneous actions 45 Grenoble, June 19th, 2011.
56. ... or exploration ?
SCORE =
0/2
+ k.sqrt( log(10)/2 )
Replace it
with
EXP3 / INF
57. The game of Go is a part of AI.
Computers are ridiculous in front of children.
Easy situation.
Termed “semeai”.
Requires a little bit
of abstraction.
58. The game of Go is a part of AI.
Computers are ridiculous in front of children.
800 cores, 4.7 GHz,
top level program.
Plays a stupid move.
59. The game of Go is a part of AI.
Computers are ridiculous in front of children.
8 years old;
little training;
finds the good move
60. MoGo(TW): games vs pros
in the game of Go
First win in 9x9
First draw (a few days ago!) over 6 games
First win over 4 games in 9x9 blind Go
First win with H2.5 in 13x13 Go
First win with H6 in 19x19 Go in 2009 (also done by Zen)
First win with H7 in 19x19 Go vs top pro in 2009 (also
done by Pachi in 2011)
61. Monte-Carlo Tree Search
1. Games (a bit of formalism)
2. Hidden information <==> SA
3. Decidability / complexity
4. Real implementation
==> Dark Chess endgames
==> appli to UrbanRivals
Games with simultaneous actions 61 Grenoble, June 19th, 2011.
62. Let's have fun with Urban Rivals (4 cards)
Each player has
- four cards (each one can be used once)
- 12 pilz (each one can be used once)
- 12 life points
Each card has:
- one attack level
- one damage
- special effects (forget it for the moment)
Four turns:
- P1 attacks P2
- P2 attacks P1
- P1 attacks P2
- P2 attacks P1
Games with simultaneous actions Grenoble, June 19th, 2011. 62
63. Let's have fun with Urban Rivals
First, attacker plays:
- chooses a card
- chooses ( PRIVATELY ) a number of pilz
Attack level = attack(card) x (1+nb of pilz)
Then, defender plays:
- chooses a card
- chooses a number of pilz
Defense level = attack(card) x (1+nb of pilz)
Result:
If attack > defense
Defender looses Power(attacker's card)
Else
Attacker looses Power(defender's card)
Games with simultaneous actions Grenoble, June 19th, 2011. 63
64. Let's have fun with Urban Rivals
==> The MCTS-based AI is now at the best human level.
Experimental (only) remarks on EXP3:
- discard strategies with small number of sims = better approx
of the Nash
- also an improvement by taking into
account the other bandit
- not yet compared to INF
- virtual simulations (inspired by Kummer)
Games with simultaneous actions Grenoble, June 19th, 2011. 64
65. Conclusions
New stuff:
Undecidability of optimal play for 2-player games with hidden information
Transformation “PO periodically revealed ==> simultaneous action game
with full observation”
Open problems
Complexity: simultaneous action and infinite horizon (in progress)
Complexity with PO: same information for both cases ?
Nash of matrix games with strong dominance
Mathematical validation of variants of Exp3 / Inf
Consistent “realistic” approaches for PO games (H finite)
66. Conclusions
New stuff:
Undecidability of optimal play for 2-player games with hidden information
Transformation “PO periodically revealed ==> simultaneous action game
with full observation”
Open problems
Complexity: simultaneous action and infinite horizon (in progress)
Complexity with PO: same information for both cases ?
Nash of matrix games with strong dominance
Mathematical validation of variants of Exp3 / Inf
Consistent “realistic” approaches for PO games (H finite)
67. Conclusions
New stuff:
Undecidability of optimal play for 2-player games with hidden information
Transformation “PO periodically revealed ==> simultaneous action game
with full observation”
Open problems
Complexity: simultaneous action and infinite horizon (in progress)
Complexity with PO: same information for both cases ?
Nash of matrix games with strong dominance
Mathematical validation of variants of Exp3 / Inf
Consistent “realistic” approaches for PO games (H finite)
68. Conclusions
New stuff:
Undecidability of optimal play for 2-player games with hidden information
Transformation “PO periodically revealed ==> simultaneous action game
with full observation”
Open problems
Complexity: simultaneous action and infinite horizon (in progress)
Complexity with PO: same information for both players ?
Nash of matrix games with strong dominance
Mathematical validation of variants of Exp3 / Inf
Consistent “realistic” approaches for PO games (H finite)
69. Conclusions
New stuff:
Undecidability of optimal play for 2-player games with hidden information
Transformation “PO periodically revealed ==> simultaneous action game
with full observation”
Open problems
Complexity: simultaneous action and infinite horizon (in progress)
Complexity with PO: same information for both players ?
Nash of matrix games with strong dominance
Mathematical validation of variants of Exp3 / Inf
Consistent “realistic” approaches for PO games (H finite)
70. Conclusions
New stuff:
Undecidability of optimal play for 2-player games with hidden information
Transformation “PO periodically revealed ==> simultaneous action game
with full observation”
Open problems
Complexity: simultaneous action and infinite horizon (in progress)
Complexity with PO: same information for both players ?
Nash of matrix games with strong dominance
Mathematical validation of variants of Exp3 / Inf
Consistent “realistic” approaches for PO games (H finite)
71. Conclusions
New stuff:
Undecidability of optimal play for 2-player games with hidden information
Transformation “PO periodically revealed ==> simultaneous action game
with full observation”
Open problems
Complexity: simultaneous action and infinite horizon (in progress)
Complexity with PO: same information for both players ?
Nash of matrix games with strong dominance
Mathematical validation of variants of Exp3 / Inf
Consistent “realistic” approaches for PO games (H finite)
72. When is MCTS relevant ?
Robust in front of:
High dimension;
Non-convexity of Bellman values;
Complex models
Delayed reward
Simultaneous actions
More difficult for
High values of H;
Highly unobservable cases (Monte-Carlo, but not
Monte-Carlo Tree Search, see Cazenave et al.)
Lack of reasonable baseline for the MC
73. When is MCTS relevant ?
We should
Robust in front of: test INF and
High dimension; justify mathematically
Non-convexity of Bellman values;
our improvements
Complex models
Delayed reward Some Further
Simultaneous actions undecidability
results
work !
More difficult for
High values of H;
Highly unobservable cases (Monte-Carlo, but not
Monte-Carlo Tree Search, see Cazenave et al.)
Lack of reasonable baseline for the MC
74. When is MCTS relevant ?
Convenient.
How to apply it: Easy to check.
Implement the transition
(a function action x state → state )
Design a Monte-Carlo part (a random simulation)
(a heuristic in one-player games;
difficult if two opponents)
==> at this point you can simulate...
Implement UCT (just a bias in the simulator – no real optimizer)
Possibly parallelize (Gelly et al)
75. PO problems, approx.
Nash ==> mailing list
Challenge: outperform humans
in “Urban Rivals”
- free game
- fast games (~ 1 minute)
- 11M registered players