Board Games in Academia 2010
@article{gelly:hal-00695370,
hal_id = {hal-00695370},
url = {http://hal.inria.fr/hal-00695370},
title = {{The Grand Challenge of Computer Go: Monte Carlo Tree Search and Extensions}},
author = {Gelly, Sylvain and Kocsis, Levente and Schoenauer, Marc and Sebag, Mich{\`e}le and Silver, David and Szepesvari, Csaba and Teytaud, Olivier},
abstract = {{The ancient oriental game of Go has long been considered a grand challenge for artificial intelligence. For decades, com- puter Go has defied the classical methods in game tree search that worked so successfully for chess and checkers. How- ever, recent play in computer Go has been transformed by a new paradigm for tree search based on Monte-Carlo meth- ods. Programs based on Monte-Carlo tree search now play at human-master levels and are beginning to challenge top professional players. In this paper we describe the leading algorithms for Monte-Carlo tree search and explain how they have advanced the state of the art in computer Go.}},
language = {Anglais},
affiliation = {TAO - INRIA Saclay - Ile de France , Laboratoire de Recherche en Informatique - LRI , LPDS , Microsoft Research - Inria Joint Centre - MSR - INRIA , University of Alberta, Canada , Department of Computing Science},
publisher = {ACM},
pages = {106-113},
journal = {Communication of the ACM},
volume = {55},
number = {3 },
audience = {internationale },
year = {2012},
pdf = {http://hal.inria.fr/hal-00695370/PDF/CACM-MCTS.pdf},
}
This document discusses blind Go, a variant of the game where players do not look at the board and must memorize positions. It explores strategies for blind Go, such as playing unusual moves that are harder for the opponent to remember. Experiments found that providing an empty board as a visual aid helped players. When playing against professionals in blind 9x9 Go, the computer won 2 of 3 games. In a 19x19 game against a top human player, the computer won through an unexpected, unusual move where the human made a rare mistake due to not seeing the board. Further research is needed, but playing unconventional moves seems beneficial in blind Go.
Undecidability in partially observable deterministic gamesOlivier Teytaud
Undecidability in partially observable deterministic games
Presented in Dagstuhl 2010
Accepted in IJFCS:
@article{david:hal-00710073,
hal_id = {hal-00710073},
url = {http://hal.inria.fr/hal-00710073},
title = {{The Frontier of Decidability in Partially Observable Recursive Games}},
author = {David, Auger and Teytaud, Olivier},
abstract = {{The classical decision problem associated with a game is whether a given player has a winning strategy, i.e. some strategy that leads almost surely to a victory, regardless of the other players' strategies. While this problem is relevant for deterministic fully observable games, for a partially observable game the requirement of winning with probability 1 is too strong. In fact, as shown in this paper, a game might be decidable for the simple criterion of almost sure victory, whereas optimal play (even in an approximate sense) is not computable. We therefore propose another criterion, the decidability of which is equivalent to the computability of approximately optimal play. Then, we show that (i) this criterion is undecidable in the general case, even with deterministic games (no random part in the game), (ii) that it is in the jump 0', and that, even in the stochastic case, (iii) it becomes decidable if we add the requirement that the game halts almost surely whatever maybe the strategies of the players.}},
language = {Anglais},
affiliation = {Laboratoire de Recherche en Informatique - LRI , TAO - INRIA Saclay - Ile de France},
booktitle = {{Special Issue on "Frontier between Decidability and Undecidability"}},
publisher = {World Scinet},
journal = {International Journal on Foundations of Computer Science (IJFCS)},
volume = {Accepted},
note = {revised 2011, accepted 2011, in press },
audience = {internationale },
year = {2012},
}
This document provides an overview of various concepts in software engineering, including implementation, testing, debugging, development rules, and sayings around software development. It discusses principles like debugging and maintenance taking more time than implementation, data structures being more important than codes/algorithms, avoiding premature optimization, and releasing software often for early feedback. It also covers topics such as unit testing, avoiding obese code, intellectual property, management approaches, software development methodologies, and different types of testing.
Computers have made progress playing the game of Go but still have weaknesses. In 19x19 Go, computers have beaten professionals with handicaps of 6-9 stones. In 9x9 Go, computers have reached human professional level by beating professionals without handicaps. However, in 19x19 Go computers still require at least a 6 stone handicap against top professionals. Future improvements may allow computers to reach professional human level in 19x19 Go without handicaps.
Direct policy search (DPS) is a reinforcement learning technique that directly optimizes parametric policies for decision making. Key aspects of DPS include:
(1) Defining a parametric policy with parameters that are optimized.
(2) Choosing an optimization algorithm like cross-entropy or CMA-ES to maximize the average reward of the policy over many simulations or real interactions.
(3) Applying DPS involves overloading the default policy function to define the parametric policy, and choosing an optimization algorithm to update the policy parameters.
Bias and Variance in Continuous EDA: massively parallel continuous optimizationOlivier Teytaud
The document discusses step-size adaptation rules for continuous evolutionary algorithms. It finds that the Estimation of Multivariate Normal Algorithm (EMNA) and Mutative Self-Adaptation (SSA) are much better than Cumulative Step-Size Adaptation (CSA) for parallelization, with EMNA being the simplest and most parallelizable approach. Experimental results on test functions like the sphere confirm that EMNA and SSA are twice to three times faster than CSA and achieve near linear speed-ups with increasing processor numbers.
- The document discusses energy management in France and potential areas of research collaboration between France and Taiwan.
- Key areas discussed include optimizing long-term investment policies for electricity generation using tools like reinforcement learning and stochastic programming to account for uncertainties.
- Specific questions mentioned are around optimal connections between Europe and Africa, impacts of subsidizing solar power or switching off nuclear plants, and benefits of demand reduction contracts.
- The researcher proposes combining methods like direct policy search and Monte Carlo tree search to better optimize long-term planning while accounting for short-term effects. Plans are discussed to test new ideas, share data and codes, and potentially organize joint work between the two regions.
The document discusses derivative-free optimization and evolutionary algorithms. It begins with an introduction to derivative-free optimization, explaining why it is useful when derivatives are unavailable or functions are noisy. Evolutionary algorithms are then discussed, including their fundamental elements like populations, selection, and variation operators. Specific evolutionary algorithms are presented, such as the estimation of distribution algorithm (EDA) and the (1+1)-ES algorithm with 1/5th success rule adaptation. The slides note that evolutionary algorithms are robust to noise and difficult optimization problems but are generally slower than derivative-based methods.
This document discusses blind Go, a variant of the game where players do not look at the board and must memorize positions. It explores strategies for blind Go, such as playing unusual moves that are harder for the opponent to remember. Experiments found that providing an empty board as a visual aid helped players. When playing against professionals in blind 9x9 Go, the computer won 2 of 3 games. In a 19x19 game against a top human player, the computer won through an unexpected, unusual move where the human made a rare mistake due to not seeing the board. Further research is needed, but playing unconventional moves seems beneficial in blind Go.
Undecidability in partially observable deterministic gamesOlivier Teytaud
Undecidability in partially observable deterministic games
Presented in Dagstuhl 2010
Accepted in IJFCS:
@article{david:hal-00710073,
hal_id = {hal-00710073},
url = {http://hal.inria.fr/hal-00710073},
title = {{The Frontier of Decidability in Partially Observable Recursive Games}},
author = {David, Auger and Teytaud, Olivier},
abstract = {{The classical decision problem associated with a game is whether a given player has a winning strategy, i.e. some strategy that leads almost surely to a victory, regardless of the other players' strategies. While this problem is relevant for deterministic fully observable games, for a partially observable game the requirement of winning with probability 1 is too strong. In fact, as shown in this paper, a game might be decidable for the simple criterion of almost sure victory, whereas optimal play (even in an approximate sense) is not computable. We therefore propose another criterion, the decidability of which is equivalent to the computability of approximately optimal play. Then, we show that (i) this criterion is undecidable in the general case, even with deterministic games (no random part in the game), (ii) that it is in the jump 0', and that, even in the stochastic case, (iii) it becomes decidable if we add the requirement that the game halts almost surely whatever maybe the strategies of the players.}},
language = {Anglais},
affiliation = {Laboratoire de Recherche en Informatique - LRI , TAO - INRIA Saclay - Ile de France},
booktitle = {{Special Issue on "Frontier between Decidability and Undecidability"}},
publisher = {World Scinet},
journal = {International Journal on Foundations of Computer Science (IJFCS)},
volume = {Accepted},
note = {revised 2011, accepted 2011, in press },
audience = {internationale },
year = {2012},
}
This document provides an overview of various concepts in software engineering, including implementation, testing, debugging, development rules, and sayings around software development. It discusses principles like debugging and maintenance taking more time than implementation, data structures being more important than codes/algorithms, avoiding premature optimization, and releasing software often for early feedback. It also covers topics such as unit testing, avoiding obese code, intellectual property, management approaches, software development methodologies, and different types of testing.
Computers have made progress playing the game of Go but still have weaknesses. In 19x19 Go, computers have beaten professionals with handicaps of 6-9 stones. In 9x9 Go, computers have reached human professional level by beating professionals without handicaps. However, in 19x19 Go computers still require at least a 6 stone handicap against top professionals. Future improvements may allow computers to reach professional human level in 19x19 Go without handicaps.
Direct policy search (DPS) is a reinforcement learning technique that directly optimizes parametric policies for decision making. Key aspects of DPS include:
(1) Defining a parametric policy with parameters that are optimized.
(2) Choosing an optimization algorithm like cross-entropy or CMA-ES to maximize the average reward of the policy over many simulations or real interactions.
(3) Applying DPS involves overloading the default policy function to define the parametric policy, and choosing an optimization algorithm to update the policy parameters.
Bias and Variance in Continuous EDA: massively parallel continuous optimizationOlivier Teytaud
The document discusses step-size adaptation rules for continuous evolutionary algorithms. It finds that the Estimation of Multivariate Normal Algorithm (EMNA) and Mutative Self-Adaptation (SSA) are much better than Cumulative Step-Size Adaptation (CSA) for parallelization, with EMNA being the simplest and most parallelizable approach. Experimental results on test functions like the sphere confirm that EMNA and SSA are twice to three times faster than CSA and achieve near linear speed-ups with increasing processor numbers.
- The document discusses energy management in France and potential areas of research collaboration between France and Taiwan.
- Key areas discussed include optimizing long-term investment policies for electricity generation using tools like reinforcement learning and stochastic programming to account for uncertainties.
- Specific questions mentioned are around optimal connections between Europe and Africa, impacts of subsidizing solar power or switching off nuclear plants, and benefits of demand reduction contracts.
- The researcher proposes combining methods like direct policy search and Monte Carlo tree search to better optimize long-term planning while accounting for short-term effects. Plans are discussed to test new ideas, share data and codes, and potentially organize joint work between the two regions.
The document discusses derivative-free optimization and evolutionary algorithms. It begins with an introduction to derivative-free optimization, explaining why it is useful when derivatives are unavailable or functions are noisy. Evolutionary algorithms are then discussed, including their fundamental elements like populations, selection, and variation operators. Specific evolutionary algorithms are presented, such as the estimation of distribution algorithm (EDA) and the (1+1)-ES algorithm with 1/5th success rule adaptation. The slides note that evolutionary algorithms are robust to noise and difficult optimization problems but are generally slower than derivative-based methods.
Artificial intelligence is the study of intelligent behavior and the attempt to find ways in which such behavior could be engineered in any type of artifact. There are two main types of AI - weak AI which only simulates human intelligence and strong AI which matches or exceeds human intelligence. Some key developments in AI history include McCarthy coining the term in 1955 and the creation of languages like LISP and PROLOG. Current applications of AI include speech recognition, robotics, gaming, facial recognition and use in the military and life sciences. The future of AI could see machines matching and exceeding human intelligence and potentially merging with humans. There are both advantages like efficiency but also risks like uncontrolled self-modification that come with continued advances in AI.
This document presents an overview of artificial intelligence. It discusses the history of AI from Aristotle to modern times. Key topics covered include the limitations of human mind, robotics, applications of AI in various fields, and advantages and disadvantages of AI. The document concludes by discussing the idea of artificial life and the requirement for life to have a physical form.
This document discusses perspectives on artificial intelligence and its implications for humanity. It presents views from Ray Kurzweil and David Gelernter on whether AI can achieve human-level intelligence and emotion. The document also discusses the concept of a technological singularity where computers become more intelligent than humans. It questions whether humans would need to enhance themselves with technology to maintain control over superintelligent robots or achieve longer lives. In the end, the document cautions that within 35 years, human civilization as we know it could end if superintelligent robots surpass human abilities.
Artificial Intelligence and Socially Empathetic Robotsclairey08
The document discusses artificial intelligence and social aspects of robotics. It describes Zeno, an interactive robotic companion that can engage in conversation and convey emotion. It also discusses the "uncanny valley" theory that robots made to look very human can seem grotesque. The document outlines the KISMET robot project which aims to develop robots that can interact cooperatively with humans. It raises questions about whether AI machines could replace humans in the workplace and how to ensure their safe and reliable operation according to programming.
Artificial intelligence and intelligent systemJAKA Pradana
Teks tersebut membahas perkembangan bidang Artificial Intelligence (AI) dan sub-bidangnya seperti machine learning, knowledge representation, dan planning sejak tahun 1970-an hingga 2000-an. Pada awalnya, berbagai sub-bidang tersebut masih tergabung dalam AI, namun kemudian mulai memisahkan diri dan berfokus pada bidang masing-masing. Hal ini menyebabkan penelitian AI kehilangan fokus pada tujuan awal untuk membangun sistem cerdas seperti
Artificial Intelligence AI Topics History and Overviewbutest
The document discusses the history and concepts of artificial intelligence including machine learning. It provides definitions of key AI terms and overview of famous early AI programs. It also discusses machine learning methods and applications, dimensions of machine learning study, and issues in the field. Games AI is explored through techniques like min-max trees used in chess programs. The Turing Test proposal and its problems/proposed modifications are summarized.
This document discusses the intersections between artificial intelligence (AI) and software engineering (SE). It provides an overview of the disciplines of AI and SE, noting their origins and goals. It describes several areas of research at the intersection of the two fields, including software agents, knowledge-based systems, ambient intelligence, and computational intelligence. It also discusses conferences and workshops focused on bringing together AI and SE researchers.
Artificial Intelligence and Optimization with ParallelismOlivier Teytaud
This document discusses parallelism in artificial intelligence and evolutionary computation. It explains that comparison-based optimization algorithms, which include many evolutionary algorithms, can be naturally parallelized by speculatively running multiple branches in parallel with a branching factor of 3 or more. This allows theoretical logarithmic speedups to be achieved in practice through simple parallelization tricks.
Artificial Inteligence for Games an Overview SBGAMES 2012Bruno Duarte Corrêa
This document provides an overview of artificial intelligence concepts for games. It discusses navigation techniques like waypoints, navmeshes and pathfinding using A* and steering behaviors. It also covers perceptions, finite state machines, behavior trees, planning using GOAP and learning approaches like neural networks and genetic algorithms. Specific examples are given for how AI was implemented in the game Left 4 Dead to deliver robust behaviors, provide competent human proxies and generate dramatic pacing through an adaptive director algorithm.
Artificial Intelligence Based Mutual Authentication Technique with Four Entit...IDES Editor
4-G mobile communications system has utilized
high speed data communications technology having
connectivity to all sorts of networks including 2-G and 3-G
mobile networks. Authentication of mobile subscribers and
networks are a prime criterion to check and minimize security
threats and attacks. An artificial intelligence based mutual
authentication system with four entities is proposed. A person
talking salutation or greeting words in different times are
always consisting of a very narrow range of frequencies which
are varying in nature from person to person. Voice frequency
of the salutation or selective words used by a subscriber like
Hello, Good Morning etc is taken as first entity. Second entity
is chosen as frequency of flipping or clapping sound of the
calling subscriber. Then third entity is taken as face image of
the calling subscriber. Fourth entity is granted as probability
of salutation or greeting word from subscriber’s talking habit
(set of salutation words) while initializing a call. These four
entities such as probability of particular range of frequencies
for the salutation word, frequency of flipping sound, face
image matching of the subscriber, particular salutation or
greeting word at the time of starting a call are used with most
frequently, more frequently and less frequently by the calling
subscriber like uncertainty in Artificial Intelligence (AI). Now
different relative grades are assigned for most frequently,
more frequently and less frequently used parameters and the
grades are modified according to the assumed weightage. A
Fuzzy Rule (condition) by Fuzzy operation is invented. If the
results obtained from fuzzy operations are satisfied by the
fuzzy rule, the subscriber (MS) and the network (Switch or
Server) are mutually authenticated in 4-G mobile
communications.
Artificial intelligence (AI) is defined as the scientific understanding of intelligent behavior and its implementation in machines. AI aims to create computer systems that can perform tasks requiring intelligence, such as understanding language, learning from experience, reasoning and problem solving. While modern AI has achieved impressive performance on specific tasks, it has not yet reached human-level intelligence in general or the ability to learn from experience like children. Researchers continue to study both computational and biological aspects of intelligence to develop more capable AI systems.
Artificial Intelligence in High Content Screening and Cervical Cancer DiagnosisUniversity of Zurich
Artficial Intelligence (AI) studies and designs intelligent agents, e.g. systems that perceive their environment and take actions that maximize the chances of success. In microscopy a success is often understood when the automated image analysis effciently detects phenotypes, as in biological screens, or retrieves a diagnostically relevant statistics from images, as in the medical diagnosis. During the talk I will present two applications aimed at supporting high-content screening and diagnosis of cervical cancer where subdomains of AI, e.g. Evolutionary Algorithm, Neural Networks and Machine Learning techniques were applied.
In the realm of cybersecurity, offensive security practices act as a critical shield. By simulating real-world attacks in a controlled environment, these techniques expose vulnerabilities before malicious actors can exploit them. This proactive approach allows manufacturers to identify and fix weaknesses, significantly enhancing system security.
This presentation delves into the development of a system designed to mimic Galileo's Open Service signal using software-defined radio (SDR) technology. We'll begin with a foundational overview of both Global Navigation Satellite Systems (GNSS) and the intricacies of digital signal processing.
The presentation culminates in a live demonstration. We'll showcase the manipulation of Galileo's Open Service pilot signal, simulating an attack on various software and hardware systems. This practical demonstration serves to highlight the potential consequences of unaddressed vulnerabilities, emphasizing the importance of offensive security practices in safeguarding critical infrastructure.
Skybuffer SAM4U tool for SAP license adoptionTatiana Kojar
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
Taking AI to the Next Level in Manufacturing.pdfssuserfac0301
Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as:
1. How quickly AI is being implemented in manufacturing.
2. Which barriers stand in the way of AI adoption.
3. How data quality and governance form the backbone of AI.
4. Organizational processes and structures that may inhibit effective AI adoption.
6. Ideas and approaches to help build your organization's AI strategy.
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Driving Business Innovation: Latest Generative AI Advancements & Success StorySafe Software
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsDianaGray10
Join us to learn how UiPath Apps can directly and easily interact with prebuilt connectors via Integration Service--including Salesforce, ServiceNow, Open GenAI, and more.
The best part is you can achieve this without building a custom workflow! Say goodbye to the hassle of using separate automations to call APIs. By seamlessly integrating within App Studio, you can now easily streamline your workflow, while gaining direct access to our Connector Catalog of popular applications.
We’ll discuss and demo the benefits of UiPath Apps and connectors including:
Creating a compelling user experience for any software, without the limitations of APIs.
Accelerating the app creation process, saving time and effort
Enjoying high-performance CRUD (create, read, update, delete) operations, for
seamless data management.
Speakers:
Russell Alfeche, Technology Leader, RPA at qBotic and UiPath MVP
Charlie Greenberg, host
Artificial intelligence is the study of intelligent behavior and the attempt to find ways in which such behavior could be engineered in any type of artifact. There are two main types of AI - weak AI which only simulates human intelligence and strong AI which matches or exceeds human intelligence. Some key developments in AI history include McCarthy coining the term in 1955 and the creation of languages like LISP and PROLOG. Current applications of AI include speech recognition, robotics, gaming, facial recognition and use in the military and life sciences. The future of AI could see machines matching and exceeding human intelligence and potentially merging with humans. There are both advantages like efficiency but also risks like uncontrolled self-modification that come with continued advances in AI.
This document presents an overview of artificial intelligence. It discusses the history of AI from Aristotle to modern times. Key topics covered include the limitations of human mind, robotics, applications of AI in various fields, and advantages and disadvantages of AI. The document concludes by discussing the idea of artificial life and the requirement for life to have a physical form.
This document discusses perspectives on artificial intelligence and its implications for humanity. It presents views from Ray Kurzweil and David Gelernter on whether AI can achieve human-level intelligence and emotion. The document also discusses the concept of a technological singularity where computers become more intelligent than humans. It questions whether humans would need to enhance themselves with technology to maintain control over superintelligent robots or achieve longer lives. In the end, the document cautions that within 35 years, human civilization as we know it could end if superintelligent robots surpass human abilities.
Artificial Intelligence and Socially Empathetic Robotsclairey08
The document discusses artificial intelligence and social aspects of robotics. It describes Zeno, an interactive robotic companion that can engage in conversation and convey emotion. It also discusses the "uncanny valley" theory that robots made to look very human can seem grotesque. The document outlines the KISMET robot project which aims to develop robots that can interact cooperatively with humans. It raises questions about whether AI machines could replace humans in the workplace and how to ensure their safe and reliable operation according to programming.
Artificial intelligence and intelligent systemJAKA Pradana
Teks tersebut membahas perkembangan bidang Artificial Intelligence (AI) dan sub-bidangnya seperti machine learning, knowledge representation, dan planning sejak tahun 1970-an hingga 2000-an. Pada awalnya, berbagai sub-bidang tersebut masih tergabung dalam AI, namun kemudian mulai memisahkan diri dan berfokus pada bidang masing-masing. Hal ini menyebabkan penelitian AI kehilangan fokus pada tujuan awal untuk membangun sistem cerdas seperti
Artificial Intelligence AI Topics History and Overviewbutest
The document discusses the history and concepts of artificial intelligence including machine learning. It provides definitions of key AI terms and overview of famous early AI programs. It also discusses machine learning methods and applications, dimensions of machine learning study, and issues in the field. Games AI is explored through techniques like min-max trees used in chess programs. The Turing Test proposal and its problems/proposed modifications are summarized.
This document discusses the intersections between artificial intelligence (AI) and software engineering (SE). It provides an overview of the disciplines of AI and SE, noting their origins and goals. It describes several areas of research at the intersection of the two fields, including software agents, knowledge-based systems, ambient intelligence, and computational intelligence. It also discusses conferences and workshops focused on bringing together AI and SE researchers.
Artificial Intelligence and Optimization with ParallelismOlivier Teytaud
This document discusses parallelism in artificial intelligence and evolutionary computation. It explains that comparison-based optimization algorithms, which include many evolutionary algorithms, can be naturally parallelized by speculatively running multiple branches in parallel with a branching factor of 3 or more. This allows theoretical logarithmic speedups to be achieved in practice through simple parallelization tricks.
Artificial Inteligence for Games an Overview SBGAMES 2012Bruno Duarte Corrêa
This document provides an overview of artificial intelligence concepts for games. It discusses navigation techniques like waypoints, navmeshes and pathfinding using A* and steering behaviors. It also covers perceptions, finite state machines, behavior trees, planning using GOAP and learning approaches like neural networks and genetic algorithms. Specific examples are given for how AI was implemented in the game Left 4 Dead to deliver robust behaviors, provide competent human proxies and generate dramatic pacing through an adaptive director algorithm.
Artificial Intelligence Based Mutual Authentication Technique with Four Entit...IDES Editor
4-G mobile communications system has utilized
high speed data communications technology having
connectivity to all sorts of networks including 2-G and 3-G
mobile networks. Authentication of mobile subscribers and
networks are a prime criterion to check and minimize security
threats and attacks. An artificial intelligence based mutual
authentication system with four entities is proposed. A person
talking salutation or greeting words in different times are
always consisting of a very narrow range of frequencies which
are varying in nature from person to person. Voice frequency
of the salutation or selective words used by a subscriber like
Hello, Good Morning etc is taken as first entity. Second entity
is chosen as frequency of flipping or clapping sound of the
calling subscriber. Then third entity is taken as face image of
the calling subscriber. Fourth entity is granted as probability
of salutation or greeting word from subscriber’s talking habit
(set of salutation words) while initializing a call. These four
entities such as probability of particular range of frequencies
for the salutation word, frequency of flipping sound, face
image matching of the subscriber, particular salutation or
greeting word at the time of starting a call are used with most
frequently, more frequently and less frequently by the calling
subscriber like uncertainty in Artificial Intelligence (AI). Now
different relative grades are assigned for most frequently,
more frequently and less frequently used parameters and the
grades are modified according to the assumed weightage. A
Fuzzy Rule (condition) by Fuzzy operation is invented. If the
results obtained from fuzzy operations are satisfied by the
fuzzy rule, the subscriber (MS) and the network (Switch or
Server) are mutually authenticated in 4-G mobile
communications.
Artificial intelligence (AI) is defined as the scientific understanding of intelligent behavior and its implementation in machines. AI aims to create computer systems that can perform tasks requiring intelligence, such as understanding language, learning from experience, reasoning and problem solving. While modern AI has achieved impressive performance on specific tasks, it has not yet reached human-level intelligence in general or the ability to learn from experience like children. Researchers continue to study both computational and biological aspects of intelligence to develop more capable AI systems.
Artificial Intelligence in High Content Screening and Cervical Cancer DiagnosisUniversity of Zurich
Artficial Intelligence (AI) studies and designs intelligent agents, e.g. systems that perceive their environment and take actions that maximize the chances of success. In microscopy a success is often understood when the automated image analysis effciently detects phenotypes, as in biological screens, or retrieves a diagnostically relevant statistics from images, as in the medical diagnosis. During the talk I will present two applications aimed at supporting high-content screening and diagnosis of cervical cancer where subdomains of AI, e.g. Evolutionary Algorithm, Neural Networks and Machine Learning techniques were applied.
In the realm of cybersecurity, offensive security practices act as a critical shield. By simulating real-world attacks in a controlled environment, these techniques expose vulnerabilities before malicious actors can exploit them. This proactive approach allows manufacturers to identify and fix weaknesses, significantly enhancing system security.
This presentation delves into the development of a system designed to mimic Galileo's Open Service signal using software-defined radio (SDR) technology. We'll begin with a foundational overview of both Global Navigation Satellite Systems (GNSS) and the intricacies of digital signal processing.
The presentation culminates in a live demonstration. We'll showcase the manipulation of Galileo's Open Service pilot signal, simulating an attack on various software and hardware systems. This practical demonstration serves to highlight the potential consequences of unaddressed vulnerabilities, emphasizing the importance of offensive security practices in safeguarding critical infrastructure.
Skybuffer SAM4U tool for SAP license adoptionTatiana Kojar
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
Taking AI to the Next Level in Manufacturing.pdfssuserfac0301
Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as:
1. How quickly AI is being implemented in manufacturing.
2. Which barriers stand in the way of AI adoption.
3. How data quality and governance form the backbone of AI.
4. Organizational processes and structures that may inhibit effective AI adoption.
6. Ideas and approaches to help build your organization's AI strategy.
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Driving Business Innovation: Latest Generative AI Advancements & Success StorySafe Software
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsDianaGray10
Join us to learn how UiPath Apps can directly and easily interact with prebuilt connectors via Integration Service--including Salesforce, ServiceNow, Open GenAI, and more.
The best part is you can achieve this without building a custom workflow! Say goodbye to the hassle of using separate automations to call APIs. By seamlessly integrating within App Studio, you can now easily streamline your workflow, while gaining direct access to our Connector Catalog of popular applications.
We’ll discuss and demo the benefits of UiPath Apps and connectors including:
Creating a compelling user experience for any software, without the limitations of APIs.
Accelerating the app creation process, saving time and effort
Enjoying high-performance CRUD (create, read, update, delete) operations, for
seamless data management.
Speakers:
Russell Alfeche, Technology Leader, RPA at qBotic and UiPath MVP
Charlie Greenberg, host
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/temporal-event-neural-networks-a-more-efficient-alternative-to-the-transformer-a-presentation-from-brainchip/
Chris Jones, Director of Product Management at BrainChip , presents the “Temporal Event Neural Networks: A More Efficient Alternative to the Transformer” tutorial at the May 2024 Embedded Vision Summit.
The expansion of AI services necessitates enhanced computational capabilities on edge devices. Temporal Event Neural Networks (TENNs), developed by BrainChip, represent a novel and highly efficient state-space network. TENNs demonstrate exceptional proficiency in handling multi-dimensional streaming data, facilitating advancements in object detection, action recognition, speech enhancement and language model/sequence generation. Through the utilization of polynomial-based continuous convolutions, TENNs streamline models, expedite training processes and significantly diminish memory requirements, achieving notable reductions of up to 50x in parameters and 5,000x in energy consumption compared to prevailing methodologies like transformers.
Integration with BrainChip’s Akida neuromorphic hardware IP further enhances TENNs’ capabilities, enabling the realization of highly capable, portable and passively cooled edge devices. This presentation delves into the technical innovations underlying TENNs, presents real-world benchmarks, and elucidates how this cutting-edge approach is positioned to revolutionize edge AI across diverse applications.
Conversational agents, or chatbots, are increasingly used to access all sorts of services using natural language. While open-domain chatbots - like ChatGPT - can converse on any topic, task-oriented chatbots - the focus of this paper - are designed for specific tasks, like booking a flight, obtaining customer support, or setting an appointment. Like any other software, task-oriented chatbots need to be properly tested, usually by defining and executing test scenarios (i.e., sequences of user-chatbot interactions). However, there is currently a lack of methods to quantify the completeness and strength of such test scenarios, which can lead to low-quality tests, and hence to buggy chatbots.
To fill this gap, we propose adapting mutation testing (MuT) for task-oriented chatbots. To this end, we introduce a set of mutation operators that emulate faults in chatbot designs, an architecture that enables MuT on chatbots built using heterogeneous technologies, and a practical realisation as an Eclipse plugin. Moreover, we evaluate the applicability, effectiveness and efficiency of our approach on open-source chatbots, with promising results.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
"Choosing proper type of scaling", Olena SyrotaFwdays
Imagine an IoT processing system that is already quite mature and production-ready and for which client coverage is growing and scaling and performance aspects are life and death questions. The system has Redis, MongoDB, and stream processing based on ksqldb. In this talk, firstly, we will analyze scaling approaches and then select the proper ones for our system.
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...Alex Pruden
Folding is a recent technique for building efficient recursive SNARKs. Several elegant folding protocols have been proposed, such as Nova, Supernova, Hypernova, Protostar, and others. However, all of them rely on an additively homomorphic commitment scheme based on discrete log, and are therefore not post-quantum secure. In this work we present LatticeFold, the first lattice-based folding protocol based on the Module SIS problem. This folding protocol naturally leads to an efficient recursive lattice-based SNARK and an efficient PCD scheme. LatticeFold supports folding low-degree relations, such as R1CS, as well as high-degree relations, such as CCS. The key challenge is to construct a secure folding protocol that works with the Ajtai commitment scheme. The difficulty, is ensuring that extracted witnesses are low norm through many rounds of folding. We present a novel technique using the sumcheck protocol to ensure that extracted witnesses are always low norm no matter how many rounds of folding are used. Our evaluation of the final proof system suggests that it is as performant as Hypernova, while providing post-quantum security.
Paper Link: https://eprint.iacr.org/2024/257
Dandelion Hashtable: beyond billion requests per second on a commodity serverAntonios Katsarakis
This slide deck presents DLHT, a concurrent in-memory hashtable. Despite efforts to optimize hashtables, that go as far as sacrificing core functionality, state-of-the-art designs still incur multiple memory accesses per request and block request processing in three cases. First, most hashtables block while waiting for data to be retrieved from memory. Second, open-addressing designs, which represent the current state-of-the-art, either cannot free index slots on deletes or must block all requests to do so. Third, index resizes block every request until all objects are copied to the new index. Defying folklore wisdom, DLHT forgoes open-addressing and adopts a fully-featured and memory-aware closed-addressing design based on bounded cache-line-chaining. This design offers lock-free index operations and deletes that free slots instantly, (2) completes most requests with a single memory access, (3) utilizes software prefetching to hide memory latencies, and (4) employs a novel non-blocking and parallel resizing. In a commodity server and a memory-resident workload, DLHT surpasses 1.6B requests per second and provides 3.5x (12x) the throughput of the state-of-the-art closed-addressing (open-addressing) resizable hashtable on Gets (Deletes).
Leveraging the Graph for Clinical Trials and Standards
Artificial intelligence and the game of Go
1. Bandit-based Monte-Carlo planning: the game
of Go and beyond
The game of Go: recent
progress for an old game
Olivier.Teytaud@inria.fr + too many people for being all cited. Includes Inria, Cnrs, Univ.
Paris-Sud, LRI, CMAP, Univ. Amsterdam, Taiwan universities (including NUTN)
TAO, Inria-Saclay IDF, Cnrs 8623, Lri, Univ. Paris-Sud,
Digiteo Labs, Pascal Network of Excellence.
Paris,
April 2010.
2. Outline: the game of Go
The history
The rules
Variants of go and complexity
Computers playing Go
Go and power plants
3. Outline: the game of Go
The history
The rules
Variants of go and complexity
Computers playing Go
Go and power plants
4. History: I'm not an expert
Origins: I don't know. Too many dates in
the literature. Someone knows ?
8th century: the game of Go in Japan ?
9th century: symmetric game ?
16th century: first schools ?
Recently:
huge progress thanks to cultural differences in
teaching ?
becomes known in Europe (cf interest for Asian
cultures and Ikaru No Go)
5. Outline: the game of Go
The history
The rules
Variants of go and complexity
Computers playing Go
Go and power plants
6. Rules
Only recently
formalized in a
mathematical
sense.
For some rules:
winner not always clearly defined (comment
by a strong japanese friend: in asian
cultures this is not so important).
Recently: “komi” modified, superko adapted
so that no draw.
Time settings get smaller and smaller (TV +
younger people)
7. Rules
Only recently
formalized in a
mathematical
sense.
For some rules:
winner not always clearly defined (comment
by a strong japanese friend: in asian
cultures this is not so important).
Recently: “komi” modified, superko adapted
so that no draw.
Time settings get smaller and smaller (TV +
younger people)
8. Rules
Only recently
formalized in a
mathematical
sense.
For some rules:
winner not always clearly defined (comment
by a strong japanese friend: in asian
cultures this is not so important).
Recently: “komi” modified, superko adapted
so that no draw.
Time settings get smaller and smaller (TV +
younger people)
9. Rules
Only recently
formalized in a
mathematical
sense.
For some rules:
winner not always clearly defined (comment
by a strong japanese friend: in asian
cultures this is not so important).
Recently: “komi” modified, superko adapted
so that no draw.
Time settings get smaller and smaller (TV +
younger people)
10. Game of Go: the rules
Black plays at the blue circle: the
white group dies (it is removed)
It's impossible to kill white (two “eyes”).
“Ko” rules: we don't come back to the same situation.
At the end, we count territories
==> black starts, so +7.5 for white.
18. Game of Go: counting territories
(white has 7.5 “bonus” as black starts)
19. Outline: the game of Go
The history
The rules
Variants of go and complexity
Computers playing Go
Go and power plants
20. Introduction to games
Partially or fully observable
Randomized or not
Iterated or not
1,2,3,... players
Decentralized or not
Continuous or not
Infinite time or not
21. Complexity measures
(not always well defined)
State-space complexity
Combinatorial
Game-tree size complexity
Decision complexity measures
Game-tree complexity
Computational complexity Computational
complexity
Perfect-play complexity measures
State of the art level
22. Complexity measures
(not always well defined)
State-space complexity = number
of possible states
Game-tree size
Decision complexity
Game-tree complexity
Computational complexity
Perfect-play complexity
State of the art level
23. Complexity measures
(not always well defined)
State-space complexity
Game-tree size = number of leafs
Decision complexity
Game-tree complexity
Computational complexity
Perfect-play complexity
State of the art level
24. Complexity measures
(not always well defined)
State-space complexity
Game-tree size
Decision complexity = min # of
leafs of tree showing perfect play
Game-tree complexity
Computational complexity
Perfect-play complexity
State of the art level
25. Complexity measures
(not always well defined)
State-space complexity
Game-tree size
Decision complexity
Game-tree complexity = # of leafs
for perfect play with constant depth
Computational complexity
Perfect-play complexity
State of the art level
26. Complexity measures
(not always well defined)
State-space complexity
Game-tree size
Decision complexity
Game-tree complexity
Computational complexity (=
complexity classes, later)
Perfect-play complexity
State of the art level
27. Computational complexity:
Main reasons for this measure ?
Good feeling of understanding
(disagree if you want :-) )
Explicit families of problems
(extracted by reduction)
Fun
Connections
with classical complexity measures
Much better for looking clever
(when you speak about NP-complete
problems you look clever)
29. Computational complexity
Given a class X, a problem q can be
in X
or harder than pbs in X (X-hard)
or both (X-complete)
or neither
NP
NP -difficile
NP -complete
30. Computational complexity
For evaluating the complexity of your
problem:
1. Generalize your game to any size
(non trivial for chess)
2. Consider the problem:
- here is a board
- is the situation a win in perfect play ?
NP
NP
NP -complete -difficile
31. Computational complexity
==> cast into a decision problem (binary question)
==> can be used for choosing optimal move
(but not necessary)
==> trivial games can be EXPTIME-hard
==> no clear correlation with the fact that a game is difficult
for a computer (when compared to humans)
NP
NP
NP -complete -difficile
33. PSPACE vs EXPTIME
==> many important games are either PSPACE or EXPTIME
Theorem: If playing = filling a location
for eternity, then it is PSPACE.
(not necessarily PSPACE-complete!)
Proof: Depth-first search.
Applis: Hex, Havannah, Tic-Tac-Toe,
Ponnuki-Go...
35. NP / PSPACE / EXPTIME in Go
Tsumegos with no ko, forced moves only for
W, 2 moves for B, polynomial length: NP-
complete
Ponnuki-Go : PSPACE
Go without ko: PSPACE-hard
Go with ko + japanese rules:
EXPTIME-complete
Go with ko + superko: unknown
Some phantom-rengo undecidable ?
If Go with ko > Go without ko, then
PSPACE EXPTIME
36. Complexity measures
(not always well defined)
State-space complexity
Game-tree size
Decision complexity
Game-tree complexity
Computational complexity
Perfect-play complexity (complexity
of perfect algorithm)
State of the art level
37. Complexity measures
(not always well defined)
State-space complexity
Game-tree size
Decision complexity
Game-tree complexity
Computational complexity
Perfect-play complexity
State of the art level
38. State of the art level
Very weak solving
Means that we know who should win
Typically proved by strategy-stealing
E.g.: hex (first player wins), hex + swap
(second player wins)
Weak solving
Strong solving
Best results so far
39. State of the art level
Very weak solving
Weak solving
Perfect play reached with reasonnable computation
time
Biggest success: draughts (tenths of years of
computation on tenths of machines)
Strong solving
Best results so far
40. State of the art level
Very weak solving
Weak solving
Strong solving
Perfect play from any situation in
reasonable time (variants of Tic-Tac-Toe)
Best results so far
41. State of the art level
Very weak solving
Weak solving
Strong solving
Best results so far
Shi-Fu-Mi: humans loose
English draughts: humans + machines reach perfect
play
Chess: nobody can compete with machines
Ponnuki-Go: some variants solved
9x9 Go: MoGoTW won with the disadvantageous side
with a top player
42. Go: from 29 to 6 stones
1998: loss against amateur (6d) 19x19 H29
2008: win against a pro (8p) 19x19, H9 MoGo
2008: win against a pro (4p) 19x19, H8 CrazyStone
2008: win against a pro (4p) 19x19, H7 CrazyStone
2009: win against a pro (9p) 19x19, H7 MoGo
2009: win against a pro (1p) 19x19, H6 MoGo
2007: win against a pro (5p) 9x9 (blitz) MoGo
2008: win against a pro (5p) 9x9 white MoGo
2009: win against a pro (5p) 9x9 black MoGo
2009: win against a pro (9p) 9x9 white Fuego
2009: win against a pro (9p) 9x9 black MoGoTW
==> still 6 stones at least!
43. Outline: the game of Go
The history
The rules
Variants of go and complexity
Computers playing Go
Go and power plants
44. Monte-Carlo Tree Search
Monte-Carlo Tree Search (MCTS) appeared
in games.
Its most well-known variant is termed Upper
Confidence Tree (UCT).
I here present UCT.
Bandits;
Monte-Carlo approach for tree-search;
UCT.
45. A ``bandit'' problem
p1,...,pN unknown probabilities ∈ [0,1]
At each time step i∈ [1,n]
choose ui∈ {1,...,N} (as a function of uj and rj, j<i)
With probability pui
win ( ri=1 )
loose ( ri=0 )
46. A ``bandit'' problem: the target
p1,...,pN unknown probabilities ∈ [0,1]
At each time step i∈ [1,n]
choose ui∈ {1,...,N} (as a function of uj and rj, j<i)
With probability pui
win ( ri=1 )
loose ( ri=0 )
Regret: Rn=n max{pi} - ∑ rj (j<n)
How to minimize the regret (worst case on p) ?
47. Bandits – a classical solution
Regret: Rn=n max{pi} - ∑ rj (j<i)
UCB1: Choose u maximizing the compromise:
Empirical average for decision u
+ √( log(i)/ number of trials with decision u )
==> optimal regret O(log(n))
(Lai et al; Auer et al)
48. Infinite bandit: progressive
widening
UCB1: Choose u maximizing the compromise:
Empirical average for decision u
+ √( log(i)/ number of trials with decision u )
==> argmax only on the i first arms
( [ 0.25 0.5 ] )
(Coulom, Chaslot et al, Wang et al)
49. Bandits: much more
What is a bandit:
- a criterion (here a bandit)
defines the problem
- usually a score (typically
exploration+exploitation)
defines a criterion
==> an optimal score for a criterion is not optimal
for another ==> a wide literature
50. Bandits and trees
- we have seen the
definition of discrete
time control problems;
- we have seen what are
bandits
- we now introduce trees and UCT
62. Go: from 29 to 6 stones
Formula for
simulation
Asymptotically optimal move.
But all the tree is visited infinitely often!
What is used in implementations which work ?
63. Go: from 29 to 6 stones
Formula for
simulation
64. Go: from 29 to 6 stones
Formula for
simulation
Not consistent! Sometimes:
- Good move might have 0/1
- Bad move 1/(N-1) after N simulations
==> we only simulate bad move!
65. Go: from 29 to 6 stones
Formula for
simulation
Other (better) estimates,
but still inconsistent
66. Go: from 29 to 6 stones
Formula for
simulation
nbWins + 1
argmax ---------------
nbLosses + 2
==> consistency
==> frugality
67. Outline
Discrete time control: various approaches
Monte-Carlo Tree Search (UCT, MCTS; 2006)
Extensions
Weakness
Games as benchmarks ?
70. Why UCT is suboptimal for games ?
(boring version)
71. Why UCT is suboptimal for games ?
(clear version)
Monte-Carlo Tree Search, under mild
conditions on games (including
deterministic two-player zero-sum
games), can be
consistent (→ best move);
frugal (if there is a good move, it does not
visit infinitely often all the tree).
72. Why UCT is suboptimal for games ?
(clear version)
Frugal algorithms: folklore results (many
people implement “frugal” MCTS).
However, these algorithms are (usually)
not consistent.
What is new is
sufficient
conditions for
consistency + frugal.
73. Extensions
``Standard'' UCT:
score(situation,move) = compromise (in [0,1+] )
between
a) empirical quality
P ( win | nextMove(situation) = move )
estimated in simulations
b) exploration term
(UCT is not fundamental in Go)
Remarks:
1) No offline learning
2) No learning from one situation to another
3) No expert rules
74. Extension 1: offline learning
score(situation,move) = compromise between
a) empirical quality
b) exploration term
c) offline value (Chaslot et al, Coulom) =
empirical estimate P ( played | pattern )
for patterns with big support
==> estimated on database
At first, (c) is the most important; later, (a) dominates.
75. Extensions
``Standard'' UCT:
score(situation,move) = compromise between
a) empirical quality
b) exploration term
Remarks:
1) No offline learning
2) No learning from one situation to another
3) No expert rules
76. Extension 2: transient values
score(situation,move) = compromise between
a) empirical quality
P' ( win | nextMove(situation) = move )
estimated in simulations
b) exploration term
c) offline value
d) ``transient'' value: (Gelly et al, 07)
P' (win | same player plays “move” later)
==> brings information from node N to ancestor node M
==> does not bring information from node N to
descendants or cousins (many people have tried)
77. Extensions
``Standard'' UCT:
score(situation,move) = compromise between
a) empirical quality
b) exploration term
Remarks:
1) No offline learning
2) No learning from one situation to another
3) No expert rules
78. Extension 3: expert rules
score(situation,move) = compromise between
a) empirical quality
b) exploration term
c) offline value
d) transient value
e) expert rules
==> empirically derived linear combination
Most important terms,
(e)+(c) first,
then (d) becomes stronger,
finally (a) only
79. Go: from 29 to 6 stones
2007: win against a pro (5p) 9x9 (blitz) MoGo
2008: win against a pro (5p) 9x9 white MoGo
2009: win against a pro (5p) 9x9 black MoGo
2009: win against a pro (9p) 9x9 white Fuego
2009: win against a pro (9p) 9x9 black MoGoTW
2008: win against a pro (8p) 19x19, H9 MoGo
2008: win against a pro (4p) 19x19, H8 CrazyStone
2008: win against a pro (4p) 19x19, H7 CrazyStone
2009: win against a pro (9p) 19x19, H7 MoGo
2009: win against a pro (1p) 19x19, H6 MoGo
==> still 6 stones at least!
81. A trivial semeai
Plenty of equivalent
situations!
They are randomly
sampled, with
no generalization.
50% of estimated
win probability!
82. Semeai
Plenty of equivalent
situations!
They are randomly
sampled, with
no generalization.
50% of estimated
win probability!
83. Semeai
Plenty of equivalent
situations!
They are randomly
sampled, with
no generalization.
50% of estimated
win probability!
84. Semeai
Plenty of equivalent
situations!
They are randomly
sampled, with
no generalization.
50% of estimated
win probability!
85. Semeai
Plenty of equivalent
situations!
They are randomly
sampled, with
no generalization.
50% of estimated
win probability!
86. Semeai
Plenty of equivalent
situations!
They are randomly
sampled, with
no generalization.
50% of estimated
win probability!
87. Semeai
Plenty of equivalent
situations!
They are randomly
sampled, with
no generalization.
50% of estimated
win probability!
88. Semeai
Plenty of equivalent
situations!
They are randomly
sampled, with
no generalization.
50% of estimated
win probability!
89. A trivial semeai
Plenty of equivalent
situations!
They are randomly
sampled, with
no generalization.
50% of estimated
win probability!
90. A trivial semeai
Plenty of equivalent
situations!
They are randomly
sampled, with
no generalization.
50% of estimated
win probability!
91. A trivial semeai
Plenty of equivalent
situations!
They are randomly
sampled, with
no generalization.
50% of estimated
win probability!
92. It does not work. Why ?
50% of estimated
win probability!
In the first node:
The first simulations give ~ 50%
The next simulations go to 100% or 0% (depending
on the chosen move)
But, then, we switch to another node
(~ 8! x 8! such nodes)
93. And the humans ?
50% of estimated
win probability!
In the first node:
The first simulations give ~ 50%
The next simulations go to 100% or 0% (depending
on the chosen move)
But, then, we DON'T switch to another node
98. Outline: the game of Go
The history
The rules
Variants of go and complexity
Computers playing Go
Go and power plants
99. What is high-dimensional
discrete time control ?
There are time steps: 0, 1, 2, ..., H.
There are states and transitions:
xi+1 = f( xi, di)
di is the decision at time step i:
di=u(xi)
There is a cost:
C = C(xH)
==> We look for u(.) such that C is as
small as possible.
100. High-dimensional discrete time
control
xi+1 = f( xi, di)
di=u(xi)
C = C(xH)
==> We look for u(.) such that C is as
small as possible.
101. Discrete time + high dimension
+ uncertainty
xi+1 = f( xi, di, Ai) di=u(xi)
C = C(xH)
Ai might be:
- a Markov model
- an opponent: Ai maximizes inf C
==> we look for u(.) such that C is as small
as possible (e.g. on average).
102. Summary
High dimensional discrete time control is an
important problem
Many problems have no satisfactory
solution.
A new approach: Bandit-Based Monte-Carlo
Planning
103. High-dimensional discrete time
control
A main application: the management of
many energy stocks in front of randomness
At each time step we see random outcomes
We have to take decisions
After H time steps, we observe a cost
104. What are the approaches ?
Dynamic programming (Massé – Bellman 50's)
(still the main approach in industry)
Reinforcement learning (some promising results,
less used in industry)
Some tree exploration tools (less usual in
stochastic or continuous cases)
Bandit-Based Monte-Carlo planning (MCTS/UCT)
105. What are the approaches ?
Dynamic programming (Massé – Bellman 50's)
(still the main approach in industry)
Where we are:
Done: Presentation of the problem.
Now: We briefly present dynamic
programming
Thereafter: We present MCTS / UCT.
106. Dynamic programming
V(x) = expectation of C(xH) if optimal
strategy. (well defined)
u(x) such that the expectation of
V(f(x,u(x),A)) is minimal
Computation by dynamic programming
We compute V for all the X with horizon H.
107. Dynamic programming
V(x) = expectation of C(xH) if optimal
strategy. (well defined)
u(x) such that the expectation of
V(f(x,u(x),A)) is minimal
Computation by dynamic programming
We compute V for all the X with horizon H.
We compute V for all the X with horizon H-1.
108. Dynamic programming
V(x) = expectation of C(xH) if optimal
strategy. (well defined)
u(x) such that the expectation of
V(f(x,u(x),A)) is minimal
Computation by dynamic programming
We compute V for all the X with horizon H.
We compute V for all the X with horizon H-1.
... ... ...
109. Extensions
Approximate dynamic programming (e.g.
for continuous domains)
Reinforcement learning
Case where f(...) or A is black-box
Huge state spaces
==> but lack of stability
...
==> there is room for improvements
110. Conclusion : games = great for
artificial intelligence
Very difficult
for computers.
114. “Real” games
Assumption: if a computer understands and guesses spins, then
this robot will be efficient for something else than just games.
(holds true for Go)
Frédéric Lemoine MIG 11/07/2008 114
115. “Real” games
Assumption: if a computer understands and guesses spins, then
this robot will be efficient for something else than just games.
VS
Frédéric Lemoine MIG 11/07/2008 115
117. Conclusion
Essentially asymptotically proved only
Empirically good for
The game of Go
Some other games
Non-linear expensive optimization
Active learning
Not (yet) tested industrially
Understood weaknesses: plenty of very similar nodes!
Next challenge:
Solve these weaknesses
Industrial applications
Partially observable cases : cf Cazenave, Rolet
118. Biblio
Bandits: Lai, Robbins, Auer, Cesa-Bianchi...
UCT: Kocsis, Szepesvari, Coquelin, Munos...
MCTS (Go): Coulom, Chaslot, Fiter, Gelly, Hoock, Silver, Muller,
Pérez, Rimmel, Wang...
Tree + DP for industrial applicationl: Péret, Garcia...
Bandits with infinitely many arms:
Audibert, Coulom, Munos, Wang...
Applications far from Go: Rolet,
Teytaud (F), Rimmel, De Mesmay
...
Links with “macro-actions” ?
Parallelization, mixing with offline
learning, bias...
119. Paul Veyssière Vincent Berthier
Contributors
Amine Bourki Hassen Doghmen
Matthieu Coulm Univ. Taiwan
Univ. Paris
Bandits: Lai, Robbins, Auer, Cesa-Bianchi...
UCT: Kocsis, Szepesvari, Coquelin, Munos...
MCTS (Go): Coulom, Chaslot, Fiter, Gelly, Hoock, Silver, Muller,
Pérez, Rimmel, Wang...
Tree + DP for industrial applicationl: Péret, Garcia...
Bandits with infinitely many arms:
Audibert, Coulom, Munos, Wang...
Applications far from Go: Rolet,
Teytaud (F), Rimmel, De Mesmay
...
Links with “macro-actions” ?
Parallelization, mixing with offline
learning, bias...