Mdpcg presentation

•

0 likes•73 views

Sarah H.Q. Li is a Ph.D. candidate at the University of Washington studying incentivizing autonomous vehicles and spacecraft toward constraint satisfaction. Her research focuses on autonomous decision making in stochastic environments, including UAV delivery systems, in-orbit satellite constellations, and routing of competitive autonomous vehicles. She presents work on applying her research to in-orbit satellite service planning and congestion control of competitive ride-share systems.

Engineering

Sarah H.Q. Li
Ph.D. Candidate
Department of Aeronautics and Astronautics, University of Washington
Incentivizing autonomous
vehicles/spacecraft toward
constraint satisfaction

2 of 27
Autonomous decision making in stochastic environments
- UAV delivery
- In-orbit servicing satellite constellation
- Routing of competitive autonomous vehicles
Motivations

3 of 27
Application: In-orbit satellite
service planning

4 of 27
Extension to MDP
congestion games
Outline
Finite time MDP
01
03
02
04
05
06
Application: congestion
control of competitive
ride-share systems
Incentive design for MDP
congestion games
Frank-Wolfe-Based
Learning method
Bibliography

10 of 27
Potential game formulation
[1]: Calderone, Dan, and S. Shankar Sastry. "Markov decision process routing games." In 2017 ACM/IEEE 8th International
Conference on Cyber-Physical Systems (ICCPS), pp. 273-280. IEEE, 2017.

12 of 27
Algorithms for MDP congestion
game

13 of 27
Frank-Wolfe algorithm for MDP
congestion games

14 of 27
Learning algorithm for MDP
Wardrop equilibrium
[2] Yu, Yue, Dan Calderone, Sarah HQ Li, Lillian J. Ratliff, and Behçet Açıkmeşe. "A Primal-Dual Approach to Markovian Network
Optimization." under review for Automatica.

17 of 27
Incentivizing desirable player
behavior

18 of 27
Incentivizing desirable player
behavior
[3] Li, Sarah HQ, Yue Yu, Daniel Calderone, Lillian Ratliff, and Behçet Açikmeşe. "Tolling for constraint satisfaction in markov decision
process congestion games." In 2019 American Control Conference (ACC), pp. 1238-1243. IEEE, 2019.

19 of 27
Finding the minimum tolling value

20 of 27
Finding the minimum tolling value

21 of 27
Finding the minimum tolling value

22 of 27
Finding the minimum tolling value
[4] Li, Sarah HQ, Yue Yu, Dan Calderone, Lillian J. Ratliff, and Behcet Acikmese. "Online Constraint Satisfaction via Tolls in MDP
Congestion Games." arXiv preprint arXiv:1907.08912 (2021).

23 of 27
Incentivizing desirable player
behavior

CREDITS: This presentation template was created by Slidesgo, including
icons by Flaticon, and infographics & images by Freepik.
Please keep this slide for attribution.
25 of 27
Does anyone have any questions?
sarahli@uw.edu
THANKS
[1]: Calderone, Dan, and S. Shankar Sastry. "Markov decision process routing games." In 2017
ACM/IEEE 8th International Conference on Cyber-Physical Systems (ICCPS), pp. 273-280. IEEE, 2017.
[2] Yu, Yue, Dan Calderone, Sarah HQ Li, Lillian J. Ratliff, and Behçet Açıkmeşe. "A Primal-Dual
Approach to Markovian Network Optimization." under review for Automatica.
[3] Li, Sarah HQ, Yue Yu, Daniel Calderone, Lillian Ratliff, and Behçet Açikmeşe. "Tolling for constraint
satisfaction in markov decision process congestion games." In 2019 American Control Conference (ACC),
pp. 1238-1243. IEEE, 2019.
[4] Li, Sarah HQ, Yue Yu, Dan Calderone, Lillian J. Ratliff, and Behcet Acikmese. "Online Constraint
Satisfaction via Tolls in MDP Congestion Games." arXiv preprint arXiv:1907.08912 (2021).

27 of 27
Application: In-orbit satellite
service planning

HORUS: Peering into Lunar Shadowed Regions with AI

Advanced-Concepts-Team

During this pandemic we have experienced the devastating effects of isolation. Although astronauts undergo exhaustive training, psychological strain has been observed during space missions resulting in stress, home-sickness, anxiety, etc. Being confined with the same people poses challenges that could result in interpersonal antagonisms or sleeping problems. This effect could be also emphasized during long-term missions or Earth-out-of-view phenomenon. The aim of our system is to alleviate some of those issues. A standalone VR headset with physiological sensors will be used to collect and integrate different inputs such as psycho-physiological signals, oculomotor patterns, voice, and behavioral actions. Afterwards, an adaptive model will process and analyze the signals to detect specific psychological states (e.g., stress, fatigue). Finally, the system will stream personalized content (user’s favorite Earth locations or revisiting family/friends memories) or provide individual games for entertainment and training. The system will also allow the integration of other crew members in the same VR. The expected result is to provide astronauts with a tool to escape the feeling of isolation, to promote positive wellbeing timely delivered, and to boost crew cohesion and teamwork.

META-SPACE: Psycho-physiologically Adaptive and Personalized Virtual Reality ...

Advanced-Concepts-Team

The LIFE initiative has the goal to develop the science, the technology and a roadmap for an aspiring space mission that will allow humankind to detect and characterize, via nulling interferometry, the atmospheres of hundreds of nearby extrasolar planets including dozens that may be similar to Earth. This follow-up talk will tackle more of the techniques and technologies that will enable such an ambitious undertaking. I will outline the underlying measuring principle, and provide some overview over essential technologies, their current status and necessary developments.

The Large Interferometer For Exoplanets (LIFE) II: Key Methods and Technologies

Advanced-Concepts-Team

Black holes have evolved from theoretical prediction to accepted hypothesis, due to the wealth of new discoveries in the last decades. In this talk I will discuss the observational evidence for the existence of black holes of different sizes and what we know about their evolution based on observations and theory. I will also describe what Quasars and Active Galactic Nuclei are, and how these extremely luminous objects can be used to study black holes at the early ages of the Universe.

Black Holes and Bright Quasars

Advanced-Concepts-Team

Long-term exposure to microgravity, ionizing radiation and increased levels of psychological stress can cause changes in the astronauts’ skin, resulting in skin rashes, itches and delayed wound healing during space missions. There is still a lack of understanding how the complex spaceflight environment induces these defects. This PhD project aims to investigate how exposure to a combination of spaceflight stressors can affect the structure and function of the skin, and how they can hamper wound healing. For this we have developed in vitro simulation models and are exposing primary human dermal fibroblasts to hydrocortisone, ionizing radiation and simulated microgravity. Results indicate a significant negative effect of hydrocortisone as well as simulated microgravity on wound healing capability of dermal fibroblasts. Furthermore, a project has been initiated with the support of the European Space Agency Academy “Spin Your Thesis!” Campaign, aiming to investigate the effects of an increased gravitational force on fibroblast function related to wound healing. Altogether the results of this PhD project will give more insights into the effects of combined spaceflight stressors on dermal skin cells, and improve risk assessment for human deep space exploration.

In vitro simulation of spaceflight environment to elucidate combined effect o...

Advanced-Concepts-Team

Studying the atmospheres of a statistically significant number of rocky, terrestrial exoplanets - including the search for habitable and potentially inhabited planets - is one of the major goals of exoplanetary science and possibly the most challenging question in 21st century astrophysics. However, despite being at the top of the agenda of all major space agencies and ground-based observatories, none of the currently planned projects or missions worldwide has the technical capabilities to achieve this goal. In this talk we present new results from the LIFE Mission initiative, which addresses this issue by investigating the scientific potential of a mid infrared nulling interferometer observatory. Here we will focus on the mission's yield estimates, our simulator software as well as various exemplary science cases such as observing Earth- and Venus-twins or searching for phosphine in exoplanetary atmospheres.

The Large Interferometer For Exoplanets (LIFE): the science of characterising...

Advanced-Concepts-Team

Ephemeral wetlands – vernal pools, are temporal isolated bodies of freshwater that host amphibious (aquatic & terrestrial) lifeforms. Herein rare plant species and living animal fossils have adapted to the extreme conditions of water saturation and rapid desiccation events. In many ways, vernal pools hold extreme properties analogous to conditions that could be found across the solar system; they are natural laboratories to understand life adaptations to extreme conditions, colonization events and ecological life patterns. Vernal pools are very appropriate to study life macroecology, the diversity and distribution patterns of the taxa across the globe including microbes. Microbial organisms in vernal pools had never been studied before. Using high-throughput sequencing technology we addressed the diversity of bacteria and archaea in soils, water and plant tissues, exploring community assembly mechanisms and diversity patterns. In addition, the microbes living inside plants —endophytes, which are known to alleviate plants under environmental stress, were unraveled for the first time from the vernal pool-amphibious plant species Eryngium castrense. Discrete diversity patterns were found across latitudinal and anthropogenic transects, but this last one pattern is still part of further investigation and analysis. Vernal pools represent a new frontier to understand the dynamics of life, holding important lessons about adaptation processes, including mutualisms. They can be seen as naturally enclosed ecosystems on planetary bodies or even spacecraft in our quest for extraterrestrial habitats.Protecting and understanding these understudied and threatened ecosystems may hold the key of future human survival.

Vernal pools a new ecosystem for astrobiology studies

Advanced-Concepts-Team

Sentinel-1 satellites, ESA’s Synthetic Aperture Radar (SAR) mission, provide continuous data from the Earth surface in weekly to biweekly time intervals. This data availability provides an unprecedented opportunity to continuously monitor the Earth surface motion in areas prone to geohazards; such as regions of high seismic and volcanic activities, with the end goal of supporting the Early Warning Systems. However, the great challenge is to derive insights from Terabytes of satellite image sequences, in a computationally-efficient and time-critical manner. We’ve risen to this challenge by designing innovative signal processing and deep learning algorithms to efficiently mine this invaluable wealth of data. This talk gives on overview of our designed solutions, as well as a demonstration of these solutions in the Tectonic and Volcanic monitoring of South America (TecVolSA) project.

Keeping a Sentinel Eye on the Volcanoes – from Space!

Advanced-Concepts-Team

More from Advanced-Concepts-Team (20)

2024.03.22 - Mike Heddes - Introduction to Hyperdimensional Computing.pdf

Isabelle Diacaire - From Ariadnas to Industry R&D in optics and photonics

The ExoGRAVITY project - observations of exoplanets from the ground with opti...

MOND_famaey.pdf

Pablo Gomez - Solving Large-scale Challenges with ESA Datalabs

Jonathan Sauder - Miniaturizing Mechanical Systems for CubeSats: Design Princ...

Towards an Artificial Muse for new Ideas in Quantum Physics

EDEN ISS - A space greenhouse analogue in Antarctica

How to give a robot a soul

Information processing with artificial spiking neural networks

Exploring Architected Materials Using Machine Learning

Electromagnetically Actuated Systems for Modular, Self-Assembling and Self-Re...

HORUS: Peering into Lunar Shadowed Regions with AI

META-SPACE: Psycho-physiologically Adaptive and Personalized Virtual Reality ...

The Large Interferometer For Exoplanets (LIFE) II: Key Methods and Technologies

Black Holes and Bright Quasars

In vitro simulation of spaceflight environment to elucidate combined effect o...

The Large Interferometer For Exoplanets (LIFE): the science of characterising...

Vernal pools a new ecosystem for astrobiology studies

Keeping a Sentinel Eye on the Volcanoes – from Space!

Recently uploaded

Introduction to Machine Learning Unit-5 Notes for II-II Mechanical Engineering

C Sai Kiran

İTÜ CAD and Reverse Engineering Workshop

Emre Günaydın

Introduction to Casting Processes in Manufacturing

ssuser0811ec

Halogenation process of chemical process industries

MuhammadTufail242431

Introduction to Machine Learning Unit-4 Notes for II-II Mechanical Engineering

C Sai Kiran

Blood Donation Management System is a web database application that enables the public to make online session reservation, to view nationwide blood donation events online and at the same time provides centralized donor and blood stock database. This application is developed by using ASP.NET technology from Visual Studio with the MySQL 5.0 as the database management system. The methodology used to develop this system as a whole is Object Oriented Analysis and Design; whilst, the database for BDMS is developed by following the steps in Database Life Cycle. The targeted users for this application are the public who is eligible to donate blood ,'system moderator, administrator from National Blood Center and the staffs who are working in the blood banks of the participating hospitals. The main objective of the development of this application is to overcome the problems that exist in the current system, which are the lack of facilities for online session reservation and online advertising on the nationwide blood donation events, and also decentralized donor and blood stock database. Besides, extra features in the system such as security protection by using password, generating reports, reminders of blood stock shortage and workflow tracking can even enhance the efficiency of the management in the blood banks. The final result of this project is the development of web database application, which is the BDMS.

Online blood donation management system project.pdf

Kamal Acharya

2024 DevOps Pro Europe - Growing at the edge

Paco Orozco

Welcome to WIPAC Monthly the magazine brought to you by the LinkedIn Group Water Industry Process Automation & Control. In this month's edition, along with this month's industry news to celebrate the 13 years since the group was created we have articles including A case study of the used of Advanced Process Control at the Wastewater Treatment works at Lleida in Spain A look back on an article on smart wastewater networks in order to see how the industry has measured up in the interim around the adoption of Digital Transformation in the Water Industry.

Water Industry Process Automation and Control Monthly - May 2024.pdf

Water Industry Process Automation & Control

In present era, the scopes of information technology growing with a very fast .We do not see any are untouched from this industry. The scope of information technology has become wider includes: Business and industry. Household Business, Communication, Education, Entertainment, Science, Medicine, Engineering, Distance Learning, Weather Forecasting. Carrier Searching and so on. My project named “Event Management System” is software that store and maintained all events coordinated in college. It also helpful to print related reports. My project will help to record the events coordinated by faculties with their Name, Event subject, date & details in an efficient & effective ways. In my system we have to make a system by which a user can record all events coordinated by a particular faculty. In our proposed system some more featured are added which differs it from the existing system such as security.

Event Management System Vb Net Project Report.pdf

Kamal Acharya

Toll Tax Management System is a web based application that can provide all the information related to toll plazas and the passenger checks in and pays the amount, then he/she will be provided by a receipt. With this receipt he/she can leave the toll booth without waiting for any verification call. The information would also cover registration of staff, toll plaza collection, toll plaza collection entry for vehicles, date wise report entry, Vehicle passes and passes reports b/w dates.

Toll tax management system project report..pdf

Kamal Acharya

The proposed project is developed to manage the automobile in the automobile dealer company. The main module in this project is login, automobile management, customer management, sales, complaints and reports. The first module is the login. The automobile showroom owner should login to the project for usage. The username and password are verified and if it is correct, next form opens. If the username and password are not correct, it shows the error message. When a customer search for a automobile, if the automobile is available, they will be taken to a page that shows the details of the automobile including automobile name, automobile ID, quantity, price etc. “Automobile Management System” is useful for maintaining automobiles, customers effectively and hence helps for establishing good relation between customer and automobile organization. It contains various customized modules for effectively maintaining automobiles and stock information accurately and safely. When the automobile is sold to the customer, stock will be reduced automatically. When a new purchase is made, stock will be increased automatically. While selecting automobiles for sale, the proposed software will automatically check for total number of available stock of that particular item, if the total stock of that particular item is less than 5, software will notify the user to purchase the particular item. Also when the user tries to sale items which are not in stock, the system will prompt the user that the stock is not enough. Customers of this system can search for a automobile; can purchase a automobile easily by selecting fast. On the other hand the stock of automobiles can be maintained perfectly by the automobile shop manager overcoming the drawbacks of existing system.

Automobile Management System Project Report.pdf

Kamal Acharya

This paper addresses the vulnerability of deep learning models, particularly convolutional neural networks (CNN)s, to adversarial attacks and presents a proactive training technique designed to counter them. We introduce a novel volumization algorithm, which transforms 2D images into 3D volumetric representations. When combined with 3D convolution and deep curriculum learning optimization (CLO), itsignificantly improves the immunity of models against localized universal attacks by up to 40%. We evaluate our proposed approach using contemporary CNN architectures and the modified Canadian Institute for Advanced Research (CIFAR-10 and CIFAR-100) and ImageNet Large Scale Visual Recognition Challenge (ILSVRC12) datasets, showcasing accuracy improvements over previous techniques. The results indicate that the combination of the volumetric input and curriculum learning holds significant promise for mitigating adversarial attacks without necessitating adversary training.

Immunizing Image Classifiers Against Localized Adversary Attacks

gerogepatton

Digital Signal Processing Lecture notes n.pdf

AbrahamGadissa

CME397 Surface Engineering- Professional Elective

karthi keyan

Explosives Industry manufacturing process.pdf

884710SadaqatAli

Overview of the fundamental roles in Hydropower generation and the components involved in wider Electrical Engineering. This paper presents the design and construction of hydroelectric dams from the hydrologist’s survey of the valley before construction, all aspects and involved disciplines, fluid dynamics, structural engineering, generation and mains frequency regulation to the very transmission of power through the network in the United Kingdom. Author: Robbie Edward Sayers Collaborators and co editors: Charlie Sims and Connor Healey. (C) 2024 Robbie E. Sayers

HYDROPOWER - Hydroelectric power generation

Robbie Edward Sayers

Maintaining high-quality standards in the production of TMT bars is crucial for ensuring structural integrity in construction. Addressing common defects through careful monitoring, standardized processes, and advanced technology can significantly improve the quality of TMT bars. Continuous training and adherence to quality control measures will also play a pivotal role in minimizing these defects.

Quality defects in TMT Bars, Possible causes and Potential Solutions.

PrashantGoswami42

Online movie ticket booking system for movies is a web-based program. This application allows users to purchase cinema tickets over the portal. To buy tickets, people must first register or log in. This website's backend is PHP and JavaScript, and the front end is HTML and CSS. All phases of the software development life cycle are efficiently managed in order to design and implement software. On the website, there are two panels: one for administrators and one for customers/users. The admin has the ability to add cinemas, movies, delete, halt execution, and add screens, among other things. The website is simple to navigate and appealing, saving the end user time.

A CASE STUDY ON ONLINE TICKET BOOKING SYSTEM PROJECT.pdf

Kamal Acharya

shape functions of 1D and 2 D rectangular elements.pptx

VishalDeshpande27

Architectural Portfolio Sean Lockwood

seandesed

Recently uploaded (20)

Introduction to Machine Learning Unit-5 Notes for II-II Mechanical Engineering

İTÜ CAD and Reverse Engineering Workshop

Introduction to Casting Processes in Manufacturing

Halogenation process of chemical process industries

Introduction to Machine Learning Unit-4 Notes for II-II Mechanical Engineering

Online blood donation management system project.pdf

2024 DevOps Pro Europe - Growing at the edge

Water Industry Process Automation and Control Monthly - May 2024.pdf

Event Management System Vb Net Project Report.pdf

Toll tax management system project report..pdf

Automobile Management System Project Report.pdf

Immunizing Image Classifiers Against Localized Adversary Attacks

Digital Signal Processing Lecture notes n.pdf

CME397 Surface Engineering- Professional Elective

Explosives Industry manufacturing process.pdf

HYDROPOWER - Hydroelectric power generation

Quality defects in TMT Bars, Possible causes and Potential Solutions.

A CASE STUDY ON ONLINE TICKET BOOKING SYSTEM PROJECT.pdf

shape functions of 1D and 2 D rectangular elements.pptx

Architectural Portfolio Sean Lockwood

Mdpcg presentation

1. Sarah H.Q. Li Ph.D. Candidate Department of Aeronautics and Astronautics, University of Washington Incentivizing autonomous vehicles/spacecraft toward constraint satisfaction

2. 2 of 27 Autonomous decision making in stochastic environments - UAV delivery - In-orbit servicing satellite constellation - Routing of competitive autonomous vehicles Motivations

3. 3 of 27 Application: In-orbit satellite service planning

4. 4 of 27 Extension to MDP congestion games Outline Finite time MDP 01 03 02 04 05 06 Application: congestion control of competitive ride-share systems Incentive design for MDP congestion games Frank-Wolfe-Based Learning method Bibliography

5. 5 of 27 Markov decision processes

6. 6 of 27 Dynamic programming

7. 7 of 27 Extension to multiple players

8. 8 of 27 MDP Wardrop equilibrium

9. 9 of 27 MDP Wardrop equilibrium

10. 10 of 27 Potential game formulation [1]: Calderone, Dan, and S. Shankar Sastry. "Markov decision process routing games." In 2017 ACM/IEEE 8th International Conference on Cyber-Physical Systems (ICCPS), pp. 273-280. IEEE, 2017.

11. 11 of 27 Summary

12. 12 of 27 Algorithms for MDP congestion game

13. 13 of 27 Frank-Wolfe algorithm for MDP congestion games

14. 14 of 27 Learning algorithm for MDP Wardrop equilibrium [2] Yu, Yue, Dan Calderone, Sarah HQ Li, Lillian J. Ratliff, and Behçet Açıkmeşe. "A Primal-Dual Approach to Markovian Network Optimization." under review for Automatica.

15. 15 of 27 Application: urban ride-share

16. 16 of 27 Application: urban ride-share

17. 17 of 27 Incentivizing desirable player behavior

18. 18 of 27 Incentivizing desirable player behavior [3] Li, Sarah HQ, Yue Yu, Daniel Calderone, Lillian Ratliff, and Behçet Açikmeşe. "Tolling for constraint satisfaction in markov decision process congestion games." In 2019 American Control Conference (ACC), pp. 1238-1243. IEEE, 2019.

19. 19 of 27 Finding the minimum tolling value

20. 20 of 27 Finding the minimum tolling value

21. 21 of 27 Finding the minimum tolling value

22. 22 of 27 Finding the minimum tolling value [4] Li, Sarah HQ, Yue Yu, Dan Calderone, Lillian J. Ratliff, and Behcet Acikmese. "Online Constraint Satisfaction via Tolls in MDP Congestion Games." arXiv preprint arXiv:1907.08912 (2021).

23. 23 of 27 Incentivizing desirable player behavior

24. 24 of 27 Collaborators

25. CREDITS: This presentation template was created by Slidesgo, including icons by Flaticon, and infographics & images by Freepik. Please keep this slide for attribution. 25 of 27 Does anyone have any questions? sarahli@uw.edu THANKS [1]: Calderone, Dan, and S. Shankar Sastry. "Markov decision process routing games." In 2017 ACM/IEEE 8th International Conference on Cyber-Physical Systems (ICCPS), pp. 273-280. IEEE, 2017. [2] Yu, Yue, Dan Calderone, Sarah HQ Li, Lillian J. Ratliff, and Behçet Açıkmeşe. "A Primal-Dual Approach to Markovian Network Optimization." under review for Automatica. [3] Li, Sarah HQ, Yue Yu, Daniel Calderone, Lillian Ratliff, and Behçet Açikmeşe. "Tolling for constraint satisfaction in markov decision process congestion games." In 2019 American Control Conference (ACC), pp. 1238-1243. IEEE, 2019. [4] Li, Sarah HQ, Yue Yu, Dan Calderone, Lillian J. Ratliff, and Behcet Acikmese. "Online Constraint Satisfaction via Tolls in MDP Congestion Games." arXiv preprint arXiv:1907.08912 (2021).

26. 26 of 27 Connection to routing games

27. 27 of 27 Application: In-orbit satellite service planning

Mdpcg presentation

Recommended

Recommended

More Related Content

Similar to Mdpcg presentation

Similar to Mdpcg presentation (20)

More from Advanced-Concepts-Team

More from Advanced-Concepts-Team (20)

Recently uploaded

Recently uploaded (20)

Mdpcg presentation