Online platforms are emerging as a powerful mechanism for matching resources to requests. In the setting of freight, the requests arrive from shippers, who have a diverse collection of goods. The resources are supplied by shippers (trucks), and have various physical constraints (driver’s route preferences, carrying capacity, geographic preferences, etc.). Online platforms are emerging that (a) learn the characteristics of shippers and carriers, and (b) efficiently match goods to trucks based on such learning.
Our project will develop algorithms for such online resource allocation. This is a challenging problem, due to the complexity of the learning tasks. Such algorithms can have considerable impact on efficiently using trucking resources.
Tracking the tracker: Time Series Analysis in Python from First Principleskenluck2001
This document summarizes a presentation on time series analysis in Python. It discusses two main applications: forecasting and anomaly detection. It provides an overview of various time series models and algorithms discussed in the presentation, including ARIMA, Kalman filters, and their extensions. It also covers concepts like stationarity, Box-Jenkins methodology, and challenges in predicting time series like stock prices.
survey slides for contextual bandit
main reference: Li Zhou. A Survey on Contextual Multi-armed Bandits. arXiv, 2015. (https://arxiv.org/abs/1508.03326)
The document discusses multi-armed bandits and their applications. It provides an overview of multi-armed bandits, describing the exploration-exploitation dilemma. It then discusses the optimal UCB algorithm and how it balances exploration and exploitation. Finally, it summarizes two applications of multi-armed bandits: using them for learning to rank in recommendation systems and addressing the cold-start problem in recommender systems.
Multi-Armed Bandits: Intro, examples and tricksIlias Flaounas
In this talk Ilias will discuss some variations of the Multi-Armed Bandits (MABs), a less popular although important area of Machine Learning. MABs enable us to build adaptive systems capable of finding solutions for tasks based on the interactions with their environment. MABs solve a task by acquiring useful knowledge at every step of an iterative process while they balance the exploration-exploitation dilemma. They are used to tackle practical problems like selecting appropriate online ads and personalized content for presentation to users; assigning people to cohorts in controlled trials; supporting decision making and more. To solve these kinds of problems solutions need to be identified as fast as possible since accepting errors can be costly. Ilias will discuss some examples from industry and academia as well as some of the related work at Atlassian.
This document discusses linear time-invariant (LTI) systems and convolution. It begins by defining LTI systems and convolution for both continuous and discrete time. Convolution is described as a way to construct the output of a system given its impulse response. Applications in digital signal processing and image processing are mentioned. Convolution filtering plays an important role in edge detection and related image processing algorithms. The mathematical definition of discrete time convolution is provided. An example problem calculating outputs for different inputs using convolution is given at the end.
Track 6 - Mobile Apps and computational systems as learning tools
Authors: Santiago E. Moll, José-A. Moraño, Luis M. Sánchez-Ruiz and Nuria Llobregat-Gómez
Recent developments on SMT solvers for non-linear polynomial constraints have become crucial to make the template-based (or constraint-based) method for program analysis effective in practice. Moreover, using Max-SMT (its optimization version) is the key to extend this approach to develop an automated compositional program verification method based on generating conditional inductive invariants. We build a bottom-up program verification framework that propagates preconditions of small program parts as postconditions for preceding program parts and can recover from failures when some precondition is not proved. These techniques have successfully been implemented within the VeryMax tool which currently can check safety, reachability and termination properties of C++ code. In this talk we will provide an overview of the Max-SMT solving techniques and its application to compositional program analysis.
Tracking the tracker: Time Series Analysis in Python from First Principleskenluck2001
This document summarizes a presentation on time series analysis in Python. It discusses two main applications: forecasting and anomaly detection. It provides an overview of various time series models and algorithms discussed in the presentation, including ARIMA, Kalman filters, and their extensions. It also covers concepts like stationarity, Box-Jenkins methodology, and challenges in predicting time series like stock prices.
survey slides for contextual bandit
main reference: Li Zhou. A Survey on Contextual Multi-armed Bandits. arXiv, 2015. (https://arxiv.org/abs/1508.03326)
The document discusses multi-armed bandits and their applications. It provides an overview of multi-armed bandits, describing the exploration-exploitation dilemma. It then discusses the optimal UCB algorithm and how it balances exploration and exploitation. Finally, it summarizes two applications of multi-armed bandits: using them for learning to rank in recommendation systems and addressing the cold-start problem in recommender systems.
Multi-Armed Bandits: Intro, examples and tricksIlias Flaounas
In this talk Ilias will discuss some variations of the Multi-Armed Bandits (MABs), a less popular although important area of Machine Learning. MABs enable us to build adaptive systems capable of finding solutions for tasks based on the interactions with their environment. MABs solve a task by acquiring useful knowledge at every step of an iterative process while they balance the exploration-exploitation dilemma. They are used to tackle practical problems like selecting appropriate online ads and personalized content for presentation to users; assigning people to cohorts in controlled trials; supporting decision making and more. To solve these kinds of problems solutions need to be identified as fast as possible since accepting errors can be costly. Ilias will discuss some examples from industry and academia as well as some of the related work at Atlassian.
This document discusses linear time-invariant (LTI) systems and convolution. It begins by defining LTI systems and convolution for both continuous and discrete time. Convolution is described as a way to construct the output of a system given its impulse response. Applications in digital signal processing and image processing are mentioned. Convolution filtering plays an important role in edge detection and related image processing algorithms. The mathematical definition of discrete time convolution is provided. An example problem calculating outputs for different inputs using convolution is given at the end.
Track 6 - Mobile Apps and computational systems as learning tools
Authors: Santiago E. Moll, José-A. Moraño, Luis M. Sánchez-Ruiz and Nuria Llobregat-Gómez
Recent developments on SMT solvers for non-linear polynomial constraints have become crucial to make the template-based (or constraint-based) method for program analysis effective in practice. Moreover, using Max-SMT (its optimization version) is the key to extend this approach to develop an automated compositional program verification method based on generating conditional inductive invariants. We build a bottom-up program verification framework that propagates preconditions of small program parts as postconditions for preceding program parts and can recover from failures when some precondition is not proved. These techniques have successfully been implemented within the VeryMax tool which currently can check safety, reachability and termination properties of C++ code. In this talk we will provide an overview of the Max-SMT solving techniques and its application to compositional program analysis.
Stochastic optimization from mirror descent to recent algorithmsSeonho Park
The document discusses stochastic optimization algorithms. It begins with an introduction to stochastic optimization and online optimization settings. Then it covers Mirror Descent and its extension Composite Objective Mirror Descent (COMID). Recent algorithms for deep learning like Momentum, ADADELTA, and ADAM are also discussed. The document provides convergence analysis and empirical studies of these algorithms.
The issues about maneuvering target track prediction were discussed in this paper. Firstly, using Kalman filter which based on current statistical model describes the state of maneuvering target motion, thereby analyzing time range of the target maneuvering occurred. Then, predict the target trajectory in real time by the improved gray prediction model. Finally, residual test and posterior variance test model accuracy, model accuracy is accurate.
The document discusses three topics in data assimilation: sea ice modeling, the role of unstable subspaces, and the role of model error. It describes challenges in assimilating data into sea ice models with changing state space dimensions due to adaptive meshes. It discusses using a fixed dimensional state space defined by a supermesh to apply the Ensemble Kalman Filter to sea ice models. It also summarizes the Kalman filter and introduces exploring the convergence and asymptotic properties of the Kalman filter estimates.
This document discusses hardware architectures for deep learning. It covers various integer and floating point datatypes used in neural networks and their associated ranges and accuracy. It also discusses the evolution of Intel's SIMD instruction sets and their support for different data widths. The document introduces the roofline performance model and uses it to analyze the performance of a fully connected layer implementation on different hardware. It covers techniques for improving computational intensity, like loop splitting and reordering. Finally, it discusses computational transforms like Strassen's algorithm and Winograd that can reduce the operation count of matrix multiplication and convolution, respectively, though with some tradeoffs in numerical stability and memory usage.
A Strategic Model For Dynamic Traffic AssignmentKelly Taylor
This document proposes a strategic model for dynamic traffic assignment. The key elements are:
1) Users follow strategies that assign preference orders to outgoing arcs from each node based on arrival time and congestion.
2) A time-space network is constructed to model flow variations over time on the original road network.
3) An equilibrium is achieved when expected delays are minimal for each origin-destination pair given the strategies and capacities.
A prospect theory model of route choice with context dependent reference pointsPablo Guarda
This document presents a study comparing a standard route choice model (SRUM) to prospect theory models of route choice (CPT models) with context-dependent reference points. The CPT models better fit the experimental route choice data than the SRUM, particularly models using relative reference points based on average time outcomes. Estimation of the CPT models confirmed loss aversion for both travel time attributes. The study provides empirical support for using prospect theory to model travelers' risk attitudes in time-related route choice decisions.
Sensor Fusion Study - Ch3. Least Square Estimation [강소라, Stella, Hayden]AI Robotics KR
This document discusses Wiener filtering and recursive least squares estimation. It begins with an introduction to Wiener filtering, providing an overview of its history and development. It then discusses how the power spectrum of a stochastic process changes when passed through a linear time-invariant system. Next, it formulates the problem of using a linear filter to extract a signal from additive noise. It derives expressions for the power spectrum of the error and its variance. Finally, it considers optimizing a parametric filter by assuming the optimal filter is a first-order low-pass filter and that the signal and noise spectra are known forms. It derives an expression for the optimal parameter T based on minimizing the error variance.
This document describes three methods - eigenvalue decomposition, uniformization, and matrix exponentiation - for computing sufficient statistics for continuous-time Markov chains (CTMCs) that are needed for maximum likelihood estimation. The eigenvalue decomposition method is prone to large errors, while the uniformization method sums many small numbers and the matrix exponentiation method is most accurate but also the slowest. The document implemented these methods and compared their performance and accuracy for computing statistics in an expectation-maximization algorithm.
The document presents an algorithm for cooperative particle filtering for sensor network localization. It describes a distributed cooperative particle filter (CoopPF) that allows nodes to estimate their unknown locations by exploiting inter-node ranging measurements and communicating location probability distributions. The algorithm factorizes weight calculations to allow an iterative distributed implementation. It also proposes parametric distribution approximations to further reduce communication costs. Simulation results show the CoopPF and variants achieve accurate localization and perform better than existing methods in terms of mean square error over time and ranging noise levels.
Kyeong Soo Kim, "Clock skew compensation algorithm immune to floating-point precision loss," Invited talk, to be delivered at 2022 International Workshop on Mathematics and Its Applications, Chungnam National University (CNU), Daejoen, Korea, August 4-5, 2022.
A car sharing auction with temporal-spatial OD connection conditionsharapon
This document summarizes a research presentation about designing a mobility sharing auction mechanism that considers temporal-spatial origin-destination (OD) connection conditions. The researchers propose a framework for mobility services that uses "knots" to represent connections between OD pairs across time slots. They describe a setting where a service supplier issues permits to users based on vehicle capacity limits and aims to maximize social welfare by satisfying the temporal-spatial OD connection condition. They also discuss extending the classic Vickrey-Clarke-Groves mechanism to this mobility sharing context, where users' payments are based on how their participation affects the social welfare optimization problem. The researchers claim this mechanism provides incentives for users to bid their true values.
RSC: Mining and Modeling Temporal Activity in Social MediaAlceu Ferraz Costa
Presentation of the KDD 2015 paper describing the RSC model:
RSC: Mining and Modeling Temporal Activity in Social Media
Alceu Ferraz Costa, Yuto Yamaguchi, Agma Juci Machado Traina, Caetano Traina Jr., and Christos Faloutsos
The 21st SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2015
In this paper, implementations of three Hough Transform based fingerprint alignment algorithms are analyzed with respect to time complexity on Java Card environment. Three algorithms are: Local Match Based Approach (LMBA), Discretized Rotation Based Approach
(DRBA), and All Possible to Match Based Approach (APMBA). The aim of this paper is to present the complexity and implementations of existing work of one of the mostly used method of fingerprint alignment, in order that the complexity can be simplified or find the best algorithm with efficient complexity and implementation that can be easily implemented on Java Card environment for match on card. Efficiency involves the accuracy of the implementation, time taken to perform fingerprint alignment, memory required by the implementation and instruction operations required and used.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Dynamic Kohonen Network for Representing Changes in InputsJean Fecteau
The document describes a system that uses a self-organizing map (Kohonen network) to dynamically represent changes to a set of inputs over time. The system is able to recognize new inputs, remove unlikely inputs, and merge similar inputs while continuously updating its representation. It was tested on simulated 3D color vector inputs with added noise. The system generally converged quickly and accurately but was sensitive to noise and struggled with similar inputs due to its binary region definitions. While flawed, it demonstrated the ability to adapt its knowledge to changes in inputs without reinitializing.
This document discusses estimating the inverse covariance matrix for compositional data, which represents relative abundance measurements that are constrained to sum to a constant. It introduces the concept of compositional data analysis and describes how relative abundance data can be modeled as a log-ratio transformation of absolute count data. It reviews existing approaches for sparse precision matrix estimation and proposes relaxing the constraints to account for the compositional nature of the data, in order to estimate a sparse inverse covariance specifically for compositional datasets.
This document discusses time series forecasting techniques for multivariate and hierarchical time series data. It presents several cases involving energy consumption forecasting, sales forecasting, and freight transportation forecasting. For each case, it describes the time series data and components, discusses feature generation methods like nonparametric transformations and the Haar wavelet transform to extract features, and evaluates different forecasting models and their ability to generate consistent forecasts while respecting any hierarchical relationships in the data. The focus is on generating accurate forecasts while maintaining properties like consistency, minimizing errors, and handling complex time series structures.
Updates provided to the D-STOP Business Advisory Council at the 2017 Symposium and Board Meeting: https://ctr.utexas.edu/2018/04/12/d-stop-2017-symposium-archive/
Updates provided to the D-STOP Business Advisory Council at the 2017 Symposium and Board Meeting: https://ctr.utexas.edu/2018/04/12/d-stop-2017-symposium-archive/
Stochastic optimization from mirror descent to recent algorithmsSeonho Park
The document discusses stochastic optimization algorithms. It begins with an introduction to stochastic optimization and online optimization settings. Then it covers Mirror Descent and its extension Composite Objective Mirror Descent (COMID). Recent algorithms for deep learning like Momentum, ADADELTA, and ADAM are also discussed. The document provides convergence analysis and empirical studies of these algorithms.
The issues about maneuvering target track prediction were discussed in this paper. Firstly, using Kalman filter which based on current statistical model describes the state of maneuvering target motion, thereby analyzing time range of the target maneuvering occurred. Then, predict the target trajectory in real time by the improved gray prediction model. Finally, residual test and posterior variance test model accuracy, model accuracy is accurate.
The document discusses three topics in data assimilation: sea ice modeling, the role of unstable subspaces, and the role of model error. It describes challenges in assimilating data into sea ice models with changing state space dimensions due to adaptive meshes. It discusses using a fixed dimensional state space defined by a supermesh to apply the Ensemble Kalman Filter to sea ice models. It also summarizes the Kalman filter and introduces exploring the convergence and asymptotic properties of the Kalman filter estimates.
This document discusses hardware architectures for deep learning. It covers various integer and floating point datatypes used in neural networks and their associated ranges and accuracy. It also discusses the evolution of Intel's SIMD instruction sets and their support for different data widths. The document introduces the roofline performance model and uses it to analyze the performance of a fully connected layer implementation on different hardware. It covers techniques for improving computational intensity, like loop splitting and reordering. Finally, it discusses computational transforms like Strassen's algorithm and Winograd that can reduce the operation count of matrix multiplication and convolution, respectively, though with some tradeoffs in numerical stability and memory usage.
A Strategic Model For Dynamic Traffic AssignmentKelly Taylor
This document proposes a strategic model for dynamic traffic assignment. The key elements are:
1) Users follow strategies that assign preference orders to outgoing arcs from each node based on arrival time and congestion.
2) A time-space network is constructed to model flow variations over time on the original road network.
3) An equilibrium is achieved when expected delays are minimal for each origin-destination pair given the strategies and capacities.
A prospect theory model of route choice with context dependent reference pointsPablo Guarda
This document presents a study comparing a standard route choice model (SRUM) to prospect theory models of route choice (CPT models) with context-dependent reference points. The CPT models better fit the experimental route choice data than the SRUM, particularly models using relative reference points based on average time outcomes. Estimation of the CPT models confirmed loss aversion for both travel time attributes. The study provides empirical support for using prospect theory to model travelers' risk attitudes in time-related route choice decisions.
Sensor Fusion Study - Ch3. Least Square Estimation [강소라, Stella, Hayden]AI Robotics KR
This document discusses Wiener filtering and recursive least squares estimation. It begins with an introduction to Wiener filtering, providing an overview of its history and development. It then discusses how the power spectrum of a stochastic process changes when passed through a linear time-invariant system. Next, it formulates the problem of using a linear filter to extract a signal from additive noise. It derives expressions for the power spectrum of the error and its variance. Finally, it considers optimizing a parametric filter by assuming the optimal filter is a first-order low-pass filter and that the signal and noise spectra are known forms. It derives an expression for the optimal parameter T based on minimizing the error variance.
This document describes three methods - eigenvalue decomposition, uniformization, and matrix exponentiation - for computing sufficient statistics for continuous-time Markov chains (CTMCs) that are needed for maximum likelihood estimation. The eigenvalue decomposition method is prone to large errors, while the uniformization method sums many small numbers and the matrix exponentiation method is most accurate but also the slowest. The document implemented these methods and compared their performance and accuracy for computing statistics in an expectation-maximization algorithm.
The document presents an algorithm for cooperative particle filtering for sensor network localization. It describes a distributed cooperative particle filter (CoopPF) that allows nodes to estimate their unknown locations by exploiting inter-node ranging measurements and communicating location probability distributions. The algorithm factorizes weight calculations to allow an iterative distributed implementation. It also proposes parametric distribution approximations to further reduce communication costs. Simulation results show the CoopPF and variants achieve accurate localization and perform better than existing methods in terms of mean square error over time and ranging noise levels.
Kyeong Soo Kim, "Clock skew compensation algorithm immune to floating-point precision loss," Invited talk, to be delivered at 2022 International Workshop on Mathematics and Its Applications, Chungnam National University (CNU), Daejoen, Korea, August 4-5, 2022.
A car sharing auction with temporal-spatial OD connection conditionsharapon
This document summarizes a research presentation about designing a mobility sharing auction mechanism that considers temporal-spatial origin-destination (OD) connection conditions. The researchers propose a framework for mobility services that uses "knots" to represent connections between OD pairs across time slots. They describe a setting where a service supplier issues permits to users based on vehicle capacity limits and aims to maximize social welfare by satisfying the temporal-spatial OD connection condition. They also discuss extending the classic Vickrey-Clarke-Groves mechanism to this mobility sharing context, where users' payments are based on how their participation affects the social welfare optimization problem. The researchers claim this mechanism provides incentives for users to bid their true values.
RSC: Mining and Modeling Temporal Activity in Social MediaAlceu Ferraz Costa
Presentation of the KDD 2015 paper describing the RSC model:
RSC: Mining and Modeling Temporal Activity in Social Media
Alceu Ferraz Costa, Yuto Yamaguchi, Agma Juci Machado Traina, Caetano Traina Jr., and Christos Faloutsos
The 21st SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2015
In this paper, implementations of three Hough Transform based fingerprint alignment algorithms are analyzed with respect to time complexity on Java Card environment. Three algorithms are: Local Match Based Approach (LMBA), Discretized Rotation Based Approach
(DRBA), and All Possible to Match Based Approach (APMBA). The aim of this paper is to present the complexity and implementations of existing work of one of the mostly used method of fingerprint alignment, in order that the complexity can be simplified or find the best algorithm with efficient complexity and implementation that can be easily implemented on Java Card environment for match on card. Efficiency involves the accuracy of the implementation, time taken to perform fingerprint alignment, memory required by the implementation and instruction operations required and used.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Dynamic Kohonen Network for Representing Changes in InputsJean Fecteau
The document describes a system that uses a self-organizing map (Kohonen network) to dynamically represent changes to a set of inputs over time. The system is able to recognize new inputs, remove unlikely inputs, and merge similar inputs while continuously updating its representation. It was tested on simulated 3D color vector inputs with added noise. The system generally converged quickly and accurately but was sensitive to noise and struggled with similar inputs due to its binary region definitions. While flawed, it demonstrated the ability to adapt its knowledge to changes in inputs without reinitializing.
This document discusses estimating the inverse covariance matrix for compositional data, which represents relative abundance measurements that are constrained to sum to a constant. It introduces the concept of compositional data analysis and describes how relative abundance data can be modeled as a log-ratio transformation of absolute count data. It reviews existing approaches for sparse precision matrix estimation and proposes relaxing the constraints to account for the compositional nature of the data, in order to estimate a sparse inverse covariance specifically for compositional datasets.
This document discusses time series forecasting techniques for multivariate and hierarchical time series data. It presents several cases involving energy consumption forecasting, sales forecasting, and freight transportation forecasting. For each case, it describes the time series data and components, discusses feature generation methods like nonparametric transformations and the Haar wavelet transform to extract features, and evaluates different forecasting models and their ability to generate consistent forecasts while respecting any hierarchical relationships in the data. The focus is on generating accurate forecasts while maintaining properties like consistency, minimizing errors, and handling complex time series structures.
Updates provided to the D-STOP Business Advisory Council at the 2017 Symposium and Board Meeting: https://ctr.utexas.edu/2018/04/12/d-stop-2017-symposium-archive/
Updates provided to the D-STOP Business Advisory Council at the 2017 Symposium and Board Meeting: https://ctr.utexas.edu/2018/04/12/d-stop-2017-symposium-archive/
Updates provided to the D-STOP Business Advisory Council at the 2017 Symposium and Board Meeting: https://ctr.utexas.edu/2018/04/12/d-stop-2017-symposium-archive/
This document discusses ongoing research projects related to collaborative sensing and heterogeneous networking leveraging vehicular fleets. Specifically, it discusses:
1) How increased cluster density of vehicles improves overall data rates and reduces variability in individual user rates.
2) Modeling what collaborative sensing systems can "see" or be aware of in obstructed environments and how coverage benefits scale with increased penetration of collaborative vehicles.
3) Developing optimal information sharing policies to maximize situational awareness for autonomous nodes in resource-constrained network environments.
Updates provided to the D-STOP Business Advisory Council at the 2017 Symposium and Board Meeting: https://ctr.utexas.edu/2018/04/12/d-stop-2017-symposium-archive/
Updates provided to the D-STOP Business Advisory Council at the 2017 Symposium and Board Meeting: https://ctr.utexas.edu/2018/04/12/d-stop-2017-symposium-archive/
Updates provided to the D-STOP Business Advisory Council at the 2017 Symposium and Board Meeting: https://ctr.utexas.edu/2018/04/12/d-stop-2017-symposium-archive/
Updates provided to the D-STOP Business Advisory Council at the 2017 Symposium and Board Meeting: https://ctr.utexas.edu/2018/04/12/d-stop-2017-symposium-archive/
Updates provided to the D-STOP Business Advisory Council at the 2017 Symposium and Board Meeting: https://ctr.utexas.edu/2018/04/12/d-stop-2017-symposium-archive/
Updates provided to the D-STOP Business Advisory Council at the 2017 Symposium and Board Meeting: https://ctr.utexas.edu/2018/04/12/d-stop-2017-symposium-archive/
Through this project, the research team will leverage the computing resources and expertise at UT to develop a “data discovery environment” for transportation data to aid decision-making. Many efforts focus on leveraging transportation data to help travelers make decisions, but less thought has gone into a framework for using big data to help transportation agency staff and decision makers. The team will start by building the DDE for the Central Texas region, in collaboration with the local MPO, the City of Austin, and the local transit agency. Initially, the project will focus on creating more meaning from existing data sources, and as the project progresses, it will grow to include more novel data sources and methods. The data platform will be web-based and part of the research includes not only building the tool but developing appropriate protocols for access and governance.
This document discusses modeling strategies for autonomous and connected vehicles. It proposes modifying traditional four-step transportation models to account for autonomous vehicle adoption rates and different trip types. Autonomous vehicle passenger car equivalents and flow ratios are modeled based on vehicle speed, market penetration, and other factors. The document also describes plans for a 4G deployment test bed to demonstrate connected vehicle technologies on managed lanes in Dallas-Fort Worth and Virginia.
Advanced driver assistance systems (ADAS) are a key technology for improving road safety. But both current and proposed ADAS are limited in important ways. Vision- and lidar-based ADAS performs poorly in heavy rain, snow, or fog. Lack of vehicle situational awareness due to these sensing limitations will unfortunately be the cause of many accidents, including fatalities, for connected and automated vehicles in the years to come. The goal of this research is to develop and test a sensing strategy with robust perception: No blind spots, applicable to all driveable environments, and available in all weather conditions. We believe there are three key requirements for collaborative all-weather sensing:
– Precise vehicle positioning within a common reference frame
– Decimeter-accurate vision and radar mapping
– A means of quantifying the benefits of collaborative sensing
Vehicular radar and communication are the two primary means of using radio frequency (RF) signals in transportation systems. Automotive radars provide high-resolution sensing using proprietary waveforms in millimeter wave (mmWave) bands and vehicular communications allow vehicles to exchange safety messages or raw sensor data. Both the techniques can be used for applications such as forward collision warning, cooperative adaptive cruise control, and pre-crash applications.
Many areas of machine learning and data mining focus on point estimates of key parameters. In transportation, however, the inherent variance, and, critically, the need to understand the limits of that variance and the impact it may have, have long been understood to be important. Indeed, variance and other risk measures that capture the cost of the spread around the mean, are critical factors in understanding how people act. Thus they are critical for prediction, as well as for purposes of long term planning, where controlling risk may be equally important to controlling the mean (the point estimate).
There has been tremendous progress on large scale optimization techniques to enable the solution of large scale machine learning and data analytics problems. Stochastic Gradient Descent and its variants is probably the most-used large-scale optimization technique for learning. This has not yet seen an impact on the problem of statistical inference — namely, obtaining distributional information that might allow us to control the variance and hence the risk of certain solutions.
Investigation and findings on reservation-based intersections and managed lanes
Real-Time Signal Control and Traffic Stability
Congestion on urban arterials is largely centered around intersection control. Traditional traffic signal schemes are limited in their ability to adapt in real time to traffic conditions or by their ability to coordinate with each other to ensure adequate performance. Specifically, there is a tension between adaptivity (as with actuated signals) and coordination through pre-timed signals (signal progression). We propose to investigate whether routing protocols in telecommunications networks can be applied to resolve these problems. Specifically, the backpressure algorithm of Tassiulas & Emphremides (1992) can ensure system stability through decentralized control under relatively weak regularity conditions. It is as yet unknown whether this algorithm can be adapted to traffic signal systems, and if so, what modifications are needed. Traffic systems differ in several significant ways from telecommunication networks: each intersection approach has relatively few queues (lanes) that must be shared among traffic to various definitions. First-in, first-out constraints lead to head-of-line blocking effects, traffic waves move at a much slower speed than data packets, and traffic queues are tightly limited by physical space (finite buffers). Determining whether (and how) the backpressure concept can be adapted to traffic networks requires significant research, and has the potential to dramatically improve signal performance.
Improved Models for Managed Lane Operations
Managed lanes (ML) are increasingly being considered as a tool to mitigate congestion on highways with limited areas for capacity expansion. Managed lanes are dynamically priced based on the congestion level, and can be set either with the objective of maximum utilization (e.g., a public operator) or profit maximization (e.g., a private operator). Optimization models for determining these pricing policies make restrictive assumptions about the layout of these corridors (often a single entrance and exit) or knowledge of traveler characteristics on behalf of the modeler (e.g., distribution of willingness to pay). Developing new models to address these issues would allow for better utilization of these facilities.
Professor Robert W. Heath Jr. is the director of UT SAVES (Situation-Aware Vehicular Engineering Systems), which combines expertise in wireless communications, signal processing, and transportation research. UT SAVES collaborates with automotive companies like Honda R&D Americas on projects involving sensing, communication, and analytics for applications such as automated driving. Membership provides access to UT SAVES research and facilities, including graduate research assistants and experimental capabilities in areas like millimeter wave communication and sensor fusion. Current research projects focus on cooperative sensing, vehicle-to-everything communication, and applying 5G cellular networks to driving assistance technologies.
The Business Advisory Council meeting covered the following topics in 3 sentences or less:
The meeting covered updates on education and workforce development programs at the Engineering Education and Research Center including summer internships and distinguished lectures. Research updates were provided on 30 completed projects and 18 ongoing projects covering topics like connected corridors and autonomous vehicles. New proposed research was presented on topics such as video data analytics, traffic signal optimization, and modeling willingness to share trips in autonomous vehicles.
The document discusses managing mobility during the design-build reconstruction of the Dallas Horseshoe highway interchange project. It describes the project's high traffic volumes and constraints. It highlights the contractor's successes in maintaining access and maximizing work during limited closures. It stresses the importance of collaboration between the agency and contractor in developing traffic control plans and finding solutions to difficult situations.
The document summarizes research on the use of natural pozzolans and reclaimed/remediated fly ashes in concrete. Key findings include:
1) Natural pozzolans like pumice and metakaolin reduced heat of hydration and provided good strength and ASR resistance, while zeolites and shale also performed well.
2) Reclaimed and remediated fly ashes reduced heat of hydration and met ASTM standards, with fineness impacting performance.
3) Future research will assess blended fly ashes and develop rapid screening tests for supplementary cementitious materials.
More from Center for Transportation Research - UT Austin (20)
The Building Blocks of QuestDB, a Time Series Databasejavier ramirez
Talk Delivered at Valencia Codes Meetup 2024-06.
Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds.
It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeWalaa Eldin Moustafa
Dynamic policy enforcement is becoming an increasingly important topic in today’s world where data privacy and compliance is a top priority for companies, individuals, and regulators alike. In these slides, we discuss how LinkedIn implements a powerful dynamic policy enforcement engine, called ViewShift, and integrates it within its data lake. We show the query engine architecture and how catalog implementations can automatically route table resolutions to compliance-enforcing SQL views. Such views have a set of very interesting properties: (1) They are auto-generated from declarative data annotations. (2) They respect user-level consent and preferences (3) They are context-aware, encoding a different set of transformations for different use cases (4) They are portable; while the SQL logic is only implemented in one SQL dialect, it is accessible in all engines.
#SQL #Views #Privacy #Compliance #DataLake
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataKiwi Creative
Harness the power of AI-backed reports, benchmarking and data analysis to predict trends and detect anomalies in your marketing efforts.
Peter Caputa, CEO at Databox, reveals how you can discover the strategies and tools to increase your growth rate (and margins!).
From metrics to track to data habits to pick up, enhance your reporting for powerful insights to improve your B2B tech company's marketing.
- - -
This is the webinar recording from the June 2024 HubSpot User Group (HUG) for B2B Technology USA.
Watch the video recording at https://youtu.be/5vjwGfPN9lw
Sign up for future HUG events at https://events.hubspot.com/b2b-technology-usa/
Learn SQL from basic queries to Advance queriesmanishkhaire30
Dive into the world of data analysis with our comprehensive guide on mastering SQL! This presentation offers a practical approach to learning SQL, focusing on real-world applications and hands-on practice. Whether you're a beginner or looking to sharpen your skills, this guide provides the tools you need to extract, analyze, and interpret data effectively.
Key Highlights:
Foundations of SQL: Understand the basics of SQL, including data retrieval, filtering, and aggregation.
Advanced Queries: Learn to craft complex queries to uncover deep insights from your data.
Data Trends and Patterns: Discover how to identify and interpret trends and patterns in your datasets.
Practical Examples: Follow step-by-step examples to apply SQL techniques in real-world scenarios.
Actionable Insights: Gain the skills to derive actionable insights that drive informed decision-making.
Join us on this journey to enhance your data analysis capabilities and unlock the full potential of SQL. Perfect for data enthusiasts, analysts, and anyone eager to harness the power of data!
#DataAnalysis #SQL #LearningSQL #DataInsights #DataScience #Analytics
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Aggregage
This webinar will explore cutting-edge, less familiar but powerful experimentation methodologies which address well-known limitations of standard A/B Testing. Designed for data and product leaders, this session aims to inspire the embrace of innovative approaches and provide insights into the frontiers of experimentation!
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Round table discussion of vector databases, unstructured data, ai, big data, real-time, robots and Milvus.
A lively discussion with NJ Gen AI Meetup Lead, Prasad and Procure.FYI's Co-Found
The Ipsos - AI - Monitor 2024 Report.pdfSocial Samosa
According to Ipsos AI Monitor's 2024 report, 65% Indians said that products and services using AI have profoundly changed their daily life in the past 3-5 years.
1. Regret of Queueing Bandits
Sanjay Shakkottai
Department of Electrical and Computer Engineering
The University of Texas at Austin
Joint with Subhashini Krishnasamy, Rajat Sen, Ari Arapostathis (UT
Austin); Ramesh Johari (Stanford Univ.)
SAVES Meeting
April 10, 2018
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 1 / 44
2. Motivation (1/3)
Stream of multiple types of tasks
(jobs)
Multiple agents (servers) with
varying task dependent expertise
Match (schedule) tasks to agents
Dynamic decision making problem
because the number of tasks
changes with time based on past
decisions
Queueing Models
Rich history for such decision making
through queueing and scheduling for
various performance metrics
μ1
μ2
μ3
μK
agents / servers
μ1 = (μ11 μ12 … μ1U)
Queue U
Queue 2
Queue 1
tasks
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 2 / 44
3. Motivation (2/3)
Emerging Setting: Agent and task
characteristics unknown
Joint online learning and dynamic
optimization
Online Learning: Learn agent and task
characteristics/statistics
Formally, learn task dependent service
rates of agents
Dynamic Optimization: Using the
learned statistics, iteratively optimize to
achieve performance goals
μ1
μ2
μ3
μK
agents / servers
μ1 = (μ11 μ12 … μ1U)
Queue U
Queue 2
Queue 1
tasks
Applications
Online service systems (Uber, Lyft, Airbnb, Upwork); Scheduling in wireless
networks; Crowdsourced task allocation for human-machine systems
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 3 / 44
4. Motivation, Questions and Approach (3/3)
How well do we need to learn the
statistics?
What is the time-scale of learning?
How much resources are need for
learning?
Algorithms for joint online learning
and optimization?
Bandit Approach
Rich history for online learning and
optimization through bandits and regret
μ1
μ2
μ3
μK
agents / servers
μ1 = (μ11 μ12 … μ1U)
Queue U
Queue 2
Queue 1
tasks
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 4 / 44
5. Bandit Overview
μ1 μ2 μ3 μK
Arm 1 Arm 2 Arm 3 Arm K
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 5 / 44
6. Bandit Overview (1/3)
μ1 μ2 μ3 μK
Arm 1 Arm 2 Arm 3 Arm K
Multi-armed Bandit: K arms, each arm returns a random Bernoulli
reward if the arm is played
Can play one arm at each (discrete) time t
Associate a rv Xi (t) with arm i; with P(Xi (t) = 1) = µi
WLOG 1 > µ1 > µ2 ≥ . . . ≥ µK > 0
Reward: Accumulate reward at time t if the chosen arm returns ’1’
Key Question
Suppose {µi }K
i=1 are unknown. Which arm to play at each time to
maximize expected cumulative reward?
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 6 / 44
7. Bandit Overview (2/3)
μ1 μ2 μ3 μK
Arm 1 Arm 2 Arm 3 Arm K
As we play arms over time, we learn the values of {µi } (with varying
reliabilities)
Explore vs. Exploit: At time t should we play unknown arms (explore
to discover the arm with maximum µi ) OR play best known arm
(exploit past information)
Applications – Optimizing while Learning: Online advertising, drug
trials, wireless spectrum probing/sharing, finance, ...
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 7 / 44
8. Bandit Overview (3/3)
Policy π plays a sequence of arms {i1, i2, . . .} over time
Arm selection can depend on all past arm selections and reward
observations
Regret of a policy R(t): The expected accumulated loss of reward with
respect to a genie that knows the best arm (i.e. genie knows {µi })
R(t) = tµ1 − E
t
s=1
Xis (s)
Key Results (Lai and Robbins; Auer, Cesa-Bianchi and Fischer)
1. R(t) scales as K log(t)
2. Simple algorithms along with finite time upper and lower regret bounds
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 8 / 44
9. Part 1: Queue Regret for Server Selection/Matching
μ1
μ2
μ3
μK
arrivals
agents / servers
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 9 / 44
10. Queueing + Bandits
Arms as servers; departure from queue
if reward equals ’1’
Bernoulli job arrivals at rate λ ∈ (0, 1);
job backlogged in queue until served
Genie is stable: λ < µ1
Bandit algorithm schedules server
(’plays arm’) whenever queue is
backlogged
Applications: Online service systems
(Uber, Lyft, Airbnb, Upwork); financial
markets (limit order books);
communication networks, ...
μ1
μ2
μ3
μK
arrivals
agents / servers
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 10 / 44
11. Queue-Regret
time
Queue length
Regenerative cycle
Q(t) queue length at time t under bandit algorithm,
Q∗(t) queue length under “genie” policy
Always schedules the best server (here, server ’1’)
Ψ(t) is the queue-regret
Ψ(t) := E [Q(t) − Q∗
(t)] .
Interpretation: Ψ(t) traditional regret with caveat that reward
accumulated only if queue is backlogged
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 11 / 44
12. The Bandit vs. the Queueing Viewpoint
Q(t) = t
s=1 ((A(s) − D(s)) : The queue length equals [cumulative
arrivals - cumulative departures]
Q∗(t) = t
s=1 ((A(s) − D∗(s))
Ψ(t) = E t
s=1 (D∗(s) − D(s)) : Accumulated difference in service
In bandit terms, this is the difference in accumulated rewards
Bandit Viewpoint
Regret increases over time: Ψ(t) ≤ R(t) ∼ K log(t)
Queueing Viewpoint
As t → ∞ in steady state, E[Q(t) − Q∗(t)] ∼ 0
Key Question
How do we bridge these two different viewpoints?
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 12 / 44
13. Intuition – Bridging these Viewpoints
t
0 500 1000 1500 2000 2500 3000 3500 4000
Ψ(t)
0
5
10
15
20
25
30
35
40
Ω 1
t
O log3
t
t
O log3
t
O log t
log log t
Early Stage Late Stage
μ1
μ2
μ3
μK
arrivals
agents / servers
Over time, we (approximately) learn the values of {µi }
Eventually, learn “well enough” so that “effective service rate” exceeds
λ (arrival rate)
Queue length hits zero periodically =⇒ sample path queue-regret
“resets” at these epochs!
Takeaway
We should anticipate a phase transition in queue-regret behavior
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 13 / 44
14. Main Results – The Late Stage (1/3)
t
0 500 1000 1500 2000 2500 3000 3500 4000
Ψ(t)
0
5
10
15
20
25
30
35
40
Ω 1
t
O log3
t
t
O log3
t
O log t
log log t
Early Stage Late Stage
μ1
μ2
μ3
μK
arrivals
agents / servers
Queue length hits zero infinitely often; at these epochs the
sample-path regret “resets”
Queue-regret approximately a (discrete) derivative of the bandit
cumulative regret
Since the optimal cumulative regret scales like log(t), asymptotically
the optimal queue-regret should scale like 1/t
Takeaway
Order-wise matching upper and lower bounds showing O(1/t) behavior
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 14 / 44
15. Main Results – The Early Stage (2/3)
t
0 500 1000 1500 2000 2500 3000 3500 4000
Ψ(t)
0
5
10
15
20
25
30
35
40
Ω 1
t
O log3
t
t
O log3
t
O log t
log log t
Early Stage Late Stage
μ1
μ2
μ3
μK
arrivals
agents / servers
Still cannot (even approximately) identify the best server
Expected service rate is smaller than arrival rate λ
Queue continuously backlogged; queue-regret similar to bandit regret
Takeaway
Order-wise matching upper and lower bounds showing O(log(t)) behavior
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 15 / 44
16. Main Results – The Transition (3/3)
t
0 500 1000 1500 2000 2500 3000 3500 4000
Ψ(t)
0
5
10
15
20
25
30
35
40
Ω 1
t
O log3
t
t
O log3
t
O log t
log log t
Early Stage Late Stage
μ1
μ2
μ3
μK
arrivals
agents / servers
Time to switch scales at least as t = Ω(K/ ),
= (µ1 − λ) : Gap between the arrival rate and best service rate
Transition analysis through a heavily loaded setting as → 0
Scale K and ; demonstrate algorithm with queue-regret
O poly(log t)/ 2t for times that are arbitrarily close to Ω(K/ )
Takeaway
Phase transition time scales as (K/ ). Smaller means harder to learn
optimal server, and pushes out the phase transition time.
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 16 / 44
17. Implications
Scheduling: Much of scheduling literature focuses on steady-state or
long-time-scale behavior (e.g. Lyapunov arguments)
With emerging systems (online matching markets, wireless systems),
short-time behavior is equally important
In Online service systems, the number of jobs per customer might
reach steady-state only after a long time
In wireless 5G, much more flux between base-stations due to
densification
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 17 / 44
18. Related Work
Bandits and Cumulative Regret: Vast literature started with Lai &
Robbins 1985, and UCB (finite time bounds and simple algorithm)
Auer, Cesa-Bianchi, & Fischer 2002; See Bubeck and Cesa-Bianchi
2012 for a survey
Bandit and Queues: Rich history with focus on infinite horizon costs
and optimality of index policies (Gittins index 1979): Cox & Smith
1961, Buyukkoc, Varaiya and Walrand 1985, Van Mieghem 1995, Lott
& Teneketzis 2000, Mahajan & Teneketzis 2008, Nino-Mora 2006
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 18 / 44
19. Achieving the Bounds: −Greedy Thompson Sampling
time steps
exploit
explore
Bandit algorithms trade-off between explore and exploit steps
t− Greedy: With probability t, choose a server uniformly at random;
other-wise use Thompson sampling (here, t = 3K log2
(t)/t)
Thompson Sampling: Sampling and Bayesian update algorithm to
model and update {µi }K
i=1
Jointly used to both update “belief” on best arm as well as determine
the next arm to sample
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 19 / 44
20. −Greedy Structured Exploration
time steps
exploit
explore
Explore Step: t− Greedy algorithm provides structured exploration
t− Greedy: With probability t, choose a server uniformly at random;
other-wise use Thompson sampling (here, t = 3K log2
(t)/t)
Ensures that a poly-logarithmic amount of time used for “pure”
learning
Provides high probability upper bounds on number of sub-optimal
schedules in the exploit steps
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 20 / 44
21. A Primer on Thompson Sampling
0 0.2 0.4 0.6 0.8 1
pdf
0
2
4
6
8
10
12
β(1, 1)
β(3, 2)
β(10, 4)
β(100, 34)
Model µi as a random variable Qi for each i ∈ [K]
Initially, Uniform [0, 1] prior distribution for each Qi
Sample each distribution, choose arm/server with largest sample
Update the sampled arm’s distribution (Bayesian posterior
distribution) based on {0, 1} observations
Observations until time t: Arm i has seen Ai (t) ’1’s and Bi (t) ’0’s
Conjugate Prior: The posterior distribution of Qi given the
observations is Beta(Ai (t) + 1, Bi (t) + 1)
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 21 / 44
22. Achievability in the Late Stage
Theorem
Consider any problem instance (λ,µµµ). Then,
Ψ(t) = O K
log3
t
2t
for all t large enough (precise bounds available).
K = number of servers/arms
= (λ − µ1) : Gap between the arrival rate and best service rate
Scaling of K and (Heavy-Load Scaling)
For any β ∈ (0, 1), there exists a scaling of K with such that the
queue-regret scales as O poly(log t)/ 2t for all t > (K/ )β
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 22 / 44
23. Sketch of Proof: Achievability in the Late Stage (1/3)
time
Queue length
Regenerative cycle
Main Challenge: Coupled Cycles
Queues usually go through regenerative cycles which are independent
BUT HERE ...
Queue length evolution is dependent on the past history of bandit arm
schedules (cycles are coupled by the bandit)
Our Approach
1. High probability bound on the number of sub-optimal schedules
2. Bounds on length of regenerative cycle
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 23 / 44
24. Sketch of Proof: Achievability in the Late Stage (2/3)
time
Queue length
Regenerative cycle
Sub-optimal Schedules: Structured exploration via t−Greedy shows
that all servers including the sub-optimal ones, are sampled a
sufficiently large number of times
Ensures that algorithm schedules the correct link in the exploit phase
in the late stages with high probability
Busy Cycle of Queue: Coarse high probability upper bound on the
queue-length =⇒ coarse upper bound on busy cycle
Recursive Bound: Use above bound to get tighter bounds on the
queue-length, and in turn, the start of the current regenerative cycle
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 24 / 44
25. Sketch of Proof: Bounding the Busy Cycle (3/3)
time
Queue length
Regenerative cycle
Bandit System: Bandit bounds suggest that expected number of
sub-optimal arm pulls until time t is bounded by log(t)
Queueing System: There is a linear gap between arrival and best
service rate (scales as t)
Combination implies that even in the “worst case”, the current regenerative
cycle cannot extend too far into the past
Use this as a first cut bound on busy cycle, use this bound to bound
queue length, and again use this queue length bound to sharpen busy
cycle bound
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 25 / 44
26. Converse in the Late Stage
Theorem
For any problem instance (λ,µµµ) and any “reasonable” policy, the
queue-regret Ψ(t) satisfies
Ψ(t) ≥
λ
4
D(µµµ)(1 − α)(K − 1)
1
t
for infinitely many t, where D(µµµ) = ∆
KL µmin, µ∗+1
2
.
λ = arrival rate
∆ = rate gap between best and second best server/arm
α ∈ (0, 1), characterizes “reasonable” policy (formally, α-consistency in
bandit literature)
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 26 / 44
27. Sketch of Proof: Converse in the Late Stage
Sample Path Coupling: Construct a new bandit system (Bandit-Alt)
for which:
(a) Queue-regret is unchanged from bandit system
(b) Queue length of genie system (sample-path-wise) smaller than
Bandit-Alt
Bandit Bounds: (Roughly) use one time-step argument to show that
probability of using wrong server is O(1/t) for Bandit-Alt
Actually more delicate argument, as one step bounds not attainable
Show average (over time) bounds using bandit measure-change
arguments, and use pigeon-holing to get infinitely often bounds on
queue-regret
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 27 / 44
28. Numerical Results (1/2)
t
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
Ψ(t)
0
50
100
150
Phase Transition
Shift
ǫ = 0.05
ǫ = 0.1
ǫ = 0.15
System with 5 servers with ∈ {0.05, 0.1, 0.15}
The phase-transition point shifts towards the right as decreases
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 28 / 44
29. Numerical Results (2/2)
t
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
Ψ(t)
0
2
4
6
8
10
Q-ThS(Exp. Prob. = 3K log2
(t)
t )
Q-UCB
UCB-1
Thompson
Q-Ths(Exp. Prob. = 0.4K log2
(t)
t )
Comparison of queue-regret performance of Q-ThS, Q-UCB, UCB-1
amd Thompson Sampling
5 server system with u = 0.15 and ∆ = 0.17
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 29 / 44
30. Algorithm Design Questions
Learning with Queues: Should we explore more aggressively in initial
stages, because regret “resets” and we do not have an “asymptotic”
penalty?
Learning and Matching: More complex resource allocation tasks such
as matching
Interactions with multiple users/queues
Low dimensional structure across users and serves
Learning with Agent Dynamics: Agents/servers change over time
(agents arrive and depart)
How much to “trust” current “learned” agents and how much to explore
new agents?
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 30 / 44
31. Part 2: Holding-Cost Regret for Multi-Class Systems
μ1
μ2
μ3
μK
agents / servers
μ1 = (μ11 μ12 … μ1U)
Queue U
Queue 2
Queue 1
tasks
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 31 / 44
32. System Model: Multi-Class Queues
K servers and U queues (job
classes)
Bernoulli job arrivals at rate
λu ∈ (0, 1) to class u ∈ U
Service rate matrix
(µuk, 1 ≤ u ≤ U, 1 ≤ k ≤ K)
µuk ∈ (0, 1) is the unknown
service rate (success probability)
of server k for a job of type u
Models servers/agents whose
service rate depends on the type
of job/task
μ1
μ2
μ3
μK
agents / servers
μ1 = (μ11 μ12 … μ1U)
Queue U
Queue 2
Queue 1
tasks
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 32 / 44
33. System Model: Holding Costs
Holding cost: The expected total
waiting cost over finite time T
J(T) := E
T
t=1
βt
U
i=1
ci Qi (t)
β ∈ (0, 1] a discount factor
(useful when considering
T → ∞)
ck > 0 a waiting time cost in
queue class k
μ1
μ2
μ3
μK
agents / servers
μ1 = (μ11 μ12 … μ1U)
Queue U
Queue 2
Queue 1
tasks
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 33 / 44
34. The cµ Rule
Algorithm 1 The cµ Algorithm with costs {ck} and rates {µk}
At time t,
Choose job-server pairs according to the Max-Weight rule with weights
given by {ci µi,j } (product of the waiting cost and success probability)
Single Server Case
Serve the non-empty queue with largest
cuµu, u = 1, 2, . . . , U
Buyukkoc, Varaiya and Walrand 1985
Single server cµ rule is holding cost optimal
for any β ∈ (0, 1] and any T > 0
μ1
μ1 = (μ11 μ12 … μ1U)Queue U
Queue 2
Queue 1
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 34 / 44
35. Background: A History of the cµ Rule
The single server system (with known statistics)
Optimal expected holding costs for finite/infinite horizon (Cox and
Smith 1961, Buyukkoc, Varaiya and Walrand 1985)
Asymptotically optimal for convex costs in heavy traffic (Van Mieghem
1995)
The multi-server system – optimal policies for infinite horizon and
heavy traffic (with known statistics)
’N’, ’W’ networks – Harrison 1998, Bell and Williams 2001
Homogeneous servers – Lott and Teneketzis 2000, Glazebrook and
Nino Mora 2001
Generalized cµ rule (with convex costs, i.e. c = c(q) is convex) –
Mandelbaum and Stolyar 2004
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 35 / 44
36. Single Server Case: Main Result (1/2)
Service rates unknown – estimate using observed samples from past
allocation decisions, and use these parameters {ˆµu}
Unlike bandit algorithms, there is no explicit explore for forced learning
of server rates
J∗(T) is the holding cost with cµ rule
J(T) is the holding cost with empirical c ˆµ rule
Main Result: Constant Holding Cost Regret
J∗(T) − J(T) = O(1), and does not depend on T
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 36 / 44
37. Single Server Case: Main Result (2/2)
time
Queue length
Regenerative cycle
Intuition: Busy cycles are identical for all work conserving policies
Intuition: Within each cycle, all jobs need to be scheduled by any
policy (in some order). Implies sufficient number of server “samples”
Stability (busy cycle are sample-path identical) + Server samples gives
“free explore”
Implication: Learned priority order of queues “couples” with genie by
time τ = O(log t) w.p. 1/t3
Explore-Free System
No “random exploration” regret incurred unlike typical bandit systems
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 37 / 44
38. Multi Server Case: Instability
Stability of multi-server cµ rule
previously unknown
New Result: In general, the cµ
rule is unstable
Queue 1 has strict priority:
c1µ1j > c2µ2j , j = 1, 2
Server 1 has higher rate:
µ11 > µ12
π1 : stationary distribution of Q1
Queue 1
Queue 2
μ12
μ11
μ21
μ22
!1
!2
Result: Q2 is strongly unstable
If λ2 > π1(0)µ21 + π1(0, 1)µ22, then there exists positive constants
b0, b1, t0 s.t. ∀t ≥ t0,
P (Q2(t) < b2t) ≤ exp(−b1t)
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 38 / 44
39. Multi Server Case: Sufficient Conditions for Stability
Stability and Tails Bounds on Busy Cycles
λ · α < min
q∈QK
(R(q) · α), for some α > 0, α ∈ PU,
where PU is the probability simplex, Ql := {q ∈ ZU
+ : |q1| = l} for l ∈ Z+.
R(q) is the service rate to queues as a function of queue lengths
Condition in addition leads to exponential tails on busy cycles – uses
drift analysis from (Hajek 1982)
In limiting cases, provides close to complete stability region
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 39 / 44
40. Multi Server Case: A Conditional Explore Algorithm (1/2)
Algorithm 2 Conditional Explore c ˆµ Algorithm
At time t,
ε(t) ← 1 Nmin(t) < Υ(t) ,
B(t) ← independent Bernoulli sample of mean min{1, 3U log2
t
t }.
if ε(t) ∧ B(t) = 1 then
Explore: Schedule from E uniformly at random.
else
Exploit: Schedule according to the cµ rule with parameters ˆµ(t).
end if
Nmin(t) = mini,j Ni,j (t), Υ(t) = polylog(t)
Intuition: Explore only if the (worst case) number of samples is
sub-logarithmic with respect to time
Algorithm initially explores aggressively, but falls off quickly
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 40 / 44
41. Multi Server Case: A Conditional Explore Algorithm (2/2)
Algorithm 3 Conditional Explore c ˆµ Algorithm
At time t,
ε(t) ← 1 Nmin(t) < Υ(t) ,
B(t) ← independent Bernoulli sample of mean min{1, 3U log2
t
t }.
if ε(t) ∧ B(t) = 1 then
Explore: Schedule from E uniformly at random.
else
Exploit: Schedule according to the cµ rule with parameters ˆµ(t).
end if
Key point: Can show that with sufficiently high probability, algorithm
does not explore after sufficient time elapses
Every link has a constant probability of being scheduled in a busy cycle
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 41 / 44
42. Multi Server Case: The Main Result
time
Queue length
Regenerative cycle
O(1) Holding Cost Regret
For any (λ, µ) such that the cµ rule with known parameters has
exponential tails (satisfies stability), the holding cost regret with the
Conditional Explore c ˆµ Algorithm is O(1), i.e. independent of time.
Takeaway: Explore strategies different from traditional bandit settings
Intuition: Asymptotic free-learning in a queueing job/task allocation
setting
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 42 / 44
43. Conclusion
t
0 500 1000 1500 2000 2500 3000 3500 4000
Ψ(t)
0
5
10
15
20
25
30
35
40
Ω 1
t
O log3
t
t
O log3
t
O log t
log log t
Early Stage Late Stage
μ1
μ2
μ3
μK
arrivals
agents / servers
Queues + Bandits over finite time horizons
Phase transition in queue-regret
Initially increases logarithmically over time
Asymptotically goes down as 1/t
Learning based variants of the cµ rule
Conditional Explore to cut off asymptotic explore
Free-learning in queueing + learning
Holding cost regret does not scale with time
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 43 / 44
44. References
“Regret of Queueing Bandits”, S. Krishnasamy, R. Sen, R. Johari and
S. Shakkottai. Proceedings of the Thirtieth Annual Conference on
Neural Information Processing Systems (NIPS), Barcelona, Spain,
December 2016. Available at: https://arxiv.org/abs/1604.06377
“On Learning the c mu Rule: Single and Multiserver Settings”, S.
Krishnasamy, A. Arapostathis, R. Johari and S. Shakkottai, UT Austin
Technical Report, February 2018. Available at:
https://arxiv.org/abs/1802.06723
Sanjay Shakkottai (ECE, UT Austin) Regret of Queueing Bandits March 1, 2018 44 / 44