Machine Learning and Optimization For Traffic and Emergency Resource Management. Milos Hauskrecht Department of Computer Science University of Pittsburgh Students: Branislav Kveton, Tomas Singliar UPitt collaborators: Louise Comfort, JS Lin   External: Eli Upfal (Brown), Carlos Guestrin (CMU)
S-CITI related projects Modeling multivariate distributions of traffic variables  Optimization of (emergency) resources over unreliable transportation network Traffic monitoring and traffic incident detection Optimization of distributed systems with discrete and continuous variables: Traffic light control
S-CITI related projects Modeling multivariate distributions of traffic variables  Optimization of (emergency) resources over unreliable transportation network Traffic monitoring and traffic incident detection Optimization of control of distributed systems with discrete and continuous variables: Traffic light control
Traffic network Traffic network systems are  stochastic  (things happen at random) distributed  (at many places concurrently) Modeling and computational challenges Very complex structure Involved interactions High dimensionality PITTSBURGH
Challenges Modeling  the behavior of a large stochastic system Represent relations between traffic  variables Inference  (Answer queries about model) Estimate congestion in unobserved area using limited information Useful for a variety of optimization tasks Learning  (Discovering the model automatically) Interaction patterns not known Expert knowledge difficult to elicit Use Data Our solutions: probabilistic graphical models, statistical  Machine learning methods
Road traffic data We use PennDOT sensor network 155 sensors for volume and speed every 5 minutes
Models of traffic data Local interactions Markov random field Effects are circular Solution: Break the cycles
The all-independent assumption Unre a listic !
Mixture of trees A tree structure retains many dependencies but still loses some Have many trees to represent interactions
Latent variable model A combination of latent factors represent interactions
Four projects Modeling multivariate distributions of traffic variables   Optimization of (emergency) resources over unreliable transportation network Traffic monitoring and traffic incident detection Optimization of distributed systems with discrete and continuous variables: Traffic light control
Optimizations in unreliable transportation networks Unreliable network – connections (or nodes) may fail E.g. traffic congestion, power line failure
Optimizations in unreliable transportation networks Unreliable network – connections (nodes) may fail more than one connection may go down to
Optimizations in unreliable transportation networks Unreliable network – connections (nodes) may fail many connections may go down together
Optimizations in unreliable transportation networks Unreliable network – connections (nodes) may fail parts of the network may become disconnected
Optimizations of resources in unreliable transportation networks Example: emergency system.  Emergency vehicles use the network system to get from one location to the other
Optimizations of resources in unreliable transportation networks One failure here won’t prevent us from reaching the target, though the path taken can be longer
Optimizations of resources in unreliable transportation networks Two failures can get the two nodes disconnected
Optimizations of resources in unreliable transportation networks Emergencies can occur at different locations and they can come with different priorities
Optimizations of resources in unreliable transportation networks …  considering all possible emergencies, it may be better to change the initial location of the vehicle to get a better coverage
Optimizations of resources in unreliable transportation networks …  If emergencies are concurrent and/or some connections are very unreliable it may be better to use two vehicles  …
Optimizations of resources in unreliable transportation networks where to place the vehicles and how many of them to achieve the coverage with  the best expected cost-benefit tradeoff ? ? ? ? ? ? ? ? ? ?
Solving the problem A two stage stochastic program with recourse   Problem stages: Find optimal allocations  of resources (em. vehicles) Match (repeatedly) emergency demands with allocated vehicles  after failures  occur Curse of dimensionality:  many possible failure configurations in the second stage Our solution:  Stochastic (MC) approximations (UAI-2001, UAI-2003) Current:   adapt to continuous random quantities (congestion rates,traffic flows and their relations)
Four projects Modeling multivariate distributions of traffic variables   Optimization of (emergency) resources over unreliable transportation network Traffic monitoring and traffic incident detection Optimization of distributed systems with discrete and continuous variables: Traffic light control
Incident detection on dynamic data incident incident no incident
Incident detection algorithms Incidents detected indirectly through caused congestion State of the art:  California 2 algorithm If OCC(up) – OCC(down) > T1, next step If [OCC(up) – OCC(down)]/ OCC(up) > T2, next step If [OCC(up) – OCC(down)]/ OCC(down) > T3, possible accident If previous condition persists for another time step,  sound alarm Hand-calibrated  for the specific section of the road Occupancy spikes Occupancy falls
Incident detection algorithms Machine Learning approach (ICML 2006) Use a set of simple feature detectors and learn the classifier from the data Improved performance California 2 SVM based model
Four projects Modeling multivariate distributions of traffic variables   Optimization of (emergency) resources over unreliable transportation network Traffic monitoring and traffic incident detection Optimization of control of distributed systems with discrete and continuous variables: Traffic light control
Dynamic traffic management A set of intersections A set of connection (roads)   in between intersections Traffic lights regulating  the traffic flow on roads Traffic lights are controlled independently Objective:  coordinate  traffic lights to minimize congestions and maximize the throughput
Solutions  Problems:   how to model the dynamic behavior of the system how to optimize the plans  Our solutions  (NIPS 03,ICAPS 04, UAI 04, IJCAI 05, ICAPS 06, AAAI 06) Model:  Factored hybrid Markov decision processes  continuous and discrete variables  Optimization:   Hybrid Approximate Linear Programming optimizations over 30 dimensional continuous state spaces and 25 dimensional action spaces Goals:  hundreds of state and action variables
Thank you  Questions

Machine Learning and Optimization For Traffic and Emergency ...

  • 1.
    Machine Learning andOptimization For Traffic and Emergency Resource Management. Milos Hauskrecht Department of Computer Science University of Pittsburgh Students: Branislav Kveton, Tomas Singliar UPitt collaborators: Louise Comfort, JS Lin External: Eli Upfal (Brown), Carlos Guestrin (CMU)
  • 2.
    S-CITI related projectsModeling multivariate distributions of traffic variables Optimization of (emergency) resources over unreliable transportation network Traffic monitoring and traffic incident detection Optimization of distributed systems with discrete and continuous variables: Traffic light control
  • 3.
    S-CITI related projectsModeling multivariate distributions of traffic variables Optimization of (emergency) resources over unreliable transportation network Traffic monitoring and traffic incident detection Optimization of control of distributed systems with discrete and continuous variables: Traffic light control
  • 4.
    Traffic network Trafficnetwork systems are stochastic (things happen at random) distributed (at many places concurrently) Modeling and computational challenges Very complex structure Involved interactions High dimensionality PITTSBURGH
  • 5.
    Challenges Modeling the behavior of a large stochastic system Represent relations between traffic variables Inference (Answer queries about model) Estimate congestion in unobserved area using limited information Useful for a variety of optimization tasks Learning (Discovering the model automatically) Interaction patterns not known Expert knowledge difficult to elicit Use Data Our solutions: probabilistic graphical models, statistical Machine learning methods
  • 6.
    Road traffic dataWe use PennDOT sensor network 155 sensors for volume and speed every 5 minutes
  • 7.
    Models of trafficdata Local interactions Markov random field Effects are circular Solution: Break the cycles
  • 8.
  • 9.
    Mixture of treesA tree structure retains many dependencies but still loses some Have many trees to represent interactions
  • 10.
    Latent variable modelA combination of latent factors represent interactions
  • 11.
    Four projects Modelingmultivariate distributions of traffic variables Optimization of (emergency) resources over unreliable transportation network Traffic monitoring and traffic incident detection Optimization of distributed systems with discrete and continuous variables: Traffic light control
  • 12.
    Optimizations in unreliabletransportation networks Unreliable network – connections (or nodes) may fail E.g. traffic congestion, power line failure
  • 13.
    Optimizations in unreliabletransportation networks Unreliable network – connections (nodes) may fail more than one connection may go down to
  • 14.
    Optimizations in unreliabletransportation networks Unreliable network – connections (nodes) may fail many connections may go down together
  • 15.
    Optimizations in unreliabletransportation networks Unreliable network – connections (nodes) may fail parts of the network may become disconnected
  • 16.
    Optimizations of resourcesin unreliable transportation networks Example: emergency system. Emergency vehicles use the network system to get from one location to the other
  • 17.
    Optimizations of resourcesin unreliable transportation networks One failure here won’t prevent us from reaching the target, though the path taken can be longer
  • 18.
    Optimizations of resourcesin unreliable transportation networks Two failures can get the two nodes disconnected
  • 19.
    Optimizations of resourcesin unreliable transportation networks Emergencies can occur at different locations and they can come with different priorities
  • 20.
    Optimizations of resourcesin unreliable transportation networks … considering all possible emergencies, it may be better to change the initial location of the vehicle to get a better coverage
  • 21.
    Optimizations of resourcesin unreliable transportation networks … If emergencies are concurrent and/or some connections are very unreliable it may be better to use two vehicles …
  • 22.
    Optimizations of resourcesin unreliable transportation networks where to place the vehicles and how many of them to achieve the coverage with the best expected cost-benefit tradeoff ? ? ? ? ? ? ? ? ? ?
  • 23.
    Solving the problemA two stage stochastic program with recourse Problem stages: Find optimal allocations of resources (em. vehicles) Match (repeatedly) emergency demands with allocated vehicles after failures occur Curse of dimensionality: many possible failure configurations in the second stage Our solution: Stochastic (MC) approximations (UAI-2001, UAI-2003) Current: adapt to continuous random quantities (congestion rates,traffic flows and their relations)
  • 24.
    Four projects Modelingmultivariate distributions of traffic variables Optimization of (emergency) resources over unreliable transportation network Traffic monitoring and traffic incident detection Optimization of distributed systems with discrete and continuous variables: Traffic light control
  • 25.
    Incident detection ondynamic data incident incident no incident
  • 26.
    Incident detection algorithmsIncidents detected indirectly through caused congestion State of the art: California 2 algorithm If OCC(up) – OCC(down) > T1, next step If [OCC(up) – OCC(down)]/ OCC(up) > T2, next step If [OCC(up) – OCC(down)]/ OCC(down) > T3, possible accident If previous condition persists for another time step, sound alarm Hand-calibrated for the specific section of the road Occupancy spikes Occupancy falls
  • 27.
    Incident detection algorithmsMachine Learning approach (ICML 2006) Use a set of simple feature detectors and learn the classifier from the data Improved performance California 2 SVM based model
  • 28.
    Four projects Modelingmultivariate distributions of traffic variables Optimization of (emergency) resources over unreliable transportation network Traffic monitoring and traffic incident detection Optimization of control of distributed systems with discrete and continuous variables: Traffic light control
  • 29.
    Dynamic traffic managementA set of intersections A set of connection (roads) in between intersections Traffic lights regulating the traffic flow on roads Traffic lights are controlled independently Objective: coordinate traffic lights to minimize congestions and maximize the throughput
  • 30.
    Solutions Problems: how to model the dynamic behavior of the system how to optimize the plans Our solutions (NIPS 03,ICAPS 04, UAI 04, IJCAI 05, ICAPS 06, AAAI 06) Model: Factored hybrid Markov decision processes continuous and discrete variables Optimization: Hybrid Approximate Linear Programming optimizations over 30 dimensional continuous state spaces and 25 dimensional action spaces Goals: hundreds of state and action variables
  • 31.
    Thank you Questions

Editor's Notes

  • #5 The road network of a major city like Pittsburgh is an incredibly complex thing, both in structure and behavior. Things happen there at random * an at many places at the same time. So if we want to model it, we must deal with complex spatial structure, involved interactions between traffic flows, high dimensionality of data. And what happens is of course strongly confounded by variables we do not see, such as weather and time. With this in mind, we undertake the study of Pittsburgh roads.
  • #6 To successfully model a system behavior, we need to get these three things right. We want to be able to capture the general nature of how traffic interacts spatially. The sensor coverage is not all that great, especially since they break; also the sensors and cameras are not on the roads very densely. We can have a computer tell us what is the most likely picture of traffic where we do not see. Experts have intuitions from simulations , but nobody really knows what traffic d oes. For all this types of problems, the probabilistic paradigm of modeling provides a well understood and precise formulation, and often an algorithm.
  • #7 Let us discuss for a while how the data is collected and what kind of structure it has. Pennsylvania Department of Transportation, PennDOT for short, operates about 200 sensors in Pittsburgh. They look like this * pole at the side of the road and measure speed and number of cars every 5 minutes. In this work we focus on car counts but our models directly generalize to other quantities such as speed. In the picture, the red dots are positions of the sensors.
  • #8 Here we have a photo that we all recognize; it shows central Pittsburgh. I am going to draw the sensors as the bluish circles and in particular I am going to talk about the highlighted sensor in the middle. The lines connecting the sensors roughly correspond to major connectivity of the road network. Now, let us choose a direction traffic, say southbound, and look at two sensors upstream and downstream from our highlighted sensor. Certainly, if the traffic backs up at the downstream sensor *, it may back up all the way to the Point State Park sensor. On the other hand, if there is a lot of cars coming from the upstream sensor, the road certainly will not be empty here *. But you will agree with me that once you know the status in these two places, and any other places that feed traffic to our sensor, you know all there is to know about the surrounding situation. Right? It doesn’t matter if traffic is jammed here *; if it flow s freely here * , the traffic jam need not interest us as far as the highlighted sensor is concerned. In other words, the MARKOV PROPERTY holds, which implies the Markov Random Field is the correct model. These effects can become circular . The circularity that is at the heart of the trouble with markov Random Field computational complexity. So basically if you are going to hope for any simplification, you have to break the cycle s. That will be the leitmotif for the rest of the talk and I’ll show you three different models that do it.
  • #9 Now people do all sorts of analyses when it comes to traffic. For example physicists look at phase transitions, which is really cool, since you certainly could say that traffic froze and it would be a good metaphor. But when it comes to modeling probabilities, typically the simplifying assumption is that everything is independent. [click] I told you why this is not the case in the previous slide and I’ll show you later how strong of an assumption it is. So it would be good to retain maximum dependencies possible, while maintaining that acyclicity property. What is a maximal acyclic graph? A tree, of course.
  • #10 So here I drew one of the possible trees. We still lose some dependencies though and we would like to account for them. So we have another tree account for them, whose edges are orange color in this picture. Now nobody had developed a machine learning algorithm for this, because we are working with continuous variables. But Marina Meila has developed something similar for discrete variables. So we sat down and redid her derivations for the continuous case. And I’m going to show you what kind of tree structures we learn.
  • #11 Now I will show you an alternative representation of the dependencies. We don’t like the links so we remove them and we introduce abstract hidden variables. They represent the links we had there indirectly. The good thing about them is that they don’t have to be just that, they may represent external variables such as the weather that would have links to all sensors, or the presence of a Steelers’ game that would have links to around here *. The mathematical name for the concrete model we are using is Mixture of Factor Analyzers (so it’s again a mixture model.)