Streets4MPI (Parallel Programming Project)
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Streets4MPI (Parallel Programming Project)

on

  • 596 views

 

Statistics

Views

Total Views
596
Views on SlideShare
596
Embed Views
0

Actions

Likes
0
Downloads
6
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Streets4MPI (Parallel Programming Project) Presentation Transcript

  • 1. Streets4MPI(Parallel Programming Project) Julian Fietkau Joachim Nitschke University of Hamburg April 4th, 2012
  • 2. Agenda Julian Fietkau, Joachim Nitschke Agenda Introduction Simulation Concept Traffic load Traffic jam tolerance Implementation Parallelization Results Summary 2 / 34
  • 3. Introduction Julian Fietkau, Joachim Nitschke Project task, revisited Decide on a problem that may be solved using parallel processing, and implement a solution. → Street traffic simulation Main Caveat Realistic traffic predictions can only be made using an exceedingly detailed model. This makes things prohibitively complicated. 3 / 34
  • 4. Introduction Julian Fietkau, Joachim Nitschke Streets4MPI is here We can simulate thousands of cars on the streets of Hamburg: We use OpenStreetMap data for navigation. We incorporate shortest-path algorithms. We model road congestion and its effect on the actual driving speed. We optimize the road system based on which roads are heavily used and which one are empty. We can visualize all of this as dynamic heatmaps. Also, we can do most of this in parallel using MPI. 4 / 34
  • 5. Simulation: Concept Julian Fietkau, Joachim Nitschke Revision: Discrete macroscopic simulation Simulation runs in steps: Traffic load in one step influences the driver’s behavior in the next step Abstract from single cars, traffic lights etc. to daily traffic Display traffic development over longer time periods and influences on street network 5 / 34
  • 6. Simulation: Concept Julian Fietkau, Joachim Nitschke Street network 6 / 34
  • 7. Simulation: Concept Julian Fietkau, Joachim Nitschke Trips Representation for a resident’s daily traffic Shortest path between two nodes 7 / 34
  • 8. Simulation: Concept Julian Fietkau, Joachim Nitschke Trips 8 / 34
  • 9. Simulation: Traffic load Julian Fietkau, Joachim Nitschke Traffic load: Effect on driving speed Heavy traffic slows cars down How can we calculate the deceleration? Assumption: Cars keep safe braking distances ⇒ By looking at the braking distance, we calculate the actual speed Braking distance and actual speed lbraking = lntrips − lcar street vactual = lbraking · abraking · 2 9 / 34
  • 10. Simulation: Traffic load Julian Fietkau, Joachim Nitschke Oscillation Problem: Drivers show oscillating behavior 10 / 34
  • 11. Simulation: Traffic jam tolerance Julian Fietkau, Joachim Nitschke Idea Solution: Each driver gets an individual traffic jam tolerance Traffic jam tolerance vperceived = vactual + (videal − vactual ) · ftolerance Compromise: We assign a (random) traffic jam tolerance to each process and all its residents This is because one copy of the street network graph can accomodate one traffic jam tolerance factor, and each MPI process has its own copy anyway 11 / 34
  • 12. Simulation: Traffic jam tolerance Julian Fietkau, Joachim Nitschke Result 12 / 34
  • 13. Simulation: Traffic jam tolerance Julian Fietkau, Joachim Nitschke Improvement: Reworking redundant nodes Problem: Redundant nodes are used to model curved streets Solution: Merge edges 13 / 34
  • 14. Implementation Julian Fietkau, Joachim Nitschke Python Streets4MPI is written in Python (2.6 compatible) External modules: imposm.parser, pygraph, mpi4py, PIL 14 / 34
  • 15. Implementation Julian Fietkau, Joachim Nitschke OSM parser Import data from OpenStreetMap XML-based semi-structured data format: OSM May provide additional information: street types/sizes, speed limits, residential/industrial/commercial zones 15 / 34
  • 16. Implementation Julian Fietkau, Joachim Nitschke <?xml version="1.0" encoding="UTF-8"?> <osm version="0.6" generator="CGImap 0.0.2"> <bounds minlat="54.0889580" minlon="12.2487570" maxlat="54.0913900" maxlon="12.2524800"/> <node id="298884269" lat="54.0901746" lon="12.2482632" user="SvenHRO" uid="46882" visible="true" version="1" changeset="676636" timestamp="2008-09-21T21:37:45Z"/> <node id="261728686" lat="54.0906309" lon="12.2441924" user="PikoWinter" uid="36744" visible="true" version="1" changeset="323878" timestamp="2008-05-03T13:39:23Z"/> ... <node id="298884272" lat="54.0901447" lon="12.2516513" user="SvenHRO" uid="46882" visible="true" version="1" changeset="676636" timestamp="2008-09-21T21:37:45Z"/> <way id="26659127" user="Masch" uid="55988" visible="true" version="5" changeset="4142606" timestamp="2010-03-16T11:47:08Z"> <nd ref="292403538"/> <nd ref="298884289"/> ... <nd ref="261728686"/> <tag k="highway" v="unclassified"/> <tag k="name" v="Pastower Straße"/> </way> ... </osm> 16 / 34
  • 17. Implementation Julian Fietkau, Joachim Nitschke Visualization Simulation and visualization run independantly Visualization uses the Python Imaging Library to render the map and traffic load data to image files Supports many different data modes and two color modes (heatmap and grayscale) 17 / 34
  • 18. Parallelization Julian Fietkau, Joachim Nitschke mpi4py Object oriented interface on top of the MPI specifications Provides all usual MPI routines communicator = MPI.COMM_WORLD object = None if communicator.Get_rank() == 0: object = Object() object = communicator.bcast(object, root=0) Single program multiple data (mostly) MPI code contained within our main class 19 / 34
  • 19. Parallelization Julian Fietkau, Joachim Nitschke Algorithm Each MPI process. . . . . . generates its own copy of the street network . . . generates trips for its (equally divided) subset of all residents . . . gets its own traffic jam resistance . . . calculates the shortest paths for its residents and the resulting traffic load After every simulation step, each process gets sent the traffic loads from all other processes (via mpi.allgather) Complete results are saved to disk by process #0 20 / 34
  • 20. Parallelization Julian Fietkau, Joachim Nitschke Algorithm: visualization OSM python #0 iteration 0 python #1 iteration 0 ... MPI python #n iteration 0 results exchange traffic data time iteration 1 iteration 1 iteration 1 results exchange traffic data iteration 2 iteration 2 iteration 2 results exchange traffic data iteration 3 iteration 3 iteration 3 results 21 / 34
  • 21. Parallelization Julian Fietkau, Joachim Nitschke Performance (I) Run times of Streets4MPI (Stellingen dataset, 1000 residents, 50 simulation steps) 3000 100 total time efficiency 2500 75 2000 efficiency [%] total time [s] 1500 50 1000 25 500 0 0 1 4 8 12 16 # of processes 22 / 34
  • 22. Parallelization Julian Fietkau, Joachim Nitschke Performance (II) Run times of Streets4MPI (Stellingen dataset, 100 residents, 200 simulation steps) 1200 100 total time efficiency 1000 75 800 efficiency [%] total time [s] 600 50 400 25 200 0 0 1 4 8 12 16 # of processes 23 / 34
  • 23. Parallelization Julian Fietkau, Joachim Nitschke Weaknesses Some activities (e.g. initial I/O, road construction simulation) are not easily parallelized using our current model Disk activity by process #0 makes it drag behind and leave others waiting for synchronization Shortest path calculation is not optimal for the distributed case 24 / 34
  • 24. Parallelization Julian Fietkau, Joachim Nitschke Improvement: Shortest path revisited Highest calculation costs are due to the shortest path calculations Current implementation: Dijkstra’s algorithm 2 Complexity: O(nnodes ) nresidents Executed ∼ nprocesses times Static shortest path vs. dynamic shortest path 25 / 34
  • 25. Parallelization Julian Fietkau, Joachim Nitschke Dynamic shortest path Idea: Calculate shortest paths once and update them only when the edge weights change Performance gain through local influence of changes 26 / 34
  • 26. Parallelization Julian Fietkau, Joachim Nitschke Dynamic shortest path: Increasing a weight +𝚫w 27 / 34
  • 27. Parallelization Julian Fietkau, Joachim Nitschke Dynamic shortest path: Decreasing a weight -𝚫w 28 / 34
  • 28. Results Julian Fietkau, Joachim Nitschke Simulation speed-up results Results are mostly satisfactory, but could very likely be improved There are constant time elements not yet parallelized It bears mentioning that using the current model (traffic jam resistance per process) increases simulation quality with number of processes, so real efficiency is slightly better than measured 29 / 34
  • 29. Results Julian Fietkau, Joachim Nitschke Project Goals Simulation working and producing nontrivial results Parallel processing in Python working Visualization working Further work needed: better parallelization(?), documentation 30 / 34
  • 30. Summary Julian Fietkau, Joachim Nitschke Most Important Points Simple traffic simulation Macro level with congestion analysis, street development, visualization MPI on Python 31 / 34
  • 31. Miscellaneous: Literature Julian Fietkau, Joachim Nitschke Literature Weber, B.; MÃŒller, P.; Wonka, P.; Gross, M.: Interactive Geometric Simulation of 4D Cities In: EUROGRAPHICS 28 (2009), Nr. 2 Chandy, K.M.; Misra, J.: Distributed computation on graphs: Shortest path algorithms In: Commun. ACM, vol. 25, no. 11, pp. 833 – 837, Nov. 1982 Antonio, J. K.; Huang, G. M.; Tsai, W. K.: A fast distributed shortest path algorithm for a class of hierar- chically clustered data networks In: IEEE Trans. Comput., vol. 41, pp. 710 – 724, June 1992 32 / 34
  • 32. Miscellaneous: Weblinks Julian Fietkau, Joachim Nitschke Weblinks Project website http://jfietkau.github.com/Streets4MPI/ (some time soon) GitHub repository http://github.com/jfietkau/Streets4MPI (available right now!) Project wiki http://pwiki.julian-fietkau.de/ (might go offline soon-ish) 33 / 34
  • 33. Miscellaneous: Download and Usage Julian Fietkau, Joachim Nitschke Download and Usage These slides are published under the CC-BY-SA 3.0 license. All pictures and illustrations not created by Streets4MPI are based on content from the OpenClipArt Project. Download these slides and give feedback: http://www.julian-fietkau.de/streets4mpi_final 34 / 34