SlideShare a Scribd company logo
PUNCH
Partitioning Using Natural-Cut Heuristics

       Daniel Delling (Microsoft Research)
    Andrew V. Goldberg (Microsoft Research)
     Ilya Razenshteyn (Moscow University)
    Renato F. Werneck (Microsoft Research)



                 May 19, 2010
Motivation




      Goal: process a continental-sized road network in parallel
      (Europe: 18M nodes and 43M arcs).
      The first natural step: divide it into “small” parts with few
      arcs between them.
      Partition problems are NP-hard, but routinely solved using
      different heuristics.
An example
Applications: routing on road networks
   Idea: Precompute distances between boundary nodes of each cell.


   Overlay Graph:
       Nodes — boundary nodes
       Edges between boundary nodes,


                               Search Graph:
                         t
                                    Source and target cell,
          s
                                    Overlay graph,
                                    Use bidirectional Dijkstra
       Number of cut edges affects the performance heavily.
       More applications: arc-flags and reach.
Existing solvers




       METIS [KK’99], SCOTCH [PR’96], KAPPA [HSS’10],
       KASPAR [OS’10], KAFFPA [SS’10]. General purpose, some
       are fast, some produce very good solution.
       There are many more . . .
       Our goal: partitioner tailored to road networks,
       emphasize quality, still fast enough in practice.
Formal definition



      Input: undirected graph G = (V , E ).
      Result: partition (V = V1 ∪ V2 ∪ . . . ∪ Vk , with Vi ∩ Vj = ∅).
      Goal: minimize number of edges between Vi .
      Two common variants:
          given U, require |Vi | ≤ U for every i,
          given k and , require |Vi | ≤ (1 + ) n/k .
      We focus on the first one.
      Rebalancing is possible.
Intuition




   Road networks: dense regions (grids, cities) interleaved with
   natural cuts (mountains, parks, rivers, deserts, sparse areas,
   freeways).
Summary of the algorithm




      Filtering:
           Contracts dense regions,
           Reduces graph size,
           Preserves natural cuts structure.
      Assembly phase:
           Works with much smaller graph,
           Finds actual partition.
Outline of the talk


   Introduction


   Natural cuts


   Assembly phase


   Experiments


   Conclusion
Outline of the talk


   Introduction


   Natural cuts


   Assembly phase


   Experiments


   Conclusion
Natural cuts


      Sparse sets that
      separate dense areas.
      Minimum s–t cuts are
      trivial (average degree
      < 3).
      Sparsest cuts would be
      OK, but they are
      intractable.
      Our notion of natural
      cut is both tractable and
      useful.
Natural cuts



                   Pick centers in a
                   randomized manner.
               v   Compute minimum cut
                   between the core and the
                   ring.
                   Repeat until every node
                   is inside of at least two
                   cores.
Natural cuts



                       Pick centers in a
                       randomized manner.
               v       Compute minimum cut
                       between the core and the
          U/10 nodes   ring.
                       Repeat until every node
                       is inside of at least two
                       cores.
Natural cuts



                       Pick centers in a
                       randomized manner.
               v       Compute minimum cut
                       between the core and the
          U/10 nodes   ring.
                       Repeat until every node
                       is inside of at least two
                       cores.
           U nodes
Natural cuts



                       Pick centers in a
                       randomized manner.
               v       Compute minimum cut
                       between the core and the
          U/10 nodes   ring.
                       Repeat until every node
                       is inside of at least two
                       cores.
           U nodes
Natural cuts
   Take a union of all natural cuts found and contract everything
   between them.
   The resulting graph is much smaller than the original one.
       U = 106 — 18M nodes to 10K nodes
       U = 103 — 18M nodes to 1.3M nodes
Natural cuts
   Take a union of all natural cuts found and contract everything
   between them.
   The resulting graph is much smaller than the original one.
       U = 106 — 18M nodes to 10K nodes
       U = 103 — 18M nodes to 1.3M nodes
Natural cuts
   Take a union of all natural cuts found and contract everything
   between them.
   The resulting graph is much smaller than the original one.
       U = 106 — 18M nodes to 10K nodes
       U = 103 — 18M nodes to 1.3M nodes
Tiny cuts


   The most obvious natural cuts — 1-cuts and 2-cuts.
       We handle them explicitly before processing natural cuts.
       Greatly decreases graph size (by half) and overall running
       time,
Outline of the talk


   Introduction


   Natural cuts


   Assembly phase


   Experiments


   Conclusion
Assembly phase




   Three ingredients:
       Greedy algorithm,
       Local search,
       Multistart and combination heuristics (optional).
Greedy algorithm

      We combine well-connected small fragments in a randomized
      fashion.
      Repeat until maximal.
      Finds initial partition.
Greedy algorithm

      We combine well-connected small fragments in a randomized
      fashion.
      Repeat until maximal.
      Finds initial partition.
Greedy algorithm

      We combine well-connected small fragments in a randomized
      fashion.
      Repeat until maximal.
      Finds initial partition.
Greedy algorithm

      We combine well-connected small fragments in a randomized
      fashion.
      Repeat until maximal.
      Finds initial partition.
Greedy algorithm

      We combine well-connected small fragments in a randomized
      fashion.
      Repeat until maximal.
      Finds initial partition.
Greedy algorithm

      We combine well-connected small fragments in a randomized
      fashion.
      Repeat until maximal.
      Finds initial partition.
The local search

      Pick two neighboring cells, disassemble them, apply greedy
      algorithm to the subproblem.
      Repeat several times for every pair of neighboring cells.
The local search

      Pick two neighboring cells, disassemble them, apply greedy
      algorithm to the subproblem.
      Repeat several times for every pair of neighboring cells.
The local search

      Pick two neighboring cells, disassemble them, apply greedy
      algorithm to the subproblem.
      Repeat several times for every pair of neighboring cells.
Multistart and combination heuristics




   Since the local search is typically much faster than the natural cuts
   detection, we can use the following two heuristics:
       Multistart: since the local search is randomized, we can
       repeat it several times.
       Combination: keep track of several solutions, and combine
       them from time to time.
Outline of the talk


   Introduction


   Natural cuts


   Assembly phase


   Experiments


   Conclusion
Experimental evaluation




      C++/OpenMP
      Tested on Western Europe map (18M nodes, 43M arcs).
      Machine: Intel Xeon X5680 (two six-core 3.33GHz CPUs)
      with DDR3-1333MHz RAM.
A typical use-case




       Europe, U = 64K .
       Tiny cuts contraction: 25 seconds (18M nodes to 9M nodes).
       Natural cuts identification: 50 seconds (12 cores, 9M nodes to
       100K nodes).
       Greedy + local search: only 5 seconds (12 cores).
Running times on Europe


                                                                Tiny cuts
              200




                                                                Natural cuts
                                                                Greedy + Local search
              180




                                                                                   q
              160
              140
              120




                                                                        q
   Time (s)

              100
              80




                                                          q
              60




                                             q
              40




                    q

                                q
                    q     q     q            q            q             q          q
              20




                          q
                          q
                    q
                                q
                                             q
                                                          q             q          q
              0




                    210   212   214          216          218          220        222

                                      maximum cell size
Influence of ϕ
   The local search tries every edge ϕ times.
              15000




                                                                                   Dependence on phi

                            q
              14000
              13000
   cut size

              12000




                                    q

                                        q
                                            q
                                                q
                                                    q
                                                        q
                                                            q
                                                                 q
              11000




                                                                     q   q
              10000




                      0.1       1                           10               100    1000        10000

                                                                 time (s)
Multistart and combination
   Combination helps!
              15000




                                                                                         Dependence on phi
                                                                                         Multistart
                            q
                                                                                         Combination
              14000
              13000
   cut size

              12000




                                    q

                                        q
                                            q
                                                q
                                                    q
                                                        q                    q       q
                                                            q                                  q
                                                                 q
              11000




                                                                     q   q
                                                                                 q
                                                                                         q
                                                                                                    q
              10000




                      0.1       1                           10               100             1000       10000

                                                                 time (s)
Balanced partitions



       Recall that there are two variants of requirements on |Vi |:
           given U, require |Vi | ≤ U for every i,
           given k and , require |Vi | ≤ (1 + ) n/k .
       PUNCH solves the first, but most existing solvers find
        -balanced partitions.
       Rebalancing:
           Run PUNCH with U = (1 + ) n/k ,
           If there are too many regions, redistribute them.
Balanced partitions: Europe, = 0.03 (quality and time)

                              Cut size
     K   PUNCH    KAFFPA    KASPAR       KAPPA    SCOTCH    METIS
     2      129       130       133          —        469       —
     4      309       412       355         543       952      846
     8      634       749       774         986      1667     1675
    16     1293      1454      1401        1760      2922     3519
    32     2289      2428      2595        3186      4336     7424
    64     3828      4240      4502        5290      6772    11313

                               Time
     K   PUNCH    KAFFPA    KASPAR       KAPPA    SCOTCH    METIS
     2      255      1013      1946          —        12       —
     4      215      1823      2168         441       25       29
     8      176      2067      2232         418        39      29
    16      151      2340      2553         498        52      31
    32      130      2445      2599         418        65      31
    64      145      2533      2534         308        77      30
Concluding remarks




      PUNCH is good for partitioning road networks.
      It doesn’t work as well on instances without natural cuts.
Seattle by METIS
Seattle by PUNCH
Portland by METIS
Portland by PUNCH
Vancouver by METIS
Vancouver by PUNCH
Thank you for your attention!

More Related Content

What's hot

Object-oriented Programming-with C#
Object-oriented Programming-with C#Object-oriented Programming-with C#
Object-oriented Programming-with C#
Doncho Minkov
 
Abstract class in c++
Abstract class in c++Abstract class in c++
Abstract class in c++
Sujan Mia
 
Deep Learning A-Z™: Boltzmann Machines - Restricted Boltzmann Machine
Deep Learning A-Z™: Boltzmann Machines  - Restricted Boltzmann MachineDeep Learning A-Z™: Boltzmann Machines  - Restricted Boltzmann Machine
Deep Learning A-Z™: Boltzmann Machines - Restricted Boltzmann Machine
Kirill Eremenko
 
What is Python Lambda Function? Python Tutorial | Edureka
What is Python Lambda Function? Python Tutorial | EdurekaWhat is Python Lambda Function? Python Tutorial | Edureka
What is Python Lambda Function? Python Tutorial | Edureka
Edureka!
 
Inheritance in c++
Inheritance in c++Inheritance in c++
Inheritance in c++
Vineeta Garg
 
Inheritance in java
Inheritance in javaInheritance in java
Inheritance in java
Lovely Professional University
 
Gremlin's Anatomy
Gremlin's AnatomyGremlin's Anatomy
Gremlin's Anatomy
Stephen Mallette
 
How to use Map() Filter() and Reduce() functions in Python | Edureka
How to use Map() Filter() and Reduce() functions in Python | EdurekaHow to use Map() Filter() and Reduce() functions in Python | Edureka
How to use Map() Filter() and Reduce() functions in Python | Edureka
Edureka!
 
L7 fuzzy relations
L7 fuzzy relationsL7 fuzzy relations
L7 fuzzy relations
Mohammad Umar Rehman
 
Chapter 02: Classes Objects and Methods Java by Tushar B Kute
Chapter 02: Classes Objects and Methods Java by Tushar B KuteChapter 02: Classes Objects and Methods Java by Tushar B Kute
Chapter 02: Classes Objects and Methods Java by Tushar B Kute
Tushar B Kute
 
Deep Learning A-Z™: Convolutional Neural Networks (CNN) - Module 2
Deep Learning A-Z™: Convolutional Neural Networks (CNN) - Module 2Deep Learning A-Z™: Convolutional Neural Networks (CNN) - Module 2
Deep Learning A-Z™: Convolutional Neural Networks (CNN) - Module 2
Kirill Eremenko
 
MobileNet V3
MobileNet V3MobileNet V3
MobileNet V3
Wonbeom Jang
 
Command line arguments
Command line argumentsCommand line arguments
Command line arguments
Ashok Raj
 
Learn C# Programming - Encapsulation & Methods
Learn C# Programming - Encapsulation & MethodsLearn C# Programming - Encapsulation & Methods
Learn C# Programming - Encapsulation & Methods
Eng Teong Cheah
 
Deep Learning using Keras
Deep Learning using KerasDeep Learning using Keras
Deep Learning using Keras
Aly Abdelkareem
 
StarGAN
StarGANStarGAN
StarGAN
Joonhyung Lee
 
MobileNet - PR044
MobileNet - PR044MobileNet - PR044
MobileNet - PR044
Jinwon Lee
 
Deep Learning A-Z™: Artificial Neural Networks (ANN) - Module 1
Deep Learning A-Z™: Artificial Neural Networks (ANN) - Module 1Deep Learning A-Z™: Artificial Neural Networks (ANN) - Module 1
Deep Learning A-Z™: Artificial Neural Networks (ANN) - Module 1
Kirill Eremenko
 
chapter22.ppt
chapter22.pptchapter22.ppt
chapter22.ppt
Tareq Hasan
 
Inheritance in c++ by Manan Pasricha
Inheritance in c++ by Manan PasrichaInheritance in c++ by Manan Pasricha
Inheritance in c++ by Manan Pasricha
MananPasricha
 

What's hot (20)

Object-oriented Programming-with C#
Object-oriented Programming-with C#Object-oriented Programming-with C#
Object-oriented Programming-with C#
 
Abstract class in c++
Abstract class in c++Abstract class in c++
Abstract class in c++
 
Deep Learning A-Z™: Boltzmann Machines - Restricted Boltzmann Machine
Deep Learning A-Z™: Boltzmann Machines  - Restricted Boltzmann MachineDeep Learning A-Z™: Boltzmann Machines  - Restricted Boltzmann Machine
Deep Learning A-Z™: Boltzmann Machines - Restricted Boltzmann Machine
 
What is Python Lambda Function? Python Tutorial | Edureka
What is Python Lambda Function? Python Tutorial | EdurekaWhat is Python Lambda Function? Python Tutorial | Edureka
What is Python Lambda Function? Python Tutorial | Edureka
 
Inheritance in c++
Inheritance in c++Inheritance in c++
Inheritance in c++
 
Inheritance in java
Inheritance in javaInheritance in java
Inheritance in java
 
Gremlin's Anatomy
Gremlin's AnatomyGremlin's Anatomy
Gremlin's Anatomy
 
How to use Map() Filter() and Reduce() functions in Python | Edureka
How to use Map() Filter() and Reduce() functions in Python | EdurekaHow to use Map() Filter() and Reduce() functions in Python | Edureka
How to use Map() Filter() and Reduce() functions in Python | Edureka
 
L7 fuzzy relations
L7 fuzzy relationsL7 fuzzy relations
L7 fuzzy relations
 
Chapter 02: Classes Objects and Methods Java by Tushar B Kute
Chapter 02: Classes Objects and Methods Java by Tushar B KuteChapter 02: Classes Objects and Methods Java by Tushar B Kute
Chapter 02: Classes Objects and Methods Java by Tushar B Kute
 
Deep Learning A-Z™: Convolutional Neural Networks (CNN) - Module 2
Deep Learning A-Z™: Convolutional Neural Networks (CNN) - Module 2Deep Learning A-Z™: Convolutional Neural Networks (CNN) - Module 2
Deep Learning A-Z™: Convolutional Neural Networks (CNN) - Module 2
 
MobileNet V3
MobileNet V3MobileNet V3
MobileNet V3
 
Command line arguments
Command line argumentsCommand line arguments
Command line arguments
 
Learn C# Programming - Encapsulation & Methods
Learn C# Programming - Encapsulation & MethodsLearn C# Programming - Encapsulation & Methods
Learn C# Programming - Encapsulation & Methods
 
Deep Learning using Keras
Deep Learning using KerasDeep Learning using Keras
Deep Learning using Keras
 
StarGAN
StarGANStarGAN
StarGAN
 
MobileNet - PR044
MobileNet - PR044MobileNet - PR044
MobileNet - PR044
 
Deep Learning A-Z™: Artificial Neural Networks (ANN) - Module 1
Deep Learning A-Z™: Artificial Neural Networks (ANN) - Module 1Deep Learning A-Z™: Artificial Neural Networks (ANN) - Module 1
Deep Learning A-Z™: Artificial Neural Networks (ANN) - Module 1
 
chapter22.ppt
chapter22.pptchapter22.ppt
chapter22.ppt
 
Inheritance in c++ by Manan Pasricha
Inheritance in c++ by Manan PasrichaInheritance in c++ by Manan Pasricha
Inheritance in c++ by Manan Pasricha
 

Similar to Graph Partition with Natural Cuts

Clustering of graphs and search of assemblages
Clustering of graphs and search of assemblagesClustering of graphs and search of assemblages
Clustering of graphs and search of assemblages
Data-Centric_Alliance
 
Scalable membership management
Scalable membership management Scalable membership management
Scalable membership management
Vinay Setty
 
Randomness conductors
Randomness conductorsRandomness conductors
Randomness conductors
wtyru1989
 
Finding Dense Subgraphs
Finding Dense SubgraphsFinding Dense Subgraphs
Finding Dense Subgraphs
Carlos Castillo (ChaTo)
 
Advances in Directed Spanners
Advances in Directed SpannersAdvances in Directed Spanners
Advances in Directed Spanners
Grigory Yaroslavtsev
 
Lecture16
Lecture16Lecture16
Lecture16
Dharmesh Goyal
 
Vlsi physical design automation on partitioning
Vlsi physical design automation on partitioningVlsi physical design automation on partitioning
Vlsi physical design automation on partitioning
Sushil Kundu
 
My pp tno sound
My pp tno soundMy pp tno sound
My pp tno sound
dicosmo178
 
Mesh Generation and Topological Data Analysis
Mesh Generation and Topological Data AnalysisMesh Generation and Topological Data Analysis
Mesh Generation and Topological Data Analysis
Don Sheehy
 
Average Sensitivity of Graph Algorithms
Average Sensitivity of Graph AlgorithmsAverage Sensitivity of Graph Algorithms
Average Sensitivity of Graph Algorithms
Yuichi Yoshida
 
ACM 2013-02-25
ACM 2013-02-25ACM 2013-02-25
ACM 2013-02-25
Ted Dunning
 
NMR Hardware3.pptx
NMR Hardware3.pptxNMR Hardware3.pptx
NMR Hardware3.pptx
MahmoudSakkal2
 
An automated and user-friendly optical tweezers for biomolecular investigat...
An automated and user-friendly optical  tweezers for biomolecular  investigat...An automated and user-friendly optical  tweezers for biomolecular  investigat...
An automated and user-friendly optical tweezers for biomolecular investigat...
Dr. Pranav Rathi
 
Separation of Macromolecules by Their Size: The Mean Span Dimension
Separation of Macromolecules by Their Size: The Mean Span DimensionSeparation of Macromolecules by Their Size: The Mean Span Dimension
Separation of Macromolecules by Their Size: The Mean Span Dimension
cypztm
 
Perimeter Institute Talk 2009
Perimeter Institute Talk 2009Perimeter Institute Talk 2009
Perimeter Institute Talk 2009
Dave Bacon
 
Lecture 02 internet video search
Lecture 02 internet video searchLecture 02 internet video search
Lecture 02 internet video search
zukun
 
Connected Dominating Set and Short Cycles
Connected Dominating Set and Short CyclesConnected Dominating Set and Short Cycles
Connected Dominating Set and Short Cycles
Neeldhara Misra
 
Oxford 05-oct-2012
Oxford 05-oct-2012Oxford 05-oct-2012
Oxford 05-oct-2012
Ted Dunning
 
Introduction to Hadron Structure from Lattice QCD
Introduction to Hadron Structure from Lattice QCDIntroduction to Hadron Structure from Lattice QCD
Introduction to Hadron Structure from Lattice QCD
Christos Kallidonis
 
Vlsiphysicaldesignautomationonpartitioning 120219012744-phpapp01
Vlsiphysicaldesignautomationonpartitioning 120219012744-phpapp01Vlsiphysicaldesignautomationonpartitioning 120219012744-phpapp01
Vlsiphysicaldesignautomationonpartitioning 120219012744-phpapp01
Hemant Jha
 

Similar to Graph Partition with Natural Cuts (20)

Clustering of graphs and search of assemblages
Clustering of graphs and search of assemblagesClustering of graphs and search of assemblages
Clustering of graphs and search of assemblages
 
Scalable membership management
Scalable membership management Scalable membership management
Scalable membership management
 
Randomness conductors
Randomness conductorsRandomness conductors
Randomness conductors
 
Finding Dense Subgraphs
Finding Dense SubgraphsFinding Dense Subgraphs
Finding Dense Subgraphs
 
Advances in Directed Spanners
Advances in Directed SpannersAdvances in Directed Spanners
Advances in Directed Spanners
 
Lecture16
Lecture16Lecture16
Lecture16
 
Vlsi physical design automation on partitioning
Vlsi physical design automation on partitioningVlsi physical design automation on partitioning
Vlsi physical design automation on partitioning
 
My pp tno sound
My pp tno soundMy pp tno sound
My pp tno sound
 
Mesh Generation and Topological Data Analysis
Mesh Generation and Topological Data AnalysisMesh Generation and Topological Data Analysis
Mesh Generation and Topological Data Analysis
 
Average Sensitivity of Graph Algorithms
Average Sensitivity of Graph AlgorithmsAverage Sensitivity of Graph Algorithms
Average Sensitivity of Graph Algorithms
 
ACM 2013-02-25
ACM 2013-02-25ACM 2013-02-25
ACM 2013-02-25
 
NMR Hardware3.pptx
NMR Hardware3.pptxNMR Hardware3.pptx
NMR Hardware3.pptx
 
An automated and user-friendly optical tweezers for biomolecular investigat...
An automated and user-friendly optical  tweezers for biomolecular  investigat...An automated and user-friendly optical  tweezers for biomolecular  investigat...
An automated and user-friendly optical tweezers for biomolecular investigat...
 
Separation of Macromolecules by Their Size: The Mean Span Dimension
Separation of Macromolecules by Their Size: The Mean Span DimensionSeparation of Macromolecules by Their Size: The Mean Span Dimension
Separation of Macromolecules by Their Size: The Mean Span Dimension
 
Perimeter Institute Talk 2009
Perimeter Institute Talk 2009Perimeter Institute Talk 2009
Perimeter Institute Talk 2009
 
Lecture 02 internet video search
Lecture 02 internet video searchLecture 02 internet video search
Lecture 02 internet video search
 
Connected Dominating Set and Short Cycles
Connected Dominating Set and Short CyclesConnected Dominating Set and Short Cycles
Connected Dominating Set and Short Cycles
 
Oxford 05-oct-2012
Oxford 05-oct-2012Oxford 05-oct-2012
Oxford 05-oct-2012
 
Introduction to Hadron Structure from Lattice QCD
Introduction to Hadron Structure from Lattice QCDIntroduction to Hadron Structure from Lattice QCD
Introduction to Hadron Structure from Lattice QCD
 
Vlsiphysicaldesignautomationonpartitioning 120219012744-phpapp01
Vlsiphysicaldesignautomationonpartitioning 120219012744-phpapp01Vlsiphysicaldesignautomationonpartitioning 120219012744-phpapp01
Vlsiphysicaldesignautomationonpartitioning 120219012744-phpapp01
 

Recently uploaded

RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
KAMESHS29
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Neo4j
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
DianaGray10
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 

Recently uploaded (20)

RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 

Graph Partition with Natural Cuts

  • 1. PUNCH Partitioning Using Natural-Cut Heuristics Daniel Delling (Microsoft Research) Andrew V. Goldberg (Microsoft Research) Ilya Razenshteyn (Moscow University) Renato F. Werneck (Microsoft Research) May 19, 2010
  • 2. Motivation Goal: process a continental-sized road network in parallel (Europe: 18M nodes and 43M arcs). The first natural step: divide it into “small” parts with few arcs between them. Partition problems are NP-hard, but routinely solved using different heuristics.
  • 4. Applications: routing on road networks Idea: Precompute distances between boundary nodes of each cell. Overlay Graph: Nodes — boundary nodes Edges between boundary nodes, Search Graph: t Source and target cell, s Overlay graph, Use bidirectional Dijkstra Number of cut edges affects the performance heavily. More applications: arc-flags and reach.
  • 5. Existing solvers METIS [KK’99], SCOTCH [PR’96], KAPPA [HSS’10], KASPAR [OS’10], KAFFPA [SS’10]. General purpose, some are fast, some produce very good solution. There are many more . . . Our goal: partitioner tailored to road networks, emphasize quality, still fast enough in practice.
  • 6. Formal definition Input: undirected graph G = (V , E ). Result: partition (V = V1 ∪ V2 ∪ . . . ∪ Vk , with Vi ∩ Vj = ∅). Goal: minimize number of edges between Vi . Two common variants: given U, require |Vi | ≤ U for every i, given k and , require |Vi | ≤ (1 + ) n/k . We focus on the first one. Rebalancing is possible.
  • 7. Intuition Road networks: dense regions (grids, cities) interleaved with natural cuts (mountains, parks, rivers, deserts, sparse areas, freeways).
  • 8. Summary of the algorithm Filtering: Contracts dense regions, Reduces graph size, Preserves natural cuts structure. Assembly phase: Works with much smaller graph, Finds actual partition.
  • 9. Outline of the talk Introduction Natural cuts Assembly phase Experiments Conclusion
  • 10. Outline of the talk Introduction Natural cuts Assembly phase Experiments Conclusion
  • 11. Natural cuts Sparse sets that separate dense areas. Minimum s–t cuts are trivial (average degree < 3). Sparsest cuts would be OK, but they are intractable. Our notion of natural cut is both tractable and useful.
  • 12. Natural cuts Pick centers in a randomized manner. v Compute minimum cut between the core and the ring. Repeat until every node is inside of at least two cores.
  • 13. Natural cuts Pick centers in a randomized manner. v Compute minimum cut between the core and the U/10 nodes ring. Repeat until every node is inside of at least two cores.
  • 14. Natural cuts Pick centers in a randomized manner. v Compute minimum cut between the core and the U/10 nodes ring. Repeat until every node is inside of at least two cores. U nodes
  • 15. Natural cuts Pick centers in a randomized manner. v Compute minimum cut between the core and the U/10 nodes ring. Repeat until every node is inside of at least two cores. U nodes
  • 16. Natural cuts Take a union of all natural cuts found and contract everything between them. The resulting graph is much smaller than the original one. U = 106 — 18M nodes to 10K nodes U = 103 — 18M nodes to 1.3M nodes
  • 17. Natural cuts Take a union of all natural cuts found and contract everything between them. The resulting graph is much smaller than the original one. U = 106 — 18M nodes to 10K nodes U = 103 — 18M nodes to 1.3M nodes
  • 18. Natural cuts Take a union of all natural cuts found and contract everything between them. The resulting graph is much smaller than the original one. U = 106 — 18M nodes to 10K nodes U = 103 — 18M nodes to 1.3M nodes
  • 19. Tiny cuts The most obvious natural cuts — 1-cuts and 2-cuts. We handle them explicitly before processing natural cuts. Greatly decreases graph size (by half) and overall running time,
  • 20. Outline of the talk Introduction Natural cuts Assembly phase Experiments Conclusion
  • 21. Assembly phase Three ingredients: Greedy algorithm, Local search, Multistart and combination heuristics (optional).
  • 22. Greedy algorithm We combine well-connected small fragments in a randomized fashion. Repeat until maximal. Finds initial partition.
  • 23. Greedy algorithm We combine well-connected small fragments in a randomized fashion. Repeat until maximal. Finds initial partition.
  • 24. Greedy algorithm We combine well-connected small fragments in a randomized fashion. Repeat until maximal. Finds initial partition.
  • 25. Greedy algorithm We combine well-connected small fragments in a randomized fashion. Repeat until maximal. Finds initial partition.
  • 26. Greedy algorithm We combine well-connected small fragments in a randomized fashion. Repeat until maximal. Finds initial partition.
  • 27. Greedy algorithm We combine well-connected small fragments in a randomized fashion. Repeat until maximal. Finds initial partition.
  • 28. The local search Pick two neighboring cells, disassemble them, apply greedy algorithm to the subproblem. Repeat several times for every pair of neighboring cells.
  • 29. The local search Pick two neighboring cells, disassemble them, apply greedy algorithm to the subproblem. Repeat several times for every pair of neighboring cells.
  • 30. The local search Pick two neighboring cells, disassemble them, apply greedy algorithm to the subproblem. Repeat several times for every pair of neighboring cells.
  • 31. Multistart and combination heuristics Since the local search is typically much faster than the natural cuts detection, we can use the following two heuristics: Multistart: since the local search is randomized, we can repeat it several times. Combination: keep track of several solutions, and combine them from time to time.
  • 32. Outline of the talk Introduction Natural cuts Assembly phase Experiments Conclusion
  • 33. Experimental evaluation C++/OpenMP Tested on Western Europe map (18M nodes, 43M arcs). Machine: Intel Xeon X5680 (two six-core 3.33GHz CPUs) with DDR3-1333MHz RAM.
  • 34. A typical use-case Europe, U = 64K . Tiny cuts contraction: 25 seconds (18M nodes to 9M nodes). Natural cuts identification: 50 seconds (12 cores, 9M nodes to 100K nodes). Greedy + local search: only 5 seconds (12 cores).
  • 35. Running times on Europe Tiny cuts 200 Natural cuts Greedy + Local search 180 q 160 140 120 q Time (s) 100 80 q 60 q 40 q q q q q q q q q 20 q q q q q q q q 0 210 212 214 216 218 220 222 maximum cell size
  • 36. Influence of ϕ The local search tries every edge ϕ times. 15000 Dependence on phi q 14000 13000 cut size 12000 q q q q q q q q 11000 q q 10000 0.1 1 10 100 1000 10000 time (s)
  • 37. Multistart and combination Combination helps! 15000 Dependence on phi Multistart q Combination 14000 13000 cut size 12000 q q q q q q q q q q q 11000 q q q q q 10000 0.1 1 10 100 1000 10000 time (s)
  • 38. Balanced partitions Recall that there are two variants of requirements on |Vi |: given U, require |Vi | ≤ U for every i, given k and , require |Vi | ≤ (1 + ) n/k . PUNCH solves the first, but most existing solvers find -balanced partitions. Rebalancing: Run PUNCH with U = (1 + ) n/k , If there are too many regions, redistribute them.
  • 39. Balanced partitions: Europe, = 0.03 (quality and time) Cut size K PUNCH KAFFPA KASPAR KAPPA SCOTCH METIS 2 129 130 133 — 469 — 4 309 412 355 543 952 846 8 634 749 774 986 1667 1675 16 1293 1454 1401 1760 2922 3519 32 2289 2428 2595 3186 4336 7424 64 3828 4240 4502 5290 6772 11313 Time K PUNCH KAFFPA KASPAR KAPPA SCOTCH METIS 2 255 1013 1946 — 12 — 4 215 1823 2168 441 25 29 8 176 2067 2232 418 39 29 16 151 2340 2553 498 52 31 32 130 2445 2599 418 65 31 64 145 2533 2534 308 77 30
  • 40. Concluding remarks PUNCH is good for partitioning road networks. It doesn’t work as well on instances without natural cuts.
  • 47. Thank you for your attention!