Graph Data Science for
Digital Twin -
Wire Management
Graph Pattern Matching + GDS algorithms: computing at scale on your graph
Pierre Halftermeyer
Sr. Solutions Engineer
PhD in Graph Theory
3½ years at
pierre[AT]neo4j.com
Neo4j Inc. All rights reserved 2023
2
Use case:
Optimal constrained
wire management
Neo4j Inc. All rights reserved 2023
3
Neo4j Inc. All rights reserved 2023
4
Neo4j Inc. All rights reserved 2023
5
Neo4j Inc. All rights reserved 2023
6
Neo4j Inc. All rights reserved 2023
7
Neo4j Inc. All rights reserved 2023
8
Neo4j Inc. All rights reserved 2023
9
Neo4j Inc. All rights reserved 2023
10
Neo4j Inc. All rights reserved 2023
11
Neo4j Inc. All rights reserved 2023
12
Roadmap
Modeling and
Ingestion 1
3
2
4
Setting the problem
instance (sources and
targets)
Building full set of
possible connections
(constraint-compliant)
Apply GDS
algorithm
Neo4j Inc. All rights reserved 2023
13
Roadmap
Modeling and
Ingestion 1
3
2
4
Setting the problem
instance (sources and
targets)
Building full set of
possible connections
(constraint-compliant)
Apply GDS
algorithm
Ingest Minecraft-like
data
Unit voxels 3D grid modeling
Neo4j Inc. All rights reserved 2023
14
....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ......................
....................+⍐+++++++++++⍐+......................
....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ......................
....................+⍐+++++++++++⍐+......................
....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ......................
.........................................................
# Floor 4
.........................................................
....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ......................
....................+⍐+++++++++++⍐+......................
....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ......................
....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ......................
....................+⍐+++++++++++⍐+......................
....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ......................
....................+⍐+++++++++++⍐+......................
....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ......................
Neo4j Inc. All rights reserved 2023
15
▢ means :Voxel / Room by default
+ means :Lobby
⍐ means :Elevator:Bottom
Ingestion of (x, y, z) Voxels
Neo4j Inc. All rights reserved 2023
16
// parse csv
LOAD CSV FROM "file:///building.txt" AS line
WITH collect(line[0]) as lines
WITH [l IN lines | CASE WHEN l STARTS WITH "#" THEN "#" ELSE l
END] AS lines
CALL apoc.coll.split(lines, "#")
YIELD value
WITH value
WITH collect(value) AS building
WITH range(0, size(building)-1) AS ixs, building
WITH [ix IN ixs | {floor_num:ix+1, floor:building[ix]}] AS building
UNWIND building AS floor
WITH floor.floor_num AS z, floor.floor AS floor
UNWIND range(0, size(floor)-1) AS x
WITH z, x, floor[x] AS line
UNWIND range(0, size(line)-1) AS y
WITH z, x, y, substring(line,y,1) AS val
CREATE (:Voxel {x:x, y:y, z:z, val:val});
// Delete non relevant voxels
MATCH (n {val:'.'})
DELETE n;
// label Lobbys and Elevators
MATCH (s:Voxel WHERE s.val IN ["+", "⍐"])
SET s:Lobby;
MATCH (s:Voxel WHERE s.val IN ["⍐"])
SET s:Elevator:Bottom;
Connect adjacent
Voxels in 3D
Neo4j Inc. All rights reserved 2023
17
MATCH (s:Voxel)
WITH s.x AS x, s.y AS y, s.z AS z, s
ORDER BY z
WITH x, y, collect(s) AS line
CALL apoc.nodes.link(line, "HAS_ABOVE");
MATCH (a)-[r:HAS_ABOVE]->(b)
WHERE b.z - a.z <> 1
DELETE r;
MATCH (s:Voxel)
WITH s.x AS x, s.y AS y, s.z AS z, s
ORDER BY y
WITH x, z, collect(s) AS line
CALL apoc.nodes.link(line, "HAS_EAST");
MATCH (a)-[r:HAS_EAST]->(b)
WHERE b.y - a.y <> 1
DELETE r;
MATCH (s:Voxel)
WITH s.x AS x, s.y AS y, s.z AS z, s
ORDER BY x
WITH y, z, collect(s) AS line
CALL apoc.nodes.link(line, "HAS_NORTH");
MATCH (a)-[r:HAS_NORTH]->(b)
WHERE b.x - a.x <> 1
Neo4j Inc. All rights reserved 2023
18
Roadmap
Modeling and
Ingestion 1
3
2
4
Setting the problem
instance (sources and
targets)
Building the full set of
possible wires
(constraint-compliant)
Apply GDS
algorithm
Rules
Let’s PoC with mock data
Neo4j Inc. All rights reserved 2023
19
If a wire has to join two Voxels from distinct
floors, pass the wire via the elevator.
No vertical drilling.
Our customer.
Neo4j Inc. All rights reserved 2023
20
Neo4j Inc. All rights reserved 2023
21
Vertical connections use the Elevator
MATCH (a:!(Elevator&Bottom))-[r:HAS_ABOVE]->(b)
DELETE r;
MATCH (a:Elevator&Bottom)-[r:HAS_ABOVE]->(b)
SET b:Elevator:Top;
Neo4j Inc. All rights reserved 2023
22
Cables crossing room are possible but not
more than 3 rooms in a row. If more, you have
to pass via the lobby. Cable shouldn’t be
longer than 10 units of distance. This is
possible:
Voxel-Voxel-Voxel-Lobby-Lobby-Lobby-Voxel-Voxel
Our customer.
Neo4j Inc. All rights reserved 2023
23
Neo4j Inc. All rights reserved 2023
24
Neo4j Inc. All rights reserved 2023
25
1
0
u
n
i
t
s
m
a
x
Possible cable laying via Lobby.
MATCH (s:Voxel&!Lobby)
CALL {
WITH s
MATCH p=(s)(
()-[r1:HAS_NORTH|HAS_EAST]-(:Voxel&!Lobby)
){0,2}(:Voxel&!Lobby)-[:HAS_NORTH|HAS_EAST]-(:Lobby)(
()-[r2:HAS_EAST|HAS_NORTH|HAS_ABOVE]-(:Lobby)
){0,5}(:Lobby)-[:HAS_NORTH|HAS_EAST]-(:Voxel&!Lobby)(
()-[r3:HAS_NORTH|HAS_EAST]-(:Voxel&!Lobby)
){0,2}(t WHERE t<>s)
WITH s, t, size(r1+r2+r3)+2 AS wire_length,
[n IN nodes(p)|elementId(n)] AS intermediary_voxels
WHERE wire_length <= 10
CREATE (s)-[:LAYABLE_WIRE {cost:1+log(wire_length), intermediary_voxels:intermediary_voxels}]->(t)
} IN TRANSACTIONS OF 10 rows;
Neo4j Inc. All rights reserved 2023
26
Possible cable laying not via Lobby.
MATCH (s:Voxel)
CALL {
WITH s
MATCH p=(s)(
()-[r1:HAS_NORTH|HAS_EAST]-(:Voxel&!Lobby)
){1,2}(t:Voxel&!Lobby)
WITH s, t, size(r1) AS wire_length,
[n IN nodes(p)|elementId(n)] AS intermediary_voxels
WHERE wire_length <= 10
CREATE (s)-[:LAYABLE_WIRE {cost:log(1+wire_length), intermediary_voxels:intermediary_voxels}]->(t)
} IN TRANSACTIONS OF 10 rows;
Neo4j Inc. All rights reserved 2023
27
Neo4j Inc. All rights reserved 2023
28
Roadmap
Modeling and
Ingestion 1
3
2
4
Setting the problem
instance (sources and
targets)
Building full set of
possible connections
(constraint-compliant)
Apply GDS
algorithm
IT Room and Target
Voxel
Neo4j Inc. All rights reserved 2023
29
MATCH (s:Voxel WHERE s.val = "▢")
WITH s, rand() AS ord
ORDER BY ord
LIMIT 3
SET s:ITRoom;
Neo4j Inc. All rights reserved 2023
30
Define IT Locals (randomly for demo)
MATCH (s:Voxel&!Lobby)
WHERE rand() < 0.50
SET s:Target;
Neo4j Inc. All rights reserved 2023
31
Define Target Locals (randomly for demo)
Each target room must be connected to at
least one IT room.
Our customer.
Neo4j Inc. All rights reserved 2023
32
Neo4j Inc. All rights reserved 2023
33
Roadmap
Modeling and
Ingestion 1
3
2
4
Setting the problem
instance (sources and
targets)
Building full set of
possible connections
(constraint-compliant)
Run a GDS
algorithm
GDS Toolbox shortlist
Neo4j Inc. All rights reserved 2023
34
Neo4j Inc. All rights reserved 2023
35
Neo4j Inc. All rights reserved 2023
36
Neo4j Inc. All rights reserved 2023
37
CREATE (sr:Voxel:Virtual:SteinerRoot);
MATCH (sr:SteinerRoot)
MATCH (li:ITRoom)
CREATE (sr)-[:LAYABLE_WIRE {cost:0.0}]->(li);
GDS in action
Neo4j Inc. All rights reserved 2023
38
Projecting the graph
CALL gds.graph.project(
'voxel_layable_wires_graph',
'Voxel',
{
LAYABLE_WIRE: {
properties: 'cost',
orientation: 'UNDIRECTED'
}
}
);
Neo4j Inc. All rights reserved 2023
39
Grow a Steiner Tree
MATCH (t:Target)
WITH collect(t) AS targets
MATCH (s:SteinerRoot)
WITH s, targets
CALL gds.beta.steinerTree.stream('voxel_layable_wires_graph',
{
sourceNode: id(s),
targetNodes: [t IN targets | id(t)],
relationshipWeightProperty:"cost"
})
YIELD nodeId, parentId, weight
Neo4j Inc. All rights reserved 2023
40
Build prettified Steiner Tree (Elbow Connector)
// remove Steiner root
MATCH (sr:SteinerRoot)
DETACH DELETE sr;
// create sections of cables to show elbow connections in Bloom
MATCH (a)-[r:STEINER_FOREST_WIRE]->(b)
WITH r, r.intermediary_voxels AS intermediary_voxels
WITH r, apoc.coll.zip(intermediary_voxels[..-1],intermediary_voxels[1..]) AS wire_lines
UNWIND wire_lines AS wire_line
WITH wire_line[0] AS line_from, wire_line[1] AS line_to, elementId(r) AS wire
MATCH (f WHERE elementId(f) = line_from), (t WHERE elementId(t) = line_to)
CREATE (f)-[:STEINER_FOREST_WIRE_CHUNK {connexion:wire}]->(t);
// Identify switches
MATCH (sw)
WHERE count{(sw)-[r:STEINER_FOREST_WIRE]->()} > 1
SET sw:Switch;
Neo4j Inc. All rights reserved 2023
41
Neo4j Inc. All rights reserved 2023
42
Neo4j Inc. All rights reserved 2023
43
Neo4j Inc. All rights reserved 2023
44
Neo4j Inc. All rights reserved 2023
45
Thank you!
name.name@neotechnology.com
Neo4j Inc. All rights reserved 2023
46
About yours truly
Neo4j Inc. All rights reserved 2023
47

Démonstration Digital Twin Building Wire Management

  • 1.
    Graph Data Sciencefor Digital Twin - Wire Management Graph Pattern Matching + GDS algorithms: computing at scale on your graph
  • 2.
    Pierre Halftermeyer Sr. SolutionsEngineer PhD in Graph Theory 3½ years at pierre[AT]neo4j.com Neo4j Inc. All rights reserved 2023 2
  • 3.
    Use case: Optimal constrained wiremanagement Neo4j Inc. All rights reserved 2023 3
  • 4.
    Neo4j Inc. Allrights reserved 2023 4
  • 5.
    Neo4j Inc. Allrights reserved 2023 5
  • 6.
    Neo4j Inc. Allrights reserved 2023 6
  • 7.
    Neo4j Inc. Allrights reserved 2023 7
  • 8.
    Neo4j Inc. Allrights reserved 2023 8
  • 9.
    Neo4j Inc. Allrights reserved 2023 9
  • 10.
    Neo4j Inc. Allrights reserved 2023 10
  • 11.
    Neo4j Inc. Allrights reserved 2023 11
  • 12.
    Neo4j Inc. Allrights reserved 2023 12 Roadmap Modeling and Ingestion 1 3 2 4 Setting the problem instance (sources and targets) Building full set of possible connections (constraint-compliant) Apply GDS algorithm
  • 13.
    Neo4j Inc. Allrights reserved 2023 13 Roadmap Modeling and Ingestion 1 3 2 4 Setting the problem instance (sources and targets) Building full set of possible connections (constraint-compliant) Apply GDS algorithm
  • 14.
    Ingest Minecraft-like data Unit voxels3D grid modeling Neo4j Inc. All rights reserved 2023 14
  • 15.
    ....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ...................... ....................+⍐+++++++++++⍐+...................... ....................▢▢▢▢▢▢▢+▢▢▢▢▢▢▢ ...................... ....................+⍐+++++++++++⍐+...................... ....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ...................... ......................................................... # Floor 4 ......................................................... ....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ...................... ....................+⍐+++++++++++⍐+...................... ....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ...................... ....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ...................... ....................+⍐+++++++++++⍐+...................... ....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ...................... ....................+⍐+++++++++++⍐+...................... ....................▢▢▢▢▢▢▢ +▢▢▢▢▢▢▢ ...................... Neo4j Inc. All rights reserved 2023 15 ▢ means :Voxel / Room by default + means :Lobby ⍐ means :Elevator:Bottom
  • 16.
    Ingestion of (x,y, z) Voxels Neo4j Inc. All rights reserved 2023 16 // parse csv LOAD CSV FROM "file:///building.txt" AS line WITH collect(line[0]) as lines WITH [l IN lines | CASE WHEN l STARTS WITH "#" THEN "#" ELSE l END] AS lines CALL apoc.coll.split(lines, "#") YIELD value WITH value WITH collect(value) AS building WITH range(0, size(building)-1) AS ixs, building WITH [ix IN ixs | {floor_num:ix+1, floor:building[ix]}] AS building UNWIND building AS floor WITH floor.floor_num AS z, floor.floor AS floor UNWIND range(0, size(floor)-1) AS x WITH z, x, floor[x] AS line UNWIND range(0, size(line)-1) AS y WITH z, x, y, substring(line,y,1) AS val CREATE (:Voxel {x:x, y:y, z:z, val:val}); // Delete non relevant voxels MATCH (n {val:'.'}) DELETE n; // label Lobbys and Elevators MATCH (s:Voxel WHERE s.val IN ["+", "⍐"]) SET s:Lobby; MATCH (s:Voxel WHERE s.val IN ["⍐"]) SET s:Elevator:Bottom;
  • 17.
    Connect adjacent Voxels in3D Neo4j Inc. All rights reserved 2023 17 MATCH (s:Voxel) WITH s.x AS x, s.y AS y, s.z AS z, s ORDER BY z WITH x, y, collect(s) AS line CALL apoc.nodes.link(line, "HAS_ABOVE"); MATCH (a)-[r:HAS_ABOVE]->(b) WHERE b.z - a.z <> 1 DELETE r; MATCH (s:Voxel) WITH s.x AS x, s.y AS y, s.z AS z, s ORDER BY y WITH x, z, collect(s) AS line CALL apoc.nodes.link(line, "HAS_EAST"); MATCH (a)-[r:HAS_EAST]->(b) WHERE b.y - a.y <> 1 DELETE r; MATCH (s:Voxel) WITH s.x AS x, s.y AS y, s.z AS z, s ORDER BY x WITH y, z, collect(s) AS line CALL apoc.nodes.link(line, "HAS_NORTH"); MATCH (a)-[r:HAS_NORTH]->(b) WHERE b.x - a.x <> 1
  • 18.
    Neo4j Inc. Allrights reserved 2023 18 Roadmap Modeling and Ingestion 1 3 2 4 Setting the problem instance (sources and targets) Building the full set of possible wires (constraint-compliant) Apply GDS algorithm
  • 19.
    Rules Let’s PoC withmock data Neo4j Inc. All rights reserved 2023 19
  • 20.
    If a wirehas to join two Voxels from distinct floors, pass the wire via the elevator. No vertical drilling. Our customer. Neo4j Inc. All rights reserved 2023 20
  • 21.
    Neo4j Inc. Allrights reserved 2023 21
  • 22.
    Vertical connections usethe Elevator MATCH (a:!(Elevator&Bottom))-[r:HAS_ABOVE]->(b) DELETE r; MATCH (a:Elevator&Bottom)-[r:HAS_ABOVE]->(b) SET b:Elevator:Top; Neo4j Inc. All rights reserved 2023 22
  • 23.
    Cables crossing roomare possible but not more than 3 rooms in a row. If more, you have to pass via the lobby. Cable shouldn’t be longer than 10 units of distance. This is possible: Voxel-Voxel-Voxel-Lobby-Lobby-Lobby-Voxel-Voxel Our customer. Neo4j Inc. All rights reserved 2023 23
  • 24.
    Neo4j Inc. Allrights reserved 2023 24
  • 25.
    Neo4j Inc. Allrights reserved 2023 25 1 0 u n i t s m a x
  • 26.
    Possible cable layingvia Lobby. MATCH (s:Voxel&!Lobby) CALL { WITH s MATCH p=(s)( ()-[r1:HAS_NORTH|HAS_EAST]-(:Voxel&!Lobby) ){0,2}(:Voxel&!Lobby)-[:HAS_NORTH|HAS_EAST]-(:Lobby)( ()-[r2:HAS_EAST|HAS_NORTH|HAS_ABOVE]-(:Lobby) ){0,5}(:Lobby)-[:HAS_NORTH|HAS_EAST]-(:Voxel&!Lobby)( ()-[r3:HAS_NORTH|HAS_EAST]-(:Voxel&!Lobby) ){0,2}(t WHERE t<>s) WITH s, t, size(r1+r2+r3)+2 AS wire_length, [n IN nodes(p)|elementId(n)] AS intermediary_voxels WHERE wire_length <= 10 CREATE (s)-[:LAYABLE_WIRE {cost:1+log(wire_length), intermediary_voxels:intermediary_voxels}]->(t) } IN TRANSACTIONS OF 10 rows; Neo4j Inc. All rights reserved 2023 26
  • 27.
    Possible cable layingnot via Lobby. MATCH (s:Voxel) CALL { WITH s MATCH p=(s)( ()-[r1:HAS_NORTH|HAS_EAST]-(:Voxel&!Lobby) ){1,2}(t:Voxel&!Lobby) WITH s, t, size(r1) AS wire_length, [n IN nodes(p)|elementId(n)] AS intermediary_voxels WHERE wire_length <= 10 CREATE (s)-[:LAYABLE_WIRE {cost:log(1+wire_length), intermediary_voxels:intermediary_voxels}]->(t) } IN TRANSACTIONS OF 10 rows; Neo4j Inc. All rights reserved 2023 27
  • 28.
    Neo4j Inc. Allrights reserved 2023 28 Roadmap Modeling and Ingestion 1 3 2 4 Setting the problem instance (sources and targets) Building full set of possible connections (constraint-compliant) Apply GDS algorithm
  • 29.
    IT Room andTarget Voxel Neo4j Inc. All rights reserved 2023 29
  • 30.
    MATCH (s:Voxel WHEREs.val = "▢") WITH s, rand() AS ord ORDER BY ord LIMIT 3 SET s:ITRoom; Neo4j Inc. All rights reserved 2023 30 Define IT Locals (randomly for demo)
  • 31.
    MATCH (s:Voxel&!Lobby) WHERE rand()< 0.50 SET s:Target; Neo4j Inc. All rights reserved 2023 31 Define Target Locals (randomly for demo)
  • 32.
    Each target roommust be connected to at least one IT room. Our customer. Neo4j Inc. All rights reserved 2023 32
  • 33.
    Neo4j Inc. Allrights reserved 2023 33 Roadmap Modeling and Ingestion 1 3 2 4 Setting the problem instance (sources and targets) Building full set of possible connections (constraint-compliant) Run a GDS algorithm
  • 34.
    GDS Toolbox shortlist Neo4jInc. All rights reserved 2023 34
  • 35.
    Neo4j Inc. Allrights reserved 2023 35
  • 36.
    Neo4j Inc. Allrights reserved 2023 36
  • 37.
    Neo4j Inc. Allrights reserved 2023 37 CREATE (sr:Voxel:Virtual:SteinerRoot); MATCH (sr:SteinerRoot) MATCH (li:ITRoom) CREATE (sr)-[:LAYABLE_WIRE {cost:0.0}]->(li);
  • 38.
    GDS in action Neo4jInc. All rights reserved 2023 38
  • 39.
    Projecting the graph CALLgds.graph.project( 'voxel_layable_wires_graph', 'Voxel', { LAYABLE_WIRE: { properties: 'cost', orientation: 'UNDIRECTED' } } ); Neo4j Inc. All rights reserved 2023 39
  • 40.
    Grow a SteinerTree MATCH (t:Target) WITH collect(t) AS targets MATCH (s:SteinerRoot) WITH s, targets CALL gds.beta.steinerTree.stream('voxel_layable_wires_graph', { sourceNode: id(s), targetNodes: [t IN targets | id(t)], relationshipWeightProperty:"cost" }) YIELD nodeId, parentId, weight Neo4j Inc. All rights reserved 2023 40
  • 41.
    Build prettified SteinerTree (Elbow Connector) // remove Steiner root MATCH (sr:SteinerRoot) DETACH DELETE sr; // create sections of cables to show elbow connections in Bloom MATCH (a)-[r:STEINER_FOREST_WIRE]->(b) WITH r, r.intermediary_voxels AS intermediary_voxels WITH r, apoc.coll.zip(intermediary_voxels[..-1],intermediary_voxels[1..]) AS wire_lines UNWIND wire_lines AS wire_line WITH wire_line[0] AS line_from, wire_line[1] AS line_to, elementId(r) AS wire MATCH (f WHERE elementId(f) = line_from), (t WHERE elementId(t) = line_to) CREATE (f)-[:STEINER_FOREST_WIRE_CHUNK {connexion:wire}]->(t); // Identify switches MATCH (sw) WHERE count{(sw)-[r:STEINER_FOREST_WIRE]->()} > 1 SET sw:Switch; Neo4j Inc. All rights reserved 2023 41
  • 42.
    Neo4j Inc. Allrights reserved 2023 42
  • 43.
    Neo4j Inc. Allrights reserved 2023 43
  • 44.
    Neo4j Inc. Allrights reserved 2023 44
  • 45.
    Neo4j Inc. Allrights reserved 2023 45
  • 46.
  • 47.
    About yours truly Neo4jInc. All rights reserved 2023 47