Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Loom at Clojure/West

3,659 views

Published on

Slides from the talk on Loom and Graphs in Clojure by Aysylu Greenberg at Clojure/West.

Published in: Technology

Loom at Clojure/West

  1. 1. Loom and Graphs in Clojure github.com/aysylu/loom Aysylu Greenberg @aysylu22; http://aysy.lu March 25th 2014
  2. 2. Graphs
  3. 3. Loom Graph Algorithms + Visualization
  4. 4. In this Talk Loom Overview Loom’s Graph API Functional Graph Algorithms Your Graphs in Loom
  5. 5. In this Talk Loom Overview Loom’s Graph API Functional Graph Algorithms Your Graphs in Loom
  6. 6. Loom Overview: Supported Graphs •  Undirected Graph •  Directed Graph (Digraph) •  Weighted Graph •  FlyGraph o  nodes + successors edges o  successors + start nodes + edges
  7. 7. Loom Overview: Graph Algorithms • DFS/BFS (+ bidirectional)
  8. 8. Loom Overview: Graph Algorithms • DFS/BFS (+ bidirectional) • Topological Sort
  9. 9. Loom Overview: Graph Algorithms • DFS/BFS (+ bidirectional) • Topological Sort • Single Source Shortest Path (Dijkstra, Bellman-Ford)
  10. 10. Loom Overview: Graph Algorithms • DFS/BFS (+ bidirectional) • Topological Sort • Single Source Shortest Path (Dijkstra, Bellman-Ford) • Strongly Connected Components (Kosaraju)
  11. 11. Loom Overview: Graph Algorithms • DFS/BFS (+ bidirectional) • Topological Sort • Single Source Shortest Path (Dijkstra, Bellman-Ford) • Strongly Connected Components (Kosaraju) • Density (edges/nodes) • Loner Nodes • DFS/BFS (+ bidirectional) • Topological Sort • Single Source Shortest Path (Dijkstra, Bellman-Ford) • Strongly Connected Components (Kosaraju)
  12. 12. Loom Overview: Graph Algorithms • DFS/BFS (+ bidirectional) • Topological Sort • Single Source Shortest Path (Dijkstra, Bellman-Ford) • Strongly Connected Components (Kosaraju) • Density (edges/nodes) • Loner Nodes • Two Coloring
  13. 13. Loom Overview: Graph Algorithms • DFS/BFS (+ bidirectional) • Topological Sort • Single Source Shortest Path (Dijkstra, Bellman-Ford) • Strongly Connected Components (Kosaraju) • Density (edges/nodes) • Loner Nodes • Two Coloring • Max-Flow (Edmonds-Karp)
  14. 14. Loom Overview: Visualizing Graphs
  15. 15. In this Talk Loom Overview Loom’s Graph API Functional Graph Algorithms Your Graphs in Loom
  16. 16. Graph API: Representations
  17. 17. Graph API: Representations • Adjacency List {A: [B], B: [C, D], C: [], …}
  18. 18. Graph API: Representations • Adjacency List • Incidence List {A: [e1], B: [e2, e3], …, e1: [A, B], e2: [B, C], …}
  19. 19. Graph API: Representations • Adjacency List • Incidence List • Adjacency Matrix Source Destination A B … E A 0 1 … ∞ B ∞ 0 … ∞ … … … … … E ∞ 1 … 0
  20. 20. Graph API: Representations • Adjacency List • Incidence List • Adjacency Matrix • Incidence Matrix Nodes Edges e1 e2 .. e5 A S N … N B D S … D … … … … … E N N … S
  21. 21. Graph API: Representations • Adjacency List sparse graphs • Incidence List • Adjacency Matrix dense graphs • Incidence Matrix
  22. 22. Graph API: Graph (defprotocol Graph! (nodes [g])! (edges [g])! (has-node? [g node])! (has-edge? [g n1 n2])! (successors [g] [g node])! (out-degree [g node]))!
  23. 23. Graph API: Digraph (defprotocol Digraph! (predecessors [g] [g node])! (in-degree [g node])! (transpose [g]))
  24. 24. Graph API: WeightedGraph (defprotocol WeightedGraph! (weight [g] [g n1 n2]))
  25. 25. Graph API: EditableGraph (defprotocol EditableGraph! (add-nodes* [g nodes])! (add-edges* [g edges])! (remove-nodes* [g nodes])! (remove-edges* [g edges])! (remove-all [g]))
  26. 26. Graph API: Complex Graphs How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])!
  27. 27. Graph API: Complex Graphs How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])! (extend BasicEditableDigraph! Graph …! Digraph …! EditableGraph …)!
  28. 28. Graph API: Complex Graphs How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])! (extend BasicEditableDigraph! Graph! ! ! ! ! )! (nodes [g])! (edges [g])! (has-node? [g node])! (has-edge? [g n1 n2])! (successors [g] [g node])! (out-degree [g node]) Protocol definition (not valid code!)
  29. 29. Graph API: Complex Graphs How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])! (extend BasicEditableDigraph! Graph default-graph-impl! Digraph …! EditableGraph …)!
  30. 30. Graph API: Complex Graphs (def default-graph-impl! {:edges! (fn [g]! (for [n1 (nodes g)! n2 (successors g n1)]! [n1 n2]))! :nodes (fn [g] ...)! ...})! (nodes [g])! (edges [g])! (has-node? [g node])! (has-edge? [g n1 n2])! (successors [g] [g node])! (out-degree [g node])
  31. 31. Graph API: Complex Graphs How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])! (extend BasicEditableDigraph! Graph default-graph-impl! Digraph …! EditableGraph …)!
  32. 32. Graph API: Complex Graphs How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])! (extend BasicEditableDigraph! Graph default-graph-impl! Digraph default-digraph-impl! EditableGraph …)!
  33. 33. Graph API: Complex Graphs (def default-digraph-impl! {:transpose! (fn [g]! (assoc g! :succs (:preds g)! :preds (:succs g)))! …}) (predecessors [g]! ! ! [g node])! (in-degree [g node])! (transpose [g])
  34. 34. Graph API: Complex Graphs How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])! (extend BasicEditableDigraph! Graph default-graph-impl! Digraph default-digraph-impl! EditableGraph …)!
  35. 35. Graph API: Complex Graphs How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])! (extend BasicEditableDigraph! Graph default-graph-impl! Digraph default-digraph-impl! EditableGraph editable-graph-impl)!
  36. 36. In this Talk Loom Overview Loom’s Graph API Functional Graph Algorithms Your Graphs in Loom
  37. 37. Functional Graph Algorithms: Bellman-Ford
  38. 38. Functional Graph Algorithms: Bellman-Ford CLRS Introduction to Algorithms
  39. 39. Functional Graph Algorithms: Bellman-Ford CLRS Introduction to Algorithms
  40. 40. Functional Graph Algorithms: Bellman-Ford (defn- init-estimates! [graph start]! (let [nodes (disj (nodes graph) start)! costs (interleave nodes! (repeat 1e999M))! paths (interleave nodes (repeat nil))]! [(apply assoc {start 0} costs)! (apply assoc {start nil} paths)]))
  41. 41. Functional Graph Algorithms: Bellman-Ford
  42. 42. Functional Graph Algorithms: Bellman-Ford
  43. 43. Functional Graph Algorithms: Bellman-Ford (defn- can-relax-edge?! [[u v :as edge] edge-cost costs]! (let [vd (get costs v)! ud (get costs u)! sum (+ ud edge-cost)]! (> vd sum)))
  44. 44. Functional Graph Algorithms: Bellman-Ford (defn- relax-edge! [[u v :as edge]! uvcost! [costs paths :as estimates]]! (let [ud (get costs u)! sum (+ ud uvcost)]! (if (can-relax-edge? edge uvcost costs)! [(assoc costs v sum)! (assoc paths v u)]! estimates)))
  45. 45. Functional Graph Algorithms: Bellman-Ford
  46. 46. Functional Graph Algorithms: Bellman-Ford (defn- relax-edges! [g start estimates]! (reduce (fn [estimates [u v :as edge]]! (relax-edge! edge! (weight g u v)! ! estimates))! estimates! (edges g)))!
  47. 47. Functional Graph Algorithms: Bellman-Ford (defn bellman-ford! [g start]! (let [initial-estimates (init-estimates g start)! ;relax-edges is calculated for all edges V-1 times! [costs paths] (reduce (fn [estimates _] (relax-edges g start estimates))! initial-estimates! (-> g nodes count dec range))! edges (edges g)]! (if (some (fn [[u v :as edge]] (can-relax-edge? edge (wt g u v) costs))! edges)! false! [costs! (->> (keys paths)! ;remove vertices that are unreachable from source! (remove #(= Double/POSITIVE_INFINITY (get costs %)))! (reduce! (fn [final-paths v]! (assoc final-paths v! ; follows the parent pointers! ; to construct path from source to node v! (loop [node v path ()]! (if node! (recur (get paths node) (cons node path))! path))))! {}))])))
  48. 48. Functional Graph Algorithms: Bellman-Ford (defn bellman-ford! [g start]! (let [initial-estimates (init-estimates g start)! ;relax-edges is calculated for all edges V-1 times! [costs paths]! (reduce! (fn [estimates _] (relax-edges g start estimates))! initial-estimates! (-> g nodes count dec range))! edges (edges g)]!
  49. 49. Functional Graph Algorithms: Bellman-Ford (if (some (fn [[u v :as edge]]! (can-relax-edge? edge! ! ! ! ! ! (weight g u v)! ! ! ! ! ! costs))! ! ! edges)! false! ! !;;; return paths))) !
  50. 50. Functional Graph Algorithms: loom.alg-generic • No knowledge of graph representation • Requires only successors o + start for DFS/BFS, topological sort, Dijkstra o + end for BFS path
  51. 51. In this Talk Loom Overview Loom’s Graph API Functional Graph Algorithms Your Graphs in Loom
  52. 52. Your Graphs in Loom core.async Single Static Assignment Form Titanium-Loom Github And more!
  53. 53. Your Graphs in Loom core.async Single Static Assignment Form Titanium-Loom Github And more!
  54. 54. SSA Loom •  Single Static Assignment (SSA) form produced by core.async •  Generated by parse-to-state-machine function
  55. 55. SSA Loom (parse-to-state-machine! '[(if (> (+ 1 2 x y) 0)! (+ x 1)! (+ x 2))]) {76 [{:value :clojure.core.async.impl.ioc-macros/ value, :id inst_4937} {:value inst_4937, :id inst_4938}], 74 [{:refs [clojure.core/+ x 1], :id inst_4933} {:value inst_4933, :block 76, :id inst_4934}], 73 [{:refs [clojure.core/+ x 1 2 y], :id inst_4930} {:refs [clojure.core/> inst_4930 0], :id inst_4931} {:test inst_4931, :then-block 74, :else-block 75, :id inst_4932}]}}]
  56. 56. SSA Loom (view (ssa->loom ssa ssa-node-fn ssa-edges-fn)) if (> (+ 1 2 x y) 0)! V (+ x 1)!V (+ x 2) V!
  57. 57. SSA Loom (view (ssa->loom ssa ssa-node-fn ssa-edges-fn))! ! (defn ssa->loom! ([ssa get-nodes get-edges]!
  58. 58. SSA Loom (view (ssa->loom ssa ssa-nodes-fn ssa-edges-fn))! ! (reify! Graph! (nodes [g] (get-nodes ssa))! …! (successors [g node]! (->> edges! (filter (fn [[n1 n2]]! (= n1 node)))! (map second)))! …!
  59. 59. SSA Loom (view (ssa->loom ssa ssa-nodes-fn ssa-edges-fn))! ! Digraph! (predecessors [g node]! (->> edges! (filter (fn [[n1 n2]]! (= n2 node)))! (map first)))! …))
  60. 60. Your Graphs in Loom core.async Single Static Assignment Form Titanium-Loom Github And more!
  61. 61. Titanium Loom •  Titanium by Clojurewerkz (titanium.clojurewerkz.org) •  Built on top of Aurelius Titan (thinkaurelius.github.com/titan) •  Storage backends: Cassandra, HBase, BerkeleyDB Java Edition •  No graph visualization
  62. 62. Titanium Loom (let [in-mem-graph! (open {"storage.backend” "inmemory"})]! (tg/transact!! (let [a (nodes/create! {:name "Node A"})! b (nodes/create! {:name "Node B"})! c (nodes/create! {:name "Node C"})! e1 (edges/connect! a b)! e2 (edges/connect! b c)! e3 (edges/connect! c a)]))! (titanium->loom in-mem-graph))
  63. 63. Titanium Loom (view graph)
  64. 64. Your Graphs in Loom Titanium-Loom core.async Single Static Assignment Form Github And more!
  65. 65. Data Flow Analysis Framework • Liveness analysis (dead code elimination) • Constant propagation • Taint analysis • Clojure Repos + Relationships
  66. 66. Github: Clojure Repos + Relations •  For each Clojure project, solve system of equations until reaching fixed point:
  67. 67. Github: Clojure Repos + Relations Quil math.numeric-tower
  68. 68. Github: Clojure Repos + Relations core.async clojure-ring clout > compojure core.match
  69. 69. Loom’s Vision The Graph Library in Clojure github.com/aysylu/loom

×