Big Data with Hadoop & Spark Training: http://bit.ly/2IYeuvF This CloudxLab Introduction to GraphX tutorial helps you to understand GraphX in detail. Below are the topics covered in this tutorial: 1) Introduction to GraphX 2) What is Graph? 3) Examples of Graph Computation 4) Pagerank using GraphX

1. GraphX
GraphX

2. GraphX
What is a graph

3. GraphX
Examples of graph computations
● Finding common friends
● Finding the page rank
● And Many more…

4. GraphX
Finding common friends
Examples of graph computations

5. GraphX
Finding Page Rank
Examples of graph computations

6. GraphX
● Unifies Graph Computation
○ ETL
○ Exploratory analysis
○ Iterative
● View the same Data as Graph and Collections
● Transform and join graphs with RDDs efficiently
● Extends the Spark RDD by introducing a new Graph
abstraction
GraphX

7. GraphX
GraphX
○ PageRank
■ If important pages link you, you are more important
○ Connected Components
■ Clusters amongst your facebook friends
○ Triangle Counting
■ Triangles passing through each vertex => measure of
clustering.
○ Label propagation
○ SVD++
○ Strongly connected components
Has library of algorithms

8. GraphX
GraphX
● subgraph
● joinVertices
● aggregateMessages
● And more….
Provides set of fundamental operations
https://spark.apache.org/docs/latest/graphx-programming-guide.html

9. GraphX
GraphX - Pagerank
1. BarackObama
2. Lady Gaga
3. John Resig
4. Justin Bieber
6. Matei Zaharia
6. Martin Odersky
7. anonsys
PR(A) = 0.15 + 0.85 * ( rank of node / outgoing)

10. GraphX
GraphX - Pagerank
$ hadoop fs -cat /data/spark/graphx/followers.txt
2 1
4 1
1 2
6 3
7 3
7 6
6 7
3 7
https://github.com/cloudxlab/bigdata/blob/master/spark/examples/graphx/pagerank.scala

11. GraphX
GraphX - Pagerank
import org.apache.spark.graphx.GraphLoader
// Load the edges as a graph
val graph = GraphLoader.edgeListFile(sc, "/data/spark/graphx/followers.txt")
// Run PageRank
val ranks = graph.pageRank(0.0001).vertices
// Join the ranks with the usernames
val users = sc.textFile("/data/spark/graphx/users.txt").map { line =>
val fields = line.split(",")
(fields(0).toLong, fields(1))
}
val ranksByUsername = users.join(ranks).map {
case (id, (username, rank)) => (username, rank)
}
// Print the result
println(ranksByUsername.collect().mkString("n"))
See more

12. GraphX
GraphX - Pagerank
1. BarackObama
2. Lady Gaga
3. John Resig
4. Justin Bieber
6. Matei Zaharia
6. Martin Odersky
7. anonsys
0.15
0.70
1.39
1.46
1.0
1.3

13. Thank you!
GraphX
reachus@cloudxlab.com