The document discusses Pregel, a system for large-scale graph processing. Pregel uses a message passing model where computation is organized into supersteps. In each superstep, vertices can send messages to other vertices and modify their own state. The document also discusses Giraph, an open source implementation of Pregel built on Hadoop. Giraph runs as a single Map-only job to avoid disk I/O between supersteps. It uses a master node to coordinate workers and assign graph partitions.

1. Pregel

2. Pregel
• A System for Large-Scale Graph Processing
• Sufficiently flexible to express arbitrary graph
algorithms
• So easy

3. Pregel: Model Of Computation
• Vertex state
• Terminate codition: all vertex are inactive

4. Pregel: Model Of Computation
• Sequence of supersteps
• Invoke compute() for each active vertex
• Each vertex can
– Modify its state, its outgoing edges
– Recive messages
– Send messages to another

5. Pregel: Model Of Computation

6. Pregel: Model Of Computation

7. Pregel API

8. Pregel API
• Combiners
• Aggregators
• Topology Mutations
• Input and Output

9. Giraph

10. Why not implement Giraph with
multiple MapReduce jobs
• Too much disk, no in-memory caching, a
superstep becomes a job!

11. Giraph is a single Map-only job in
Hadoop
• Hadoop is purely a resource manager for Giraph, all
communication is done through Netty-based IPC

12. Maximum vertex value
implementation

13. Giraph components
• Master
– One active master at a time
– Assign partition owners to workers prior to each
superstep
– Synchronize supersteps
• Worker
– Load the graph from input
– Does the computation/messaging of its assigned
partitions

14. Graph distribution