Pregel: A System for Large-Scale Graph Processing

“Pregel: A System for Large-Scale Graph Processing – Malewicz et al. (Google) 2010

“Many practical computing problems concern large graphs.”

Yesterday we looked at some of the models for understanding networks and graphs. Today’s paper focuses on processing of graphs, especially the efficient processing of large graphs where large can mean billions of vertices and trillions of edges. Pregel is the system at Google that powers PageRank, which makes it a very interesting system to study. It is also the inspiration for Apache Giraph, which Facebook use to analyze their social graph.

There are single machine-sized graph processing problems, and then there are distributed graph processing problems. You probably don’t want to be using a distributed graph processing platform to solve a problem that would fit on one machine (See Musketeer), but if you really do have a large graph, then…”

Pregel: A System for Large-Scale Graph Processing