| Modification dateThis property is a special property in this wiki.
|
24 February 2009 16:45:24 +
|
| Question answer
|
('''Jesse Weaver''') You make a good point … ('''Jesse Weaver''') You make a good point. While they provide a parallel algorithm for reasoning ''after'' partitioning, they do not propose a parallel algorithm for data partitioning itself. Algorithm one seems to indicate that a single computational node does the partitioning and then sends the tuples to their assigned owners. Such a paradigm is actually well known to significantly affect performance due to the lack of parallelization in the partitioning part of the computation and the amount of overhead spent sending tuples to each computational node. This is particularly expensive on large data sets distributed across a large number of computational nodes. (I don't believe this problem is clearly reflected in the evaluation because they only try scaling to 16 nodes. Such an approach is unlikely to scale well to tens of thousands of computational nodes like on a Blue Gene/L.) My intuition is that, unless there is a parallel algorithm for efficiently and sufficiently partitioning a graph, then it is unlikely that this approach will scale well to very large data sets and/or very large numbers of computational nodes. very large numbers of computational nodes.
|
| Question asked
|
After criticizing state of the art tools i … After criticizing state of the art tools in the introduction for not being "web-scale", the authors discuss their graph partitioning algorithm and its use the Metis package which "has been shown to work for graphs with millions of nodes." This hardly seems "web scale", either. Do you have any sense of how the graph partitioning component of their system might scale to hundreds of millions or billions of nodes, and if it might end up negating the speedup gained from parallelization? g the speedup gained from parallelization?
|
| Question asked by
|
Gregory Todd Williams +
|
| Question for the Presentation
|
Questions for Distributed Reasoning +
|
| Categories |
Presentation Questions
|
