Is your feature request related to a problem? Please describe.
Currently, GraphFrames provides a connected-components algorithm based on the "big / small star" algorithm (2016), or the even older choice from GraphX. There is yet another method available, from 2020, described in a paper titled "In-database connected component analysis" and called "Randomised Contraction" which has potentially better runtime and space efficiency. In our testing it seems to outperform the GraphFrames version by 10-20%.
A full description of the algorithm, with implementation code, is found in this paper: https://arxiv.org/abs/1802.09478 (view PDF link, top right)
https://ieeexplore.ieee.org/document/9101609
Describe the solution you would like
It would be nice to see GraphFrames provide more choices for algorithms for CC, this one in particular - selectable by spark.graphframes.connectedComponents.algorithm = randomised_contraction or similar.
Component
Additional context
n/a
Are you planning on creating a PR?
Is your feature request related to a problem? Please describe.
Currently, GraphFrames provides a connected-components algorithm based on the "big / small star" algorithm (2016), or the even older choice from GraphX. There is yet another method available, from 2020, described in a paper titled "In-database connected component analysis" and called "Randomised Contraction" which has potentially better runtime and space efficiency. In our testing it seems to outperform the GraphFrames version by 10-20%.
A full description of the algorithm, with implementation code, is found in this paper: https://arxiv.org/abs/1802.09478 (view PDF link, top right)
https://ieeexplore.ieee.org/document/9101609
Describe the solution you would like
It would be nice to see GraphFrames provide more choices for algorithms for CC, this one in particular - selectable by
spark.graphframes.connectedComponents.algorithm=randomised_contractionor similar.Component
Additional context
n/a
Are you planning on creating a PR?