|
| 1 | +# GraphFrames 0.12.0 release |
| 2 | + |
| 3 | +- **Published:** 2026-06-12T00:00:00Z |
| 4 | +- **Title:** GraphFrames 0.12.0 release |
| 5 | +- **Summary:** This release brings new Community Detection algorithm, new API to find all simple paths between subset of vertices, approximate neighbor functions and significant performance improvements for the Two-Phase Connected Components Algorithm. |
| 6 | + |
| 7 | +## New Contributors |
| 8 | + |
| 9 | +- [@slavlotski](https://github.com/slavlotski) -- `asReversed` helper API to reverse all the edges of the graph |
| 10 | + |
| 11 | +## New Community Detection Algorithm |
| 12 | + |
| 13 | +Previous versions of GraphFrames relied entirely on the most naive implementation of the Label Propagation algorithm. While this implementation is fast and well-known, the quality of the output clusters is questionable, and the algorithm itself is unstable. Even small changes in the local structure can alter the output. |
| 14 | + |
| 15 | +The new algorithm significantly modifies the original Label Propagation algorithm. While it follows the same idea that allows for efficient implementation on distributed graphs, it also provides more flexibility. The inspiration came from [Xie, Jierui, and Boleslaw K. Szymanski. "Community detection using a neighborhood strength driven label propagation algorithm." 2011 IEEE Network Science Workshop. IEEE, 2011.](https://arxiv.org/abs/1105.3264) |
| 16 | + |
| 17 | +The core idea is that, during propagation, vertices choose a community based not only on their local neighborhood, but also on the number of neighbors they have in common with other community members. Compared to existing label propagation, the new algorithm also supports passing initial labels, which allows it to be used incrementally or for semi-supervised community detection. |
| 18 | + |
| 19 | +Credits to [@SemyonSinchenko](https://github.com/SemyonSinchenko). |
| 20 | + |
| 21 | +## New all paths API |
| 22 | + |
| 23 | +After introducing the `AggregateNeighbors` API in version `0.11.0`, which is a generic, multi-hop aggregation API, GraphFrames is receiving built-in implementations based on neighbor aggregation. The first is the long-awaited API that finds all simple paths between a subset of vertices. |
| 24 | + |
| 25 | +Credits to [@SemyonSinchenko](https://github.com/SemyonSinchenko). |
| 26 | + |
| 27 | +## Aproximate Neighbor Functions |
| 28 | + |
| 29 | +This release brings a foundation API for the approximate neighbor functions. Users can use it to cpmoute an approximate graph diameter, HyperBALL or approximate closeness centrality. |
| 30 | + |
| 31 | +Credits to [@SemyonSinchenko](https://github.com/SemyonSinchenko). |
| 32 | + |
| 33 | +## Performance optimizations in Connected Components |
| 34 | + |
| 35 | +The Two-Phase algorithm is based on the idea of rewiring edges to end up with a star-like graph structure. However, during the rewiring process, a large number of leaves, or vertices with no outgoing edges, appear. Although determining components for these vertices is trivial, and they do not participate in the main algorithm loop, they still shuffle and join until full convergence. The new optimization adds an efficient way to determine the optimal time to remove such leaves and offset the cost of rejoining them after convergence. Based on initial benchmarks, the optimization delivers a ~25% performance boost. |
| 36 | + |
| 37 | +This optimization was part of the Databricks' internal fork of GraphFrames. It was donated to the open-source GraphFrames by the company. |
| 38 | + |
| 39 | +Credits to [@WeichenXu123](https://github.com/WeichenXu123) and [Databricks](https://www.databricks.com/) |
| 40 | + |
| 41 | +## Important note |
| 42 | + |
| 43 | +Previous versions of Graphframes had an unspecified contract within the Pregel API regarding the handling of edge attributes. All edge attributes, including the IDs of the source (`src`) and destination (`dst`) vertices, were implicitly packed into a `StructType` and persisted. Although persisting was required for performance, it blocked the Catalyst optimizer from eliminating these columns if they were not used. This resulted in an almost twofold increase in peak memory load in all scenarios and was considered a bug. Starting with version 0.12.0, users who want to use edge attributes in the low-level Pregel should specify them explicitly using `requiredEdgeColumns(...)` in Scala or `required_edge_columns(...)` in the Python API. |
| 44 | + |
| 45 | +## Future steps |
| 46 | + |
| 47 | +- Moving in the direction of support of full-featured graph queries |
| 48 | +- Improving GraphFrames capabilities in Graph ML |
| 49 | +- Adding features useful in Spatial Graphs analysis |
0 commit comments