Commit df4a0ea

<p align="center">

<img src="docs/src/img/GraphFrames-Logo-Large.png" alt="GraphFrames Logo" width="500"/>

</p>

<p align="center">

<a href="https://github.com/graphframes/graphframes/actions/workflows/scala-ci.yml"><img src="https://github.com/graphframes/graphframes/actions/workflows/scala-ci.yml/badge.svg" alt="Scala CI"></a> <a href="https://github.com/graphframes/graphframes/actions/workflows/python-ci.yml"><img src="https://github.com/graphframes/graphframes/actions/workflows/python-ci.yml/badge.svg" alt="Python CI"></a> <a href="https://github.com/graphframes/graphframes/actions/workflows/pages/pages-build-deployment"><img src="https://github.com/graphframes/graphframes/actions/workflows/pages/pages-build-deployment/badge.svg" alt="pages-build-deployment"></a> <a href="https://github.com/graphframes/graphframes/actions/workflows/scala-publish.yml"><img src="https://github.com/graphframes/graphframes/actions/workflows/scala-publish.yml/badge.svg" alt="scala-central-publish"></a> <a href="https://github.com/graphframes/graphframes/actions/workflows/python-publish.yml"><img src="https://github.com/graphframes/graphframes/actions/workflows/python-publish.yml/badge.svg" alt="python-pypi-publish"></a> <img src="https://img.shields.io/github/v/release/graphframes/graphframes" alt="GitHub Release"> <img src="https://img.shields.io/github/license/graphframes/graphframes" alt="GitHub License"> <img src="https://img.shields.io/pypi/dm/graphframes-py" alt="PyPI - Downloads">

</p>

This is a package for graphs processing and analytics on scale. It is built on top of Apache Spark and relies on DataFrame abstraction. It provides built-in and easy to use distributed graph algorithms as well as a flexible APIs like `Pregel` or `AggregateMessages` to make custom graph processing. Users can write highly expressive queries by leveraging the DataFrame API, combined with a new API for network motif finding. The user also benefits from DataFrame performance optimizations within the Spark SQL engine. GraphFrames works in Java, Scala, and Python.

There are some popular use cases when GraphFrames is almost irreplaceable, including, but not limited to:

- Compliance analytics with a scalable shortest paths algorithm and motif analysis;

- Anti-fraud with scalable cycles detection in large networks;

- Identity resolution on the scale of billions with highly efficient connected components;

- Search result ranking with a distributed, Pregel-based PageRank;

- Clustering huge graphs with Label Propagation and Power Iteration Clustering;

- Building a knowledge graph systems with Property Graph Model.

- [Installation](https://graphframes.io/02-quick-start/01-installation.html)

- [Creating Graphs](https://graphframes.io/04-user-guide/01-creating-graphframes.html)

- [Basic Graph Manipulations](https://graphframes.io/04-user-guide/02-basic-operations.html)

- [Centrality Metrics](https://graphframes.io/04-user-guide/03-centralities.html)

- [Motif finding](https://graphframes.io/04-user-guide/04-motif-finding.html)

- [Traversals and Connectivity](https://graphframes.io/04-user-guide/05-traversals.html)

- [Community Detection](https://graphframes.io/04-user-guide/06-graph-clustering.html)

- [Scala API](https://graphframes.io/api/scaladoc/)

- [Python API](https://graphframes.io/api/python/)

- [Apache Spark compatibility](https://graphframes.io/02-quick-start/01-installation.html#spark-versions-compatibility)

- [GraphFrames Network Motif Finding Tutorial](https://graphframes.github.io/graphframes/docs/_site/motif-tutorial.html)

- [Introducing GraphFrames](https://databricks.com/blog/2016/03/03/introducing-graphframes.html)

- [GraphFrames Google Group](https://groups.google.com/forum/#!forum/graphframes)

- [#graphframes Discord Channel on GraphGeeks](https://discord.com/channels/1162999022819225631/1326257052368113674)

- [Graph Operations in Apache Spark Using GraphFrames](https://www.pluralsight.com/courses/apache-spark-graphframes-graph-operations)

- [Executing Graph Algorithms with GraphFrames on Databricks](https://www.pluralsight.com/courses/executing-graph-algorithms-graphframes-databricks)

- [On-Time Flight Performance with GraphFrames for Apache Spark](https://databricks.com/blog/2016/03/16/on-time-flight-performance-with-graphframes-for-apache-spark.html)

- [Sustainability in Aluminum Production](https://www.databricks.com/blog/sustainability-aluminum-production)

- [A top level overview of GraphFrames internals](https://graphframes.io/01-about/02-architecture.html)

- [GraphFrames: An Integrated API for Mixing Graph and Relational Queries, Dave et al. 2016](https://people.eecs.berkeley.edu/~matei/papers/2016/grades_graphframes.pdf).

See [contribution guide](./CONTRIBUTING.md) and the [local development setup walkthrough](https://graphframes.io/06-contributing/01-contributing-guide.html) for step-by-step instructions on preparing your environment, running tests, and submitting changes.

[](https://www.star-history.com/#graphframes/graphframes&Date)

graph.connectedComponents

.setUseLocalCheckpoints(true)

.setAlgorithm("graphframes")

.setBroadcastThreshold(-1)

.run()

@Benchmark

def benchmarkCDLPGraphX(blackhole: Blackhole): Unit = {

val cdlpResults = graph.labelPropagation

.setAlgorithm("graphx")

.maxIter(10)

.run()

val res: Unit = cdlpResults.write.format("noop").mode("overwrite").save()

blackhole.consume(res)

}

- `graphframes-spark3_2.12`

- `graphframes-spark3_2.13`

- `graphframes-connect-spark3_2.12`

- `graphframes-connect-spark3_2.13`

- `graphframes-graphx-spark3_2.12`

- `graphframes-graphx-spark3_2.13`

- `graphframes-spark4_2.13`

- `graphframes-connect-spark4_2.13`

- `graphframes-graphx-spark4_2.13`

- **Size Category:** _XS_

| Algorithm | Measurements | Time (s) |

| -------------------------------- | ---------------------------------------------- | ---------------------------------------- |

| Shortest Paths Graphframes | ${benchmarks.benchmarkSP.measurements} | ${benchmarks.benchmarkSP.metric} |

| Shortest Paths GraphX | ${benchmarks.benchmarkSPGraphX.measurements} | ${benchmarks.benchmarkSPGraphX.metric} |

| Connected Components Graphframes | ${benchmarks.benchmarkCC.measurements} | ${benchmarks.benchmarkCC.metric} |

| Connected Components GraphX | ${benchmarks.benchmarkCCGraphX.measurements} | ${benchmarks.benchmarkCCGraphX.metric} |

| Label Propagation GraphFrames | ${benchmarks.benchmarkCDLP.measurements} | ${benchmarks.benchmarkCDLP.metric} |

| Label Propagation GraphX | ${benchmarks.benchmarkCDLPGraphX.measurements} | ${benchmarks.benchmarkCDLPGraphX.metric} |