pregel-rs

pregel-rs is a Graph processing library written in Rust that features a Pregel-based Framework for implementing your own algorithms in a message-passing fashion. It is designed to be efficient and scalable, making it suitable for processing large-scale graphs.

Features

• Pregel-based framework: pregel-rs is a powerful graph processing model that allows users to implement graph algorithms in a message-passing fashion, where computation is performed on vertices and messages are passed along edges. pregel-rs provides a framework that makes it easy to implement graph algorithms using this model.

• Rust-based implementation: pregel-rs is implemented in Rust, a systems programming language known for its safety, concurrency, and performance. Rust's strong type system and memory safety features help ensure that pregel-rs is robust and reliable.

• Efficient and scalable: pregel-rs designed to be efficient and scalable, making it suitable for processing large-scale graphs. It uses parallelism and optimization techniques to minimize computation and communication overhead, allowing it to handle graphs with millions or even billions of vertices and edges. For us to achieve this, we have built it on top of polars a blazingly fast DataFrames library implemented in Rust using Apache Arrow Columnar Format as the memory model.

• Graph abstraction: pregel-rs provides a graph abstraction that makes it easy to represent and manipulate graphs in Rust. It supports both directed and undirected graphs, and provides methods for adding, removing, and querying vertices and edges.

• Customizable computation: pregel-rs allows users to implement their own computation logic by defining vertex computation functions. This gives users the flexibility to implement their own graph algorithms and customize the behavior of pregel-rs to suit their specific needs.

Getting started

To get started with pregel-rs, you can follow these steps:

1. Install Rust: pregel-rs requires Rust to be installed on your system. You can install Rust by following the instructions on the official Rust website: https://www.rust-lang.org/tools/install

2. Create a new Rust project: Once Rust is installed, you can create a new Rust project using the Cargo package manager, which is included with Rust. You can create a new project by running the following command in your terminal:

cargo new my_pregel_project

3. Add pregel-rs as a dependency: Next, you need to add pregel-rs as a dependency in your Cargo.toml file, which is located in the root directory of your project. You can add the following line to your Cargo.toml file:

[dependencies]
pregel-rs = "0.0.13"

4. Implement your graph algorithm: Now you can start implementing your graph algorithm using the pregel-rs framework. You can define your vertex computation functions and use the graph abstraction provided by pregel-rs to manipulate the graph.

5. Build and run your project: Once you have implemented your graph algorithm, you can build and run your project using the Cargo package manager. You can build your project by running the following command in your terminal:

You could also run one of the examples to check how this library works:

cargo build
cargo run --example pagerank

Acknowledgments

Read Pregel: A System for Large-Scale Graph Processing for a reference on how to implement your own Graph processing algorithms in a Pregel fashion. If you want to take some inspiration from some curated-sources, just explore the /examples folder of this repository.

Related projects

1. GraphX is a library enabling Graph processing in the context of Apache Spark.
2. GraphFrames is the DataFrame-based equivalent to GraphX.

License

Copyright © 2023 Ángel Iglesias Préstamo ([email protected])

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.

By contributing to this project, you agree to release your contributions under the same license.