Skip to content
This repository has been archived by the owner on Dec 20, 2022. It is now read-only.

SparkRDMA Release Version 2.0
Compare
Choose a tag to compare
@petro-rudenko petro-rudenko released this 31 Mar 02:15
· 37 commits to master since this release
v2.0
5519dd9
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

All-new implementation of SparkRDMA, redesigned from the ground up to further increase scalability, robustness and most importantly - performance.

Among the new features and capabilities introduced in this release:

  • All-new Metadata (Map Output) fetching protocol - now allows scaling to the tens of thousands of partitions, with superior performance and recoverability
  • Software-level flow control in RdmaChannel - eliminates pause storms in the fabric
  • ODP (On-Demand Paging) support - improves memory efficiency

Attached are pre-built binaries. Please follow the README page for instructions.

Assets 3
Loading