This repository has been archived by the owner on Dec 20, 2022. It is now read-only.
SparkRDMA Release Version 2.0
All-new implementation of SparkRDMA, redesigned from the ground up to further increase scalability, robustness and most importantly - performance.
Among the new features and capabilities introduced in this release:
- All-new Metadata (Map Output) fetching protocol - now allows scaling to the tens of thousands of partitions, with superior performance and recoverability
- Software-level flow control in RdmaChannel - eliminates pause storms in the fabric
- ODP (On-Demand Paging) support - improves memory efficiency
Attached are pre-built binaries. Please follow the README page for instructions.