Skip to content

Commit 0b16321

Browse files
fabianbormannfabianbormann
committed
chore: refacotring and clean-up
1 parent 097d308 commit 0b16321

File tree

9 files changed

+200
-243
lines changed

9 files changed

+200
-243
lines changed

‎README.md

Lines changed: 66 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Cardano Rewards Calculation
1+
# Cardano Rewards Calculation 🧮
22

33
<p align="left">
44
<img alt="Tests" src="https://github.com/cardano-foundation/cf-java-rewards-calculation/actions/workflows/tests.yaml/badge.svg?branch=main" />
@@ -27,21 +27,21 @@ We also generate for each version of this project calculation reports. These rep
2727
```mermaid
2828
flowchart
2929
A[Total Transaction Fees <br />at Epoch n] --> B[Total Reward Pot <br />at Epoch n]
30-
B --> | <b><i>treasuryGrowthRate</b></i> | C[Treasury]
31-
B --> | 1 - <b><i>treasuryGrowthRate</b></i> | D[Stake Pool Rewards Pot <br />at Epoch n]
30+
B --> | treasuryGrowthRate | C[Treasury]
31+
B --> | 1 - treasuryGrowthRate | D[Stake Pool Rewards Pot <br />at Epoch n]
3232
subgraph ADA_POTS[" "]
3333
D --> | Unclaimed Rewards | E["ADA Reserves<br /> (monetary expansion) <br /> Started at ~14B ADA"]
34-
E --> | <b><i>monetaryExpandRate</a></b></i> * Performance of all Stake Pools | B
34+
E --> | monetaryExpandRate * Performance of all Stake Pools | B
3535
C --> F[Payouts e.g. for <br />Project Catalyst]
3636
D --> | Rewards Equation<br /> for Pool 1 | G[Stake Pool 1]
3737
D --> | ewards Equation<br /> for Pool 2 | H[Stake Pool 2]
3838
D --> I[...]
3939
D --> | Rewards Equation<br /> for Pool n | J[Stake Pool n]
40-
J --> | <b><i>margin & minPoolCost</i></b> | K[Operators]
41-
J --> | <b><i>rewards</i></b> | L[Delegators]
40+
J --> | margin & minPoolCost | K[Operators]
41+
J --> | rewards | L[Delegators]
4242
D --> | Rewards going to<br /> de-registered<br /> stake addresses | C
43-
L <--> | Stake Key Registration & <br /> Deregistration | M[Deposits]
44-
K <--> | Stake Pool Registration & <br /> Deregistration | M
43+
L <--> | Stake Key Registrations + <br /> Deregistrations | M[Deposits]
44+
K <--> | Stake Pool Registrations + <br /> Deregistrations | M
4545
M --> | Unclaimed Refunds for Retired Pools | C
4646
end
4747
@@ -60,20 +60,69 @@ flowchart
6060
style ADA_POTS fill:#f6f9ff,stroke:#f6f9ff
6161
```
6262

63-
## 🚀 Getting Started
64-
65-
#### Prerequisites
63+
## 🤓Interesting Findings
64+
65+
While the flowchart above shows the calculation of the Ada pots in general, there are some more aspects that need to be considered:
66+
67+
- The point in time 🕑. The reward calculation starts at slot `(4 * 2160) / 0.05) = 172800` (48h) each epoch. Before the Babbage era,
68+
accounts that were de-registered **before** this point in time were not considered for the rewards calculation.
69+
This means that the rewards for the de-registered stake addresses are not considered in the rewards calculation.
70+
Those rewards were not distributed and went therefore back to the **reserves**. Accounts that deregistered **after**
71+
this point in time, but before the end of the epoch, were considered for the rewards calculation. But as it is not
72+
possible to send rewards to a de-registered stake address, the rewards went back to the **treasury**.
73+
- At the Allegra hard fork, the (pre-Shelley) bootstrap addresses were removed from the UTxO and the Ada contained in them was returned to the reserves.
74+
- There was a different behavior (pre-Allegra): If a pool reward address had been used for multiple pools,
75+
the stake account only received the reward for one of those pools and did also not get any member rewards.
76+
This **behavior has changed in mainnet epoch 236** with the Allegra hard fork. Now, the stake account receives the rewards for all pools (including member rewards).
77+
- Transaction fees and a part of the reserve is used to build the total reward pot. It is often mentioned that
78+
the monetary expansion rate (protocol parameter) is used to calculate the part coming from the reserves.
79+
While this is true, the actual calculation is a multiplication of the monetary expansion rate and the performance of all stake pools `eta`.
80+
`Eta` is the ratio of the blocks produced **by pools** in an epoch and the expected blocks `(432000 slots per epoch / 20 sec per block = 21600)`.
81+
With beginning of Shelly the blocks were produced by OBFT nodes and not by pools. Therefore, the performance of all stake pools would be 0.
82+
The decentralization parameter `d` has been introduced to slightly increase the amount of **expected blocks produced by pools**.
83+
In the time when `d` was above 0.8 `eta` was set to 1.0. `d` decreased over time from 1.0 to 0.0 and disappeared completely with the Vasil hard fork.
84+
- The pool deposit is currently 500 Ada. This deposit will be returned (on the next epoch boundary) to the pool reward address when the pool is retired.
85+
However, if the deposit could not be returned (e.g. because the pool reward address is de-registered), the **deposit will be added to the treasury** instead.
86+
- Pool updates override pool deregistrations. This means that if a pool is updated before the end of the epoch, the pool will not be retired and the deposit will not be returned.
6687

67-
Java 17
88+
## 🚀 Getting Started
6889

69-
#### Build & Test
90+
Make sure to have Java 17 installed and run the following commands:
7091

7192
```
7293
git clone https://github.com/cardano-foundation/cf-java-rewards-calculation.git
7394
cd cf-java-rewards-calculation
7495
./mvnw clean test
7596
```
97+
98+
## 📦 Usage
99+
100+
In the near future you can use the calculation part of this repository as a library in your own project as it will be accessible
101+
through maven central.
102+
103+
For now the structure of the repository is divided in two parts:
104+
105+
- calculation package
106+
- rewards
107+
- [EpochCalculation](./src/main/java/org/cardanofoundation/rewards/calculation/EpochCalculation.java)
108+
- [DepositsCalculation](./src/main/java/org/cardanofoundation/rewards/calculation/DepositsCalculation.java)
109+
- [PoolRewardsCalculation](./src/main/java/org/cardanofoundation/rewards/calculation/PoolRewardsCalculation.java)
110+
- [TreasuryCalculation](./src/main/java/org/cardanofoundation/rewards/calculation/TreasuryCalculation.java)
111+
112+
- validation package
113+
- [EpochValidation](./src/main/java/org/cardanofoundation/rewards/validation/EpochValidation.java)
114+
- [DepositsValidation](./src/main/java/org/cardanofoundation/rewards/validation/DepositsValidation.java)
115+
- [PoolRewardValidation](./src/main/java/org/cardanofoundation/rewards/validation/PoolRewardValidation.java)
116+
- [TreasuryValidation](./src/main/java/org/cardanofoundation/rewards/validation/TreasuryValidation.java)
117+
- ...
118+
119+
While the calculation package is used as a pure re-implementation of the ledger specification,
120+
the validation package is used to get the needed data (e.g. registration/deregistration certificates, epoch stakes, etc.)
121+
from the data provider and execute the calculation. Furthermore, the validation package is used to compare the calculated
122+
values with the actual values using DB Sync as ground truth.
123+
76124
#### Data Provider
125+
77126
The pool rewards calculation and also the treasury calculation requires a data provider to perform the calculation.
78127
This repository offers different data providers and also an interface if you want to add your own provider. The following data providers are available:
79128

@@ -106,16 +155,15 @@ JSON_DATA_SOURCE_FOLDER=/path/to/your/rewards-calculation-test-data
106155
`⚠️ The actual rewards data will also be fetched but only used from the validator and not within the calculation package.`
107156

108157
## 🫡 Roadmap
109-
- [ ] Create REST endpoints to get the rewards as a service
110-
- [X] Include MIR certificates
158+
- [ ] Provide a library through maven central to use the calculation in other projects
159+
- [ ] Enhance reporting and add values for the other pots as well. Display the flow of Ada within an epoch
160+
- [ ] Find out the root cause of the difference between the actual rewards and the calculated rewards beginning with epoch 350
111161
- [ ] Add a `/docs` folder containing parsable Markdown files to explain MIR certificates and edge cases
112-
- [ ] Enhance reporting and add values for the other pots as well. Include information from the `/docs` folder
162+
- [X] Include MIR certificates
113163
- [X] Calculate member and operator rewards
114164
- [X] Add deposits and utxo pot
115165
- [X] Calculate unclaimed rewards that need to go back to the reserves
116166
- [X] Put rewards to unregistered stake addresses into the treasury
117-
- [ ] Create a web ui to visualize the rewards calculation
118-
- [ ] Find out the root cause of the difference between the actual rewards and the calculated rewards beginning with epoch 350
119167

120168
## 📖 Sources
121169
- [Shelley Cardano Delegation Specification](https://github.com/input-output-hk/cardano-ledger/releases/download/cardano-ledger-spec-2023-04-03/shelley-ledger.pdf)

‎src/main/java/org/cardanofoundation/rewards/calculation/TreasuryCalculation.java

Lines changed: 95 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,96 @@
22

33
import org.cardanofoundation.rewards.calculation.domain.*;
44
import org.cardanofoundation.rewards.calculation.enums.AccountUpdateAction;
5+
import org.cardanofoundation.rewards.calculation.enums.MirPot;
56

67
import java.math.BigDecimal;
78
import java.math.BigInteger;
89
import java.math.MathContext;
10+
import java.util.HashSet;
911
import java.util.List;
12+
import java.util.stream.Collectors;
1013

1114
import static org.cardanofoundation.rewards.calculation.constants.RewardConstants.*;
12-
import static org.cardanofoundation.rewards.calculation.util.BigNumberUtils.multiplyAndFloor;
15+
import static org.cardanofoundation.rewards.calculation.util.BigNumberUtils.*;
1316

1417
public class TreasuryCalculation {
1518

19+
public static TreasuryCalculationResult calculateTreasuryInEpoch(int epoch, ProtocolParameters protocolParameters,
20+
AdaPots adaPotsForPreviousEpoch, Epoch epochInfo,
21+
List<PoolDeregistration> retiredPools,
22+
List<MirCertificate> mirCertificates,
23+
final HashSet<String> deregisteredAccounts,
24+
final HashSet<String> lateDeregisteredAccounts,
25+
final HashSet<String> registeredAccountsUntilNow,
26+
BigInteger unspendableEarnedRewards) {
27+
// The Shelley era and the ada pot system started on mainnet in epoch 208.
28+
// Fee and treasury values are 0 for epoch 208.
29+
if (epoch <= MAINNET_SHELLEY_START_EPOCH) {
30+
return TreasuryCalculationResult.builder()
31+
.treasury(BigInteger.ZERO)
32+
.epoch(epoch)
33+
.totalRewardPot(BigInteger.ZERO)
34+
.treasuryWithdrawals(BigInteger.ZERO)
35+
.unspendableEarnedRewards(BigInteger.ZERO)
36+
.build();
37+
}
38+
39+
BigInteger totalFeesForCurrentEpoch = BigInteger.ZERO;
40+
int totalBlocksInEpoch = 0;
41+
42+
BigDecimal treasuryGrowthRate = protocolParameters.getTreasuryGrowRate();
43+
BigDecimal monetaryExpandRate = protocolParameters.getMonetaryExpandRate();
44+
BigDecimal decentralizationParameter = protocolParameters.getDecentralisation();
45+
46+
if (epochInfo != null) {
47+
totalFeesForCurrentEpoch = epochInfo.getFees();
48+
totalBlocksInEpoch = epochInfo.getBlockCount();
49+
if (isLower(decentralizationParameter, BigDecimal.valueOf(0.8)) && isHigher(decentralizationParameter, BigDecimal.ZERO)) {
50+
totalBlocksInEpoch = epochInfo.getNonOBFTBlockCount();
51+
}
52+
}
53+
54+
final BigInteger reserveInPreviousEpoch = adaPotsForPreviousEpoch.getReserves();
55+
final BigInteger treasuryInPreviousEpoch = adaPotsForPreviousEpoch.getTreasury();
56+
57+
final BigInteger totalRewardPot = calculateTotalRewardPotWithEta(
58+
monetaryExpandRate, totalBlocksInEpoch, decentralizationParameter, reserveInPreviousEpoch, totalFeesForCurrentEpoch);
59+
60+
final BigInteger treasuryCut = multiplyAndFloor(totalRewardPot, treasuryGrowthRate);
61+
BigInteger treasuryForCurrentEpoch = treasuryInPreviousEpoch.add(treasuryCut);
62+
63+
if (retiredPools.size() > 0) {
64+
List<String> rewardAddressesOfRetiredPools = retiredPools.stream().map(PoolDeregistration::getRewardAddress).toList();
65+
HashSet<String> deregisteredRewardAccounts = deregisteredAccounts.stream()
66+
.filter(rewardAddressesOfRetiredPools::contains).collect(Collectors.toCollection(HashSet::new));
67+
deregisteredRewardAccounts.addAll(lateDeregisteredAccounts.stream()
68+
.filter(rewardAddressesOfRetiredPools::contains).collect(Collectors.toSet()));
69+
List<String> ownerAccountsRegisteredInThePast = registeredAccountsUntilNow.stream()
70+
.filter(rewardAddressesOfRetiredPools::contains).toList();
71+
72+
BigInteger unclaimedRefunds = calculateUnclaimedRefundsForRetiredPools(retiredPools, deregisteredRewardAccounts, ownerAccountsRegisteredInThePast);
73+
treasuryForCurrentEpoch = treasuryForCurrentEpoch.add(unclaimedRefunds);
74+
}
75+
76+
BigInteger treasuryWithdrawals = BigInteger.ZERO;
77+
for (MirCertificate mirCertificate : mirCertificates) {
78+
if (mirCertificate.getPot() == MirPot.TREASURY) {
79+
treasuryWithdrawals = treasuryWithdrawals.add(mirCertificate.getTotalRewards());
80+
}
81+
}
82+
83+
treasuryForCurrentEpoch = treasuryForCurrentEpoch.subtract(treasuryWithdrawals);
84+
treasuryForCurrentEpoch = treasuryForCurrentEpoch.add(unspendableEarnedRewards);
85+
86+
return TreasuryCalculationResult.builder()
87+
.treasury(treasuryForCurrentEpoch)
88+
.epoch(epoch)
89+
.totalRewardPot(totalRewardPot)
90+
.treasuryWithdrawals(treasuryWithdrawals)
91+
.unspendableEarnedRewards(unspendableEarnedRewards)
92+
.build();
93+
}
94+
1695
/*
1796
* Calculate the reward pot for epoch e with the formula:
1897
*
@@ -49,7 +128,7 @@ private static BigDecimal calculateEta(int totalBlocksInEpochByPools, BigDecimal
49128
// instead of the OBFT (Ouroboros Byzantine Fault Tolerance) nodes. It was introduced close before the Shelley era:
50129
// https://github.com/input-output-hk/cardano-ledger/commit/c4f10d286faadcec9e4437411bce9c6c3b6e51c2
51130
BigDecimal expectedBlocksInNonOBFTSlots = new BigDecimal(EXPECTED_SLOT_PER_EPOCH )
52-
.multiply(activeSlotsCoeff).multiply (BigDecimal.ONE.subtract(decentralizationParameter));
131+
.multiply(activeSlotsCoeff).multiply(BigDecimal.ONE.subtract(decentralizationParameter));
53132

54133
// eta is the ratio between the number of blocks that have been produced during the epoch, and
55134
// the expectation value of blocks that should have been produced during the epoch under
@@ -64,17 +143,24 @@ private static BigDecimal calculateEta(int totalBlocksInEpochByPools, BigDecimal
64143
https://github.com/input-output-hk/cardano-ledger/blob/9e2f8151e3b9a0dde9faeb29a7dd2456e854427c/eras/shelley/formal-spec/epoch.tex#L546C9-L547C87
65144
*/
66145
public static BigInteger calculateUnclaimedRefundsForRetiredPools(List<PoolDeregistration> retiredPools,
67-
List<AccountUpdate> latestAccountUpdates) {
68-
BigInteger refunds = BigInteger.ZERO;
69-
146+
HashSet<String> deregisteredRewardAccounts,
147+
List<String> ownerAccountsRegisteredInThePast) {
148+
BigInteger unclaimedRefunds = BigInteger.ZERO;
70149
if (retiredPools.size() > 0) {
71-
for (AccountUpdate lastAccountUpdate : latestAccountUpdates) {
72-
if (lastAccountUpdate.getAction() == AccountUpdateAction.DEREGISTRATION) {
73-
refunds = refunds.add(POOL_DEPOSIT_IN_LOVELACE);
150+
/* Check if the reward address of the retired pool has been unregistered before
151+
or if the reward address has been unregistered after the randomness stabilization window
152+
or if the reward address has not been registered at all */
153+
for (PoolDeregistration retiredPool : retiredPools) {
154+
String rewardAddress = retiredPool.getRewardAddress();
155+
if (deregisteredRewardAccounts.contains(rewardAddress) ||
156+
!ownerAccountsRegisteredInThePast.contains(rewardAddress)) {
157+
// If the reward address has been unregistered, the deposit can not be returned
158+
// and will be added to the treasury instead (Pool Reap see: shelley-ledger.pdf p.53)
159+
unclaimedRefunds = unclaimedRefunds.add(POOL_DEPOSIT_IN_LOVELACE);
74160
}
75161
}
76162
}
77163

78-
return refunds;
164+
return unclaimedRefunds;
79165
}
80166
}

‎src/main/java/org/cardanofoundation/rewards/validation/PoolRewardValidation.java

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,9 @@ public static PoolRewardCalculationResult computePoolRewardInEpoch(String poolId
4141

4242
int totalBlocksInEpoch = epochInfo.getBlockCount();
4343

44-
if (epoch > 212 && epoch < 255) {
44+
BigDecimal decentralizationParameter = protocolParameters.getDecentralisation();
45+
BigDecimal decentralizationThreshold = BigDecimal.valueOf(0.8);
46+
if (isLower(decentralizationParameter, decentralizationThreshold) && isHigher(decentralizationParameter, BigDecimal.ZERO)) {
4547
totalBlocksInEpoch = epochInfo.getNonOBFTBlockCount();
4648
}
4749

@@ -110,7 +112,8 @@ public static PoolRewardCalculationResult computePoolRewardInEpoch(String poolId
110112
BigDecimal monetaryExpandRate = protocolParameters.getMonetaryExpandRate();
111113
BigDecimal treasuryGrowRate = protocolParameters.getTreasuryGrowRate();
112114

113-
if (epoch > 212 && epoch < 255) {
115+
BigDecimal decentralizationThreshold = BigDecimal.valueOf(0.8);
116+
if (isLower(decentralizationParameter, decentralizationThreshold) && isHigher(decentralizationParameter, BigDecimal.ZERO)) {
114117
totalBlocksInEpoch = epochInfo.getNonOBFTBlockCount();
115118
}
116119

@@ -130,7 +133,7 @@ public static PoolRewardCalculationResult computePoolRewardInEpoch(String poolId
130133
rewardAddresses = new HashSet<>(List.of(poolHistoryCurrentEpoch.getRewardAddress()));
131134
}
132135

133-
HashSet<String> accountsRegisteredInThePast = dataProvider.getRegisteredAccountsUntilLastEpoch(epoch, rewardAddresses, RANDOMNESS_STABILISATION_WINDOW);
136+
HashSet<String> accountsRegisteredInThePast = dataProvider.getRegisteredAccountsUntilLastEpoch(epoch + 2, rewardAddresses, RANDOMNESS_STABILISATION_WINDOW);
134137

135138
return computePoolRewardInEpoch(poolId, epoch, protocolParameters, epochInfo, stakePoolRewardsPot, adaInCirculation, poolHistoryCurrentEpoch,
136139
accountDeregistrations, lateAccountDeregistrations, accountsRegisteredInThePast, sharedPoolRewardAddressesWithoutReward);

0 commit comments

Comments
 (0)