Vaults Release #815

F4ever · 2025-11-07T16:31:59Z

Description

Describe what this pull request does. Explain the purpose of the changes and the problem they solve.

Related Issue/Task

Related task: #[task number] (e.g., Bump urllib3 from 1.26.2 to 1.26.4 #123)
Epic: [epic name or link, if applicable]

How Has This Been Tested?

Describe how you tested the changes:

Local tests (e.g., pytest)
Manual testing (describe steps)
Not tested (explain why)

Checklist

Documentation updated (if required)
New tests added (if applicable)
CSM_STATE_VERSION is bumped (if the new version affects data in the cache)

…le into feat/vaults-pectra

…culation

F4ever · 2025-11-10T17:39:48Z

src/providers/execution/contracts/lazy_oracle.py

+        """
+        Returns the Vaults
+        """
+        response = self.functions.batchVaultsInfo(offset, limit).call(block_identifier=block_identifier)


Suggested change

response = self.functions.batchVaultsInfo(offset, limit).call(block_identifier=block_identifier)

response = self.functions().batchVaultsInfo(offset, limit).call(block_identifier=block_identifier)

skhomuti · 2025-11-10T12:34:34Z

src/variables.py

 CACHE_PATH: Final = Path(os.getenv("CACHE_PATH", "."))

+VAULT_PAGINATION_LIMIT: Final = int(os.getenv("VAULT_PAGINATION_LIMIT", 1_000))
+VAULT_VALIDATOR_STAGES_BATCH_SIZE: Final = int(os.getenv("VAULT_VALIDATOR_STAGES_BATCH_SIZE", 1_00))


@tamtamchik you've not renamed this variable stages -> statuses
also formatting suggestion (just in case is it supposed to be 100 or 1_000?)

Suggested change

VAULT_VALIDATOR_STAGES_BATCH_SIZE: Final = int(os.getenv("VAULT_VALIDATOR_STAGES_BATCH_SIZE", 1_00))

VAULT_VALIDATOR_STAGES_BATCH_SIZE: Final = int(os.getenv("VAULT_VALIDATOR_STAGES_BATCH_SIZE", 100))

F4ever · 2025-11-10T19:19:54Z

src/variables.py


 CACHE_PATH: Final = Path(os.getenv("CACHE_PATH", "."))

+VAULT_PAGINATION_LIMIT: Final = int(os.getenv("VAULT_PAGINATION_LIMIT", 1_000))


Too high. Estimated gas consumtion > 80kk. I propose to set it in 100.

F4ever · 2025-11-10T19:20:35Z

src/variables.py

 CACHE_PATH: Final = Path(os.getenv("CACHE_PATH", "."))

+VAULT_PAGINATION_LIMIT: Final = int(os.getenv("VAULT_PAGINATION_LIMIT", 1_000))
+VAULT_VALIDATOR_STAGES_BATCH_SIZE: Final = int(os.getenv("VAULT_VALIDATOR_STAGES_BATCH_SIZE", 1_00))


Suggested change

VAULT_VALIDATOR_STAGES_BATCH_SIZE: Final = int(os.getenv("VAULT_VALIDATOR_STAGES_BATCH_SIZE", 1_00))

VAULT_VALIDATOR_STAGES_BATCH_SIZE: Final = int(os.getenv("VAULT_VALIDATOR_STAGES_BATCH_SIZE", 100))

Can be increased to 1_000

F4ever · 2025-11-10T19:22:12Z

src/providers/execution/contracts/lazy_oracle.py

+    def get_validator_statuses(
+        self,
+        pubkeys: list[str],
+        batch_size: int = variables.VAULT_VALIDATOR_STAGES_BATCH_SIZE,


Better provide this variable from staking vault service. Do not use as default, like this

F4ever · 2025-11-10T19:22:47Z

src/providers/execution/contracts/lazy_oracle.py

+        out: dict[str, ValidatorStatus] = {}
+
+        for i in range(0, len(pubkeys), batch_size):
+            batch = list(map(HexBytes, pubkeys[i : i + batch_size]))


Why are we applying HexBytes here? Let's remove it

F4ever · 2025-11-10T19:24:41Z

src/providers/execution/contracts/lazy_oracle.py

+                for pk, status in zip(batch, response)
+            })
+
+        return out


Let's add info log after all responses were received

F4ever · 2025-11-10T19:26:19Z

src/providers/execution/contracts/lazy_oracle.py

+            batch = self.get_vaults(
+                block_identifier=block_identifier, offset=offset, limit=variables.VAULT_PAGINATION_LIMIT
+            )
+            if not batch:


Is there a case when we can receive empty response here?

F4ever · 2025-11-10T19:27:07Z

src/providers/execution/contracts/lazy_oracle.py

+
+        return out
+
+    def get_all_vaults(self, block_identifier: BlockIdentifier = 'latest') -> list[VaultInfo]:


Need tests for pagination in get_all_vaults and for get_validator_statuses

F4ever · 2025-11-10T19:30:19Z

src/providers/execution/contracts/accounting.py

+
+        logger.info(
+            {
+                'msg': 'Call `simulateOracleReport({}, {}, {}, {}, {}, {}, {}, {}, {})`.'.format(  # pylint: disable=consider-using-f-string


Suggested change

'msg': 'Call `simulateOracleReport({}, {}, {}, {}, {}, {}, {}, {}, {})`.'.format( # pylint: disable=consider-using-f-string

'msg': 'Call `simulateOracleReport({}, {}, {}, {}, {}, {}, {}, {}, {})`.'.format(*payload.as_tuple())

F4ever · 2025-11-10T19:31:30Z

src/providers/execution/contracts/accounting.py

+
+        response = self.functions.simulateOracleReport(payload.as_tuple()).call(block_identifier=block_identifier)
+
+        response = ReportSimulationResults(*response)


Better assign field by field:
eg balance=balance

or use named_tuple_to_dataclass

F4ever · 2025-11-10T19:34:13Z

src/providers/execution/contracts/lazy_oracle.py

+
+        out: list[VaultInfo] = []
+        for vault in response:
+            out.append(VaultInfo(


can we use named_tuple_to_dataclass here?

or just

@list_of_dataclasses(VaultInfo)

F4ever · 2025-11-10T19:40:04Z

src/providers/execution/contracts/vault_hub.py

+    ) -> list[MintedSharesOnVaultEvent]:
+        logs = get_events_in_range(self.events.MintedSharesOnVault(), from_block_number, to_block_number)
+
+        return [MintedSharesOnVaultEvent.from_log(log) for log in logs]


Need some tests on all from_log methods

F4ever · 2025-11-10T19:45:10Z

src/providers/ipfs/storacha.py

+    BRIDGE_URL = 'https://up.storacha.network/bridge'
+    GATEWAY_URL = 'https://storacha.link/ipfs/'


Maybe move this to variables?

F4ever · 2025-11-10T19:58:51Z

src/providers/consensus/client.py

-    API_GET_VALIDATORS = 'eth/v1/beacon/states/{}/validators'
    API_GET_SPEC = 'eth/v1/config/spec'
    API_GET_GENESIS = 'eth/v1/beacon/genesis'
+    API_GET_VALIDATOR = 'eth/v1/beacon/states/{}/validators/{}'


Add additional check with this api to checks module

F4ever · 2025-11-10T20:01:18Z

tests/integration/contracts/test_accounting.py

+                'simulate_oracle_report',
+                (
+                    ReportSimulationPayload(
+                        timestamp=0,


can we write a test with non zero vaules?

F4ever · 2025-11-11T17:54:19Z

src/services/staking_vaults.py

+        """
+        validators_by_vault = self._get_validators_by_vault(validators, vaults)
+        total_pending_amount_by_pubkey = self._get_total_pending_amount_by_pubkey(pending_deposits)
+        inactive_validator_statuses = self._get_non_activated_validator_stages(validators, vaults, block_identifier)


_get_non_activated_validator_stages -> _get_non_activated_validator_statuses

Maybe get_non_aligable_to_activate_validators?

F4ever · 2025-11-11T18:24:51Z

src/services/staking_vaults.py

+            and v.validator.withdrawal_credentials in vault_wcs
+        ]
+
+        return self.w3.lido_contracts.lazy_oracle.get_validator_statuses(


I propose to filter all not found keys here

F4ever · 2025-11-11T18:41:16Z

src/services/staking_vaults.py

+        return {pubkey: Gwei(deposits[pubkey]) for pubkey in deposits}
+
+    @staticmethod
+    def _get_validators_by_vault(validators: list[Validator], vaults: VaultsMap) -> VaultToValidators:


Let's add unit tests for this func

F4ever · 2025-11-13T03:48:57Z

src/utils/apr.py

+
+
+def calculate_gross_core_apr(
+    post_internal_ether: int,


I think here should be weis and shares everywhere

TODO: check why typecheking not working

F4ever · 2025-11-13T03:50:54Z

src/utils/apr.py

+from src.modules.accounting.types import SECONDS_IN_YEAR, Shares
+
+
+def calculate_gross_core_apr(


Maybe it's net apr since we're removing fees from calculations?

F4ever · 2025-11-13T03:52:30Z

src/utils/apr.py

+    rate_diff: Decimal = post_share_rate_no_fees - pre_share_rate
+
+    if rate_diff < 0:
+        return Decimal(0)


why it can't be negative?

F4ever · 2025-11-13T03:55:32Z

src/modules/accounting/accounting.py

+            time_elapsed_seconds=self._get_time_elapsed_seconds_from_prev_report(blockstamp),
+        )
+
+        latest_onchain_ipfs_report_data = self.staking_vaults.get_latest_onchain_ipfs_report_data(blockstamp.block_hash)


maybe call self.w3.lido_contracts.lazy_oracle.get_latest_report_data(blockstamp.block_hash) directly here. Function with one call is redundant

F4ever · 2025-11-13T03:57:12Z

src/services/staking_vaults.py

+        latest_onchain_ipfs_report_data: OnChainIpfsVaultReportData,
+        core_apr_ratio: Decimal,
+        pre_total_pooled_ether: int,
+        pre_total_shares: int,


Suggested change

pre_total_shares: int,

pre_total_shares: Shares,

F4ever · 2025-11-17T15:48:23Z

src/services/staking_vaults.py

+        bb = self.w3.ipfs.fetch(CID(tree_cid), current_frame)
+        return StakingVaultIpfsReport.parse_merkle_tree_data(bb)
+
+    def get_latest_onchain_ipfs_report_data(self, block_identifier: BlockIdentifier) -> OnChainIpfsVaultReportData:


lets remove this function. Since and replace with direct contract call. Also report is not from IPFS, so the name confuses a little bit

F4ever · 2025-11-17T15:49:45Z

src/services/staking_vaults.py

+        chain_config: ChainConfig,
+        current_frame: FrameNumber,
+    ) -> tuple[Optional[StakingVaultIpfsReport], BlockNumber]:
+        slots_per_frame = frame_config.epochs_per_frame * chain_config.slots_per_epoch


Web3Converter can be reused to calculate slots and other staff

F4ever · 2025-11-17T15:51:12Z

src/services/staking_vaults.py

+        accounting_oracle = self.w3.lido_contracts.accounting_oracle
+
+        if latest_onchain_ipfs_report_data.report_cid != "":
+            prev_ipfs_report = self.get_ipfs_report(latest_onchain_ipfs_report_data.report_cid, current_frame)


can we remove part with current frame from ipfs methods?

F4ever · 2025-11-17T15:51:58Z

src/services/staking_vaults.py

+            prev_ipfs_report = self.get_ipfs_report(latest_onchain_ipfs_report_data.report_cid, current_frame)
+            tree_root_hex = Web3.to_hex(latest_onchain_ipfs_report_data.tree_root)
+
+            if not self.is_tree_root_valid(tree_root_hex, prev_ipfs_report):


Can we validate data in ipfs provider, so we can correctly use fallbacks in case in one of provider got broken

F4ever · 2025-11-17T15:53:26Z

src/services/staking_vaults.py

+        return output
+
+    def get_ipfs_report(self, ipfs_report_cid: str, current_frame: FrameNumber) -> StakingVaultIpfsReport:
+        if ipfs_report_cid == "":


we already called this method in function below.
I propose to remove this function and call self.get_vault_report directly from _get_start_point_for_fee_calculations

F4ever · 2025-11-17T15:54:52Z

src/services/staking_vaults.py

+        current_frame: FrameNumber,
+    ) -> tuple[Optional[StakingVaultIpfsReport], BlockNumber]:
+        slots_per_frame = frame_config.epochs_per_frame * chain_config.slots_per_epoch
+        accounting_oracle = self.w3.lido_contracts.accounting_oracle


let's inline this line into line 511

chasingrainbows and others added 30 commits June 25, 2025 14:17

feat(vault-pectra): move part of test_lido_locator_contract to testnet

2233a01

Merge branch 'feat/vaults-pectra' of github.com:lidofinance/lido-orac…

dc06082

…le into feat/vaults-pectra

refactor(vaults): Add test_infra_fee, test_reservation_liquidity_fee

ef2f9dd

fix(vaults): calculation of liquidity_fee. Missed last sector for cal…

7bf36ea

…culation

feat(vaults): calc fees

16090b0

feat(vaults): fix review attempt 1

b7fdf92

feat(vaults): temp commit for debugging

74b4e42

feat(vaults): cast prevFee to int

7826823

feat(vaults): Add decimal for calc fees

87dc35b

test(vaults): cross testing for taking fees

b4a1c03

test(vaults): cross testing for taking fees

778d1c9

feat(vaults): get previous report from last processing refSlot

126a400

fix(vaults): calc vault fees on mainet, tesnet at the first time

dd041ef

fix(vaults): calc vault fees on mainet, tesnet at the first time

ca4f3b6

fix(vaults): calc vault fees on mainet, tesnet at the first time

f327382

fix(vaults): rename constants for getting slashing period

b7f3265

fix(vaults): use PRECISION_E27 for taking fees

eaf9739

feat(vaults): move fees into staking_vaults

ade7592

chore(vaults): black formatting

6f721ee

chore(vaults): black formatting

64c3706

fix(vaults): test apr by using 27 decimals instead of 28

caead36

fix(vaults): review marks

adbd808

fix(vaults): review marks

fb137f3

fix(vaults): review marks

f6cfe1e

chore(vaults): fix lint

3af705b

chore(vaults): fix lint

45fe3b3

fix(vaults): fix VaultInfo getting inOutDelta

1d54f5c

chore(vaults): lint

ea209e1

chore(vaults): lint

0060cf8

feat(vaults): add split up fees into ipfs report

38dfbbf

F4ever commented Nov 10, 2025

View reviewed changes

skhomuti reviewed Nov 10, 2025

View reviewed changes

F4ever commented Nov 10, 2025

View reviewed changes

F4ever commented Nov 11, 2025

View reviewed changes

F4ever commented Nov 13, 2025

View reviewed changes

F4ever commented Nov 17, 2025

View reviewed changes

	response = self.functions.batchVaultsInfo(offset, limit).call(block_identifier=block_identifier)
	response = self.functions().batchVaultsInfo(offset, limit).call(block_identifier=block_identifier)

	VAULT_VALIDATOR_STAGES_BATCH_SIZE: Final = int(os.getenv("VAULT_VALIDATOR_STAGES_BATCH_SIZE", 1_00))
	VAULT_VALIDATOR_STAGES_BATCH_SIZE: Final = int(os.getenv("VAULT_VALIDATOR_STAGES_BATCH_SIZE", 100))


		CACHE_PATH: Final = Path(os.getenv("CACHE_PATH", "."))

		VAULT_PAGINATION_LIMIT: Final = int(os.getenv("VAULT_PAGINATION_LIMIT", 1_000))


		return out

		def get_all_vaults(self, block_identifier: BlockIdentifier = 'latest') -> list[VaultInfo]:

	'msg': 'Call `simulateOracleReport({}, {}, {}, {}, {}, {}, {}, {}, {})`.'.format( # pylint: disable=consider-using-f-string
	'msg': 'Call `simulateOracleReport({}, {}, {}, {}, {}, {}, {}, {}, {})`.'.format(*payload.as_tuple())


		response = self.functions.simulateOracleReport(payload.as_tuple()).call(block_identifier=block_identifier)

		response = ReportSimulationResults(*response)

		BRIDGE_URL = 'https://up.storacha.network/bridge'
		GATEWAY_URL = 'https://storacha.link/ipfs/'

		from src.modules.accounting.types import SECONDS_IN_YEAR, Shares


		def calculate_gross_core_apr(

Vaults Release #815

Are you sure you want to change the base?

Vaults Release #815

Conversation

F4ever commented Nov 7, 2025

Description

Related Issue/Task

How Has This Been Tested?

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

F4ever Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

F4ever Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

F4ever Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

F4ever Nov 10, 2025 •

edited

Loading

F4ever Nov 10, 2025 •

edited

Loading

F4ever Nov 10, 2025 •

edited

Loading