SIMD-0197: Chili Peppers #197
Conversation
|
Can we fix the markdown lint, please? |
proposals/0197-chilli-peppers.md
Outdated
|
|
||
| Account Chili Pepper Clock Timestamp - each account in the "hot" state needs to keep track of the value of the Block Chili Pepper Clock the last time it was accessed (read or written to). This allows determination of which accounts are hot and which are cold. | ||
|
|
||
| Hot Cache Size - A new predefined constant, termed "Hot Cache Size" corresponds approximately to the size of the hot state supported |
There was a problem hiding this comment.
How is the size determined?
There was a problem hiding this comment.
Most likely by deciding what the "standard" size of a solana network box should be and backing into how much should be used for account state. For example, given the small amount of new-state actually referenced per block, I would guess we could set this at 50gb and it would be 99% good...
There was a problem hiding this comment.
ok. "hot cache size" = 50gb. then, how about "Block Chili Pepper Limit"?
There was a problem hiding this comment.
What about forking situations?
The chili pepper meter read from the sysvar will only represent the bytes loaded on current fork. But the
total "hot cache" could is global and shared by the all forks. And the bytes loaded from current fork will only be taking part of the global "hot cache".
Maybe we should allow some percentage of lee-way to take into account of forking situation?
If so, how much percentage to allow, i.e. 10%, 20%?
There was a problem hiding this comment.
Thinking a bit more. "hot cache size" (50gb) is the physical memory capacity, right?
chili pepper meter measures just the bytes of the account's data loaded from disk. But there are other extra cost in addition to the account's data bytes to load, such as account's index, account's chili-pepper store etc. The actual physical memory will be larger than the chili pepper bytes?
If we are targeting 50gb physical memory, should we take into accounts of those extra cost and reduce them from the physical memory budget? If so, how much should we reduce?
proposals/0197-chilli-peppers.md
Outdated
| block_chili_pepper_clock = prev_block_chili_pepper_clock + sum(txn.requested_chili_peppers for txn in block.txns) | ||
| ``` | ||
|
|
||
| Implemented as a 64-bit unsigned integer (uint64), this clock is updated at the beginning of every block to reflect the total Chili Peppers requested since the chain's genesis. This monotonically increasing value is stored in a dedicated system variable (sysvar), ensuring that it remains accessible and immutable throughout the blockchain's operation. |
There was a problem hiding this comment.
begining or end?
Without actually looking up the account, how would a block know how many chili peper are actually used?
There was a problem hiding this comment.
accessing the meta-data for an account needs to be accounted for via chili-peppers as well?
proposals/0197-chilli-peppers.md
Outdated
|
|
||
| ## Impact | ||
|
|
||
| How will the implemented proposal impacts dapp developers, validators, and core contributors? |
There was a problem hiding this comment.
We will need to expand this section.
How will this impact dapp developers?
There was a problem hiding this comment.
calculating chilli peppers could be mostly hidden by the client sdk so I would guess once people update to them, there should be almost zero effect
There was a problem hiding this comment.
every transaction on the Solana network will be required to use the ComputeBudget program to request the maximum number of Chili Peppers they require.
Wouldn't the dapp have to specify the Chili pepper in the transactions? In order to do so, will they need to know whether the account is hot or cold?
There was a problem hiding this comment.
How many chili-pepper bytes should the user specify when requesting a transactions?
Should it be the account's data + accounts' index data + account's chili-pepper store, because bringing an account into global cache will incur other costs than jus the accounts' data?
|
|
||
| ## Security Considerations | ||
|
|
||
| What security implications/considerations come with implementing this feature? |
There was a problem hiding this comment.
Answers for these questions?
|
|
||
| - **Error Description**: A transaction attempts to interact with an account that has been designated as "cold" due to its Chili Pepper Clock Timestamp falling below the "Hot Cache Size" threshold relative to the current Block Chili Pepper Clock, without requesting sufficient Chili Peppers. | ||
| - **Handling**: The transaction is rejected with a "Cold Account Access Attempted" error. | ||
|
|
There was a problem hiding this comment.
Accessing a cold account and a non-existent account without sufficient chili peppers to account for the meta-data associated with an account should return the same error. This would allow us to keep the meta-data in a cold state as well and prevents the dos attack of referencing large numbers of non-existent accounts just to crush the slower back end database where the cold is stored.
|
|
||
| ## Security Considerations | ||
|
|
||
| What security implications/considerations come with implementing this feature? |
There was a problem hiding this comment.
chili peppers are a consumable resource without cost? Does this mean I can
- put a single txn with all the chili peppers possible on a block
- a high enough priority fee to guarantee it gets put into the block
and lock out everybody elses ability to retrieve cold accounts?
I would guess we would limit the max chili peppers a single txn could request to be the max number of accounts it could reference times 10m... Given ALUT, that would be a lot of chili peppers.. all of them
I would suggest we make chili peppers be representative of actual SOL so that it would cost to do this attack where we only actually charge for actual cold access?! a real head scratcher...
There was a problem hiding this comment.
A block builder sees a txn, notices it references unknown accounts, notices it does not have the chili peppers required to account for the meta data for those unknown accounts, does not include that txn
I am now thinking hitting cold meta data needs to be super expensive..
proposals/0197-chilli-peppers.md
Outdated
|
|
||
| Account Chili Pepper Clock Timestamp - each account in the "hot" state needs to keep track of the value of the Block Chili Pepper Clock the last time it was accessed (read or written to). This allows determination of which accounts are hot and which are cold. | ||
|
|
||
| Hot Cache Size - A new predefined constant, termed "Hot Cache Size" corresponds approximately to the size of the hot state supported |
There was a problem hiding this comment.
ok. "hot cache size" = 50gb. then, how about "Block Chili Pepper Limit"?
proposals/0197-chilli-peppers.md
Outdated
|
|
||
| #### Storage and Management | ||
|
|
||
| To manage the Account State Unit Clocks efficiently, Solana employs a table, associating each hot account with its respective Chili Pepper Clock Timestamp. This table enables the dynamic tracking and updating of account states, facilitating the transition of accounts between hot and cold statuses based on their activity. |
There was a problem hiding this comment.
How to handle validator restart? Should we document the restart process too?
There was a problem hiding this comment.
An account that is deleted is still considered hot until its state unit clock lapses into cold. Creating an account against a
deleted account which is still hot, will create the hot account again.
A note for ourselves.
This implies a change for our account's db and snapshot. We will need to keep track of all deleted accounts, while is still hot in the cache. This can increase the snapshot size, account's index size? What changes will be required for our accounts-db cleaning and shrinking process?
proposals/0197-chilli-peppers.md
Outdated
|
|
||
| #### Invalid Chili Pepper Request | ||
|
|
||
| - **Error Description**: A transaction specifies an invalid number of Chili Peppers, either by requesting more than a predefined maximum limit per transaction or by formatting the request improperly. |
There was a problem hiding this comment.
There are two perspectives for the usage of the chili pepper.
- leader's perspective
- replayer's perspective
Should we separately document their behaviors regarding chili peppers change?
proposals/0197-chilli-peppers.md
Outdated
|
|
||
| Account Chili Pepper Clock Timestamp - each account in the "hot" state needs to keep track of the value of the Block Chili Pepper Clock the last time it was accessed (read or written to). This allows determination of which accounts are hot and which are cold. | ||
|
|
||
| Hot Cache Size - A new predefined constant, termed "Hot Cache Size" corresponds approximately to the size of the hot state supported |
There was a problem hiding this comment.
What about forking situations?
The chili pepper meter read from the sysvar will only represent the bytes loaded on current fork. But the
total "hot cache" could is global and shared by the all forks. And the bytes loaded from current fork will only be taking part of the global "hot cache".
Maybe we should allow some percentage of lee-way to take into account of forking situation?
If so, how much percentage to allow, i.e. 10%, 20%?
|
|
||
| Computing the block Chili Pepper clock: | ||
|
|
||
| ```python |
There was a problem hiding this comment.
Will this incentivize people to over request chili pepper? What's the downside for over request chili pepper?
proposals/0197-chilli-peppers.md
Outdated
|
|
||
| #### Storage and Management | ||
|
|
||
| To manage the Account State Unit Clocks efficiently, Solana employs a table, associating each hot account with its respective Chili Pepper Clock Timestamp. This table enables the dynamic tracking and updating of account states, facilitating the transition of accounts between hot and cold statuses based on their activity. |
There was a problem hiding this comment.
nit: State Unit Clocks? Can we standardize the name to just use chili pepper clock?
It is confusing to use both state unit clocks and chili pepper clock` in the document.
|
|
||
| ```rust | ||
| struct SysvarBlockChiliPepperClock { | ||
| uint64_t state_unit_clock; |
There was a problem hiding this comment.
nit: change it to rust code (not c code), since it is using rust syntax highlight.
proposals/0197-chilli-peppers.md
Outdated
|
|
||
| ## Impact | ||
|
|
||
| How will the implemented proposal impacts dapp developers, validators, and core contributors? |
There was a problem hiding this comment.
How many chili-pepper bytes should the user specify when requesting a transactions?
Should it be the account's data + accounts' index data + account's chili-pepper store, because bringing an account into global cache will incur other costs than jus the accounts' data?
proposals/0197-chilli-peppers.md
Outdated
|
|
||
| Account Chili Pepper Clock Timestamp - each account in the "hot" state needs to keep track of the value of the Block Chili Pepper Clock the last time it was accessed (read or written to). This allows determination of which accounts are hot and which are cold. | ||
|
|
||
| Hot Cache Size - A new predefined constant, termed "Hot Cache Size" corresponds approximately to the size of the hot state supported |
There was a problem hiding this comment.
Thinking a bit more. "hot cache size" (50gb) is the physical memory capacity, right?
chili pepper meter measures just the bytes of the account's data loaded from disk. But there are other extra cost in addition to the account's data bytes to load, such as account's index, account's chili-pepper store etc. The actual physical memory will be larger than the chili pepper bytes?
If we are targeting 50gb physical memory, should we take into accounts of those extra cost and reduce them from the physical memory budget? If so, how much should we reduce?
|
@topointon-jump can you fix the linting errors? |
Initial SIMD for Chili Peppers proposal for state bandwidth used by transactions.