-
Notifications
You must be signed in to change notification settings - Fork 127
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SIMD-0082: Relax Transaction Constraints #82
Changes from 3 commits
1e4cce9
819d63a
70882ea
555d422
46ae283
3a7e79e
467c80b
6f7fe48
8f605b6
561b44f
6d6200b
dc42394
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,173 @@ | ||||||||||
--- | ||||||||||
simd: '0082' | ||||||||||
title: Relax Transaction Constraints | ||||||||||
authors: | ||||||||||
- Andrew Fitzgerald (Solana Labs) | ||||||||||
category: Standard | ||||||||||
type: Core | ||||||||||
status: Draft | ||||||||||
created: 2023-10-30 | ||||||||||
feature: | ||||||||||
--- | ||||||||||
|
||||||||||
## Summary | ||||||||||
|
||||||||||
This proposal aims to relax some of the constraints on which individual | ||||||||||
transactions can be included in a valid block. | ||||||||||
The proposal does not relax the constraints required for a transaction to be | ||||||||||
executed. | ||||||||||
|
||||||||||
## Motivation | ||||||||||
|
||||||||||
The current protocol places many constraints on the structure and contents | ||||||||||
of blocks; if any constraints are broken, block-validators will mark the block | ||||||||||
as invalid. | ||||||||||
Many of these constraints are necessary, but some of them are not, and lead to | ||||||||||
additional complexity in the protocol. | ||||||||||
This proposal aims to relax some of the constraints at the individual | ||||||||||
transaction level, in order to simplify the protocol, and give more flexibility | ||||||||||
to block-producer and block-validator implementations. | ||||||||||
|
||||||||||
More specifically, this proposal aims to relax the constraints which require | ||||||||||
account state in order to determine the validity of a transaction's inclusion. | ||||||||||
The reason these constraints are targetted specifically, is that reliance on | ||||||||||
account state necessarily requires synchronous execution within the protocol. | ||||||||||
This proposal on its' own, will not enable asynchronous execution, but it will | ||||||||||
remove one of the barriers to asynchronous execution. | ||||||||||
|
||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A bit confused by "aim" in Summary section and Motivations here. As "aim" seems to yield benefit to block producer ("...transaction can be included..."), which is not part of protocol; while Motivation talks about modifying protocol. Is it correct yo say this proposal has two parts:
or maybe I am thinking on wrong track? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think only your 2nd point is included in the proposal. Block-production changse are enabled by the changes in the proposal, but not required beyond potential changes to the fee-colection.
yes. Those bad transactions may end up having limited effect if we are still able to collect fees, or if we decide to make the block-producer cover those fees. They will not be executed though. Proposal does not suggest or require the block-producer to make any changes, but allows for changes. For several of the constraints, the producer is still incentivized to perform the checks (fee-payer, nonce) because otherwise they will not receive rewards for those transactions. There's freedom in how that is done, and the block-producer could potentially do this in a non-strict fashion. I'll cover this in the updated prop, but essentially block-validation and consensus really consists of 2 primary questions:
The changes here are intending to make answering this first question, "is this block valid?", a simpler process that does not require executing the block. This proposal separates these questions logically, but w/ current voting we still require the 2nd question, "what is the result of the block?", be answered synchronously. |
||||||||||
## Alternatives Considered | ||||||||||
|
||||||||||
1. Do nothing | ||||||||||
- This is the simplest option, as we could leave the protocol as is. | ||||||||||
However, this leaves the protocol more complex than it needs to be. | ||||||||||
2. Relax all but fee-paying constraint | ||||||||||
- This was actually the initial proposal, but it was decided that it would | ||||||||||
be better to relax as many constraints as possible, rather than just some. | ||||||||||
Any reliance on account state, means the protocol requires synchronous | ||||||||||
execution. | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. To clarify, "it would be better to relax as many constraints as possible" just means that the fee-paying constraint is also removed right? I think you could say that directly here. |
||||||||||
3. Additionally, relax the address lookup table resolution constraint | ||||||||||
- This was considered, since it is a transaction-level constraint that is | ||||||||||
depdendent on account-state. However, due to entry-level and block-level | ||||||||||
apfitzge marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||
constraints that rely on the address lookup table resolution, this | ||||||||||
constraint cannot easily be relaxed without also relaxing those | ||||||||||
constraints. | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't really understand why there is still a constraint for address lookup tables. If the lookup table is invalid for some reason can't validators just charge the fee payer some fees? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ALT resolution contributes to constraints at higher levels i.e. entry and block. Entries need to have non-conflicting transactions (without adoption of #83). In both these situations, if we allow ALTs to not resolve I think the behavior becomes very unclear. Do we only consider the static accounts? all resolvable accounts? no accounts? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see, thanks for explaining what the entry and block constraints are, I hadn't considered those. FWIW ALTs were designed to be apend only and the warmup time (currently one slot) is configurable by the runtime and can be increased in the future if it helps bankless leaders manage a cache of ALTs. So I think we could try to move forward with removing this constraint. |
||||||||||
|
||||||||||
## New Terminology | ||||||||||
|
||||||||||
None | ||||||||||
|
||||||||||
## Detailed Design | ||||||||||
|
||||||||||
Specifically, this proposal relaxes constraints that a transaction included in | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This list seems to mix "constraints on transactions which invalidate entries and blocks" and "constraints on transaction execution which invalidate the transaction". The list should be split up or the ones of the latter type should be removed. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's a very fair comment, as that is something is changing with this proposal. Currently, the list of constraints that invalidate the block is a superset of those which invalidate execution. That will no longer be the case, and I should make that more clear. |
||||||||||
a block must: | ||||||||||
|
||||||||||
1. Have a fee-payer with enough funds to pay fees | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This list seems to denote the current status of constraints on transactions. This is useful background, but we need to precisely specify the new constraints on transactions. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ... in a separate list. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Bit confused on what you're suggesting would make the document more clear here; this proposal does not add any new constraints. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The list describes what the constraints are now, but does not explicitly list what the relaxed constraints are. Consider that I or another FD dev needs to stare at this and implement the exact rules for what defines a valid transaction. |
||||||||||
2. Have a valid nonce account, if specified | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Did you consider that both normal transactions and durable nonce transactions can only be processed once to be valid? Validators and bankless leaders can use the status cache to avoid including normal transactions more than once, but they need to check that the nonce is valid or not to avoid including durable nonce transactions more than once. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Considered, but I need to rewrite sections of the proposal to be very explicit. Status cache should still be used to avoid including transactions multiple times. Duplicate inclusion of transactions is still a violation of constraints and leads to the entire block being marked as invalid. This does not depend on account state, which is why I think this should be left in. With respect to nonces, we want to avoid depdendence on account state, so cannot check the validity of the nonce account for the question of block validity. However, the nonce acount state must still be checked before execution. If the nonce account does not exist or is otherwise invalid, the transaction is not executed and fees are not charged to the fee-payer. There's an ongoing discussion of if fees should be paid by the block-producer should a transaction be unable to pay. So whether or not the producer gets charged fees in these cases is still an open question. |
||||||||||
3. Have program accounts that: | ||||||||||
1. exist | ||||||||||
2. are executable | ||||||||||
4. Have writable accounts that are: | ||||||||||
1. not executable, unless owned by | ||||||||||
`BPFLoaderUpgradeab1e11111111111111111111111` | ||||||||||
2. if owned by `BPFLoaderUpgradeab1e11111111111111111111111`, | ||||||||||
then `BPFLoaderUpgradeab1e11111111111111111111111` must be included | ||||||||||
3. if owned by `Stake11111111111111111111111111111111111111`, then the | ||||||||||
current slot must not be within the epoch stakes reward distribution period | ||||||||||
5. Have executable `BPFLoaderUpgradeab1e11111111111111111111111` owned accounts | ||||||||||
with `UpgradeableLoaderState::Program` state, and derived program data account | ||||||||||
that exist | ||||||||||
6. Have a call chain depth of 5 or less | ||||||||||
7. Have no builtin loader ownership chains | ||||||||||
(pending `4UDcAfQ6EcA6bdcadkeHpkarkhZGJ7Bpq7wTAiRMjkoi`) | ||||||||||
8. Have total loaded data size which does not exceed | ||||||||||
requested_loaded_accounts_data_size_limit | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you please add links to the feature documents. |
||||||||||
(pending `DdLwVYuvDz26JohmgSbA7mjpJFgX5zP2dkp8qsF2C33V`) | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Tried to be as specific as possible in the document itself, but these constraints (3-8) can be summarized as "transaction must be executable" There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. \(^_^)/ |
||||||||||
|
||||||||||
The intent with relaxing these constraints is to minimize the amount of | ||||||||||
account-state which is required in order to validate a block. This gives more | ||||||||||
flexibility to when and how account-state is updated during both block | ||||||||||
production and validation. | ||||||||||
|
||||||||||
With these constraints removed, there is still one account-state constraint | ||||||||||
at the transaction level: address lookup table resolution. | ||||||||||
With this proposal, this constraint is intentionally not relaxed since it is | ||||||||||
necessary for validation of entry-level and block-level constraints. | ||||||||||
However, if/when those constraints are relaxed, this constraint should be | ||||||||||
relaxed as well. | ||||||||||
|
||||||||||
The relaxation of these constraints is only relaxed for transactions being | ||||||||||
included in a block. During block validation, if any of these constraints | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How does this affect transaction simulation? Currently, we get an error when we simulate transactions where the payer does not have enough balance. This will make these kinds of errors indetectable during transaction simulation. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure what you mean. Transaction simulation is unrelated to block invalidation, and doesn't change with this proposal. |
||||||||||
are broken, the entire block is marked invalid; with this proposal, the | ||||||||||
block is not marked invalid, but the transaction will not be executed. | ||||||||||
These constraints must still be satisfied in order for the transaction to be | ||||||||||
executed, and have an effect on state. However, there are some new considerations | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe better off being explicit about side-effects on some on-chain states for these constraints-violating transactions. namely, the status cache to avoid repeated inclusion of same transactions. |
||||||||||
which must be taken into account, specifically as it relates to fees and block | ||||||||||
limits. | ||||||||||
|
||||||||||
### Fee-Paying | ||||||||||
|
||||||||||
Currently, iff a transaction's fee-payer does not have enough funds to pay the | ||||||||||
fee, the transaction cannot be included in a block. With this proposal, it is | ||||||||||
possible for such a transaction to be included in the block, and there are | ||||||||||
three different edge-cases to consider: | ||||||||||
|
||||||||||
1. The fee-payer account does not exist (0 lamports) | ||||||||||
2. The fee-payer account does not have enough funds for the entire fee | ||||||||||
3. The fee-payer account has enough funds for the fee, but would no longer be | ||||||||||
rent-exempt | ||||||||||
|
||||||||||
In case 1, the transaction should simply be ignored for execution, and have no | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should also consider that base fee will eventually change and could be much greater than now. |
||||||||||
effect on state. | ||||||||||
In case 2, the fee-paying account should be drained of all funds, but have no | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. probably a good exercise to also enumerate the normal case |
||||||||||
other effect on state. | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. With this, There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Might have missed a comment this is attached to. With what? Case 2 would probably not occur if it was a malicious actor, but could happen if some non-strict block-producer made and assumption a fee-payer was good and was not. |
||||||||||
In case 3, the fee-paying account should be drained of all funds, with fees | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That seems a bit harsh, no? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Potentially this is too harsh. Alternatively we could just drain them to the rent-exempt level. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Imho, since the commission is paid, the transaction should be executed. Deleting the account is a different event. |
||||||||||
being paid to the block-producer, and the remainder of lamports being dropped. | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||||||||||
|
||||||||||
apfitzge marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||
### Block-Limits | ||||||||||
|
||||||||||
Pending the activation of `2ry7ygxiYURULZCrypHhveanvP5tzZ4toRwVp89oCNSj`, | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please add a link. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. SIMD should stand for itself and should not have any links, as far as i know. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It at least needs to describe the feature. |
||||||||||
validators must validate a block is within block-limits. | ||||||||||
With this proposal, some transactions may not be executable, but will still | ||||||||||
count towards block-limits. | ||||||||||
If these transactions did not count towards block-limits, the validation of | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How do they specifically contribute to block limits. |
||||||||||
block-limits would require the validator to check whether or not a transaction | ||||||||||
is executable, which negates the benefits of this proposal. | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think the case here is "producer builds a block of 48M cu, but validator only spent less than 48M to execute all transactions (as some of TXs are failed and not executed)", and this should be OK (except the DoS vector - that entire block is unexecutable txs, which can be addressed by making producer pay for these unexecutable txs) |
||||||||||
|
||||||||||
Additionally, f these transactions did not count towards block-limits, a | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
malicious leader could produce a block with non-executable transactions and | ||||||||||
overload the network. | ||||||||||
|
||||||||||
## Impact | ||||||||||
|
||||||||||
- Transactions that would previously be dropped with an error, can now be | ||||||||||
included, and even charged fees. | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
I think "and even charged fees" doesn't sound declarative enough. My understanding is that the crux of this SIMD is that before we dropped txs for a variety of reasons and didn't charge fees and after we will allow invalid txs to be in a block and will charge fees for them. |
||||||||||
- Users must be more careful when constructing transactions to ensure they | ||||||||||
are executable if they don't want to waste fees | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. are there additional tools (or simulator enhancement) needed for transaction submitters ensure their transactions against those relaxed constraints? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Simulation doesn't change. It currently would tell you that your transaction had an error. That error doesn't change, it only changes what will happen to a block it may be included in. For example, if a transaction had an invalid program account, simulation will still return |
||||||||||
- The validity of a block is no longer dependent on intra-block account-state | ||||||||||
updates. This is because the only account-state required for transaction | ||||||||||
validation is the ALT resolution, which is resolved at the beginning of a slot. | ||||||||||
- Block-production could be done asynchronously | ||||||||||
- Block-validation could be done without execution, but still relies on the | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Can use some explanations/examples here. |
||||||||||
execution of previous blocks. | ||||||||||
|
||||||||||
## Security Considerations | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could any other relaxed constraints, in addition to relaxing "fee-payer requirement", lead to similar dos vector, if block producer build with bad transactions? Is it necessary to charge transactions failed due to those relaxed constraints by its transaction base fee? |
||||||||||
|
||||||||||
- Removing fee-paying requirement could allow validator clients to produce | ||||||||||
blocks that do not pay fees. If not checked by block-producer, this could | ||||||||||
allow malicious users to abuse such producers/leaders. | ||||||||||
|
||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. also, malicious leaders could create full of garbage blocks, which only contains bad transactions at no cost, burdening all the validations onto other validators, esp, heavy account state lookup. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. agreed with @ryoqun, this attack vector is the main thing that needs to be solved. Maybe this specific problem can be discussed in a separate SIMD since relaxing other transaction constraints is more straightforward. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. made a comment in this chain: #82 (comment) I don't think there's a DOS that's creating more work than would be possible within current protocol. This would change the vector, since they could have all txs that do not pay fees. But, that only makes the execution less intense, since they still have a cost that counts towards block limits. |
||||||||||
## Drawbacks | ||||||||||
|
||||||||||
- Any dynamic fees based on block utilization cannot/should not reward the | ||||||||||
block-producer directly. Otherwise, this incentivizes block-producers to fill | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Humbly not like the sound of a future proposal (eg Dynamic base fee) is already constrained by current design. But if bock producer cannot include no-paying transactions for free, say producers are liable for adding those bad transactions to block and have to pay for those, then we wouldn't have to worry about this drawback? |
||||||||||
blocks with non-fee paying transactions in order to raise fees. | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. imo, it is necessary to have someone to pay for ledger space occupied by those failed transactions (that do not update global state machine). There are several ideas, including have Producer pay. Should the discussion of how to do that be part of this SIMD? |
||||||||||
- Non-fee paying transactions included in a block will be recorded in long-term | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This could be a way to attack the cluster and use more space. A malicious leader could create 48M CU / (CU required to load payer account which currently stands at 0) number of transactions. This could be a massive number of transactions, depending on CUs charge that we decide to load the payer account. If each transaction was given the maximum size of 1280 bytes or eventually more if we include ZK transactions, then I can possibly create a huge block burdening the whole cluster's network resources because of the turbine. I feel like the leader should be somehow punished/slashed for adding transactions where the payer does not have enough balance. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Currently transactions do have a minimum cost of 1070 CUs = 720 SigVerify + 300 WriteLock. Additionally, cost has 1 CU per 4 bytes of instruction data. So large ix data txs would cost more than this bare minimum of 1070 CUs as well. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's worth noting that currently, all direct rewards are going to the block-producer anyway. There is still some discussion to be had around indirect rewards via the burning of SOL in those fees. I think that's potentially a valid reason we may want to force the block-producer into paying fees. Hoping to discuss this more There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah I agree, block producers should pay a fee as they are using network resources, storage space to transfer invalid transactions. Malicious node can just start creating blocks with invalid transactions of 1280 bytes using 44.20096 MBs/ block in network and storage. |
||||||||||
transaction history. Without any fees being paid, there is no way to reward | ||||||||||
long-term storage of these transactions. It is important to note, there is also | ||||||||||
currently no way to reward this storage. | ||||||||||
|
||||||||||
## Backwards Compatibility | ||||||||||
|
||||||||||
This proposal is backwards compatible with the current protocol, since it only | ||||||||||
relaxes constraints, and does not add any new constraints. All previously valid | ||||||||||
blocks would still be valid. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
to clarify, "relax" means "removal"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The constraints still exist in the sense we need to check "can this transaction be executed". They do not exist for the validity of the block.
Right now, those 2 different types of constraints are overlapping. I need to re-write this proposal with @lheeger-jump's suggestion on making it very explicit that these 2 things are different.