Increase Transaction Size

[jacobcreech]

Summary

Currently transaction size is capped at 1280 bytes, or 1232 bytes available for developers to use while building their applications. This is limiting for a bunch of usecases, some of them being the following:

• Transferring with fees on a token with confidential balances enabled requires 3x the size
• Untruncated Winternitz signatures can be accomplished with 37 more bytes than what is currently provided
• More routes can be added to trades using markets such as Jupiter

The current transaction cap was originally limited by the maximum IPv6 MTU size, but that is not longer a restriction. Increasing this cap would both enable new usecases like above for developers, as well as allow potential apps that were previous limited to be built.

What needs to be discussed/researched?

1. Is this a change that is reasonable to make to the Solana network?
2. What cost model should be applied to transaction size increases?

In SIMD-108, instead of having a separate fee, it was decided to pull the transaction size increase into the CU model.

3. What impact does a transaction increase have on cluster performance?

Very old research, but when testing average increased transaction sizes, it was found that as the transaction size went up, max TPS on the cluster went down exponentially. This test is likely not valid anymore and would need a retest before any consideration of writing a SIMD.

4. What should be the new max transaction size? Should we test a smaller increase followed by a later larger increase like is potentially being done with CU?

5. What is the effort to include this change?

[deanmlittle]

Honestly, the most useful way to think about this is comparing it to Jito, and the biggest thing to consider in terms of impact on the network is probably how it would affect Jito bundles, and downstream effects on products and libraries.

1. Yes. It unlocks a lot of functionality, and the lack of larger data creates a lot of avoidable inefficiencies around single-use data and atomicity. The reasonably widespread use of Jito bundles for activities other than sandwiching, like overcoming ix data limits for ZK, shows there is demand. It would generally be much more efficient to avoid the pattern of: open account -> accumulate single-use data -> use data in subsequent transaction -> close account.

2. Consider that a Jito bundle is basically a less efficient version of 5xTx, as in order to conform to the protocol, it is forced to do 5x the signature verifications with a 48-byte header overhead on each data packet, and at least one 64-byte signature and 32-byte pubkey for the fee payer of each transaction. To charge for bigger data, why not simply keep the same base cost of 5000 lamports per signature and add an additional 10 lamports per round of SHA512 after the initial 1-3 already included in the existing fee? One verification is about ~500x more expensive than a round of SHA512, so it is easy to extrapolate incremental costs (savings, in fact) to verification in this way. Don't add any additional compute/data/whatever budget instructions for data size. Existing priority fees and Jito tips already handle fighting over CUs pretty well.

3. Jito has proven that pseudo-5xTx is stable over a long period of time. If anything, as the majority of nodes are running the Jito client, the biggest question is probably "how are bundles impacted by this change?" A lot can happen in a 5x3xTx bundle, especially if compute limits are also lifted dramatically at the same time, however data and compute should really be addressed separately.

4. 2-3x is what's required for most useful cryptography. Jito has demonstrated stable 5x bundles for a long time. I don't see why 5x would not already be a reasonable baseline.

5. Implementation in Agave is straightforward. Jito protocol is a question mark. There may be some downstream effects on libraries or products that are built on the assumption of a 1232 (1280) byte limit, or the idea that 1 data packet = 1 TX.

[buffalu]

the biggest shortcoming about bundles, when compared to larger txs, is their lack of protocol-enforced sequencing and atomicity. i am in support of larger txs, the main question i have is impact on performance + how the network will handle getting them to the validator.

[oct-sheldon]

1372 is common in many enterprises using mesh VPN, MPLS and variable other solutions in a limited environment with unknown 3rd parties. You can't always know what header space you'll need, so even with a year of SIMD in-view its difficult to anticipate precisely to the byte what is desirable vs needed.

I assume it's out of scope for this discussion, but jumbo frames (9000) in a modern cluster for local traffic and anything below 1372 MTU for the public interface should be part of spec/testing. If it's not, I hope somebody is ready to dump/capture the cluster traffic while testing this to avoid scope-limited visibility to stack stability issues.

1. Yes, Solana node operators should be capable of an MTU change and to anticipate fragmentation and bus bottlenecks on a node

2. Costs should have a dynamic min-max target, most easily led by XOR in the prioritization fee as currently active in memory. Rent could be assessed to anyone wanting routine priority by declaring that higher priority per account, effectively a personal gas station to sweeten a tx offered (a later tx could be designed to pay prioritization via rent but that should be refined in a framework of block height as time). I'm clueless about implementation here except to suggest that the change should not fetch/require remote data, meaning that node operators would have a "cool kid" list, which should probably have a high floor. Significantly busy accounts with logic would be the target consumer base, not regular human consumers. Maybe min floor is 1 SOL 🤷‍♂️

3. Tx size increase should be expected to reduce total throughput in bytes. Multiple conditions should be tested/toggled, including SFP NICs, jumbo frames in the cluster inside interfaces, Intel Optane and AMD Storemi (and without as control) and upstream packet fragmentation at common global numbers like 1492, or at least log MSS for PMTU discovery on tested nodes' paths to a public core network

4. The risk of issues for smaller, less competitive operators is to lose throughput significantly. If the loss can be minimized/optimized below 15% I think it's worth the trouble to try a truer-to-max size than doing incremental changes. I assume multiple values would test in the 1300 range

5. Change MTU size, test again, rotate conditions/hardware, test again. Repeat. I would not reccomend a larger change until a framework exists for utilizing rent as parked capital for priority tx. Bus bandwidth in the tested nodes should be scrutinized and logged

[y2kappa]

Atomic liquidations or one-click DeFi operations would benefit a lot from this - things where you have one token but the operation requires a different token input, swap + deposit (single-sided liquidity deposit), swap + leverage (sol long with usdc deposits), flash loan + swap + deposit (one click leverage or atomic liquidation).

Usually it works but requires extremely tight lookup table management with very low margin of error, there are usually 100 bytes left and therefore 3 unknown accounts that are not part of preexisting lookup tables, growing this would allow a higher margin of error and better UX and fewer errors.

[MaxResnick]

If we increase the max size substantially it probably makes sense to have a seperate fee market for transaction size.

[deanmlittle]

Is it really necessary to add a fee market to a feature that reduces overhead and thus overall resource consumption? Don't get me wrong, we should charge for it as I outlined above, but a validator can already see the size of your transaction and what you're paying as a priority fee at ingress. It should be up to their own personal sequencer how it wishes to prioritize compute vs bandwidth.

[t-nelson]

if we don't make the users pay in lamports, developers will pay in tears. under-priced resources will always be used frivolously

[deanmlittle]

We must charge for it, but I don't think it needs its own separate fee market. From a transaction processing perspective, lamports are lamports and resources are resources. If a validator can guess how much X,Y,Z they will use by executing your tx upfront, the math regarding how they value their resources is subjective and should be up to them.

[jumpsiegel]

Right now a txn fits into a shred which maps cleanly onto a UDP packet or a single TCP packet inside quic. Would we change this so txn's span multiple shreds? What would we do signature verification on? If we increase the size of a txn, we create message fragmentation on the network. I agree completely there are many benifits to increasing the txn size but there are huge costs at the network layer we would need to address first

[t-nelson]

with the tpu using quic now, mtu is mostly irrelevant for solana application level packet sizes. if we're changing the tx limit, it should be bound by memory preallocation parameters. something like 4kB (one memory page) would make sense

the critical part of the discussion is if/how economics should change to account for additional load downstream

tl;dr: this is NOT a simple constant change