Filecoin Onboarding Gas Optimisation Research Summary

[rvagg]

Note

This document is not a formal FIP, but rather a synthesis of research conducted after December 2024 when high network base fee highlighted limitations in data onboarding rates. It summarises the various approaches explored to decouple onboarding from gas limitations before the decision to implement FIP-0100 as an initial solution. Although many of the items listed below would ultimately result in quite small gas improvements, this research continues to be relevant for future optimisation efforts.

Background

The Filecoin network's scalability and economic efficiency are fundamentally constrained by block gas limits, which cap the amount of computation and state changes that can be processed in each block. As the network has grown, data onboarding rates have increasingly approached these gas limits, creating bottlenecks that restrict network growth. FIP-0100 addressed a major component of this by removing the batch balancer (which previously disincentivised proof aggregation) and replacing it with a daily per-sector fee. This change lifted the theoretical maximum onboarding rate by enabling miners to always use the most gas-efficient onboarding methods. However, even with these improvements, gas optimisation remains critical for Filecoin's long-term scalability. The research summarised below represents an exploration of additional approaches to further reduce gas costs associated with data onboarding. Many of these items have negative impacts that make them unsuitable to pursue and most will result in quite small gas optimisations if pursued individually.

State Storage Optimisations

1. Compact Deadlines Block

The Deadlines block is wasteful for miners that use fewer deadlines. It's an array of 48 CIDs. Each sector insert (or update?) mutates this block by altering the CID for the deadline impacted.

The waste comes from unused deadlines, they still get a CID, the empty-deadline CID of bafy2bzacedh5oro2npo2wzxvbaemt5zmibium7gmiwavbtn3rfjztovxv7hpc. The whole block is 2,067 bytes and costs 7.4M gas to mutate each time. For miners with few occupied deadlines, this is mostly repeating the empty-deadline CID many times.

• Impact: Analysis shows an avg saving across all active miners of 3.3M gas (max 6.7M, min -20k)
• Status: Ready for implementation
• Difficulty: Easy
• Note: This change will benefit smaller miners more and may add a small incentive to run multiple, small miner actors. Miner actors using all 48 deadlines will pay the full amount—this alone may be a good reason to not pursue this change

2. Externalise SectorOnChainInfos from AMT

In the order of 50% saving for just updating / inserting sector info: ~11M gas for typical block with many inlined SectorOnChainInfos; if we externalise it the block just has CIDs and would be ~4.5M gas to update and the new external SectorOnChainInfo block would be ~833k gas (or 1M gas for a sector that's been snapped).

Saving would apply (mostly) linearly for single and batched commits and updates.

• Challenge: Migration for this change would be very expensive as it would need to touch every single sector in state! An experimental migration was run that even had difficulty completing 30% due to inefficiencies in the tooling implementations used to read and write various state data structures. Additionally, rewriting all sectors is also going to impose massive state churn costs, which mostly fall on archival node operators, which are already challenged with resource consumption of their nodes.
• Status: Explored but migration too slow to complete currently

3. Auto-compact Sparse AMT (Optimise ExpirationSet storage)

ExpirationSet, stored under a Partition, which is under a Deadline, in a miner actor, is stored in an AMT and is indexed by epoch number. AMTs become larger (taller: more blocks, each block costing between 670k and 2M gas to mutate/add, recent epochs require a height of 9 or 10). Each sector requires mutating this AMT to add/edit an ExpirationSet, costing in the order of 20M gas just for a single sector, including the parent Deadline block.

Batched inserts/updates get a large cost aggregation benefit here, regardless of how we represent it.

• Possible solutions:
  • Representing epoch as an offset from some number (perhaps the lowest epoch—and if the lowest is removed from the set, they all get updated), so that the indexes are all small.
  • Implement collapsing proposal discussed in Collapse sparse AMTs go-amt-ipld#17
  • Use the KAMT?
  • Some new data structure that can do this better: it may even be cheaper to maintain an ordered list in a vector/sharray and just update the whole thing when editing/modifying values in the middle.

4. Make Potential Empty-HAMT References Nullable

PreCommittedSectors in the miner actor; when there are no pre-committed sectors for a miner, this field contains a CID that points to the empty HAMT bafy2bzaceamp42wmmgr2g2ymg46euououzfyck7szknvfacqscohrvaikwfay. When emptying this list and setting it to the empty HAMT, the miner pays the gas cost of storing the empty HAMT (even though it's 99.9999% guaranteed to always be used elsewhere on chain already, so there's no actual write going on). This 3 byte block (824080) write costs 334000+(3340*3)+172000= 516,020 gas.
The Miner state root goes from 324 bytes, 1,588,160 gas to 286 bytes, 1,461,240 gas = 126,920 saving.
We ultimately save 807,010 gas in the miner actor by doing this, roughly 1 to 1.5% of the miner actor cost of executing a simple PCS3.

• Impact: Minimal saving (about 800k gas) but affects a small subset of miners
• Status: May be deferred for a "clear dead miner" task

5. Clean-up Power Table

Add & remove miner from power table when it get adds/remove to cron queue. There are 600k actors in the table which increases the cost of modifying it.

Whenever you register or deregister in cron, you send a message to the power actor to enable a power claim or remove one. If you're not enrolled in cron, you shouldn't have a claim.

• Impact: Currently traced as very low impact
• Benefit: Side benefit of making F3 more efficient (Fetching committee is slow on mainnet lotus#13097)

Gas Pricing and Benchmarking

6. Adjust Gas Pricelist Based on Modern Hardware Benchmarks

Would impact compute gas, not storage gas, but probably worth doing at some point soon anyway.

State Cleanup and Maintenance

7. Clean-up Expired Allocations and Claims in Verifreg

This became an optional operation since it was moved out of cron, clients can choose to pay gas to clean up their own expired allocations and miners can choose to pay gas to clean up their own expired claims. Anyone can also pay to clean up all allocations and all claims. An entity such as the Filecoin Foundation could regularly pay to submit such messages to perform cleanup.

It is currently unclear how much saving this would make on gas. Currently a single sector touching verifreg makes ~10 block writes, costing ~52.5M gas (batching benefits somewhat from overlapping writes). If the HAMTs were more compact, this number would decrease on average.

8. Incentivise or Force Miner Actor Sector Compaction

Make sure SP has an incentive to compact sectors and remove dead sector info from the chain

• Proposal: you need to clean dead sectors to get pledge back
  • Switch to check no precommit & no terminated sectors
• Difficulty: Needs SP software update, compacting sectors is a difficult operation for SP software because their internal state understanding sector layout becomes mismatched with the chain and they need to update immediately after a compaction execution

Protocol Changes

9. Encourage Non-Interactive Proof of Replication (NI-PoRep)

This is marked as having a major gas impact but is hard to incentivise and there are some technical hurdles to making this easy to adopt in the current SP software stack and workflow (e.g. Adding support to Lotus Miner for either producing NI-PoRep sectors or consuming them from a third-party, is nontrivial). An ideal future might involve NI-PoRep for committing sectors and Snaps for loading data into them.

10. Encourage DDO (non-f05) Onboarding

Depends on ecosystem tooling:

• Market software (Boost) makes DDO hard
• Curio hopefully makes it easier
• Spark v1 still using f05 labels (NOTE: this has since been addressed)
• Summary of what it takes for "SPs that want to use DDO deals and get Spark to test the retrievability of their new DDO deals must meet the following requirements" Onboarding rate and gas optimizations #1092 (comment)

Protocol:

• No alternative on-chain market (limitation of only being able to notify f05 still applies to PCS3 and PRU3)

Tools and Education

11. Tool to Educate Miners on Gas Saving Options

Turn gas modelling code into a tool miners can run for themselves on their own data. Answer questions like:

• How much could I save by batching precommits, commits, PSD (for various batch sizes)

• Batching vs aggregation savings (for various batch sizes)

• How much would I save if I removed expired verifreg claims

• How much would I save if I compacted my partitions

• Status: Short exploratory sprint: harder than expected but could push further

Radical Approach: Special-Casing the Miner Actor

12. Special-Case the Miner Actor for Storage Gas

A more fundamental redesign of how gas works for the miner actor specifically:

• Current situation: The miner actor has many options for how it structures state (compaction strategies, sector numbering schemes, etc.)
• Problem: This optionality means some miners can optimise state usage more than others, making storage gas charges appropriate to incentivise efficiency
• Proposal: If we eliminated all optionality in the miner actor such that there is just one way for a miner (given a specific list of sectors and configuration) to store its state, then storage gas incentives would become unnecessary
• Implementation: Special-case the miner actor to have reduced or zero storage gas costs, since there would be no way for miners to be inefficient with state
• Example prerequisites:
  • Perform cleanup on the miner actor to enforce optimal state organisation
  • Remove or greatly reduce optionality in data layout
  • Implement automatic compaction of partitions
  • Enforce sequential sector numbering or other optimal schemes?
  • Auto-clean expired sectors
• Benefits:
  • Lower and more predictable gas costs for miners
  • Simpler reasoning about state size and optimisation
  • Fairer playing field between sophisticated and novice miners

This approach represents a significant shift in philosophy from "use gas to incentivise efficient state usage" to "enforce efficient state usage through the protocol itself."

Already Implemented in FIP-0100

For reference, FIP-0100 implemented these key changes:

1. Removed the batch balancer fee from PreCommit and ProveCommit operations
2. Introduced a new daily per-sector fee based on quality-adjusted power
3. Removed unnecessary protocol constraints that previously limited batch sizes:
   • PRE_COMMIT_SECTOR_BATCH_MAX_SIZE: The maximum number of sector pre-commitments in a single batch.
   • PROVE_REPLICA_UPDATES_MAX_SIZE: The maximum number of sector replica updates in a single batch.
   • DECLARATIONS_MAX: Maximum number of unique "declarations" in batch operations (i.e. terminations, faults, recoveries).