Weighted tasking to cover recent deals and all miners

[bajtos]

At LabWeek'24, we were recommended to tweak the tasking algorithm (deal sampling) so that:

• New deals get tested soon after they are announced on the chain.
• All miners have a reasonably recent Spark RSR, i.e. each miner must be tested frequently enough.

A possible solution is to move away from the current uniform sampling and implement something like the following (for example):

1. Pick 1/3 of tasks by uniformly sampling deals made in the last 6 hours (assuming we already have a real-time view of all storage deals + fast finality) or the last week (in Spark v1 with manual deal ingestion)
2. Pick per-miner tasks as follows:
   • Build a list of active miners not included in the tasks defined in the first step.
   • For each miner in that list, pick one of miner's deals and add it to the tasks.
   • If the list of tasks is larger than 1/3 of tasks-per-this-round, the trim the list.
3. Pick the remaining tasks by uniformly sampling all deals older than 6 hours (or 1 week, depending on the parameter used in the first step).

[juliangruber]

This is also a topic for CEL to look at

[bajtos]

Cross-posting from filecoin-project/service-classes#6 (comment)

• Currently we are checking weighted on data stored. This means small-storing SPs won't get checked.
  • @Stebalien had the idea of blending a weighting of how much data an SP has and the time since have last checked an SP.
• Idea of making it a two round thing
• Idea of having some known failures cases where you expect nodes to fail as a way to catch "dishonest actors"