Performance optimizations for Cardano transactions signature/proof

[jpraynaud]

Why

Now that we are signing transactions on a Mithril test network for the Cardano mainnet, we want to identify bottlenecks and prepare optimizations for the signature/proving of the Cardano transactions. This process will be iterative.

What

Here are the areas that we need to optimize:

• Minimum impact of the activation of the new type of data signed for Cardano transactions on the release-mainnet network:
  • Avoid blocking the production of certificates for current signed entity types
  • Implement warm-up strategy for new comers in the network that takes advantage of the 2 first epochs downtime
  • Proofs generation does not impact the production of certificates for the network
• Minimum footprint on the signer infrastructure (disk space, memory, CPU requirements):
  • Assess the maximum disk space required by the Cardano transactions store on the signer once pruning is activated (in GB)
  • Make sure this limit is not exceeded during warm-up phase
  • Assess the memory and CPU consumption of the signer during import/signature of the transactions (maybe offload memstores of Merkle trees on disk for the signature on the signer? this would avoid too much memory consumption)
  • Update the current Mithril signer footprint documentation
• Maximum throughput of proofs served by the aggregator REST API:
  • Assess/implement optimizations needed to enhance the proof serving throughput (step by step, iterative, most efficient optimizations at first)
  • Model the expected traffic coming from a third party provider serving transactions by address for e.g. a light wallet:
    • Computation based on their average/peak traffic
    • Model proofs generation throughput (and number of transactions to prove per proof)
    • Model certificate download throughput
  • Caching strategy for transaction proofs and certificate on the client for optimal pressure on the aggregator
  • Determine maximum number of transactions allowed when creating a proof
  • Horizontal scaling strategy in worst case scenario where maximum throughput of an aggregator is not enough (e.g. aggregator 'slaves' for serving proofs)

How

Minimum impact when activated on release-mainnet:

• Impact analysis
• Design throttle proofs generation strategy
• Design warm-up strategy

Maximum proofs generation throughput

• Assess current performances of the aggregator proof route Prototype optimizations for increasing Cardano transactions proof generation throughput #1687
• Prototype first round of optimizations for increasing throughput Prototype optimizations for increasing Cardano transactions proof generation throughput #1687
• Implement optimizations:
  • Resource pooling for block range Merkle map Implement Resource Pooling for Block Range Merkle maps #1698
  • Merkle map proof computation optimization Computation of Merkle proof has bottleneck with multiple transactions #1730 and Cardano transactions prover performances drop with more than 5 transactions #1722
  • Database connection pooling Implement database connection pooling for Cardano transaction repository #1760
  • Parallelize sub Merkle trees creation and Merkle map proof computation Optimize Cardano transaction prover performances with parallelization #1756
  • Pre-compute sub Merkle proofs and use them in the block range Merkle map?
  • Cache last N certified transactions (database records, ...) (Later if needed)
• Implement boundaries on request payload sent to prover route Limit Cardano transactions prover input #1757
• Design and implement a stress test environment

Minimum SPO infrastructure footprint

• Optimize pruning/warmup on the signer for optimal disk space usage Warmup import Cardano transactions at node startup #1692 Import Cardano transactions by sequences of block ranges #1766
• Benchmark signer footprint on SPO infrastructure #1826

Infrastructure optimizations

• Plan infrastructure upgrade Slow import of Cardano transactions in testing-mainnet #1849 Release 2430 distribution #1830

Later

• Further Merkle tree optimizations: multiple storage (Sync, non Sync, low memory)?
• Design and implement client caching strategy