using keccak from constantine

[advaita-saha]

Bumps the nim-eth vendor to status-im/nim-eth#816
Essentially this replaces the keccak used in the trie, and by hashwriter to the constantine one

Preliminary small import result

ORG-run.csv vs keccak-run.csv
                       bps_x     bps_y      tps_x      tps_y  time_x  time_y   bpsd   tpsd   timed
block_number                                                                                      
(499713, 907245]    8,968.52  9,526.92  16,379.64  17,367.89     56s     53s  6.17%  6.17%  -5.64%
(907245, 1314778]   6,847.63  7,285.70  30,336.99  32,236.76    1m1s     57s  6.40%  6.40%  -5.96%
(1314778, 1722311]  5,510.92  5,708.39  36,262.85  37,502.07   1m21s   1m18s  3.49%  3.49%  -3.28%
(1722311, 2129844]  4,321.05  4,579.69  31,596.99  33,474.44   2m25s   2m20s  5.68%  5.68%  -5.34%
(2129844, 2537377]  2,910.40  3,065.92  22,426.82  23,609.88   14m7s  13m13s  6.18%  6.18%  -5.64%
(2537377, 2944910]  4,328.41  4,507.38  30,007.38  31,252.14   6m49s   6m42s  3.80%  3.80%  -3.62%
(2944910, 3352443]  4,119.18  4,282.30  34,362.03  35,754.07   1m41s   1m37s  4.06%  4.06%  -3.86%
(3352443, 3759976]  2,173.49  2,270.20  34,694.72  36,254.60   3m18s    3m9s  4.56%  4.56%  -4.34%
(3759976, 4167509]    627.38    653.73  30,560.81  31,844.98  11m51s  11m22s  4.25%  4.25%  -4.06%

blocks: 3659604, baseline: 43m32s, contender: 41m35s
Time (total): -1m57s, -4.50%

bpsd = blocks per sec diff (+), tpsd = txs per sec diff, timed = time to process diff (-)
+ = more is better, - = less is better

Note

• Using constantine, we can't do compile time hashing e.g

  nimbus-eth1/execution_chain/evm/evm_errors.nim

  Lines 79 to 83 in 1ea6f87

  	# revertSelector is a special function selector for revert reason unpacking
  	const revertSelector = keccak256(toBytes("Error(string)")).data[0..3]
  	# panicSelector is a special function selector for panic reason unpacking
  	const panicSelector = keccak256(toBytes("Panic(uint256)")).data[0..3]

[advaita-saha]

ORG-run-native-build.csv vs keccak-run.csv
                       bps_x     bps_y      tps_x     tps_y    time_x   time_y     bpsd     tpsd   timed
block_number                                                                                            
(499713, 1011095]   2,689.26  2,664.10   5,603.98  5,481.17     3m43s    3m47s   -0.64%   -0.64%   2.64%
(1011095, 1522477]  1,723.14  1,409.64   9,171.32  7,410.46      5m3s    6m21s  -18.26%  -18.26%  25.69%
(1522477, 2033859]  1,206.81    871.97   8,728.37  6,311.92     8m51s   12m26s  -27.38%  -27.38%  39.47%
(2033859, 2545241]    909.43    615.20   6,860.68  4,658.82    42m36s   56m29s  -29.67%  -29.67%  44.72%
(2545241, 3056624]  1,097.19    689.29   7,770.80  4,885.54    24m43s   35m37s  -35.43%  -35.43%  58.26%
(3056624, 3568006]    862.33    530.86   9,158.68  5,640.82    10m20s   16m39s  -37.45%  -37.45%  63.43%
(3568006, 4079388]    332.29    289.74  10,068.46  9,105.16    30m52s   33m59s   -9.58%   -9.58%  13.77%
(4079388, 4590770]    113.17    115.82   9,293.83  9,512.84  1h20m55s  1h19m9s    2.49%    2.49%  -1.98%
(4590770, 5102153]     49.46     49.72   7,162.18  7,200.59   3h6m13s  3h5m54s    0.53%    0.53%  -0.22%

blocks: 4594248, baseline: 6h33m20s, contender: 7h10m25s
Time (total): 37m5s, 9.43%

bpsd = blocks per sec diff (+), tpsd = txs per sec diff, timed = time to process diff (-)
+ = more is better, - = less is better

longer benchmarks show a ~10% slowdown @arnetheduck
Next steps would be to benchmark the individual keccak from constantine and nimcrypto

[advaita-saha]

https://hackmd.io/@advaita/Sy8nLf6hee

[bhartnett]

I performance tested this change using a clean state with no hashes at the block just after the merge and here are the results:

Master branch

INF 2025-10-06 17:04:21.464+08:00 Wrote computeKey cache                     keys=328002861 accounts=100.00%
INF 2025-10-06 17:04:21.577+08:00 Imported blocks                            blockNumber=15537395 slot=4700014 blocks=1 txs=80 mgas=29.98 bps=0.000 tps=0.008 mgps=0.003 avgBps=0.000 avgTps=0.008 avgMGps=0.003 elapsed=2h39m6s
NTC 2025-10-06 17:04:21.577+08:00 Import complete

This faster-hashes branch

INF 2025-10-06 20:02:31.045+08:00 Wrote computeKey cache                     keys=328002861 accounts=100.00%
INF 2025-10-06 20:02:31.160+08:00 Imported blocks                            blockNumber=15537395 slot=4700014 blocks=1 txs=80 mgas=29.98 bps=0.000 tps=0.008 mgps=0.003 avgBps=0.000 avgTps=0.008 avgMGps=0.003 elapsed=2h47m7s
NTC 2025-10-06 20:02:31.161+08:00 Import complete

Unfortunately there doesn't appear to be any significant speed up and in fact there is a small slow down.

This result is rather surprising because it doesn't align with the benchmark results in the link above.

[jangko]

Running eest_blockchain_test on my Ryzen 5 9600X:

• nimcrypto: 82.9s
• constantine 81.3s
• libxkcp x64 plain C: 73.0s
• libxkcp AVX2: 69.2s

a plain C library can easily beat constantine, and to put into consideration, libxkcp is not even tuned for keccak256, there is still room for improvement.

I think we should reconsider using constantine to improve nimbus EL performance, because now we have proved it at least 3 cases where other library can outperform constantine: mcl have faster bn254, libxkcp have faster keccak, and libtommath although not heavily optimized still have faster modexp.

I think what we need is a collection of specialized small library for nimbus-EL needs e.g. a keccak256, and not generic one supporting 384 or 512 bit output.

Same things goes to u256 library, we need an optimized and specialized 256 bit integer library instead of a generic multi precision like stint.

And for EVM precompiles, we often only need parts or few functions instead of a full blown crypto library.

Constantine itself does have numerous optimization, for example the modexp. Constantine does using latest algorithm such as montgomery multiplication, sliding window technique, chinese remainder theorem, separation of power of two modulus, etc. But they are high level optimization. While the lower level machine word inefficiencies make the high level optimized algorithm cannot shines.

[advaita-saha]

@jangko fully agreed, I have been reconsidering the same for the same reasons we faced for the last few months
This PR I kept open only because I was unable to find the reason for slow down in longer runs

Which also for the past week, seemed like a low priority stuff