v0.29.6

Mainnet 21

This release includes the following major changes,

1. Active pacemaker - The Flow's consensus has been updated to the Jolteon protocol, a substantially improved HotStuff derivative.
2. BFT change to support Permissionless AN.
3. Fail-fast transactions with insufficient balance to cover fees.
4. Update to Cadence v0.31.2

💥 Breaking Changes

• A new field LastViewTC has been added to the Block header. This new field is also included in the block hash calculation (#2367)

Active Pacemaker

• [Consensus] Active Pacemaker, SafetyData interface changes by @durkmurder in #2153
• [Consensus] Active Pacemaker, TimeoutAggregator interface changes by @durkmurder in #2155
• [Consensus] Timeout objects cache by @durkmurder in #2184
• [Consensus] Active Pacemaker, PaceMaker interface changes by @durkmurder in #2172
• [Consensus] Active Pacemaker multi message BLS aggregation by @durkmurder in #2243
• [Consensus] Support block-scoped and epoch-scoped identity queries in HotStuff committee by @jordanschalm in #2328
• [Consensus] Active Pacemaker signer indices by @durkmurder in #2494
• [Consensus] Active Pacemaker SafetyRules implementation by @durkmurder with contributions from @AlexHentschel in #2444
• [Consensus] SafetyRules extra test cases by @durkmurder in #2557
• [Consensus] Active Pacemaker: entities validation by @durkmurder in #2456
• [Consensus] Add method for timeout weight threshold to Committee by @jordanschalm in #2438
• [Consensus] Add Safety Threshold for EECC by @jordanschalm in #2498
• [Consensus] ActivePaceMaker implementation by @durkmurder in #2564
• [Consensus] Active Pacemaker: Timeout Collector implementation by @durkmurder in #2620
• [Consensus] Active Pacemaker: Forks cleanup by @durkmurder with contributions from @AlexHentschel in #2685
• [Consensus] Trigger Epoch Fallback Based on Epoch Commitment Deadline by @jordanschalm in #2625
• [Consensus] Active Pacemaker: TimeoutAggregator implementation by @durkmurder in #2740
• [Alternative Implementation] Update Committee for changed epoch fallback trigger condition by @jordanschalm in #2782
• [Consensus] Replace DecreasingPruningHeightError with BelowPrunedThresholdError by @durkmurder in #2754
• [Consensus] EventHandler implementation and consensus modules integration by @durkmurder in #2787
• [Consensus] EventLoop internal queues update by @durkmurder in #2883
• [Consensus] Update HotStuff Finalization Logic by @jordanschalm with contributions from @AlexHentschel in #2954
• [Consensus] Update disabled integration tests by @durkmurder in #2935
• [Consensus] Modify EpochLookup to handle epoch fallback trigger condition by @jordanschalm in #2927
• [Consensus] Consensus Compliance Engine updated to conform to Component interface by @jordanschalm with contributions from @AlexHentschel in #3121
• [Consensus] New logic for round duration calculation by @durkmurder in #3201
• [Consensus] Collectioncompliance, epochmgr Engines Implement component.Component by @jordanschalm in #3248
• [Consensus] Timeout objects rebroadcast by @durkmurder in #3277
• [Consensus] Update HotStuff Reader CLI Tool by @jordanschalm in #3363
• [Consensus] Front-Load HotStuff Verification Pt. 1: Update Compliance Engines by @jordanschalm with contributions from @AlexHentschel in #3294
• [Consensus] Front-Load HotStuff Verification Pt. 2: Remove MarkValid API by @jordanschalm in #3303
• [Consensus] Handle unsealed invalid service events by @jordanschalm in #2969
• [Consensus] MessageHub implementation by @durkmurder in #3357
• [Consensus] Add ParentView to flow.Header by @durkmurder in #3372
• [Consensus] Timeout rebroadcast de-duplication by @durkmurder in #3430
• [Consensus] Own vote & timeout message routing by @durkmurder and @AlexHentschel in #3429
• [Consensus] Follower engine message queue by @durkmurder in #3479
• [Consensus] Extract interface for compliance engine by @durkmurder in #3512
• [Consensus] Standalone startup of hotstuff participant by @durkmurder in #3513
• [Consensus] Update telemetry for EventHandler by @durkmurder in #3571
• [Consensus] Implementation of missing consensus metrics by @durkmurder in #3623
• [Consensus] TC aggregation multi-message verification fix by @durkmurder in #3698
• [Consensus] Fix bug in cluster block validation. by @jordanschalm in #3707
• [Consensus] Fix name ordering in metrics by @jordanschalm in #3710
• [Consensus] Re-enable AN test, increase test timeout, add logging, document outstanding issues by @jordanschalm in #3631
• [Consensus] Fix sync engine bug, improve observability by @jordanschalm in #3714
• [Consensus] Block header change by @durkmurder in #2367
• [Consensus] Vote/Timeout aggregation telemetry events by @durkmurder in #3763
• [Consensus] Extending sealing segment with extra information by @durkmurder in #3764

Execution node

• [FVM] SetProgram can be called with nil by @janezpodhostnik in #3808
• code gen parseRestrict wrappers by @pattyshack in #3398
• [Execution] remove duplicate blobservice on execution nodes by @peterargue in #3418
• Handle event encoding error gracefully (Port of 2562) by @m4ksio in #2565
• [Execution] Adding more metrics to the execution node by @ramtinms in #3380
• [Execution] Do not depend on string conversion in register set tracking by @SaveTheRbtz in #3529
• [Execution] Minor ingestion logging cleanup by @SaveTheRbtz in #3522
• [Execution] Randomize collection fetch order by @SaveTheRbtz in #3445
• [Execution] Use generic slice algorithms instead of interfaces by @SaveTheRbtz in #3542
• [Execution] Remove useless condition from builder by @SaveTheRbtz in #3659
• [Execution] Kill pagecache mass eviction code by @SaveTheRbtz in #3662
• [Execution] Break blobservice out to own component in en builder by @peterargue in #3613
• [Ledger] Upgrade WAL to support fdatasync(2) by @SaveTheRbtz in #3693
• [FVM] Log fees error by @janezpodhostnik in #3392
• [FVM][Execution] Adding metrics for computation effort vector by @Tonix517 in #3403
• [FVM] Change payer balance check error to a failure by @janezpodhostnik in #3679

Execution node refactoring - Prerequisite for concurrent transaction execution

• Refactor transaction invoker. by @pattyshack in #3423
• return TotalComputationUsed() as uint64 by @pattyshack in #3394
• Add Pause/Resume functionality to TransactionState by @pattyshack in #3435
• Tighten unexpected nested txn check + delay procedure result population by @pattyshack in #3437
• Switch parseRestricted to use span name instead of hardcoded strings by @pattyshack in #3443
• Clean up fvm mocks by @pattyshack in #3450
• Test ErrorsCollector against SplitError by @pattyshack in #3440
• Disable limits for all of tx seq num checker / tx verifier by @pattyshack in #3474...

v0.28.6

Mainnet 20

This release includes the following major changes,

1. Laying the ground work for Permissionless AN with several upgrades to the underlying peer-to-peer networking layer to make it more Byzantine fault tolerant.
2. A fully revamped execution node checkpointing process (v6) that is six time faster, eliminates execution node memory spikes and reduces spork downtime.
3. Rolling upgrade support for Execution and Verification nodes, working towards zero-downtime network upgrades in the future.
4. FVM performance improvements.

💥 Breaking Changes

Cadence type checking of resource has been fixed and significantly improved. Correct Cadence code should continue to function as before. However, any Cadence code that relied on implementation bugs must be corrected (onflow/cadence#2033)

Execution node

• [Exec] Add logging to collection fetch and handling by @SaveTheRbtz in #3316
• [Execution] Adding more logging and a fix to block upload retry by @Tonix517 in #3315
• [Execution] Fix TestScriptStorageMutationsDiscarded timeout by @SaveTheRbtz in #3109
• [Execution] Fixing improper use of WaitGroup in a goroutine for upload retry by @Tonix517 in #3285
• [Execution] Return error when closing gcp connection fails by @peterargue in #3290
• [Execution] Worker-based Chunk Data Pack Requests by @yhassanzadeh13 in #2951
• [Execution] state migration clean up by @ramtinms in #3192
• [EN Performance] Optimize checkpoint serialization for -37GB operational RAM, -2.7 minutes duration, -19.6 million allocs (50% fewer allocs) by @fxamacker in #3050
• [EN Performance] Reduce memory used for ledger.Payload by 32+ GB, eliminate 1+ billion allocs/op, speedup various ops by @fxamacker in #2930
• [Execution] Backport "Optimize GetEventsByBlockIDs" to master by @zhangchiqing in #3211
• [Execution] Fix error logging in script execution - port to master by @janezpodhostnik in #3071
• Refactor execution node builder (part 1) by @pattyshack in #3130
• Block computation result upload Retry implementation by @Tonix517 in https://github.com/onflow/flow-go/pull/
• Execution data format update by @smnzhu in #2841
• [Execution] remove duplicate blobservice on execution nodes by @peterargue in #3418

Sync engine

• [SyncEngine] not request pending ancestors by @zhangchiqing in #2879
• [ExecutoinStateSyn] Add bitswap metrics by @smnzhu in #2932
• Implement rate limite blob service by @smnzhu in #2859

Crypto

• [Crypto] DKG robustness against invalid messages by @tarakby in #2970
• [Crypto] Reformat using go 1.19 fmt by @SaveTheRbtz in #3141
• [Crypto] new sentinel errors and doc improvement by @tarakby in #2861
• [Crypto] update Go dependencies by @tarakby in #3085

Access and Observer node

• [Access] Option to Disable Conn Pooling by @lolpuddle in #2836
• [Access] Fix race condition in AN execution receipt metrics collection by @peterargue in #2957
• [Access] Added rpc metrics flag to Observer Node Builder by @danielholmes839 in #2973
• Observer metrics and logs using apiproxy by @danielholmes839 in #2920
• Crash Access Node if disk is full by @koko1123 in #1956
• Fix observer integration ports by @danielholmes839 in #2886

Performance benchmarking

• [Benchmark] Ability to dynamically adjust TPS by @SaveTheRbtz in #3246
• [Benchmark] Cleanup logging by @SaveTheRbtz in #3293
• [Benchmark] Expose execution rate from the loader by @SaveTheRbtz in #3080
• [Benchmark] Faster stopping of workers by @SaveTheRbtz in #3079
• [Benchmark] Fix crash on stats pusher exit by @SaveTheRbtz in #3089
• [Benchmark] Follow sealed blocks by @SaveTheRbtz in #3074
• [Benchmark] Group parameters to the loader library by @SaveTheRbtz in #3091
• [Benchmark] Keep prometheus metrics in-sync with TPS by @SaveTheRbtz in #3340
• [Benchmark] Remove noisy benchmarks by @SaveTheRbtz in #3090
• [Benchmark] Remove race in Digest() print by @SaveTheRbtz in #3076
• [Benchmark] Simplify follower inner loop by @SaveTheRbtz in #3078
• [Benchmark] Split TPS flags into initial and max by @SaveTheRbtz in #3292
• [Benchmark] upload data to BigQuery directly by @SaveTheRbtz in #3241
• [Performance] add fast path to programs cleanup by @SaveTheRbtz in #2982
• [Performance] initialize profiler after the state init by @SaveTheRbtz in #2985
• [Performance] run computation benchmarks on commit by @SaveTheRbtz in #2983
• [Benchmark] Add ability to auto-detect maximum TPS (#3338) @SaveTheRbtz
• Rewrite benchmarks for storing/loading checkpoint by @fxamacker in #3377

Cadence and FVM Performance improvement

• [FVM] Add expected event count to fuzzer by @janezpodhostnik in #3131
• [FVM] Add interaction limit to FVM fuzzing by @janezpodhostnik in #3114
• [FVM] Adding storage metering to Meter by @Tonix517 in #3236
• [FVM] Basic fuzzing and minor test change by @janezpodhostnik in #3112
• [FVM] Cleanup fvm logging by @janezpodhostnik in #3117
• [FVM] Disable limits on final state merge by @janezpodhostnik in #3182
• [FVM] Error Cleanup by @janezpodhostnik in #3244
• [FVM] Events in scripts do nothing instead of causing errors by @janezpodhostnik in #3183
• [FVM] Fix account reporter memory limit - master by @janezpodhostnik in #3105
• [FVM] Handle cadence runtime error early by @janezpodhostnik in #3204
• [FVM] Log err on TX failure by @janezpodhostnik in #3230
• [FVM] Move HandleRuntimeError to ContractFunctionInvoker by @janezpodhostnik in #3059
• [FVM] Moving event emit size metering logic to Meter interface by @Tonix517 in #3268
• [FVM] Pass context by value by @janezpodhostnik in #3220
• [FVM] Refactor checkAccountsAreNotFrozen by @janezpodhostnik in #3235
• [FVM] Remove old FVM errors by @janezpodhostnik in #3351
• [FVM] Remove old quick fixes by @janezpodhostnik in #3219
• [FVM] Replacing existing metering in state.go with new interface by @Tonix517 in #3249
• [FVM] Set ReusableCadenceRuntimePoolSize in bench tests by @janezpodhostnik in #3295
• [FVM] Simplifying FVM meter interface and implementation by @Tonix517 in #3252
• [FVM] Use embed for cadence scripts in blueprints by @janezpodhostnik in #3148
• Create one cadence runtime per ReusableCadenceRuntime object by @pattyshack in #3184
• Create virtual machine / cadence runtime inside computation.Manager by @pattyshack in #3132
• Reuse cadence runtime environment by @pattyshack in #3101
• Replace error/failure structs with CodedError/CodedFailure (Part 1) by @pattyshack in #3289
• Replace error/failure structs with CodedError/CodedFailure (Part 2) by @pattyshack in https://github.com/onflow/flow-go/pull/
• Reduce environment circular dependency by @pattyshack in #3223
• Refactor AccountKeyUpdater by @pattyshack in #3222
• Refactor GetAccountKey into AccountKeyReader by @pattyshack in #3186
• Refactor ProgramsCache into ChainPrograms by @pattyshack in #3205
• Refactor UUIDGenerator into environment by @pattyshack in #3057
• Refactor account freezer by @pattyshack in #3191
• Refactor contract reader into environment by @pattyshack in #3160
• Refacto...

v0.27.4

Mainnet 19

Major items going out this release.

💥 Breaking Changes

(updated 08/29)

• [Execution] A previous cache implementation allowed frequently queried blocks to stay in the cache for longer than originally intended. The new implementation introduced by (#2893) @ramtinms no longer allows this behaviour and causes the state to quickly become non-queryable via scripts after 500 collections, or approximately 100 blocks. With current block production rates, this translates to ~2mins.
  For script execution beyond this threshold, the Archive Node should be used. We will share the archive node details soon.
• [Execution] avoiding double encoding of event key before insertion into the event merkle trie (#1994) @ramtinms
• [FVM] merge constant size account registers in account status (#2799) @ramtinms
• [FVM] remove the use of the legacy controller by the FVM registers (#2585) @ramtinms
• Crypto: BLS ciphersuite updates (#2576) @tarakby (announcement)

Execution node checkpointing improvements

• [Execution Node] Reuse ledger state in checkpoints for -152GB RAM and -24 minutes (#2792) @fxamacker
• [Ledger] Replace LRU cache with a FIFO queue (circular buffer) (#2893) @ramtinms
• Refactor root checkpoint extraction program (#2206) @fxamacker
• Fix copylocks in TestOnBlockProposal_RemoteOrigin (#2693) @fxamacker

Execution node

• [EN Performance] Reduce memory used for ledger.Payload by 32+ GB, eliminate 1+ billion allocs/op, speedup various ops (#2930) @fxamacker
• [Execution] Data storage updates (#2292 ) @smnzhu
• [Execution] re-organizing uploader code in Execution engine (#2854) @Tonix517
• [Execution] a fix to atree reporter (#2839) @ramtinms
• [Execution] adding more debug logs (#2710) @smnzhu
• [Execution] check if state commitment exists before executing scripts (#2763) @ramtinms
• [Execution] Fixed EN debug docker image build failure (#2708) @Tonix517
• [Exec] Remove global lock from script execution (#2742) @SaveTheRbtz
• [Exec] Add manager benchmark (#2668) @SaveTheRbtz
• [Exec] improve result upload/eds tracing (#2655) @SaveTheRbtz
• [Exec] Remove Stop The World pauses from transaction/script execution (#2739) @SaveTheRbtz
• minor EN stat cleanup (#2643) @SaveTheRbtz
• [Exec] add tracing to collection view merge (#2656) @SaveTheRbtz

Cadence & FVM

• Run fvm benchmark transactions under a regular account (#2894) @pattyshack
• Move weighted Meter into meter package (#2895) @pattyshack
• [FVM] Deduct fees and limit storage in synthetic benchmarks (#2881) @SaveTheRbtz
• [FVM] Deduct fees on all networks by default (#2880) @SaveTheRbtz
• [FVM] cleanup benchstat installation (#2884) @SaveTheRbtz
• [FVM] cleanup and regenerate mocks (#2858) @SaveTheRbtz
• [FVM] Update cadence to support Executor pattern (#2853) @SaveTheRbtz
• dedup code in script/transaction environment (part 2 of many) (#2845) @pattyshack
• Split env context, unsafe rand gen, and program logger into logical units (#2849) @pattyshack
• [FVM] All service accounts should be bootstrapped with a key (#2705) @janezpodhostnik
• [FVM] removing legacy controller from storage (#2713) @ramtinms
• [FVM] Fix error handling on contract function invocations (#2531) @janezpodhostnik
• [FVM] Don't read context from state on system chunk - port (#2690) @janezpodhostnik
• [FVM] Benchstat CI improvements (#2748) @janezpodhostnik
• [FVM] Add missing tests for merging FVM state weights (#2530) @janezpodhostnik
• [FVM] contracts.go refactor (#2353) @janezpodhostnik
• [FVM] fix a bug with invalid signature error type (#2424) @tarakby
• [FVM] add extra flag for contract removal (#2584) @ramtinms
• [FVM][TEST] Splitting fvm_test.go (#2594) @Tonix517
• Fix Cadence code (#2653) @turbolent
• Switch to cadence v0.25.0 by @SupunS in #2913
• Update to Cadence 5a06fec (#2709) @turbolent

Epoch and DKG Updates

• [Fix] Update epoch test suite teardown (#2627) @kc1116
• fixed decoding error for signer indices at epoch switchover (#2830) @AlexHentschel
• Unskip flaky test - TestEpochJoinAndLeaveLN (#2802) @LLsq2
• fix flaky test: TestStaticEpochTransition (#2698) @gomisha
• [fix:TestEpochJoinAndLeaveAN] [fix:TestEpochJoinAndLeaveVN] Handle registering node without machine account (#2699) @jordanschalm

Networking

• [Networking] Fixes flakey DNS cache expiry test (#2670) @yhassanzadeh13
• Khalil/ 1725 authenticate libp2p peer before decoding message payload (#2667) @kc1116
• Khalil/2706 update middleware errors (#2726) @kc1116

Consensus / BFT

• [Consensus] Conflicting seals mitigation strategy (#2892) @durkmurder
• Update Consensus Loop (#2857) @danielholmes839
• [Sealing Segment] Handle root sealing segment with multiple blocks (#2786) @jordanschalm
• 1714 add rate limiting interceptor to the LN & EN gRPC API's (#2505) @kc1116
• 1704 add impersonation/masquerade slashing hooks (#2648) @kc1116

Sync engine

• [SyncEngine] not request pending ancestors (#2879) @zhangchiqing
• Add exponential backoff for execution data download timeouts (#2629) @peterargue

Crypto

• Update crypto scripts for Go 1.18 (#2911) @tarakby
• crypto: allocate on stack where possible (#2598) @SaveTheRbtz
• Update crypto note in README (#2781) @tarakby
• [Crypto] fix memory leak in unhappy path (#2604) @tarakby
• Crypto: BLS ciphersuite updates (#2576) @tarakby

Access and Observer node

• Observer Pull Request Candidate 0603z (#2554) @szegedim
• Observer integration tests (#2783) @danielholmes839
• AN - Lock Updates to Conn Pool (#2779) @lolpuddle
• [AN] - Trigger Cache Eviction on Overwrite (#2760) @lolpuddle
• Invalidate Pooling Logic in AN (#2691) @lolpuddle
• AN - Lock Updates to Conn Pool (#2779) @lolpuddle
• [Access] add signer ids to access API (#2762) @zhangchiqing
• Improve error reporting for access api event endpoints (#2822) @peterargue
• Add warning logs to request handler (#2671) @danielholmes839
• Add latest height metrics for execution receipts and collections (#2692) @danielholmes839
• Fix GetTransactionResultsByBlockID Panic (#2729) @
• Fix time.Sleep (#2721) @
• Add a separate channel name for push-blocks on the public network (#2641) @peterargue

Performance improvements

• Stop type casting meter before setting weights / limits (#2908) @pattyshack
• [FVM] Storage check optimisation (#2837) @janezpodhostnik
• [FVM] reducing register touches (#2502) @ramtinms

BFT Testing

• [BFT Testing] Fixes wintermute orchestrator race condition (#2780) @yhassanzadeh13
• [BFT Testing] Add BFT tests to regular CI workflows (#2725) @gomisha
• [BFT Testing] refactors gRPC interface (#2599) @yhassanzadeh13
• [BFT Testing] Wintermute attack integration test (#2463) @yhassanzadeh13

Profiler/Telemetry changes

• expose StartSpanFromRoot AccountFreezeEnabled methods to environment interface (#2900) @pattyshack
• [Profile] disable threadcreate profiler (#2898) @SaveTheRbtz
• [Profiler] move out of ./utils/debug to ./module/profiler (#2897) @SaveTheRbtz
• [Profiler] Force GC before running heap profile (#2860) @SaveTheRbtz
• [All] OpenTracing to OpenTelemetry migration (#2823) @SaveTheRbtz
• [Profiler] Automatic pprof profile uploads to Google Cloud Profiler. (#2903) @SaveTheRbtz

🛠 General Node Improvements

• [Collection Node] Add dry-run option for collection rate limiting (#2615 ) @jordanschalm
• Port of complete removal of extralogs (#2682) @m4ksio
• Backport setExecutionParameters from TransactionEnv to ScriptEnv (#2835) @pattyshack
• Adding new load test type for constant sized transactions (#2816) @Tonix517
• Add ComputationMeter interface to facilitate env code dedup-ing (#2831) @pattyshack
• Fix Unit race condition on stopping. (#2775) @pattyshack
• Remove GO111MODULE=on everywhere (#2821) @SaveTheRbtz
• chore(comment): fix IsSampled description (#2809) @SaveTheRbtz
• chore(tests): remove unused mock (#2824) @SaveTheRbtz
• chore(tests): add trace benchmark (#2806) @SaveTheRbtz
• Add staging net (#2793) @Kay-Zee
• dedup code in script/transaction environment (part 1 of many) (#2790) @pattyshack
• chore(trace): rename file (#2810) @SaveTheRbtz
• Invoke contracts directly instead of via currying (#2804) @pattyshack
• chore(localnet): fix observer's trace endpoint (#2805) @SaveTheRbtz
• chore(loader): fix Stop/Follow race (#2766) @SaveTheRbtz
• change unittest fixture to use *flow.Header instead of flow.Header (#2744) @pattyshack
• Add Andrew Meyer to performance stream (#2769) @AndrewM-SDET
• [localnet] Adding checking to EXECUTION configuration (#2788) @Tonix517
• s/Rem/Remove/g to improve code readability (#2784) @pattyshack
• chore(loader): wait for account creation results (#2778) @SaveTheRbtz
• Extend consensus follower config to support compliance settings (#2697) @jordanschalm
• Update dependency modules version and update BLS DSTs (#2603) @tarakby
• chore(loader): faster worker Stop() (#2765) @SaveTheRbtz
• chore(loader): switch from Client to Access API (#2764) @SaveTheRbtz
• loader: remove TxTracker and TxStatsTracker (#2632) @SaveTheRbtz
• Don't leak signerLock outside of Account encapsulation (#2644) @pattyshack
• remove disk size metrics (#2746) @zhangchiqing
• tweak Programs locking + minor refactoring (#2749) @pattyshack
• Fixed docker-build-loader image build failure (#2757) @Tonix517
• chore(localnet): faster account creation (#2683) @SaveTheRbtz
• Remove header caching (#2704) @SaveTheRbtz
• optimize emergency-sealing check (#2673) @zhangchiqing
• Unskipping passing flaky test - TestComponentConsumerSuite/TestHappyPath/runs_and_notifies_using_pre-notifier (#2730) @LLsq2
• Moving emulator-based tests to the integration package (#2738) @ramtinms
• Fix reversed numbers for script metrics (#2595) @dsainati1
• Simplify blockCommitter/eventHasher (#2660) @patt...

v0.26.9 - Mainnet 18

Mainnet 18

Major Changes

• [Execution] upgrade cadence to secure-cadence-m9 (#2516) @j1010001
• [Consensus & Collection] Merge Signer-indices to master (#2414) @zhangchiqing
• [Access] Add execution data requester (#1895) @peterargue

Misc

• [Util CL] fix logging current statecommitment in execution state extraction (#2610) @zhangchiqing
• [Testing] localnet: switch from docker log driver to promtail (#2609) @SaveTheRbtz
• [Access] Use Connection Pool Logic (#2613) @lolpuddle
• Add ledger checkpoint status files to gitignore (#2607) @jordanschalm
• [Logging] Fix implementation of script logging (#2528) @koko1123
• [util] update cmd to get seal for sealed block (#2587) @zhangchiqing
• [Tool] Export transactions for height range (#2442) @zhangchiqing
• [Access] Implement Connection Pooling for RPC Connections in AN (#2412) @lolpuddle
• [Networking] Downgrades libp2p pubsub version (#2592) @yhassanzadeh13
• [Crypto] performance improvement of SSWU map to curve (#2570) @tarakby
• sync engine optimization (#2548) @danielholmes839
• [State Sync] Lookup ExecutionResult directly instead of using index (#2566) @peterargue
• Update to Cadence v0.24.1 and Go SDK v0.26.1 (#2583) @turbolent
• [Protocol State] Index seal by sealed block id (#2551) @zhangchiqing
• [Testing] Moving convert fixtures to a namespace for test cases (#2533) @zhangchiqing
• [Util CLI] fix encoding in checkpoint-list-trie cmd (#2546) @zhangchiqing
• [Collection] Remove max account index (#2552) @jordanschalm
• [Testing] feat(localnet): use grafana, tempo, loki (#2510) @SaveTheRbtz
• [Testing] Fix flakey component job consumer test (#2538) @peterargue
• [Testing] chore(tests): cleanup TxTracker (#2520) @SaveTheRbtz
• [Testing] quarantine test - TestComponentConsumerSuite-runs_and_notifies_using_pre-notifier (#2555) @gomisha
• [Testing] chore(tests): cleanup worker and add tests (#2503) @SaveTheRbtz
• [Testing] chore(tests): cleanup TxStatsTracker and add tests (#2471) @SaveTheRbtz
• [Testing] chore(tests): move code back to contLoadGenerator.go (#2537) @SaveTheRbtz
• [Testing] feat(localnet): pass through tracing, extensive tracing, and cadence tracing flags (#2542) @SaveTheRbtz
• [Testing] chore(tests): cleanup worker stats tracker and add tests (#2504) @SaveTheRbtz
• Corruptible Conduit Framework Integration (#2397) @yhassanzadeh13
• [Fix] add explicit mapping of cluster prefix -> message codes (#2539) @kc1116
• PR Migrate ingestion engine to new Component interface (#2436) @danielholmes839
• chore(tests): remove unused batchLoadGenerator (#2519) @SaveTheRbtz
• chore(tests): do not go through the resolver for localhost (#2511) @SaveTheRbtz
• Add secure-cadence prefix as a tag to be pushed (#2167) @Kay-Zee
• [Crypto] improve flaky random test (#2509) @tarakby
• [All Nodes] upgrade flow to v0.3.1 (#2534) @zhangchiqing
• Exit validation early for empty collections (#2524) @jordanschalm
• Add BlockHeight to TransactionResultResponse (#2514) @lolpuddle
• Outdated blocks should not be logged as errors (#2372) @AlexHentschel
• Add Atree reporter to execution state extraction (#2461) @fxamacker
• Update to Cadence 8b113c539a2c, atree 525fbc26f40a, and CBOR ad2eae63303f (#2525) @turbolent
• add go workspace files to .dockerignore and .gitignore (#2469) @SaveTheRbtz
• [Signer Indices] Validate guarantors (#2462) @zhangchiqing
• Suggestion for PR #2462 (#2517) @AlexHentschel
• [Networking] Upgrades pubsub version of libp2p (#2515) @yhassanzadeh13
• Read memory limit from state (#2474) @robert-e-davidson3
• chore(tests): mark localnet and benchmarks owned by the performance team (#2512) @SaveTheRbtz
• add jobqueue README (#2373) @zhangchiqing
• upload log level for memory weights (#2493) @zhangchiqing
• chore(tests): fix init-light for localnet (#2508) @SaveTheRbtz
• Upgrade Cadence to secure-cadence-m8 (#2499) @janezpodhostnik
• flaky tests fix test async uploader - Test_AsyncUploader/stopping_component_stops_retrying (#2453) @gomisha
• [Fix] Adding sync request message as a cluster type message (#2495) @yhassanzadeh13
• Add tests to ensure mutations in scripts are discarded (#2482) @dsainati1
• chore(tests): cleanup unused fields from loader (#2470) @SaveTheRbtz
• [Component Manager] Fix race condition (#2349) @smnzhu
• Optimize MTrie reading single path by adding Ledger.GetSingleValue() to boost speed and reduce memory use (#2473) @fxamacker
• Optimize Ledger.Get() by making Forest.Read() use ~5x fewer allocs/op and run ~20% faster (#2477) @fxamacker
• Memory metrics (#2467) @dsainati1
• [BFT Testing] Corruptible Conduit Factory authenticating dictated messages. (#2441) @yhassanzadeh13
• Dan/1536 add cadence execution time metrics (#2394) @danielholmes839
• Upgrade linter to 1.46.2 (#2457) @zhangchiqing
• Remove unused travis yml (#2323) @Kay-Zee
• unquarantining 2 flaky tests that have been consistently passing in quarantine (#2455) @gomisha
• chore(tests): fix load test (#2454) @SaveTheRbtz
• chore(docker): fix parallel image builds (#2447) @SaveTheRbtz
• avoid race condition in validating blocks (#2427) @zhangchiqing
• Suggestion for PR #2414 (#2443) @AlexHentschel
• Memory limit exception for service account (#2437) @turbolent
• Add util command to retrieve snapshot at any height (#2429) @jordanschalm
• Use go 1.18 to build and run binary (#2363) @Kay-Zee
• Secure Cadence (#2337) @janezpodhostnik
• Secure cadence master update (#2433) @m4ksio
• Separate permission setting of contract deploy/update and removal (#2390) @robert-e-davidson3
• Robust Transaction Deduplication (#2385) @jordanschalm
• add flaky skip reason so tests can run in cron job (#2410) @kc1116
• Secure Cadence upgrade to m7 (#2423) @janezpodhostnik
• Secure Cadence flow-go-sdk upgrade (#2417) @janezpodhostnik
• [FVM] Signing errors swapped fix (#2415) @janezpodhostnik
• Log MemoryUsed by transactions. (#2396) @dsainati1
• Allow disabling/enabling permissionless deployment with a transaction (#2376) @janezpodhostnik
• Secure Cadence - merge master (#2400) @janezpodhostnik
• [Execution State] Full search json dump (#1971) @m4ksio
• improve signer indices logging (#2408) @zhangchiqing
• [Crypto] fix race conditions in DKG tests (#2391) @tarakby
• Add new code space for Canary addresses (#2388) @tarakby
• rm duplicate code in getNextAncestryLevel (#1759) @zhiqiangxu
• log bootstrap cmd error (#2259) @zhangchiqing
• fix docker build loader (#2382) @zhangchiqing
• [FVM] Merge default execution/memory weights and state weights (#2374) @janezpodhostnik
• Add test for referencing non-existent address (#2256) @janezpodhostnik
• Update execution verification tests (#2007) @janezpodhostnik
• [FVM] Collapse FVM bench results (#2111) @janezpodhostnik
• Secure Cadence - revert removing addKey api v1 tests (#2344) @janezpodhostnik
• Khalil/1689 Add authorized sender pubsub topic validator (#2317) @kc1116
• Add checksum to signer indices (#2362) @zhangchiqing
• changing the block height range for GetEventsForBlockHeightRange REST… (#2379) @vishalchangrani
• Fix race detector issues. (#2293) @SaveTheRbtz
• Add weights for cadence values and types (#2358) @robert-e-davidson3
• [FVM] signature verification refactoring (#2171) @tarakby
• Add weights for array element overhead and map pre-allocated elements (#2375) @SupunS
• [Signer Indices] upgrade onflow/flow package (#2377) @zhangchiqing
• [Collection] Log collection proposer (#2381) @zhangchiqing
• Update cd.yml to ignore daily tags created through automation (#2368) @sjonpaulbrown
• Update localnet TARGET paths after recent dockerfile changes (#2361) @peterargue
• [BFT Testing] Integrating BFT testing framework with testnet (#2308) @yhassanzadeh13
• Don't push :latest tag by default (#2355) @m4ksio
• Fix linting error (#2356) @zhangchiqing
• Secure Cadence - add memory alloc metering (#2350) @janezpodhostnik
• Secure Cadence - Merge master (#2343) @janezpodhostnik
• Secure Cadence - upgrade to m5 (#2340) @janezpodhostnik
• Secure Cadence - integration test fix (#2338) @janezpodhostnik
• Add extraction report for usage within external tooling (#2239) @sjonpaulbrown
• [BFT Testing] implements wintermute attack orchestrator (#2291) @yhassanzadeh13
• [Tool] add rollback executed height (#2334) @zhangchiqing
• [REST API] Adding missing fields to the ExecutionResult object (#2311) @vishalchangrani
• [Access] update system chunk tx handling in GetTransactionResults API (#2329) @smnzhu
• Memory weight assignment from contract (#2305) @janezpodhostnik
• Smnzhu/system tx port (#2332) @smnzhu
• remove unused test util (#2331) @robert-e-davidson3
• Fixed some tests for AddPublicKey and RemoveePublicKey (#2327) @robert-e-davidson3
• Add observer support to localnet (#2320) @shawnflow
• Fix AN crash (#2326) @m4ksio
• Tx results by block id merged (#2314) @m4ksio
• Create GetTransactionResultsByBlockID API (#2303) @smnzhu
• [Access node] Add random en lookup on receipt lookup failure (#2321) @koko1123
• Fix bootstrap constraint check (#2233) @zhangchiqing
• Add config to limit caching of new proposals while syncing (#2294) @jordanschalm
• implement result reindexing tool (#2289) @smnzhu
• Retry on conflict inserting cluster blocks (#2270) @jordanschalm
• Use block ID from seal (#2287) @Kay-Zee
• Upgrade go-retry v0.1.0->v0.2.3 (#2258) @jordanschalm
• Fix flakey test: TestComponentShutdown (#2269) @jordanschalm
• Use block ID from seal (#2288) @smnzhu
• Fix er indexing on finalization (#2284) @koko1123
• Add mutex around script tracking map (#2283) @koko1123
• Improve DKG & QC contract logging (#2263) @kc1116
• Fix result indexing by block (#2276) @koko1123
• [FVM, EN] pass request context and add timeout for script execution (#2147) @ramtinms
• Merge v0.25 to master (#2266) @turbolent
• Update the private message struct: remove Orig from DKGMessage (#2238) @kc1116
• [Networking] Fixes unicast timeo...

v0.25.7

Mainnet 17

Major Changes

Updated to Cadence version v0.23.0, which contains a breaking change for the Public Key API

There has also been a lot of improvements made to the mTrie/merkle tree storage methods, specially made around the Write Ahead Log, and Checkpointing logic. The speed improvements here are mostly noticeable for node operations side and will not be DAPP facing.

There will be an internal concept change, which will better define the concept of "Weight" in the protocol. This changed required some renaming of Stake → Weight (Node level breaking change). This means that the read the new protocol state, it will require a matching node version that has the same conceptual understanding of Stake vs Weight.

Lastly, there is a change that will make to add the code necessary for Variable Transaction Fees, but this feature will not be enabled until a community vote has been held.

All Changes

💥 Breaking Changes

• Optimize MTrie Checkpoint (regCount & regSize): -9GB alloc/op, -110 milllion allocs/op, -4GB file size (#2126) @fxamacker
• Reduce checkpoint file size by using fewer bytes to encode length of encoded payload value (#2165) @fxamacker
• Update checkpoint file format version from v4 to v5 (#2174) @fxamacker
• Optimize MTrie checkpoint: 47x speedup (11.7 hours -> 15 mins), -431 GB alloc/op, -7.6 billion allocs/op, -6.9 GB file size (#1944) @fxamacker
• Rename Stake to Weight (#2090) @jordanschalm
• Cadence upgrade (#1975) @janezpodhostnik

Cadence & FVM

• [FVM] Variable Computation Metering (#1631) @janezpodhostnik
• [FVM] Change fee deduction logic (#2097) @janezpodhostnik
• [Execution] enable cadence runtime tracing given node level flag (#2108) @ramtinms
• [FVM] refactor computation and memory metering (#2129) @ramtinms
• [Migration] Migration for Ordered Map storage change (#1839) @dsainati1
• [FVM] relax tag conditions for ECDSA runtime signature verification (#1992) @ramtinms
• [FVM] Add bls crypto functions to transaction and script env (#1605) @dsainati1
• [FVM] Execution effort weights option for bootstrap (#2228) @janezpodhostnik
• [FVM] Move execution intensities debug logging (#2229) @janezpodhostnik
• [FVM] Fix metering limits (#2217) @janezpodhostnik
• [Cadence] update to Cadence v0.23.3 (#2247) @turbolent
• [FVM] Temporary fix for addressing storage used of a non-existing account (#2253) @janezpodhostnik

BFT Testing

• Implements attack network (#2144) @yhassanzadeh13
• Adds attacker implementation (#2121) @yhassanzadeh13
• Adds orchestrator interface and attack network (#2109) @yhassanzadeh13
• Implements Corruptible Conduits (#2033) @yhassanzadeh13

Access API

• Store and retrieve transaction results by EN RPC (#2166) @m4ksio
• Create transaction result by index api (#2159) @koko1123
• Add bootstrap command to generate observer networking key (#2087) @peterargue
• Create light block response as default for AN block lookup API (#2148) @koko1123
• Add Converters between RPC and flow-go Block with nested types (#2056) @koko1123
• Create unit test for gRPC ExecutionResult response structure (#2006) @koko1123
• Support running observer standalone using standard AN image (#2037) @peterargue
• Log unique cadence scripts in AN execution engine (#1934) @koko1123
• Add ExecutionDataId field in handler (#2010) @koko1123
• [HTTP API] Transaction success status (#2106) @sideninja

🛠 Node Improvements

• [Execution] Log execution data ID (#2196)(#2055)(#2039) @smnzhu

• [Execution] Add logging for CPU, RAM, and checkpoint (#2119)(#2187) @fxamacker

• [Execution] Reduce default checkpoint distance flag from 40 to 20 to speedup EN startup (#2223) @fxamacker

• [Execution] Remove transaction results from execution data (#2098) @smnzhu

• [Consensus] VoteAggregator pruning (#2040)(#2110) @durkmurder

• [Consensus] Epoch test suite updates (#2041) @kc1116

• [Consensus] Refactor epochs related contract deployments, use blueprints pattern (#2057) @kc1116

• [Consensus] ecretsdb key soft enforcement (#2042) @kc1116

• [Collection] Add message queue to transaction ingest engine (#2035) @jordanschalm

• [All Nodes] Root snapshot QC validation (#2038) @durkmurder

• [All Nodes] Performance: improve the binary search speed (#1493) @jwinkler2083233

• [All Nodes] Implement --profiler-mem-profile-rate= (#2059) @simonhf

• [Execution] Merkle trie updates (#1909) @tarakby

• [Consensus] DKG Messaging Improvements (#1950) @jordanschalm

• [All Nodes] Performance: replace calls to Lookup with binsearch (#1513) @jwinkler2083233

• [All Nodes] Add func that will check for snapshot validity (#1940) @kc1116

• FVM error detection (#2028) @sideninja

🐞 Bug Fixes

• Fix for missing cache (#2043) @jwinkler2083233
• Fix bootstrap cluster qc generation (#1976) @jordanschalm
• Fix potential overflow in ledger metrics LatestTrieRegCountDiff() and LatestTrieMaxDepthDiff() (#2096) @fxamacker
• Fix race in epoch switchover handling (#1980) @jordanschalm
• [HTTP API] Authorizers bugfix (#2152) @sideninja

CI/CD

• Flaky Test Monitor - BigQuery update and full refactoring of unit tests (#2102) @gomisha
• Add observer as a separate docker image (#2113) @m4ksio
• Avoid setup/teardown cycle for skipped epoch integration tests; re-enable base Epoch Transition test (#1881) @jordanschalm
• Fix non-x86 linter issue (#1938) @tarakby
• changing go mod to point to the latest OpenAPI spec (#1962) @vishalchangrani

Misc

• Replace use of deprecated ioutil funcs in ledger (#2084) @fxamacker
• [Observability] Adds metrics for HeroCache (#2093)(#2218) @yhassanzadeh13 @smnzhu
• [Admin] GetIdentity command (#2191) @smnzhu
• Update sync protocol flip (#2123) @smnzhu
• Target westmere processor when compiling relic for better compatibility and performance (#1993) @yagop
• [Admin] Remove double instantiation of admin server for access node (#2141) @smnzhu
• [Util CLI] Removing transit push command; adding transit upload-transit-keys com… (#2122) @vishalchangrani
• [Admin] Add admin command to enable/disable profiler (#2076) @zhangchiqing
• [Admin] Define ToggleUploaderCommand (#2114) @smnzhu
• [Observability] Configurable metrics (#2095) @m4ksio
• Update flow-go/crypto dependency (#2107) @janezpodhostnik
• Protocol randomness use cleanup (#2001) @tarakby
• Include core contract changes in cadence-issue-1127 (#2103) @janezpodhostnik
• [Network] Add read and write deadlines for direct messages (#2079) @smnzhu
• [Util CLI] Partner node info generation tool (#1988) @kc1116
• [Network] add new metric to track number of Unicast calls in progress (#2077) @smnzhu
• [Network] add metrics and log to track blocked queue workers (#2069) @smnzhu
• [Admin] implement read execution data admin command (#1998) @smnzhu
• [Util CLI] fungible token tracker (#2012) @ramtinms
• [Logging] Fix TransactionContractFunctionInvoker error log (#2031) @janezpodhostnik
• [Logging] Make ContractHandler Commit produce deterministic errors (#2003) @janezpodhostnik
• [Logging] log vote's signer and block when vote is invalid (#1982) @zhangchiqing
• [Networking] Implements Pluggable Conduit Factory (#1986) @yhassanzadeh13
• [Util CLI]Fix account reporter (#1972) @zhangchiqing
• [Testing] Allow bench-net permissionless contract deployment (#1970) @janezpodhostnik
• Split CI integration job into categories (#2242) @smnzhu
• Update request size limit for sync engine (#2245) @smnzhu
• Fix incorrectly skipped test (#2241) @smnzhu
• Fix output of fungible token tracker test (#2237) @smnzhu
• Test monitor GitHub actions (#2151) @smnzhu
• [BFT Testing] Implements integration tests for corruptible conduit factory framework (#2203) @yhassanzadeh13

v0.24.4

Mainnet 16

Major Changes

No breaking changes for Cadence included in this update.

There are two major features included in this software upgrade:

• Consensus Voting V2: Protocol level optimization on how voting on blocks is performed, allowing more parallelism in vote processing
• Epoch Dynamic Node Addition: Adds ability to more easily adjust Protocol member list during Epoch transitions

There has also been a lot of improvements made to the mTrie/merkle tree implementation including a bug fix which will effect how the Payload hash is generated. There are also improvements to caching, such as adding a new cache type which we're currently calling HeroCache.

Lastly, there is a change that will make this software version relatively incompatible with previous Flow node software versions, which is adding the execution CID, in preparation for the improve state sync engine. This change adds an extra field to the ExecutionResult data structure, which will produce different hashes for the ExecutionResult compared to previous software versions

All Changes

Cadence & FVM

• Check and Use Contract Audit Vouchers (#1826) @tehranifar
• Fix LedgerGetRegister to avoid caching empty value (#1875) @fxamacker
• Improve error handling in MTrie checkpoint file creation (#1892) @fxamacker
• Support legacy Keccak 256 hashing algorithm (#1868) @ramtinms
• Backport: Ledger interaction error (#1878) @zhangchiqing
• Optimize Forest.Proofs() and add ValueSizes() to MTrie, Forest, and Ledger (#1870) @fxamacker
• Preallocate slices in MTrie (#1838) @fxamacker
• Update to Cadence v0.20.3 (#1829) @turbolent
• Add logs for MTrie checkpoint file creation (#1894) @fxamacker
• Update MTrie with some minor fixes and small improvements (#1801) @fxamacker
• Relaxing ledger interaction limit for service and system transactions (#1753) @ramtinms
• Tweak FVM bench to be more stable (#1722) @janezpodhostnik

Epoch and DKG Updates

• Dynamic Node Startup (#1893) @kc1116
• Improve handling of post-spork beacon key initialization (#1917) @jordanschalm
• Epochs integration test validation: consensus nodes (#1856) @kc1116
• Include staking auction length in resetEpoch (#1804) @jordanschalm
• Update deployEpoch for new FlowEpoch function (#1854) @jordanschalm
• Skip starting dkg reactor engine if node not part of dkg committee (#1849) @kc1116
• Enforce weight/ejection requirements in cluster committee (#1842) @jordanschalm
• Epochs test collection node validation (#1819) @kc1116
• Set epoch final view metrics on transition (#1806) @jordanschalm
• Epochs Integration testing: Verification node (#1763) @kc1116
• DKG Comments / Logging Changes (#1769) @jordanschalm
• DKG State (#1738) @jordanschalm
• DKG Storage Pt III: Check beacon key consistency atomically with epoch phase transition (#1732) @jordanschalm
• DKG Storage Pt II: Update components using DKGState storage (#1721) @jordanschalm
• DKG Storage Pt. I: Update storage layer (#1720) @jordanschalm
• Epochs integration tests (#1728) @kc1116
• Epochs test for joining with dynamic container addition/removal (#1603) @kc1116

Networking

• Refactors DNS cache with HeroCache (#1888) @yhassanzadeh13
• Refactoring networking received cache to HeroCache (#1908) @yhassanzadeh13

General Node and Crypto Improvements

• Update state synchronization (#1930) @smnzhu
• Chacha20-based PRG (#1782) @tarakby
• Clean up BLST tests (#1754) @tarakby

Access Node

• adding status_code to transaction result reponse in the REST API (#1959 ) @vishalchangrani
• Report execution errors for cadence scripts as OK in Access Node RPC calls (#1905) @koko1123
• Add naive error logging for Access node downstream calls (#1883) @koko1123
• REST api implementation (#1666) @vishalchangrani
• REST: Stability improvements (#1809) @sideninja

Cosensus Hotstuff

• Consensus Voting V2 (#1916) @zhangchiqing
• Sealing Core: Allow unverifiable block references below root height (#1828) @jordanschalm
• Extend Sealing Segment Structure (#1755) @jordanschalm

Sporking process improvements

• Improve state extraction (#1859) @zhangchiqing
• Validate the saved root snapshot (#1832) @zhangchiqing

Execution

• Improve transaction execution log (#1869) @zhangchiqing
• Switch eventList hash to use a Merkle trie (#1834) @ramtinms
• Better printing for keys in errors (#1781) @m4ksio
• Properly escape user-input in error messages (#1863) @m4ksio
• Log epoch number on execution node startup (#1731) @janezpodhostnik
• Remove metering computation twice (#1724) @janezpodhostnik

State sync engine

• Add execution data cid (#1775) @smnzhu
• Execution data requester (#1582) @smnzhu
• Sync engine / core updates (#1697) @smnzhu
• Allow registering custom node shutdown handlers (#1907) @peterargue
• Make NodeBuilder return runnable Nodes (#1543) @peterargue

Build, CI & Tests

• Fix flow-go failing to cross compile (#1939) @janezpodhostnik
• Fix ledger-heavy load generator (#1933) @janezpodhostnik
• Emulator build without Relic + Enable crypto no-Relic tests (#1922) @tarakby
• Fix undefined: sha3State errors when building on arm64 (#1903) @peterargue
• Support arm64 when building flow docker containers and localnet (#1904) @peterargue
• Disable machine account balance checking on test networks (#1877) @jordanschalm
• Improve Topology Factory for integration tests (#1880) @yhassanzadeh13
• Fix crypto test flakiness in randomness test (#1879) @tarakby
• Replace deprecated go get with go install (#1860) @koko1123
• Fix loader backport (#1841) @zhangchiqing
• Fix localnet loader for transaction execution success (#1816) @simonhf
• Flaky Test Monitor - CI fix - updated level 1 script to use level 1 sub-directory (#1808) @gomisha
• Stream negotiation and fallback for integration tests (#1812) @yhassanzadeh13
• Add intensive resources flag to AN tests (#1817) @kc1116
• Flaky Test Monitor - Level 3 Summary (#1793) @gomisha
• Fix bug in transfer tokens transaction in integration load tests (#1789) @janezpodhostnik
• Flaky Test Monitor - Summary Level 2 (#1707) @gomisha
• Use debian 11 (bullseye) (#1768) @Kay-Zee
• Skip flaky test conditionally (#1535) @smnzhu

🛠 Misc Improvements

• Limit range queries for sync engine @m4ksio
• Remove sudo from crypto setup (#1945) @tarakby
• Remove the crypto replace and add a crypto setup script (#1742) @tarakby
• Remove unnecessary allocations (#1596) @jwinkler2083233
• Refactors Transactions mempool to HeroCache (#1902) @yhassanzadeh13
• Adding proof to the merkle trie (#1833) @ramtinms
• Optimizes All method of BackData (#1852) @yhassanzadeh13
• Payload Patricia Trie Security fixes (#1800) @AlexHentschel
• Implements GC-optimized cache (#1778) @yhassanzadeh13
• Improve efficiency of Views and Heights gadgets (#1823) @jordanschalm
• Make hex address parsing consistent with SDK (#1792) @jordanschalm
• Removing legacy Identities interface (#1777) @AlexHentschel
• Refactor Merkle Patricia Tree used for payload hashing (#1678) @AlexHentschel
• Update min LN machine account balance (#1774) @jordanschalm
• Adding validation when constructing a Local module (#1704) @zhangchiqing
• Add readme for live debugger (#1716) @ramtinms

v0.23.3 - Mainnet 15

Mainnet 15

Two major items going out this release.

The first will be the introduction of the new Cadence Storage Layer. This new storage layer will make use of the atree, which will greatly improve the performance of storage interactions with extremly large storage paths. As part of this change, there is a breaking change in relation to the owner field. More on that in the forums

The second being automated Epoch transitions. Epochs will now auto transition once a week with an un-changing node operator list. The network will run the DKG and shuffle clusters each epoch, but node operator changes will not be enabled at epoch boundaries yet. Note that this does not yet include the reward payout, which will still be done manually until we are satisfied with the reliability of the auto epoch transitions.

Breaking Change

• bugfix for Cadence's resource owner field (link)(https://forum.onflow.org/t/breaking-change-bugfix-for-cadence-resource-owner-field/2576)

Cadence & FVM

• [Cadence] Bump Cadence to v0.20.2 (#1552)(#1676)(#1714) @SupunS @turbolent @Kay-Zee
• [Mtrie] Improve ledger write speed for newly allocated slabs (#1501) @ramtinms
• [FVM] Update ledger interaction limit for bootstrapping (#1648) @ramtinms
• [FVM] fix error messages (#1665) @tarakby
• [FVM] Recover ledger interaction error (#1625) @janezpodhostnik
• [Mtrie] Fix for Mtrie bug (#1633) @AlexHentschel
• [FVM] Clean up FVM (#1613) @turbolent
• [Mtrie] Trie pruning (#1578) @ramtinms
• [FVM] Interaction limit related issues(#1504)(#1592) @janezpodhostnik @ramtinms
• [Mtrie] On update trie pruning (complete ledger) (#1507) @ramtinms
• [FVM] Change meta script invocation to direct contract function calls (#1318) @janezpodhostnik
• [FVM] Fix fees not being deducted on gas limit reached (#1367) @janezpodhostnik

Epoch and DKG Updates

• [DKG] Do not crash for DKG failures (#1693) @tarakby
• [Epoch] Update AN builders and scaffold to use spork id (#1696) @kc1116
• [Epoch] Epoch counter reporter (#1691) @janezpodhostnik
• [DKG] Remove unnecessary fields from DKG Private Info (#1673) @tarakby
• [Epoch] Refactor fallback logic for qc voter & dkg broker (#1556) @kc1116
• [Epoch/DKG] Add jitter to all QC/DKG requests (#1637) @jordanschalm
• [Epoch] Update reset-tx-args command (#1615) @jordanschalm
• [Epoch] Epoch Emergency Fallback Improvements (#1569) @jordanschalm
• [Epoch] SporkID and ProtocolVersion (#806)(#810) @jordanschalm
• [Epoch] Extend root snapshot (#1533) @kc1116
• [Epoch] Use SporkID for network channel namespacing (#1474) @jordanschalm
• [Epoch] Use latest finalized (not sealed) reference block (#1607) @jordanschalm
• [DKG] Fixes a bug where DKG validation would continue after encountering a fatal error (#1583) @jordanschalm
• [Epoch] Add func to set approved nodes list (#1478) @kc1116
• [Epoch] Remove redundant key generation code (#1469) @kc1116

Networking

• [Networking] Update libp2p-tls to fix issue with secured gRPC certificate (#1694) @peterargue
• [Networking] Stream negotiation and fallback (#1643) @yhassanzadeh13
• [Networking] Update connection manager implementation (#1516) @smnzhu
• [Networking] Unstaked node networking updates (#1506) @smnzhu
• [Networking] Tweak-able networking received cache size (#1529) @yhassanzadeh13
• [Networking] Changing multicast to use the filtered set of node ids (#1528) @vishalchangrani
• [Networking] Logging dial addresses (#1492) @yhassanzadeh13
• [Networking] Add bitswap API (#1356) @smnzhu
• [Networking] Middleware component (#1355) @smnzhu
• [Networking] Libp2p stream compression (#1349) @yhassanzadeh13

General Node and Crypto Improvements

• [Access] Adding a block height threshold to re-request missing collections (#1711) @vishalchangrani
• [Access] Adding middlewares to the REST API implementation (#1619) @vishalchangrani
• [Execution] Using valid descendents when loading unexecuted blocks (#1588) @zhangchiqing
• [Access] REST API implementation - GetBlockByID (#1524) @vishalchangrani
• [Verification] Removes less and swap methods of chunk list (#1608) @yhassanzadeh13
• [BFT] Error handling for engines (#1568) @durkmurder
• [State Sync] state serialization module (#1483) @smnzhu
• [Consensus] Continue recover block when hitting invalid block (#1564) @zhangchiqing
• [Consensus] sealing visualization (#1457) @smnzhu
• [Crypto] light SHA3 API (#1359) @tarakby
• [Crypto] Improve Invalid input errors (#1553) @tarakby
• [Consensus] Remove legacy code in Ingestion engine (#1532) @AlexHentschel
• [Consensus] Ingestion engine inbound messages queue (#1517) @durkmurder
• [Consensus] Fix potential mem-leak from hotstuff (#1514) @zhangchiqing

🛠 Misc. Improvements

• [Misc] Do not use pointer receiver for MarshalJSON (#1652) @SaveTheRbtz
• [Misc] Update minimum go version to 1.16, and default to 1.17 (#1692) @peterargue
• [Misc] Fix code ownership for access (#1653) @SaveTheRbtz
• [Metrics] Fix randomness in execution's example (#1651) @SaveTheRbtz
• [Misc] Reenable Reporters (#1410) @janezpodhostnik
• [Admin] Add commands for reading entities (#1549) @smnzhu
• [Admin] Update live debugger to include block header data (#1587) @ramtinms
• [Misc] Adds an additional error check (#1661) @Kay-Zee
• [Documentation] Updating broken links to node docs (#1558) @j1010001
• [Verification] makes verification node fork-aware (#1538) @yhassanzadeh13
• [Documentation] Adding section on playing with localnet (#1531) @j1010001
• [Metrics] Disable cache metrics (#1518) @zhangchiqing
• [Misc] removing rlp-encoded private key string to use flow.AccountPrivateKey (#1525) @j1010001
• [Documentation] Fixing broken links in access node docs (#1481) @j1010001
• [Misc] Clean up secondary result index (#1515) @Kay-Zee
• [Misc] Adding log tracer module for debugging purposes (#1496) @ramtinms
• [Admin] Adding rpc endpoint to get registers from exec nodes (#1460) @ramtinms
• [Admin] Adding transaction and script debugger using remote registers (#1468) @ramtinms
• [Admin] add log level customization command (#1362) @smnzhu
• [Misc] Simplify address conversion (#1422) @turbolent
• [Misc] Transaction model status bugfix (#1646) @sideninja

CI & Tests

• [CI] Fix localnet docker build caching (#1712) @simonhf
• [Tests] Unskipping the TestGhostNodeExample tests (#1670) @vishalchangrani
• [CI] Add github action to make a benchstat comparison of a PR with its base (#1579) @janezpodhostnik
• [CI] Flaky test monitor (#1495)(#1500)(#1526) @gomisha @smnzhu
• [Networking] fixes flakey stream closing test (#1600) @yhassanzadeh13
• [Tests] Fix flaky TestUnstakedAccessSuite/TestReceiveBlocks tests (#1551) @peterargue
• [Tests] Add test case for storage used (#1473) @SupunS
• [CI] fixes new line bug on verification integration tests (#1498) @yhassanzadeh13
• [CI] Run CI on PRs targetting version branches (#1443) @jordanschalm
• [Tests] Add more tests for value converter (Storage Migration V6) (#1401) @SupunS

v0.22.10 Mainnet-14

• disable cache metrics (#1518) @zhangchiqing
• removing rlp-encoded private key string to use flow.AccountPrivateKey (#1525) @j1010001
• Transaction fees enable fix - port to master (#1464) @janezpodhostnik
• Backport HotStuff view metric fixes from v0.22-mainnet (#1477) @jordanschalm
• Fixing broken links in access node docs (#1481) @j1010001
• Kan/clean up secondary result index (#1515) @Kay-Zee
• Fix potential mem-leak from hotstuff (#1514) @zhangchiqing
• [Integration] fixes new line bug on verification integration tests (#1498) @yhassanzadeh13
• Update golangci, trying to fix lint (#1509) @Kay-Zee
• Flaky Test Monitor - moving result processing to main repo (#1500) @gomisha
• [Networking] logging dial addresses (#1492) @yhassanzadeh13
• add func to set approved nodes list (#1478) @kc1116
• [Networking] libp2p stream compression (#1349) @yhassanzadeh13
• Run CI on PRs targetting version branches (#1443) @jordanschalm
• [Verification] backports fix on crash loop of verification nodes upon an execution fork (#1416) @yhassanzadeh13
• remove redundant key generation code (#1469) @kc1116
• [Network] Middleware component (#1355) @smnzhu
• [Admin] add log level customization command (#1362) @smnzhu
• [FLIP] Component API (#1167) @smnzhu
• [Consensus] add hotstuff view to ping route (#1462) @zhangchiqing
• Khalil/1583 stake node util func (#1320) @kc1116
• Fix docs links (#1420) @mjudeikis
• [sync-engine, test-only] Model height management in tests of the sync engine (#1399) @huitseeker
• [FLIP] Network API change (#1306) @smnzhu
• Add blockID logging (#1408) @m4ksio
• add '*' default value for --access-node-ids flag for LN/SN nodes (#1426) @kc1116
• Fix Flakey DKG Tests (#1432) @jordanschalm
• implement pull root block and push root block vote (#1374) @smnzhu
• [Admin] simplify admin server implementation (#1364) @smnzhu
• [Bootstrap] split sign and upload into two steps (#1388) @smnzhu
• [Bootstrap] Extracted rootblock and updated finalize (#1371) @durkmurder
• [Consensus] Revert unsealed reason (#1332) @zhangchiqing
• [Consensus and Collection] Refactors guarantee dissemination (#1406) @yhassanzadeh13
• Khalil/5844 access node fallbacks (#1395) @kc1116
• [crypto] Test infra maintenance (#1368) @huitseeker
• Addition of AWS S3 uploader (#1276) @martingallagher
• [Consensus] fix startup time (#1402) @smnzhu
• [Flaky test] peer manager flakiness fix (#1392) @smnzhu
• Enable transaction fees on mainnet (#1360) @janezpodhostnik
• FVM Benchmarking (#1219) @janezpodhostnik
• [Synchronization] update HandleHeight logic (#1387) @smnzhu
• removing timeout when starting an node (#1237) @vishalchangrani
• Generate database encryption keys in bootstrap tooling (#1340) @jordanschalm
• Fix encoding user input in failed transaction message (#1376) @m4ksio
• Add jitter to DKG start and upon processing phase 1 broadcast messages (#1303) @jordanschalm
• fixing the rootblock.json download path (#1385) @vishalchangrani
• [consensus, synchronization] Fix forks preventing reception of blocks at same height (#1377) @huitseeker
• Back port state migration bug fix to master (#1336) @Kay-Zee
• Add Secrets Database (#1309) @jordanschalm
• Define error aware interface (#1275) @smnzhu
• [Network] enable subscription filter except ghost node (#1365) @smnzhu
• fixing makefile and other fixes (#1378) @vishalchangrani
• quarantine flaky test TestStreamClosing (#1351) @AlexHentschel
• Extract Accounts into an interface for easier testing (#1338) @m4ksio
• [Execution] add logger to compactor and only log when we do something (#1350) @ramtinms
• documenting reason for skipping test TestConsensus_LeaderForView (after current epoch) (#1342) @AlexHentschel
• [Consensus] Implement startup-time flag (#1304) @smnzhu
• [Network] improve error logging (#1345) @smnzhu
• Subscription filter (#1210) @smnzhu
• Khalil/async message handling (#1268) @kc1116
• Fixes some build issues when importing flow-go as a dependency (#1333) @jordanschalm
• [Networking] Async DNS Update (#1216) @yhassanzadeh13
• Alex/5857 full fix (#1325) @AlexHentschel
• Reorder middleware error checking code (#1331) @smnzhu
• Return node builder initialization errors instead of throwing fatal errors (#1293) @peterargue
• Example for using irrecoverable and RunComponent (#1327) @peterargue
• fix nil dereference (#1330) @smnzhu
• [Consensus] Log verifier ids for missing approvals (#1324) @zhangchiqing
• FVM cleanup (#1169) @janezpodhostnik
• message validation (#1289) @smnzhu
• removed outdated lingering comment (#1311) @AlexHentschel
• Validate machine account config at startup (#1236) @jordanschalm
• [module] Suggestions for #1275 (#1308) @huitseeker
• Auto Cadence Update: Add limit on call stack depth (#1313) @turbolent
• Admin interface for node operators (#1222) @smnzhu
• [All] Entity-centric tracing (#1243) @ramtinms
• Fix matching engine (#1294) @zhangchiqing
• Update core-contracts version v0.7.7->v0.7.9 (#1278) @jordanschalm
• Added balance reporter (#1154) @janezpodhostnik
• [Consensus] lower emergency sealing threshold (#1301) @zhangchiqing
• performance: cache the ID for Header entities (#1279) @jwinkler2083233
• [FVM] let cadence runtime record generic traces (#1189) @ramtinms
• Auto Cadence Update: Add PublicAccount conversion during InvokeContractFunction (#1298) @turbolent
• Add random delay to periodic peer update thread (#1277) @jordanschalm
• Auto Cadence Update: Fix export errors by properly wrapping in runtime errors (#1287) @turbolent
• Auto Cadence Update: Remove obsolete script/transaction parameter check (#1283) @turbolent
• [Verification] Adds telemetry for outstanding chunk data pack requests (#1290) @yhassanzadeh13
• [Unstaked Access Node] Bootstrapping the unstaked access node from a root snapshot that does not contain node addresses (#1266) @vishalchangrani
• [Collection] Compliance engine message queues (#1270) @durkmurder
• CLI tool to validate machine account config (#1233) @jordanschalm
• Improve error handling in access node (#1280) @peterargue

🛠 Improvements

• Simplify address conversion (#1422) @turbolent

What's Changed

• v0.22 mainnet add hotstuff view by @zhangchiqing in #1470
• Enable fees and prepare nodes to activate fees on mainnet by @ramtinms in #1471

Full Changelog: v0.22.9...v0.22.10

v0.21.2 - Mainnet 13

Mainnet 13

Main contents of this release will be the Full Observer Node, previously refered to as the unstaked access node.

We will also be include many Cadence related updates, requiring a migration, due to needing static type information to be stored for ALL types.

There are also improvements and updates to general performance, the CI/CD and bootstrapping process, as well as further improvements to Epochs, pushing towards full Mainnet roll out.

There are no breaking changes for users (at the Access Layer level).

Full Observer Node

• Clamping sync requests (#1273) @smnzhu
• Fix indexing trie updates (#1272) @m4ksio
• Disable connection pruning for staked AN that supports unstaked AN (#1254) @vishalchangrani
• Topic validator for staked channels (#1225) @smnzhu
• Add block final state commitment (#1239) @m4ksio
• Make the Access node bind to the catch-all address (#1249) @huitseeker
• Refactoring the unstaked consensus follower integration test (#1246) @vishalchangrani
• Consensus follower storage (#1228) @vishalchangrani
• Identity Provider (#1133) @smnzhu
• Fix sync engine bug (#1241) @smnzhu
• Adding some grace time to the TestNoBackoffWhenCreatingStream test (#1234) @vishalchangrani
• Port of block data uploader (#1182) @m4ksio
• Key Generation for Unstaked Access Nodes (#1180) @huitseeker

Cadence

• Add storage migration to inject static types to cadence values (#1264) @SupunS
• Update to Cadence v0.19.0 (#1265) @SupunS
• [Storage Migration] Add a flag to cleanup storage during migration (#1120) @SupunS
• [Storage Migration] Improve static type inferring for arrays and dictionaries (#1245) @SupunS
• [Storage Migration] Remove broken contracts from storage (#1110) @SupunS

Performance

• [Networking] Rolls in DNS caching (#1263) @yhassanzadeh13
• [Performance] Update the loader to use gas limits (#1242) @jwinkler2083233
• [Networking] Adds DNS cache (#1086) @yhassanzadeh13
• [Networking, Crypto] Compressed serialization, key conversions and message signing (#1129) @huitseeker

Epoch and DKG Updates

• Continue building blocks as liveness fallback in case of epoch failure (#1250) @jordanschalm (with contributions of @AlexHentschel)
• Integrate secure GRPC client to DKG and QC clients (#1224) @kc1116
• Remove flags for seeding key generation tool (#1218) @jordanschalm
• Ensure that --machine-account-address flag is valid Flow address (#1226) @jordanschalm
• Use the correct type in NewThresholdSigner (#1217) @jordanschalm
• Improve DKG logging (#1162) @tarakby
• Use account pub key encoding in machine account commands (#1181) @jordanschalm
• Fix Deploy Epoch helper utility (#1113) @jordanschalm
• Handle nil DKG Private Keyshares (#1094) @danuio
• Improve epochs metrics (#1185) @kc1116

🛠 Misc. Improvements

• [Execution, Verification] Increase event size limit for system chunk (#1252) @ramtinms
• [Synchronization] Message queue for collection sync engine (#1248) @durkmurder
• [FVM] backport of rejecting transactions with zero gas limit (#1220) @ramtinms
• [FVM] revert changes to the storage multierror (#1223) @ramtinms
• [Sealing] AssignmentCollectorTree unit tests (#1134) @durkmurder
• [Networking] removing host.Peerstore().ClearAddrs from libp2pNode (#1204) @vishalchangrani
• [Networking] Add retry logic to broadcast and forward message (#1172) @kc1116
• [Core Contracts] Update core-contracts to v0.7.7 (#1171) @jordanschalm
• [Bootstrapping] Add Migration to Update Storage Used (#1033) @janezpodhostnik
• [Bootstrapping] adding secure-rpc-addr to access node systemd file (#1028) @vishalchangrani
• [Execution] Fixing Chunk Data Response Failure (#1135) @yhassanzadeh13
• [FVM] Deduct fees on failed transactions (#1012) @janezpodhostnik
• [Networking] Check the OriginID of a libp2p message corresponds to its authenticated source (#1163) @huitseeker
• [Networking] Check the originID in sendDirect too (#1170) @huitseeker
• [Networking] Skip connecting to invalid identities in the identity table and log error instead of fatal (#1138) @vishalchangrani
• [FVM] Fix checking storage over randomly ordered list (#1156) @janezpodhostnik
• [Execution] Add configurable cache sizes for guarantees and receipts (#649) @m4ksio

CI & Tests

• Added access-tests to local integration tests (#1257) @gomisha
• Activate CI for branches (#1232) @huitseeker
• Detect CPU arch and feed that info to blst (#1227) @huitseeker
• Update check_go_version (#1212) @huitseeker
• Fixing the flaky middleware unsubscribe test (#1173) @vishalchangrani
• Fix blst cross-tests & remove race in CI (#1166) @huitseeker
• New tag and job for blst tests (#1160) @tarakby

v0.20.5 - Mainnet 12

Mainnet 12

Epochs

• Return transaction error to be checked for system chunk transaction (#1096) @kc1116
• Fix bugs with epoch phase & counter metrics (#1090) @jordanschalm
• Fixes bootstrap key Machine Account Key bug (#1092) @danuio
• Epoch Deployment CLI (#985) @danuio
• Update default values for epoch, staking and DKG length (#1063) @kc1116
• Move seed func (#1022) @kc1116
• Export machine account file func (#1015) @kc1116
• Add exported wrapper funcs for random seed util (#1014) @kc1116
• add backoff to dkg broker retry (#1080) @jordanschalm

Access Layer Updates

• Create testable example for splitter / relay engine (#1081) @smnzhu
• Split into separate request and response processing components. (#1068) @smnzhu
• Defining AccessNodeConfig and default config values to allow for non-command line node startup (#1072) @vishalchangrani
• Adding shutdown signal (#1077) @smnzhu
• Implement proxy connection factory (#1024) @smnzhu
• Running an unstaked Access Node (#911) @vishalchangrani
• Implement unstaked conduit (#994) @smnzhu

Performance

• Eject fake random (#945) @jwinkler2083233
• Binstat efficient stats in bins (#891) @simonhf

📖 Documentation

• Update Godoc for ReadyDoneAware (#1032) @smnzhu

Misc

• [Consensus] fix leader selection error (#1148) @zhangchiqing
• [Network] skip returning error is an identity address on chain cannot be parsed (#1136) @vishalchangrani
• [All] Extended events logging port to v0.20 (#1058) @jordanschalm
• [cadence] switching to the private cadence repo (#1082) @vishalchangrani
• [crypto] Add cross-testing of our BLS implementation with that of BLST (#934) @huitseeker
• [Verification] Refactors Chunk Data Pack (#968) @yhassanzadeh13
• [Consensus] Own sized leveled forest tech depth (#1046) @durkmurder
• [Crypto[ Non relic build fix (#1074) @janezpodhostnik
• [Verification] Adds type and value assertion for assigner tests (#1065) @yhassanzadeh13
• [Integration] Fixes loader chain ID (#1084) @yhassanzadeh13
• [Crypto] Remove Relic submodule from the repo (#975) @tarakby
• [All] Making scaffold ReadyDoneAware (#1067) @vishalchangrani
• [Execution] Increase ingestion engine test timeout (#1038) @smnzhu
• [Network] Refactor lifecycle manager (#1031) @smnzhu
• [Network] Making the network middleware peer manager and connection gating optional (#1040) @vishalchangrani
• [Network] Clean up DefaultServerTLSConfig & an unnecessary md5 (#990) @huitseeker
• [Execution] Adding more tests for verifiability of block execution (#969) @ramtinms
• [CMD] add make tool-transit to make file (#981) @zhangchiqing
• [Consensus] Hotfix: Fix synchornization handling response (#1001) @zhangchiqing
• [Network] Sychronize ready and done (#1026) @smnzhu
• [FVM] Simplify storage limit checker code (#950) @ramtinms
• [FVM] Separate script and transaction environments (#939) @ramtinms
• [Network] Fixing race conditions in middleware_test unit test (#1017) @vishalchangrani
• [FVM] Update DefaultStorageMBPerFLOW (#1013) @janezpodhostnik
• [Access Node] Create relay engine (#984) @smnzhu
• [Consensus] Remove legary consensus protocol 'ColdStuff' (#1010) @AlexHentschel
• [Integration] Deploy StakingProxy, LockedTokens and StakingCollection (#997) @danuio
• [Consensus] Caching of non processable approvals (#776) @durkmurder