v0.62.0

The highlights of this release are an improved wallet loading API (2~5x faster) and tools for nodes to run a "side-dag". Other than that, this release makes on step in deprecating sync-v1 by making it harder to enable by accident, along with some fixes and internal improvements.

Features

• feat(event-queue): add new VERTEX_REMOVED event #1093
• feat(side-dag): add ability to update signers list #1075
• feat(side-dag): add signers validation on peer hello #1078
• feat(side-dag): implement PoaBlockProducer #1061
• feat(side-dag): implement Proof-of-Authority #1060
• feat(side-dag): implement gen_keys CLI command #1041
• feat(side-dag): implement side_dag CLI command #1039
• feat(side-dag): implement signer round-robin #1074
• feat(side-dag): update APIs #1071
• feat(wallet): Add a vertex history streamer to the wallet websocket API #1082

Fixes

• fix(cli): regression of --help after side-dag cli #1073
• fix(indexes): efficient rocksdb mempool-tips initialization #938
• fix(logging): make different version logging less obtrusive #1087
• fix(p2p): regression after entrypoint refactor #1088
• fix(p2p): status regression after entrypoint refactor #1089
• fix(performance): using RocksDBMempoolTipsIndex made the sync too slow #1100
• fix(reward-lock): save removed txs in the consensus context #1092
• fix(side-dag): remove init timeout #1070
• fix: add peers api #1052
• fix: improve stratum logs #1049
• fix: profiler reactor initialization #1058

Chores

• chore(cli): mark sync-v1 and bridge as unsafe #1090
• chore(docker): drop support for PyPy #1062
• chore: bump version to v0.62.0 #1095
• docs(security): update link to bug bounty program #1065
• refactor(api): prevent duplicate tx earlier #1006
• refactor(p2p): split hathor.p2p.peer_discovery into separate modules #1055
• refactor(p2p): use Entrypoint type instead of str #1086
• refactor(settings): more settings injection refactors #1085
• refactor(side-dag): general improvements #1080
• refactor(side-dag): refactor node processes management #1079
• refactor(storage): change storages to use injected settings #1081
• refactor: miscellaneous small improvements #1069
• tests(event-queue): add test for invalid tx in mempool #1091
• tests(side-dag): implement missing side-dag related tests #1064

v0.61.0

This is a small release that mostly introduces a change to the block-at-height API to include the transactions (hashes or full metadata) that were confirmed by that block.

Features

• feat(api): Add include_transactions parameter to the BlockAtHeightResource #1046

Fixes

• fix(log): correctly handle errors when logging events #1040

Chores

• chore: bump version to v0.61.0 #1047
• chore: clean up feature activation logs #1035

v0.60.1

This release fixes a known issue in v0.60.0, an update is highly recommended.

Fixes

• fix(resources): regression in address history endpoint #1042
• fix(events): fix initialization on existing event storage #1038

Other changes

• docs(README): update and organize readme and supplementary documentation of the repository #944
• feat(cli): Add --log-vertex-bytes for debugging #1033
• feat(cli): Add load_from_logs cli command to load blocks and transactions as they are found in a log dump #1034
• feat: add --quit-after-n-blocks CLI argument #1031

v0.60.0

This release introduces key updates to the p2p sync, merge-mined blocks, and overall security. We've decided to disable sync-v1 by default, as support for sync-v2 is now widespread and enabling sync-v1 can lead to high CPU usage. However, sync-v1 can still be manually enabled to ensure compatibility with legacy systems.

Additionally, we've implemented an anticipated update to extend the maximum merkle path length for merge-mined blocks on the mainnet. This is a significant improvement for miners dealing with blocks that contain many transactions. Moreover, this release brings important security enhancements, further stabilizing our network and protecting against vulnerabilities.

Known issues

• address-history API returns incomplete results which causes wallets to not see all transactions of some of its addresses (this has been fixed by #1042 and is available in the release v0.60.1)

Features

• feat(cli): add argument to make sync-v1 unavailable #982
• feat(feature-activation): add feature settings validation #894
• feat(feature-activation): automatically enable support in MUST_SIGNAL #963
• feat(feature-activation): implement bit signaling sysctl #962
• feat(merged-mining): configure new max merkle path length on mainnet #1032
• feat(p2p): add ability to update peer_id.json with SIGUSR1 #981
• feat(unrecoverable-error): implement halting of full node execution #809
• feat: add first_block to the to_json_extended method of base transaction class #1003

Fixes

• fix(events): add missing signal_bits to event schema #985
• fix(indexes): Fix wrong order in MemoryTxGroupIndex #977
• fix(util): division by zero when logging progress in some cases #980
• fix: catch update_reward_lock_metadata exceptions #1029
• fix: improve unnecessary alerts #965
• fix: missing PeerId.source_file #1007
• fix: sorting execution manager callbacks #1004

Chores

• chore(cli): default to sync-v1 disabled #1011
• chore: bump version to v0.60.0 #984
• chore: fix codecov coverage upload #1005
• chore: remove unnecessary asserts #968
• docs: fix OpenAPI url of cpu profiler endpoint #979
• refactor(daa): externalize block dependencies #895
• refactor(indexes): Optimize RocksDBAddressIndex to handle pagination in O(log n) #978
• refactor(mypy): add stricter rules to more test modules [part III/VI] #972
• refactor(mypy): add stricter rules to p2p tests [part IV/VI] #973
• refactor(mypy): add stricter rules to some test modules [part II/VI] #971
• refactor(mypy): add stricter rules to specific modules [part I/VI] #969
• refactor(mypy): add stricter rules to unittest and utils [part V/VI] #974
• refactor(mypy): unify hathor and test modules strictness [part VI/VI] #975
• refactor(verification): reorganize min_height verification #1008
• refactor(vertex-handler): modularize methods [part 2/2] #988
• refactor: create VertexHandler [part 1/2] #987
• refactor: create non-null hash property #967
• refactor: remove SimpleMemoryStorage and refactor DAA #1026
• refactor: remove deprecated resource method from address history API #994
• tests(verification): add missing assertions #983

v0.59.0

This release introduces sync-v2 enabled by default alongside sync-v1, ensuring broader compatibility by using the highest common sync version per connection. For specialized use cases, the release adds --sync-v2-only and --sync-v1-only parameters, allowing exclusive use of the desired sync version based on specific network needs or compatibility considerations.

Beyond the sync protocol, this update brings smaller fixes and features across the board. Key highlights include testing the feature activation mechanism on the mainnet and experiments with increased max merkle-path length on the testnet.

To know how to upgrade your full node, see How to upgrade Hathor full node.

Features

• feat(cli): Add feature flag and CLI parameters for Nano Contracts #941
• feat(cli): Use SIGUSR2 to run sysctl commands #949
• feat(cli): mark sync-v2 parameters as safe and deprecate --x-* ones #935
• feat(debug): make it possible to connect a remote ipython shell #900
• feat(feature-activation): configure NOP features on mainnet #908
• feat(feature-activation): configure new NOP features on mainnet #958
• feat(feature-activation): decrease mainnet evaluation interval to 1 week #903
• feat(feature-activation): update NOP features config on mainnet #929
• feat(merged-mining): configure new max merkle path length on testnet #910
• feat(sync-v2): Add mempool sync status #946
• feat(sync-v2): Improve logging when unexpected TRANSACTIONS-END or BLOCKS-END are received #945
• feat(sysctl): Add core module with profiler commands to sysctl #951
• feat(sysctl): Add p2p.kill_connection to sysctl #950
• feat(sysctl): Add sysctl tools for debugging #955

Fixes

• fix(cli): Fix issue when running sysctl through USR2 in macOS #956
• fix(events): fix events simulator CLI #907
• fix(feature-activation): add missing metadata migration #927
• fix(feature-activation): fix get ancestor #930
• fix(p2p): fix update whitelist handler #885
• fix(reactor): fix reactor initialization on CLI tools #906
• fix: Fix slow tx processing on some edge cases #957
• fix: health endpoint when using --x-asyncio-reactor #940
• fix: test broke after #951 #953

Chores

• chore(cli): make sync-bridge the default #959
• chore(deps): Upgrade hathorlib to v0.5.2 #939
• chore(feature-activation): improve phased testing logs #905
• chore(feature-activation): phased testing adjustments #931
• chore: add support for Python 3.12 #844
• chore: bump version to v0.59.0 #934
• refactor(cli): improve handling of sync version from arguments #943
• refactor(feature-activation): remove enable_usage #909
• refactor(settings): remove some calls to HathorSettings #920
• refactor(settings): remove some more calls to HathorSettings #923
• refactor(settings): rename get_settings to get_global_settings #919
• refactor(storage): create StorageProtocol #922
• tests(event): fix missing hash update #921
• tests(p2p): improve protocol test #918
• tests: fix CI regression on Windows + Python 3.10 #926

v0.58.0

This is release has a large number if internal changes, some fixes and some important advances in the implementation of the sync-v2 protocol, which has been stable in our tests but is still disabled by default in this release.

To test sync-v2 you need to enable unsafe mode for the network that your node is using: --unsafe-mode=mainnet or --unsafe-mode=testnet-golf, and use one of the two options: --x-sync-bridge (for sync-v1 and sync-v2 support, sync-v2 is used when both nodes support it) or --x-sync-v2-only (will not sync with nodes that don't support sync-v2).

Features

• feat(feature-activation): configure new NOP features on testnet #879
• feat(feature-activation): implement must signal #785
• feat(reactor): implement option to use asyncio reactor #889
• feat(reliable-integration): change ordering to send voided txs first #876
• feat(reliable-integration): update peer metadata schema #804
• feat(simulator): Add StopWhenSendLineMatch trigger #847
• feat(sync-v1): remove SendDataPush queue #892
• feat(sync-v2): Add both BlockchainStreamingClient and TransactionStreamingClient to manage streamings from the client side #848
• feat(sync-v2): Always relay vertices in real-time if the peer requested it #861
• feat(sync-v2): Fix issue when a reorg occurs during a streaming of transactions #855
• feat(sync-v2): Implement new protocol for synchronizing transactions #850
• feat(sync-v2): Improve overall logging #854
• feat(sync-v2): Improve sync-v2 reliability #843
• feat(sync-v2): Stop running callLater when a block is voided in handle_get_peer_block_hashes() #873
• feat(sync-v2): Stop streaming of transactions if an unexpected vertex is received #856
• feat(sync-v2): Wait for sync internal methods to finish before initiating next syncing cycle #845
• feat(sync-v2): Watchdog to detect stale syncing #857
• feat(sysctl): add controls for enabling/disabling sync versions #869
• feat[healthcheck]: new health endpoint #807

Fixes

• fix(feature-activation): add missing metadata migration #828
• fix(mining): Fix block template not checking for MAX_FUTURE_TIMESTAMP_ALLOWED #822
• fix(mining): Wrong timestamp sent to _make_block_template #823
• fix(pubsub): Event queue becoming too long #886
• fix(storage): Fix wrong update to the best_block_tips_cache #874
• fix(sync-v2): Fix dependencies update for the first block #897
• fix(sync-v2): Fix issues caused by concurrent syncing peers #872
• fix(sync-v2): Fix n-ary search to handle reorgs during its execution #842
• fix(sync-v2): Fix stream end messages for both blockchains and transactions #870
• fix(sync-v2): Fix streamer sending more blocks than expected #839
• fix(sync-v2): Fix unhandled case caused by reorg during sync of blocks #827
• fix(sync-v2): adjust init assertion #875
• fix(sync-v2): do not consider existing vertex as unexpected #866
• fix(sync-v2): stop _process_transaction on error #877
• fix(sysctl): Fix deserializer #820
• fix(verification): avoid validation downgrade by not running it twice #853
• fix: add health endpoint to openapi #891
• fix: remove unused NOP features #901

Chores

• chore(ci): use cancel-in-progress to prevent useless CI runs #818
• chore(custom-checks): add check to prevent circular imports #887
• chore(custom_checks): Skip binary files when checking for deprecated typing #846
• chore: bump version to v0.58.0 #860
• chore: refactor python healthcheck lib #826
• chore: update dependencies #840
• refactor(cpu-mining): create CpuMiningService #803
• refactor(daa): remove global daa test mode #802
• refactor(p2p): have two internal tiers for sync: available and enabled #867
• refactor(p2p): use async/await syntax on peer discovery #883
• refactor(reactor): remove module-level global reactor #888
• refactor(scripts): implement ScriptContext #851
• refactor(scripts): modularize script files #811
• refactor(settings): change daa to use injected settings #801
• refactor(sync): remove legacy sync-v1.0 #864
• refactor(sync-v2): Modify TransactionStreamingClient to process vertices asynchronously #865
• refactor(sync-v2): Refactor sync_v2 agent to hold (block_height, block_id) information in an internal namedtuple #829
• refactor(verification): finish moving all verification methods [part 5/5] #800
• refactor(verification): implement verify_minted_tokens [part 3/9] #832
• refactor(verification): move block-only verification methods [part 3/5] #798
• refactor(verification): move token info [part 2b/9] #858
• refactor(verification): move transaction-only verification methods [part 4/5] #799
• refactor(verification): move verification methods signatures [part 2/5] #797
• refactor(verification): organization and typing improvements [part 1/9] #830
• refactor(verification): refactor verify_sum [part 2/9] #831
• refactor(verification): remove BlockVerifier inheritance [part 4/9] #833
• refactor(verification): remove TransactionVerifier inheritance [part 5/9] #834
• refactor(verification): remove VertexVerifier from BlockVerifier [part 7/9] #836
• refactor(verification): remove VertexVerifier from TransactionVerifier [part 8/9] #837
• refactor(verification): remove VertexVerifier inheritance [part 6/9] #835
• refactor(verification): remove token info duplication [part 9/9] #868
• refactor: move simulator utils functions #772
• test(simulation): Fix flaky test test_many_miners_since_beginning #815
• test(sync): Improve test_sync_rotate to prevent false positives #816
• test(sync-v2): Fix seed in some sync-v2 tests while it's under testing #817
• test: Use INFO as default log level to reduce amount of logs #814
• tests(reliable-integration): remove timestamp assertion from event scenarios test #841
• tests(verification): implement tests to improve refactor reliability #824
• tests: Fix a flaky protocol test #849
• tests: fix flaky log test #821

v0.57.0

This release contains a lot of internal restructuring along with important p2p connectivity improvements. Also support for Python 3.9 has been dropped.

Features

• feat(p2p): Add ping salt and improve rtt information #780
• feat(p2p): relay all recently-seen peers for neighbors #782
• feat(p2p): run periodic bootstrap discovery #783
• feat(reliable-integration): add decoded outputs in event responses #763
• feat(reliable-integration): emit VERTEX_METADATA_CHANGED events in reverse topological order #776
• feat(reliable-integration): implement stream ID #766
• feat(wallet): Add BaseWallet.get_balance_per_address() method #778

Fixes

• fix(feature-activation): add signal bits to minimally valid block #774
• fix(sync-v2): Stop streaming if current block becomes voided #810

Chores

• chore(ci): fix codecov main branch name #793
• chore(ci): upload codecov results on Python 3.11 test job #792
• chore(python): drop support for Python 3.9 #777
• chore: bump version to v0.57.0 #790
• chore: improve push_tx log #806
• refactor(settings): change HathorManager to use injected settings #787
• refactor(settings): refactor genesis module #794
• refactor(verification): create vertex verifiers [part 1/5] #796
• refactor(verification): move vertex verification to its own service #779
• refactor: move get settings [part 1] #768
• refactor: move get settings [part 2] #769
• refactor: move get settings [part 3] #770
• refactor: use Hathor simulator on events simulator CLI #758
• test(sync-v1): Reset rate limit hit counter after simulation #812
• tests: decrease tests duration #756

v0.56.0

This release contains important fixes and also starts using the feature activation system on the testnet so we can gather initial testing data.

Features

• feat(feature-activation): add logging of NOP features #760
• feat(feature-activation): configure NOP features on testnet #761
• feat(feature-activation): implement signal bits in mining blocks #703
• feat: add best_blockchain in /status payload #704

Fixes

• fix(event-queue): improve memory usage during the load phase #744
• fix(sync-v2): compare to local bytes broke in some cases #747
• fix(tests): make simulator test not flaky on sync-v2 #750
• fix: imports #762

Chores

• chore(event-queue): update event endpoints visibility to public #746
• chore(feature-activation): update feature endpoint visibility to public #743
• chore: add deprecation warning for Python 3.9 #759
• chore: bump version to v0.56.0 #757
• chore: improve merkle path log #745
• chore: improve miner simulator flakiness #753
• chore: improve pubsub #694
• refactor(event-queue): improve event ordering #738
• refactor(sync-v2): rename SyncAgent module from manager to agent #748
• refactor: extracts SysctlRunner from SysctlProtocol #739

v0.55.0

This release contains several behind the scene improvements that will mostly affect future features that aren't in use yet. Notably there is good progress on the feature activation mechanism and also a disabled by default preview version of sync-v2.

Features

• feat(events): add event simulator scenarios and tests #609
• feat(feature-activation): bit count optimization #633
• feat(feature-activation): block features endpoint #690
• feat(feature-activation): fix criteria validation #708
• feat(feature-activation): implement bit signaling service #702
• feat(feature-activation): implement block ancestor optimization #644
• feat(feature-activation): implement caching mechanism #645
• feat(feature-activation): implement feature service #639
• feat(feature-activation): implement get endpoint #634
• feat(feature-activation): implement signal_bits field in txs #623
• feat(feature-activation): new MUST_SIGNAL and LOCKED_IN states #675
• feat(indexes): add get_n_height_tips method to height index #734
• feat(p2p): add setting to enable/disable peer whitelist for sync-v2 #735
• feat(sync-v2): sync-v2 implemented, sync-v1 still default #275
• feat(traversal): add BFS order traversal #672
• feat: add get-best-blockchain capability #686

Fixes

• fix(event-queue): fix event simulator scenario parsing #706
• fix(events): fix bugs in the event stream and improve tests #647
• fix(metadata): actually use None for unset height #671
• fix(simulator): Patch BaseTransaction.resolve() to prevent updating timestamp #641
• fix(storage): incorrect validation store when using cache storage #663
• fix(test): regression after sync-v2 command was removed #730
• fix(tests): remove undeeded broken tests #668
• fix: consensus-fail-id marker should not be treated as a tx-id #695
• fix: send tips rate limit disconnect #650
• fix: settings initialization #666

Chores

• chore(events): move event queue option to unsafe args #656
• chore(extras): script to generate list of PRs that master is ahead of rc #701
• chore: add peer connecting event #693
• chore: bump version to v0.55.0 #661
• chore: deprecate set output #689
• chore: improve progress function #651
• chore: p2p rate limit call later max limit #657
• chore: remove --x-fast-init-beta CLI option #643
• chore: update autobahn package #713
• chore: update checkpoints #687
• chore: update default Python version to 3.10 #712
• chore: update mypy, mypy-zope, and pydantic #691
• docs: add PR templates for the new release process #674
• refactor(builder): Build IndexesManager on builders #640
• refactor(events): improve events during the load phase #652
• refactor(init): Remove duplicate code and simplify full verification method #662
• refactor(p2p): rename sync manager to sync agent #729
• refactor(sync): move sync-v1 files to a separate module #667
• refactor(sync-v2): improve deferreds handling #732
• refactor: Add common Hathor types #642
• refactor: Change HathorManager.on_new_tx() to accept only fully validated transactions #664
• refactor: Prepare for sync-v2 #697
• refactor: do not use built-in random and add proper linter check #698
• refactor: implement typed RunNodeArgs #636
• refactor: improve manager building on simulator #653
• refactor: migrate all type annotations to PEP 585 #654
• refactor: move ValidationState to its own file #635
• refactor: remove deprecated typing uses and add custom linter script #699
• refactor: use None by default for height metadata #665
• tests(events): migrate flaky event simulation tests to new format #649
• tests(feature-activation): implement reorg test #660
• tests: Do not update time by default on BaseTransaction.resolve() #669
• tests: Skip flaky event tests #646
• tests: implement new PubSub tests #696

v0.54.0

This release drops support for Python 3.8, introduces a new internal settings format in YAML (the old format is still supported in this release but will be removed in the future), and changes the default values for WS_MAX_SUBS_ADDRS_CONN and WS_MAX_SUBS_ADDRS_EMPTY (which was commonly customized by several deploys, such that now they won't need to customize it anymore). Other changes include mostly the groundwork for future features.

Features

• feat(events): reliable integration during the load phase #555
• feat(events): reset event queue command #608
• feat(feature-activation): implement basic structures #611
• feat(sysctl): Add helper command #620
• feat(ws): Change default configuration of WS_MAX_SUBS_ADDRS_CONN and WS_MAX_SUBS_ADDRS_EMPTY to infinite #625
• feat: Add safeguards for partially validated transactions [part 1/2] #577
• feat: add safeguards for partially validated transactions [part 2/2] #617

Chores

• chore(bump): v0.54.0 #567
• chore(python): drop support for Python 3.8 #615
• chore: add yamllint to all files #598
• chore: implement yaml settings inheritance #618
• chore: python 3.9 updates #616
• refactor(builder): Split resources creation from CliBuilder to ResourcesBuilder #605
• refactor(p2p): Build ConnectionsManager in builder #622
• refactor(settings): make HathorSettings always return the same instance #626
• refactor(stratum): Build StratumFactory on builder #619
• refactor: move hathor settings to yaml configuration files [part 1/2] #593