Testnet4 & Todo List Update

[chjj]

Putting another testnet up. This testnet is primarily for testing the BCH retargetting algorithm and changes that were made to the name/auction state machine.

Major Changes

• The emergency soft-fork safe guards and on-chain mitigations described in Mass Name Revocations (or, "what to do if godaddy gets hacked") #103 are implemented.
• Claimed names no longer expire within the reservation window, but are still revocable.
• We now track the registered and expired state of a name.
• In addition to the above, we track the total number of renewals.
• We no longer clear name data on expiration.
• We now use the BCH retargetting algorithm.
• Keybase has been crawled for Hacker News users older than 1.5 years. These users' keys have been added to the airdrop.
• bcrypto and goosig have been improved a lot. I think we can be a lot more confident in their consensus usage.

These might be the last major consensus changes that will go into HNS, so it's very possible this will be our final testnet. The majority of the changes left are policy-only.

Todo List Update

Updating the todo list from #92.

• Finalize the reserved names list and values. -- Tentatively done.
• Update SSH/PGP keys for the airdrop once again. -- Done.
• Remove government names from the reserved names list. -- On second thought, this appears not to be an issue (Broken README.md documentation links EOM #3).
• Improve UI/API (move some RPC calls into the REST api). -- See wallet: support the creation of unsigned auction txs #110.
• Investigate some potential wallet bugs pertaining to names.
• Write a new cBLAKE miner (hopefully with OpenCL/CUDA support). -- From what I've seen, it looks like several community members are making headway here.
• Fork & modify bstratum to work with HSD. -- See above.
• Revise/improve name record serialization and draw up a specification. -- See A referral-only root zone (almost) #125.
• Decide on which version of the urkel tree to use. -- Issue forthcoming.
• Improve some privacy aspects of the airdrop. This includes padding a recipient's subtree with dummy leaves to conceal the number of keys associated with their github account (thanks to Dan & Riad for noticing this).
• Add more policy rules regarding names.
• Implement neutrino on the consensus layer (at least). If we don't get to this, it can be deployed via softfork later. -- See below.
• Improve peer address gossip.
• Implement a decaying ban score for name resolution, similar to btcd's decaying ban score.
• Reconsider name DoS limits.
• Consider making invalid covenants not update an on-chain state (ethereum-style -- this potentially solves the race condition with OPEN transactions, but makes DoS limit counting annoying). Thanks to Jeremy Rubin for this suggestion. -- See below.
• Experiment with Bitcoin Cash's retargetting algorithm and see how it compares to DigiShield. -- Implemented.
• Switch to bthreads as our worker backend. -- See below.
• Consider running the urkel tree in a separate thread using bthreads. -- Issue forthcoming.
• Get an anycast network ready for public resolvers. -- See below.
• Drop SIG(0) in favor of a cache/proxy-friendly signing mechanism.
• Consider possible "KSK switcharoo" attacks from ICANN given the recent rollover. -- See below.
• Examine and document "BIND consensus quirks". -- See below.
• NPM-less install: start using bpkg for releases. This allows us to distribute signed tarballs instead of relying on NPM. -- I think we're ready to go for this one.
• More tests! Always more tests. -- Never check this box.

Now that a decent amount of time has passed, I think some more thought went into these things:

Neutrino

The more I think about this one, I feel it should be soft-forked in post-mainnet. The perf hit from creating filters for each block needs to be considered more. Unlike bitcoin, we wouldn't be including script data only, but also name data. Since we don't have any real data to work with at this point, it's kind of impossible to test.

Invalid covenants not updating state

I don't think we should do this one as it makes counting for DoS limits nonsensical. The assumption that there is only one action per name per block is an important one for DoS limits and many other things. The race condition is already handled gracefully in the wallet and mempool.

bthreads

We should do this post-mainnet. I'd personally prefer going into mainnet with a the same production ready backend bcoin is using.

Anycast

A few community members seem to have some experience with this and should be able to get something running to start off with. Hopefully in the future, other community members will collaborate to create more public anycast resolvers.

KSK Rollover

This was a concern of mine for a while since ICANN had not published a proper revocation of KSK-2010 immediately after the rollover. Now that they have, a rollback to the old key isn't as likely (they probably couldn't justify it to the public). For mainnet, we will use only KSK-2017. Though, we should consider whether ICANN has the ability to do an emergency rollover to a new key within the next couple years.

BIND consensus quirks

I believe we currently implement these properly. The most major quirk has to do with an unpadded RSA modulus, though I think there are a number of clients that are "out of consensus" with BIND.

---

I think the most major thing I'll be working on personally is a new serialization format for resources (#125). Note that anything DNS-related is essentially policy and can be changed post-mainnet.

Anyway, let the testing begin. I hope to see our first actual name claim this time. If you know anyone with an alexa top 100k name, please encourage them to try it!

[dos1]

I can't seem to be able to connect to any seed nodes?

...
[info] (net) Adding loader peer (172.104.177.177:13038).
[debug] (net) Refilling peers (1/8).
[debug] (net) Connecting to 139.162.183.168:13038.
[debug] (net) Connecting to 172.104.214.189:13038.
[debug] (net) Connecting to 173.255.209.126:13038.
[debug] (net) Error: Socket hangup. (139.162.183.168:13038)
[debug] (net) Error: Socket Error: ECONNRESET (172.104.214.189:13038)
[debug] (net) Error: Socket hangup. (173.255.209.126:13038)
[debug] (net) Error: Socket hangup. (172.104.177.177:13038)
[info] (net) Removed loader peer (172.104.177.177:13038).
[debug] (net) Connecting to 173.255.209.126:13038.
[info] (net) Adding loader peer (173.255.209.126:13038).
[debug] (net) Refilling peers (1/8).
[debug] (net) Connecting to 139.162.183.168:13038.
[debug] (net) Connecting to 172.104.214.189:13038.
[debug] (net) Connecting to 172.104.177.177:13038.
[debug] (net) Error: Socket Error: ECONNRESET (139.162.183.168:13038)
[debug] (net) Error: Socket hangup. (172.104.214.189:13038)
[debug] (net) Error: Socket hangup. (173.255.209.126:13038)
[info] (net) Removed loader peer (173.255.209.126:13038).
[debug] (net) Error: Socket hangup. (172.104.177.177:13038)
...

[tynes]

@dos1 It looks like your node was banned by the network. The error message Socket hangup. makes me think that your peers are destroying their connections with you. By any chance where you running a testnet3 node when the new network rolled out? If so your IP is likely banned.

I'm running a testnet4 node at anowdmaynjdigtmdjfeppmvqxdyu74rvfna6rtfuu7o2mneshubze@35.230.53.109:13038

Try to connect to me with:

$ hsd-rpc addnode anowdmaynjdigtmdjfeppmvqxdyu74rvfna6rtfuu7o2mneshubze@35.230.53.109:13038 add

[tuxcanfly]

@chjj Can you elaborate what this fix is for? Would the fix be a part of the bns which resolves names?

Implement a decaying ban score for name resolution, similar to btcd's decaying ban score.

[tynes]

Implement a decaying ban score for name resolution, similar to btcd's decaying ban score.

From my understanding, what this means is that every time a peer asks for a proof, add to their ban score and as time passes, subtract from their ban score. It will prevent a single peer from asking for too many proofs in a short period of time.

[tynes]

Implement a decaying ban score for name resolution, similar to btcd's decaying ban score.

This is WIP here: #275

[tynes]

Improve UI/API (move some RPC calls into the REST api). -- See #110.

Wallet Auction HTTP Endpoints were merged here: #163

Node HTTP Endpoints WIP here: #168