Support multiple algorithms for choosing a list of nodes

[janaakhterov]

Current behavior:

To execute a request a list of nodes account IDs must be set on the request. This list of nodes could set by the user or a list of nodes will be picked for the user from the client's network. Whenever the network picks the nodes for the request it will order the current list of nodes based on the node's "health" which is determined by when it was used last and the amount of times the node has been used. In both Java and Go the Network must be thread safe since the user has an option to make concurrent requests. To acheive thread safety the network has lock it uses whenever it sorts the list of nodes. Although the current behavior works fine for 99% of test cases, it's a huge issue for performance. Users should have the option to use different algorithms which are more performant.

Notes:

• As of Client.pingAll() being implemented the network now not only sorts the list of nodes, but also removes nodes which have surpassed max attempts.
• A user can overwrite the network list while executing requests.

Desired behavior:

When constructing the Client the user could use some method Client.setNetworkNodeChoosingAlgorithm() to set which algorithm they'd like to use.

Proposed Algorithms:
- Performance: Use a random number generator to pick a list a nodes
- RoundRobin (default): Current implementation
- None: Require a user to set the list of nodes for each request themselves

Note:

• Using Performance as algorithm name might not be the best idea since we might eventually implement some other algorithm which has better performance than the current Performance algorithm. At that point would we simply change what Performance does, or would we add another algorithm option to allow users to pick between the older performance algorithm and the new one?

[valtyr-naut]

Calling it Performance will definitely cause growth issues down the line. Something like Random or RNG is probably better. It might also be possible to implement a single nn algorithm to eval nodes on specific criteria.

[valtyr-naut]

nn = neural net. a linear weighted algo

[mehcode]

client.setNetworkBalancingAlgorithim(NetworkBalancingAlgorithim.RANDOM);

See: https://www.jscape.com/blog/load-balancing-algorithms

[tomzhenghedera]

Performance definition may need to be clarified further. Feels like it wants to use a "faster" algorithm than Round Robin to prepare the list of nodes.

None seems to provide the passive way to allow the client submitting a request to the network in fastest possible way bypassing potential sdk overheads, potentially with re-sorting list of nodes plus locking required via using a mutex. With the understanding that the client will need to be responsible for error handling/recovery. This also enables any application wants explicit control over node(s) submitting the requests to. Will benefit some diagnostic scenarios that I can think of currently.

For the current Round Robin and future Performance algorithm, I assume re-sort/re-build of the list of (healthy)nodes will always check the entire list of network nodes. This is to make sure in case of node reconnect, a node could be temporarily unavailable for few hours, but would need to be included back to the list of healthy nodes whence it is being reconnected back to the network.