Skip to content
This repository has been archived by the owner on May 10, 2021. It is now read-only.

Implement routing table computation & sketch routing #2

Open
wants to merge 13 commits into
base: handshake
Choose a base branch
from

Conversation

cache-nez
Copy link
Collaborator

@cache-nez cache-nez commented Nov 27, 2020

This PR introduces the following changes:

  • defines a Router (implementing dela's router.Router). Its methods GenerateTableFrom(handshake) and New(players, thisAddress) call NewTable, which constructs the routing table from the addresses of other participating nodes and the id of the current node. Once a routing table is computed, it is stored in the router and returned in all future calls to GenerateTableFrom (however, New constructs a new routing table).

  • implements NewTable. This function traverses shuffled addresses and if the prefix of the given address does not appear in the routing table yet, this address becomes the next hope for this prefix.

  • implements non-resilient routing. Forward simply chooses the next hop from the routing table (since all nodes are known in advance, it's guaranteed to have a next hop per destination), with one important exception: the orchestrator's client side, which is the destination of players' messages, is not participating in a protocol. The only way to reach it is to send the messages to the orchestrator's server side and route them to nil to reach the client side. Therefore, the id of the client and the server side of the orchestrator is the same and the only case the message is routed to nil is when it's routed to this node's id but not this node's address.

id/mod.go Outdated
}

func BaseAndLenFromPlayers(numPlayers int) (base byte, len int) {
return 16, 3
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A dummy implementation. This method is where the id space computation happens, the first actual implementation will probably be one for hashed ids.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Different approaches to this computation to be explored in a separate PR.

Comment on lines 31 to 78
func (h Handshake) Serialize(ctx serde.Context) ([]byte, error) {
format := hsFormats.Get(ctx.GetFormat())

data, err := format.Encode(ctx, h)
if err != nil {
return nil, xerrors.Errorf("encode: %v", err)
}

return data, nil
}

// - implements router.HandshakeFactory
type HandshakeFactory struct {
addrFac mino.AddressFactory
}

// NewHandshakeFactory creates a new factory.
func NewHandshakeFactory(addrFac mino.AddressFactory) HandshakeFactory {
return HandshakeFactory{
addrFac: addrFac,
}
}

// Deserialize implements serde.Factory. It populates the handshake if
// appropriate, otherwise it returns an error.
func (fac HandshakeFactory) Deserialize(ctx serde.Context, data []byte) (serde.Message, error) {
return fac.HandshakeOf(ctx, data)
}

// HandshakeOf implements router.HandshakeFactory. It populates the handshake if
// appropriate, otherwise it returns an error.
func (fac HandshakeFactory) HandshakeOf(ctx serde.Context, data []byte) (router.Handshake, error) {
format := hsFormats.Get(ctx.GetFormat())

ctx = serde.WithFactory(ctx, types.AddrKey{}, fac.addrFac)

msg, err := format.Decode(ctx, data)
if err != nil {
return nil, xerrors.Errorf("decode: %v", err)
}

hs, ok := msg.(Handshake)
if !ok {
return nil, xerrors.Errorf("invalid handshake '%T'", msg)
}

return hs, nil
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All methods are exactly the same as in dela's tree.Handshake, just the handshake content is different.

@cache-nez cache-nez changed the base branch from id-impl to handshake December 20, 2020 16:22
…of a handshake.

* Define a handshake, whose content will be used to initialize a routing table
  on non-orchestrator nodes

* Implement serialization and deserialization of a handshake. This is similar
  to dela's implementation, and the content is changed to represent our
  handshake's content.
This commit introduces the following changes:

* defines a Router (implementing dela's router.Router). It contains no routing
  information. Its methods GenerateTableFrom(handshake) and New(players, thisAddress)
  call NewTable(addresses, thisId), which constructs the routing table from the
  addresses of other participating nodes and the id of the current node.

* implements NewTable. This function traverses shuffled addresses and if the
  prefix of the given address does not appear in the routing table yet, this
  address becomes the next hope for this prefix.

* sketches the routing. In this prototype, RoutingTable.Forward simply chooses
  the next hop from the routing table (it assumes the next hop is always in the
  table). OnFailure does nothing.
A node can receive multiple handshakes and we shouldn't recompute the routing
table on each handshake. To keep the first computed routing table:
* Change all router's methods to have a pointer receiver
* Store the routing table in the router when it's first computed
* If a routing table is already computed when a node receives a handshake,
  do not recompute the table
…uting table.

When the node, booting the protocol, calls `Router:New` to create a routing table,
the node's address is not included in the set of players' addresses, passed to the
method. However, the node *is* a player and should be included in the routing.

This is enough to make all other players aware of this node, because the same array
of addresses, where the booting node is now included, is passed in a handshake
to any new node.
the destination address is not equal to this node's address.

This is a hack to accommodate for minogrpc's design, where the only way to reach
the client side of the orchestrator is routing a message to nil from the server
side of the orchestrator.
// OnFailure implements router.RoutingTable. It changes the next hop for a
// given destination because the provided next hop is not available.
func (t RoutingTable) OnFailure(to mino.Address) error {
// TODO: keep redundancy in the routing table, use the alternative hop
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be implemented in a separate PR.

@cache-nez cache-nez marked this pull request as ready for review January 8, 2021 13:22
@cache-nez cache-nez requested a review from gnarula January 8, 2021 13:23
Copy link
Contributor

@gnarula gnarula left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, a minor change I'd suggest is to replace fmt.Errorf with xerrors.Errorf

func (t RoutingTable) OnFailure(to mino.Address) error {
// TODO: keep redundancy in the routing table, use the alternative hop
// and mark this node as unreachable
return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return nil
return xerrors.Errorf("TODO")

}

func randomShuffle(addresses []mino.Address) {
rand.Seed(time.Now().UnixNano())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this can be moved to init() so that the Seed is initialised only once

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants