Unify engines #3405
Open
Unify engines #3405
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yacovm
force-pushed
the
unifiedEngine
branch
19 times, most recently
from
788480d
to
9d6c351
Compare
yacovm
force-pushed
the
unifiedEngine
branch
4 times, most recently
from
92df2c8
to
fd73ebb
Compare
The lifecycle of an avalanchego node can be separated to three main stages: Stage I - If supported by the VM, then a state sync occurs. Otherwise, this stage is skipped. In the X-chain, this stage is entirely substituted by a custom bootstrapping phase that replicates and executes the X-chain DAG. Stage II - A stage called bootstrapping, which entails replicating the blocks from other avalanchego nodes, and executing them. Stage III - The final form of an avalanchego node, in which it runs the snowman consensus protocol. The phases are implemented by components called "engines": - avalanche bootstrapping - snowman - state syncer - snowman bootstrapper And the handler in snow/networking/handler/ is responsible to route messages from the network into the correct engine. Engines all implement the same common.Engine interface, but the interface consists of the union of all operations across all engines. Indeed, it is often the case that a message of type `m` dispatched by engine `e`, cannot be processed by a different engine `e'. For instance, a Chits message cannot be processed by any engine other than consensus, and a message about a query of state summary is only relevant to the state sync engine. To that end, each engine simply implements a no-op dispatcher for messages it does not care about. The biggest problem with the existing aforementioned structure, is that the lifecycle of the engines imposes a strict one way movement across the various stages, and there is no single component which consolidates the transition among the stages. The movement between the various stages takes place by a callback given to each engine at every stage but the final one (Stage I and Stage II). The structure therefore makes it difficult to introduce movement from snowman consensus back to bootstrapping / state sync, or to have better control over the message dispatch. This commit unifies all engines into a single one under snow/engine/unified As a result, the implementation of the handler in snow/networking/handler/handler.go is now simpler, as it only interacts with the unified engine, and it never needs to query the snow.EngineState. The state transition between the various stages is now taken care of by the unified engine, and since the code to dispatch messages to the right engine is now all in the unified engine, it's not only more testable but it lets us move among the stages in the same place where we consider the stage we're in to dispatch the message to the correct engine. Signed-off-by: Yacov Manevich <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why this should be merged
The lifecycle of an avalanchego node can be separated to three main stages:
Stage I - If supported by the VM, then a state sync occurs. Otherwise, this stage is skipped.
In the X-chain, this stage is entirely substituted by a custom bootstrapping phase that replicates and executes the X-chain DAG.
Stage II - A stage called bootstrapping, which entails replicating the blocks from other avalanchego nodes, and executing them.
Stage III - The final form of an avalanchego node, in which it runs the snowman consensus protocol.
The phases are implemented by components called "engines":
And the handler in snow/networking/handler/ is responsible to route messages from the network into the correct engine.
Engines all implement the same common.Engine interface, but the interface consists of the union of all operations across all engines.
Indeed, it is often the case that a message of type m dispatched by engine e, cannot be processed by a different engine e'.
For instance, a Chits message cannot be processed by any engine other than consensus, and a message about a query of state summary is only relevant to the state sync engine.
To that end, each engine simply implements a no-op dispatcher for messages it does not care about.
The biggest problem with the existing aforementioned structure, is that the lifecycle of the engines imposes a strict one way movement across the various stages,
and there is no single component which consolidates the transition among the stages. The movement between the various stages takes place by a callback given to each
engine at every stage but the final one (Stage I and Stage II).
The structure therefore makes it difficult to introduce movement from snowman consensus back to bootstrapping / state sync, or to have better control over the message dispatch.
This commit unifies all engines into a single one under snow/engine/unified
As a result, the implementation of the handler in snow/networking/handler/handler.go is now simpler,
as it only interacts with the unified engine, and it never needs to query the snow.EngineState.
The state transition between the various stages is now taken care of by the unified engine, and since the code to dispatch messages to the right engine
is now all in the unified engine, it's not only more testable but it lets us move among the stages in the same place where we consider the stage we're in to dispatch
the message to the correct engine.
How this works
Just a refactoring: I shrinked the interfaces of the engines to what is absolutely required and now only the unified engine implements the common.Engine. Removed un-necessary code from the handler package and moved the logic into the unified engine.
How this was tested
CI.