Improve spec and implementation effort

[liamsi]

In my opinion, the specification and implementation feedback loop are far from optimal. In some parts the specification brushes over the reality of the implementation process and in some cases the implementation takes shortcuts that are not really aligned with the spec.

Let's discuss here how an ideal process could look like and how we could get close to achieving that ideal.

Here are some suggestions of what I think we should do:

• focus on the parts that are new and not really specified (or even implemented) already somewhere else (these can be treated as a black-box that just work™); of course this isn't always possible and there is definitely merit in repeating specification when there are minor changes but then we should focus and clearly explain the concrete diff
• the spec should be written in a way that embraces the implementation choices made (tendermint and Cosmos-SDK where it could save us time)
• if the specification changes something to core tendermint types, there must be an explanation why these changes were made and how these will be used (in a non-ambiguous way, e.g. pseudo code); also we should only merge PRs to the spec, when we opened an issue explaining what changes need to happen in lazyledger-core (similar to what @marbar3778 described here: specs -> impl #139)
• if the specification changes anything that touches on the ABCI boundary (e.g. fees), then this should only be merged if there was an extensive discussion on how to implement this first: ideally, following the now established ADR process
• if the implementation implements something that hasn't made it into the specification yet, we need to make sure, this is a) properly captured in an ADR and b) the specification catches up quickly with the non-implementation specific details
• engineers should also contribute to the spec (related to the point above)

I think if we implement all these, we will fly and ship much faster. Any thoughts?

[liamsi]

Context is the whole discussion here: celestiaorg/celestia-core#305

[liamsi]

I would add that the spec should be deployed somewhere (rendered) and should be easily searchable!

[adlerjohn]

This was resolved with #165.