Server Version Toggling

[amcgee]

This is a placeholder for specification of the version toggling feature

Proposal:

1. App shell provides a React context which fetches the server version before rendering the application
2. Apps can use a <ServerVersionSwitch> and <ServerVersionCase> to render different components based on the server version

<ServerVersionSwitch>
    <ServerVersionCase version=">=2.34">
        <ModernComponent />
    </ServerVersionCase>
    <ServerVersionCase > {/* fallback for older versions */}
        <LegacyComponent />
    </ServerVersionCase>
</ServerVersionSwitch>

The modern and legacy components could also be React.lazy with dynamic imports to reduce the bundle size of the application.

[paschalidi]

I was thinking of the api toggling or feature toggling I dont know what is the best name for it yet.
I saw how it is being applied on the reports-app and messaging-app. Specifically in the messaging-app. I looked there after @HendrikThePendric prompted me to it.

After talking with @HendrikThePendric, reading though others apps that currently are “togglers” and simply thinking of how this would work on a bigger scale I feel is a good idea over all and as it has been mentioned it will definitely decrease our effort in maintaining the 4 different versions and also help us release in a more lean fashion.

I also am seeing how this would work with an API that is being extended. In other words I can imagine how this would work when from v1 of the API we jump to the v2 where a feature such as “Search” functionality is being enabled.
We see that in here and is I would say straight forward to the extend I can imagine for now at least.

The place I am having hard time to imagine is on when the api has breaking changes sort to say. Such as renaming of an object key value or similar.

I would like to introduce a concrete example where you might help me understand our strategies better or simply fire off a discussion.

The Birthday Example

v1

Lets say we having /api/v1/... and /api/v2/... . Also let's imagine we have a Search feature. In the search feature we fetch some data and then we simply use a Card component to displays information of the users being returned.
An endpoint that returns user information on v1 API would be /api/v1/user . And the object it will return might be the following.

{ 
  firstName: "John",
  lastName: "Johnson"
}

The frontend application consumes the response and displays the first name and last name on the UI using the Card component.

v2

Now in the v2 of the API we starting support a new “feature”. The feature is that the Card displays also the the date of birth. Now from the v2 API endpoint /api/v2/user we are getting:

{ 
  firstName: "John",
  lastName: "Johnson",
  dateOfBirth: '12.5.1990'
}

Perhaps in the In the frontend we would say something like this. Pseudocode alert. 🚨

<ServerVersionSwitch>
    <ServerVersionCase version=">=v2">
        <SearchPage>
             # imagine there is a service that fetches the api and returns those values as `props` in the `SearchPage` component.
             # (look at [1] for service definition)
            <Card firstName={props.firstName} lastName={props.lastName} dateOfBirth={props.dateOfBirth}/>
        </SearchPage>
    </ServerVersionCase>
    <ServerVersionCase > {/* fallback for older versions */}
       <SearchPage>
            <Card firstName={props.firstName} lastName={props.lastName} />
        </SearchPage>
    </ServerVersionCase>
</ServerVersionSwitch>

v3

Finally we are having a v3 of the API.
Imagine now from the v3 API endpoint /api/v3/user we are getting:

{ 
  firstName: "John",
  lastName: "Johnson",
  birthDate: '12.5.1990' # changed name
}

Now we are having a more complex problem to solve by having the same API endpoint returning different keys of objects between v2 and v3 versions.

How would we solve this kind of situation gracefully?
I am afraid that we might end up with complex services (look at [1] for service definition) with multiple if statements depending on the API version.

[1] What I mean by service is a function that only fetches and returns a promise. Something between these lines.

const getUser = () => {
    return api.get('user')
}

Perhaps very simplistically put but I hope one can get the idea.

---

Maybe to answer myself one idea that comes into my head is this concept of backend for frontend (BFF) that I have been hearing some time ago. Several conference talks about it. Specifically one by some Netflix people that I could dig up is this one.
BFF is basically a server that is maintained from the FE people (at least on some organisations) and can be imagined as a layer that sanitises the RESTful API. Simplistically saying it again. For the ones who are interested in more in depth reads
and definitions of the concept there are some good reads here. For those who like images the SoundCloud blog post would be a good read in a glance.

But I am more interested in hearing other people's thoughts than on my own.

[HendrikThePendric]

Some great ideas above 🎉 . I'll first add a few points that relate to those and then also add some considerations/ideas of my own..

Toggling using Components VS helper functions

I really like the idea of using components for doing this. For me the biggest advantages would be a) semantics, and b) the ability to lazy load these and reduce bundle size.

On the other hand, I feel like having some helper functions in place would make things a bit more flexible since you could use these not only in JSX but elsewhere too. Possibly we could make these components very simple and extract most of the logic in some hooks, which we could then also expose.

Probably using components + hooks will provide enough flexibility. The only thing is that quite a lot of apps I know have some dedicated js module that abstracts API calls into functions, and then these functions are called from redux action-thunks. Since hooks are only meant to be executed from a component or from another hook, this will probably produce some challenges. Probably nothing major though.

Using version numbers VS functionality-names to toggle

My current approach for feature toggling has always been to have one dedicated file/folder for the "feature-queries", and I have always been exporting these "feature-queries" as functions with names relating to the functionality in question, and not the version number. See this example from the messaging-app:

export function supportsAttachments(dhis2CoreVersion) {
    return dhis2CoreVersion > 30
}

My main motivation for this approach was long-term maintenance. Suppose I haven't touched an app for a while, and I return to it. If I encountered some logic that essentially said "if the server supports attachments, show the attachment components", I would feel pretty comfortable with that. However, if I saw "if the server is >=31, show the attachment components", I would probably end up looking into the reason for this logic being in place and wasting a bit of time.

So, when I saw the example of switching based on the version number itself, I initially thought it was a bad idea. But thinking about this, and the additional moving parts we'd have to introduce, some more, I think we can probably provide some info about which versions introduce/break what at some other point. As such version number based switching is probably fine. Perhaps the components could even support both a version and a feature prop...

Introducing a sanitisation layer

This could be a good solution for ironing out API inconsistencies across versions, but I also have some reservations about this:

• Introducing an additional layer introduces another point of failure and another environment project to work in. It would be good to iron out inconsistencies in a central location, but I am concerned about the additional maintenance burden.
• This is quite a big undertaking but will actually only solve one part of a bigger puzzle. It would be a good way of normalising away inconsistencies, but it won't help for new features that are introduced. To go back to my example about support for attachments in the messaging app:
  • This new layer won't help us with the actual support for attachments, so there is no API sanitation possible
  • This new layer also doesn't help us conditionally rendering these attachment buttons
• I don't know whether this last point is all that relevant, but I am also slightly worried about external developers. Would they also need to be comfortable working on that new layer? If so, it will raise some barriers for contributing to the DHIS2 ecosystem. And we are actually in the process of making it easier for devs to get started...

---

Some of my own thoughts below....

Version toggle lifecycle

I already brought this up in Slack, but think it'd be good to have it here too. We need to consider the following:

• Whenever a new feature or breaking change is introduced to the API we need to introduce feature toggling code
• Feature toggling code is branching code (if/else or switch) and introducing branching code means adding complexity to an app
• Over time more toggling code will be introduced and the app will grow in complexity
• Luckily we only have to support the current version of dhis2-core and 2 versions below, so each feature toggle has a very distinct EOL: It can be removed once the dhis2-core version that introduced the breaking change or feature becomes the lowest supported version.

So in principle, each app could remain fairly constant in terms of complexity. Sometimes feature toggling code will needs to be introduced, but other times it can be removed. However, if we don't have a system in place to detect when we can remove old feature toggles, the risk of the app slowly growing into a complex monster is very real.

Just to illustrate, I think the messaging-app example above is very on point: attachment-support was introduced in v32, and our lowest supported version currently is v32. So I actually should have removed this feature-toggle by now, but as you can see it's still the 😊 . So it's very clear we need some mechanism for reminding us to remove things.

Some rough ideas about what I think we need in our apps

I'm not sure about that sanitisation layer yet, so not taking that into account. Currently I am thinking that we probably need this:

• Some components and hooks to do the actual branching
• Some sort of configuration file that holds information about the feature toggles in an app. Probably an array with a list of features/ breaking changes. Each item would probably need a version, name, description and type (i.e. FEATURE | BREAKING_CHANGE) field.
• Maybe some script that we can run locally, and/or during CI to list stale toggles (as described in the version toggle lifecycle section). We could even implement some rules, so that a check would fail if an app contains too many stale toggles (maybe a bit of an extreme measure).
• Perhaps we could even feed this configuration file to a context provider, and let components import this context via a hook, probably exposing some helper method...

const { hasFeature, hasBreakingChange } = useFeatureToggleContext()
if (!hasFeature('attachments')) {
    return null
} else {
    return <AttachmentButtons />
}

Some additional note:

• I think the word "feature-toggling" has a nice ring to it, but I think we need to think of another one. Ideally in our conversations and later in our code we need to find a word that describes both new api features and breaking changes too.
• The configuration file I describe above should be there not only to be used by the app-code but also as a source of documentation for the developer, see the "Using version numbers VS functionality-names to toggle" section for details.

[HendrikThePendric]

Another point:
I think we need to discuss versioned VS un-versioned endpoints for feature-toggling apps. As I remember it, we agreed that it would be OK/preferred for apps to do un-versioned calls, but I think the sentiments have shifted regarding that…

The reason why we (mainly @varl and I) preferred un-versioned calls (at the time) was that it completely removes to need to update the api version as releases are produced.

I can see why the maintainers of the webapi would expect apps to always do versioned calls to the api though.

FYI: We also tried doing versioned calls to the minor version of the current dhis2-core instance. I believe we did that in the usage-analytics-app, we'd do one un-versioned GET to get the system-info resource to read the minor version number, and after that we'd use that in the baseUrl. There were some practical issues with this though, most notably the problem was that just after a release the dev instance would get a minor version++, but most versioned endpoints would be unavailable at that time. There might have been other issues too, but I'm afraid they've slipped my mind.

[amcgee]

See https://jira.dhis2.org/browse/LIBS-245 for the epic which will implement this functionality.

• https://jira.dhis2.org/browse/LIBS-273 will support React sub-tree selection based on server version
• https://jira.dhis2.org/browse/LIBS-274 will support server version as input when constructing dynamic queries, and selectors to normalize the return values. This addresses what @paschalidi mentioned above - there is definitely some more thinking to do there, though.

[amcgee]

Thanks @paschalidi @HendrikThePendric for the input on this (much delayed)!!