Build the UI and bundle it in Python packages

[fabcor-maxiv]

It could be helpful to have pre-built UI in Python packages for easier deployment. This should be investigated.

[marcus-oscarsson]

Thanks,

I guess we can build the front end in the CI pipeline and then use pkg_resources.resource_filename in someway to get the default path for flask, do you think that might work ?

[fabcor-maxiv]

Yes, that could work, in principle.

I would use importlib.resources rather than pkg_resources, but that is detail.

Does any facility do customisation of the UI?

[marcus-oscarsson]

The idea is that the all customization that is needed can be done through configuration files, the removal of the .env file was a last step in this process.

[fabcor-maxiv]

Not working on this right now, but it came to my mind anyway...

The idea would be to have the UI built in the wheels (and sdist), is that right? But does anyone install mxcubeweb directly from a wheel? Currently at MaxIV we don't, we install from a git checkout.

[marcus-oscarsson]

I guess it could be done by building the UI automatically and placing it somewhere in the repository like this it can be used both by "git checked-out" sources and then possible included in a package. The first would probably be a quite straight forward thing to do (but perhaps a bit unconventional)

[fabcor-maxiv]

building the UI automatically and placing it somewhere in the repository like this it can be used [...] by "git checked-out" sources [...] (but perhaps a bit unconventional)

In this case, my vote would go for "no". Unconventional indeed :D ... I do not think the pros would outweigh the cons in this case. I am not sure how we would make sure that things do not go out of sync, that the built code is always up-to-date.

I am not against the idea of having the built UI in the sdist and wheel, though, if it helps anyone. Do we know of any setup where mxcubeweb is installed directly from a sdist or a wheel (from PyPI)?

[axelboc]

Hmm, I definitely wouldn't version-control the front-end build output. Deploying MXCuBE via git checkout is already less that ideal, so I don't think we should make this process more convenient in any way. 😝

The ideal deployment, in my opinion, should not require: 1. installing Node and pnpm, 2. installing front-end dependencies, 3. building the front-end bundle. This all has to be done in a dedicated, controlled environment ahead of time (e.g. CI machine). Embedding the front-end build output in the wheels would definitely meet this requirement, but as you said @fabcor-maxiv if nobody deploys from the wheels, there's no point in doing it.

Perhaps this issue is a bit premature and the first step should be to write high-level deployment "best practices" and let each facility find the technical solutions that suit its needs (Docker, etc.)?

[marcus-oscarsson]

Well, we are not currently deploying from wheels but we could in theory do it, we build a package for each version we deploy on the beamlines. However, the really big disadvantage of deploying a package (and the reason we are currently not doing it) is for debugging on the beamline.

[axelboc]

This is where having two side by side environments could help:

• a containerised production environment that can be restarted/reverted in a few seconds if anything wrong happens (or a wheels-based deployment);
• plus a staging environment for debugging purposes via git checkout

[fabcor-maxiv]

write high-level deployment "best practices" and let each facility find the technical solutions that suit its needs (Docker, etc.)?

Yes to this. That is why I started writing this document: https://mxcubeweb.readthedocs.io/en/latest/dev/deployment.html. And I hope the questionnaire will help uncover more things that need documenting.

The ideal deployment, in my opinion, should not require: 1. installing Node and pnpm, 2. installing front-end dependencies, 3. building the front-end bundle. This all has to be done in a dedicated, controlled environment ahead of time (e.g. CI machine).

Yes, this is feasible, but I am not sure how much can be done in a way that works for all. In other words, not sure how much can be done here on GitHub. Probably every facility needs to do this for themselves. And so we are back to: we need to write documentation.

[Aside: I am always a bit torn on writing this kind of documentation, because in the end, it is nothing specific to MXCuBE, it is just standard practices when working with Python, Node.js, web applications, software in general. There is way better documentation out there than what we could possibly write.]

we are not currently deploying from wheels but we could in theory do it, we build a package for each version we deploy on the beamlines. However, the really big disadvantage of deploying a package (and the reason we are currently not doing it) is for debugging on the beamline.

Yes, same for us. Although up to now my impression was that debugging at the beamline mostly involves debugging the Python back-end and rarely the JavaScript UI front-end. So I guess, at least in our case at MAX IV, we could built the JavaScript UI front-end in our CI pipelines, upload it to our internal npm package repository and install it at the beamlines from our internal npm package repository.

This is where having two side by side environments could help:

• a containerised production environment that can be restarted/reverted in a few seconds if anything wrong happens (or a wheels-based deployment);

• plus a staging environment for debugging purposes via git checkout

Maybe, I have not considered this... No idea if that would help at MaxIV... I would need to think about it...

[axelboc]

[Aside: I am always a bit torn on writing this kind of documentation, because in the end, it is nothing specific to MXCuBE, it is just standard practices when working with Python, Node.js, web applications, software in general. There is way better documentation out there than what we could possibly write.]

Agreed, our documentation should just point to external resources, maybe like this or that.

Yes, same for us. Although up to now my impression was that debugging at the beamline mostly involves debugging the Python back-end and rarely the JavaScript UI front-end.

Even having the Python code here and there makes it tempting to try to fix bugs directly in the production environment, but this can break things even more... The process and scope of debugging on the beamlines is definitely worth thinking about too (and documenting). Maybe something like this (though I'm far from an expert and have no experience doing this whatsoever):

1. Bug report
2. Turn it off and on again (i.e. restart the container) — if the bug remains, continue.
3. If it might be caused by a recent deployment, try rolling back to a previous version.
4. If the bug is gone then it was regression: use the staging environment to identify the problematic version and to find the root cause. Fix the bug, redeploy to staging, and then to prod.
5. If the bug appears out of nowhere or rolling back doesn't fix it, debug in production (I think there are ways to debug running containers)
6. If it's identified as an environment issue, fix it and restart the container.
7. If it's a bug in the code that had never been identified before, fix it in the codebase, redeploy to staging, and then to prod.

Note that beamline staff would easily be able to handle 2. and 3.

The steps between fixing a bug in the codebase and redeploying to staging/production could for instance be done via a dedicated, facility-hosted repository with a Docker file and some manually-triggered CI workflows designed to:

1. check out and set up the core MXCuBE packages at a given commit hash or version, build the front-end, and build/publish the Docker image to a facility-hosted Docker registry;
2. deploy a given Docker image from the registry to a given staging and/or production environment.

Sorry for going way out of scope of the initial issue... 😅