PR previews for the web apps

[paschalidi]

Foreword

The purpose of this issue is to suggest the implementation of a github action which will deploy a web application as a static website when a pull request is opened or someone commits to it. The gh action will deploy it to some domain and post back the URL as a comment on the PR. I am using this issue a phase before implementation to collect ideas and feedback on the feature itself.

Motivation

Why is this feature being proposed? There are many people who I believe would benefit from being able to click on a link when a PR is open and see the changes that the PR suggests.

Some examples of the different actors that could benefit are:

• Designers will be able to have a quick look on a new feature by clicking on the link with the deployed application. They could then propose changes before the PR is merged into master.
• The QA team might also want to use this previewed version of the app for testing. This will give them the chance to test a bug or feature against different versions of the backend. This might come handy when we have feature toggling enabled.
• Product owners also might wanna have a look on a branch that isn’t merged onto our main branch just yet.
• And finally developers who review a PR and would like to do a short manual check on the functionality their peer has worked on.

As of today our workflow doesn’t enable the above actors to view a feature branch, without having to manually checkout a branch and start the application locally. This might be second nature for developers but I can imagine that some other people would prefer to view the app deployed on the browser.

There are many different ways to achieve what I am suggesting here. Netlify comes out of the box with such functionality, also these days amazon offers it with amplify and firebase have their own solution. As far as I am aware we are currently using Netlify in some parts of our ecosystem to preview a pull request. We use the free tier of netlify which allows 90 minutes of free build time. In this issue an object storage solution can be something such as S3 of aws, Cloud storage from google or the Openstack from the Oslo university is suggested. While reading through the issue please keep in mind that there is not a specific cloud provider that is suggested. This is done on purpose so that we can all decide which one serves us better.
As is already being said above this RFC suggests utilising an object storage solution to store the artifacts of our build applications. The question is now why I am suggesting to use an object storage solution and not using existing solutions such as Netlify or aws amplify and so on. One answer is cost. Another answer is automating 🤖 and toil reduction.

About cost for example aws amplify would charge us $0.01 per build minute. When I am writing this we are more than 12 frontend developers.

Lets see how much a real world example with the following assumptions will cost us assuming the following.

1. Each day 2 new PRs are created.
2. Each application has average build time = 3 mins.
3. For each the pull request will be 3 pushes to the branch which we want to create a preview for.
4. Average number of work days/month for each dev = 17.
5. Total FE devs = 12.

Amplify will cost a month 2*3*3*0,01*17*12 = $36,72. So now, given that we usually don’t just push three times on a branch and given that we might grow the FE team I think we could all agree that amplify is expensive. On amplify’s defence you can have it up and running with just three clicks for each app. And it’s a managed solution by aws which leaves us developers with less things to maintain and let us focus on building FE apps.

Netlify has a different pricing model, they charge per developer. Now arguably we don’t need all FE developers going into netlify’s platform for the simple reason that there won't be much going on there for us to do. However if there will be 4 people going into netlify’s platform this will be around $80 a month

But how much will this cost if we use an object storage solution? Object storage is one from the oldest services cloud providers put together. S3 charges you in this case depending on your usage. Typically will cost $1-3/month. I will make an example similar to the ones I presented before: Assume that:

1. We store 4 gb of web artifacts per month on a bucket.
2. We push 20.000 times on our branches. That means we trigger 20.000 PUT requests on a bucket.
3. We visit the previewed apps 20.000 times each month. This is the same as saying making 20.000 GET requests.

This will cost less than $1 a month. You can use this calculator to see for yourself how much it will cost.

Background

Most of us are familiar with the app-platform. One of the things that app-platform does is to give us the ability to build our applications as standalone FE apps. The output of this build process is static. I say static assets in the sense that when the browser receives the page, it’s usually simple HTML without a lot of content. This then loads the scripts to pull the content into the page.

It also allows us to run the apps against a chosen version of the backend. This looks something like this

So now we have FE apps that run decoupled from the BE monolith we have. The PR preview suggestion that is made here utilises these aspects of the app-platform.

Requirements

When a Pull Request is opened or someone commits to it, we make a new build, deploy it to somedomain.com/YOUR_APP_NAME_GOES_HERE/<pr-number> and post back the URL as a comment on the pull request. This could look something like this

As I explained in the beginning of the issue different people who are interested in viewing the work that has been done in a PR will now be able to access the web application using the details link. Thanks to the app-platform the user can log in in the app using a version of the server they like.

Since we are planning on moving towards a more continuous release workflow, allowing people to do manual testing or design review on tickets while they are still on the Pull Request can create shorter development cycles. Shorter development cycles could mean improved development speed.

Implementation

Basically all comes down to github actions. Publishing an action that will build and publish each time a PR opens or a commit is made on a PR. The github action will take certain arguments that will come either hardcoded or in the case of sensitive data in the form of secrets.

The different systems we will use to achieve the pr preview will be described below.

Object storage solution

As I have mentioned already I don’t focus on a specific provider here to let people discuss which will be the best for us.

But in any case we will end up with an object storage solution where we will store the artifacts for each application for each PR. Here it is suggested to use one bucket for all our FE applications. In the bucket each application will have its own folder. The application’s folder will then contain multiple folders, one for each PR. Visually this will look something like bellow

.
.
.
├── who-data-quality-app/
│   ├── PR-1
│   ├── PR-2
│   ├── ..
│   └── PRN
├── capture-app/
│   ├── PR-1
│   ├── PR-2
│   ├── ..
│   └── PRN
├── data-visualiser/
│   ├── PR-1
│   ├── PR-2
│   ├── ..
│   └── PRN
.
.
.

Now each time a developer pushes a new change on a pull request that pull request will be built and uploaded to the corresponding folder, removing all the old artifacts.

Of course we don’t want to have all the PR artifacts kept forever and ever. To solve that we can utilise something called object lifecycle management or in other words we can set an expiration date for these folders. Simply said, we can ask the provider to delete the folder PR-X after 50 days of inactivity.

Github actions

Here it is described what the github action will do. In each push of an open PR we build and upload these built artifacts on the storage provider. The action will take some arguments as inputs. These will come either hardcoded or in the form of secrets when the information is sensitive.

A fleshed example using some third party github actions could be:

name: Deploy Preview

on: pull_request

env:
 DESTINATION: USE_YOUR_APP_NAME_HERE/${{ github.event.number }}

jobs:
 deploy:
   runs-on: ubuntu-latest
   steps:
     - uses: actions/checkout@v1

     - name: Install
       run: yarn

     - name: Build
       run: PUBLIC_URL=/${{ DESTINATION }} yarn build --standalone

     - name: Deploy
       if: success()
       uses: jakejarvis/s3-sync-action@master
       with:
         args: --delete # this will remove old artifacts and keep only the new ones
       env:
         AWS_S3_BUCKET: pr-preview-github.com
         AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
         AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
         AWS_REGION: 'us-east-1'
         SOURCE_DIR: 'build'
         DEST_DIR: ${{ DESTINATION }}

Two things that I wanna mention here before we start discussing the yaml file above. First, I don’t mean to suggest using aws here. I am only using code to communicate how this process could be done instead of writing down each step in english. And second, I think creating a gh action out of this might be a good way to reuse the above across our ecosystem. Perhaps we would like to replace the third party library with our in house solution to be sure from a security perspective but I am sure we handled more difficult things in the past.

Now the gh action has three steps. Install Build and Deploy. I wanted to bring your attention to the build step. There are more than one interesting things that happen in this line. First of all we are defining an env called PUBLIC_URL and then we pass the standalone parameter to the build script.

Why do we need the PUBLIC_URL? I talked about having all the applications in one bucket before. I suggested this for simplicity since having 40 buckets isn’t really necessary as far as I can tell. However when we trigger a simple build our applications are built to be deployed to the root of a domain. Since we are not deploying to the root /, but to some ${appName}/${prNumber} you can imagine that this creates a problem. PUBLIC_URL solves exactly this problem. Creates artifacts that will be ready to be deployed in the right directory of our object storage. The app-platform comes with this feature by default.

Now about the standalone argument. Passing this parameter as true will simply expect the user to add a backend url to login. When we build the apps for the production environment we don’t build them as standalone apps. In our case we want them to run in isolation. Then as the name indicates the app can be run without a backend server and this is why app-platform basically plays a key role in this solution.

Before I wrap up, I wanted to say that having this working will require the user to allow the domain of the previewed PR in the access setting of the instance. Something like this would do the job as far as I can tell.

[Philip-Larsen-Donnelly]

Thanks for the initiative @paschalidi !

This is something that has been on our QA "list" for a long time. I thought it was something we had in place for some repos already with Netlify, so I would like to hear the feedback from people like @amcgee and @varl who know more about that?

Note: it is generally OK for us to pay a small amount for services where we really gain a benefit. Time to set up and maintain "proprietary" low-cost solutions also has a cost - sometime a significant one. Though I think the architecture you outline is fairly low-maintenance; making use of stable and standard tools that we are already using.

[amcgee]

Hi @paschalidi - Thanks for this great overview! I'll respond in more detail this week, but first wanted to point out one thing-

We currently have Netlify's Open Source plan which gives us their Pro features and unlimited team members. We are limited in bandwidth (400Gb) and build minutes (1000) per month. We run into the build minute cap somewhat regularly, but can also add more for $7 per 500 minutes. We could also potentially use Netlify as our static site storage from a GitHub action to avoid the netlify build minutes which comes with some advantages and disadvantages over the S3 solution you outlined. I'll share more later, thanks again for putting this awesome overview together!!

[amcgee]

Also @Philip-Larsen-Donnelly We use Netlify currently for deploying all of our documentation - we have used it in the past for applications and it worked quite well, but ran into the build minute limit and some frustration around the CORS settings on debug servers (which will plague us regardless unless we set https://* or https://*.netlify.app in the debug db)

[amcgee]

@Philip-Larsen-Donnelly mentioned that it will likely soon be possible to run commands against only the debug db (not play) which would allow us to open up the CORS setting for debug servers only and support cross-domain PR and continuous deployments

[paschalidi]

Avoiding CORS issues will help go forward regardless the solution we are going with.

@amcgee it looks like I didnt know the exact numbers of netlify when I was writing this. I cant remember what made me conclude to these numbers. Thanks for providing the right numbers and pricing here. :)

I wanted to say that, of course I am only suggesting a possible way of going forward with the PR previews. If we believe collectively that some other solution is better, either because it saves us time and maintenance work or anything between these lines then proceeding with this other solution must be the way forward.

With that said, I will wait for your detailed response once you got the time for it.

[paschalidi]

After discussing with the @dhis2/frontend group we decided to go for one of the three following solutions.

1. netlify
2. surge
3. vercel

The reason is simply that these solutions require less work to begin with. Simply we can use a github action that will 1. build artefacts and 2. deploy the artefacts on a unique url for each push.

Last time when I shared the netlify and surge solution to the group, it was mentioned that we wanted to see if vercel is a better alternative than those two.

I will try to summarise the pros and the cons in my opinion for each approach in my attempt to help us reach a conclusion on the solution we want to go for.

Netlify

Props

1. dashboard where you can see all sort of information like
   • build time for each job
   • logs of each gh action - however you can ofc see the same logs in github itself
   • who pushed which commit- however you can ofc see the same logs in github itself
     and so on
2. netlify is obviously a well known provider
3. ability to customise the url and the comment on the PRs. See here

Cons

1. Build minutes cost are not free, however it is affordable.

Who uses it already

Many organisations do

Surge

Props

2. it is free

Cons

1. it does not have customisable urls. However I dont know why we might want to have custom urls for PR previes
2. it does not have a dashboard

Who uses it already

https://github.com/afc163/surge-preview#who-are-using-it

Vercel

Props

1. has a dashboard as well and provides all the functionality that netlify would provide.
2. vercel is also a well known provider
3. seems that there is no pricing for build minutes such as netlify has

Cons

1. has a cost for organisations, however it is affordable.

Who uses it already

Many organisations do

---

A close call for all our competitors above, with no knockout arguments from each side!

Now I think we will need to decide what we actually want to get out of the PR previews.
For example, If we simply wanted to have the PR being previewed all of those solutions would work equally well.
Now if we wanted to have some more advanced functionality such us custom urls for PR previews or collect information from dashboards we would have to go for either netlify or vercel.

I will leave all this information here and will ask the group to vote on their favourite.

Use 1️⃣ 2️⃣ or 3️⃣ for your favourite or even better raise your thoughts/concerns or suggestions with a comment.

Open questions

1. Does any application need to pass environment variables for it to function on PR preview? In this case we might want to see if the all the providers handle this gracefully
2. Support of monorepos is another open question, but to be frank I think all three solutions can support it.

[varl]

Update: I have a cost projection calculation based on what 2020 would have cost with Netlify in a min/max/avg model, it will be discussed with the administration team to see if we can move forward with it.

[varl]

For the curious

The average projection has been approved (~$300 per month). 🦖 🌮 🌷

[paschalidi]

Looks great :) Thanks @varl.

I think the max commits set to 5 per PR might be little. What you think?

[varl]

Maybe, I couldn't figure out a way to script figuring out how many commits we had per PR in all of 2020 so I took the numbers from 50 random PRs across our repos and came up with those numbers. 😖

[paschalidi]

I see makes sense :) Thanks for coming back @varl

[varl]

Status for PR Previews

Apps

status	name	type
✔️	app management	platform app
✔️	approval	platform app
✔️	cache cleaner	platform app
✔️	dashboard	platform app
✔️	data administration	platform app
✔️	data quality	platform app
✔️	datastore	platform app
✔️	data visualizer	platform app
✔️	ui	platform lib
✔️	data quality	platform app
✔️	data quality who	platform app
✔️	import export	platform app
✔️	menu management	platform app
✔️	messaging	platform app
✔️	reports	platform app
✔️	scheduler	platform app
✔️	settings	platform app
✔️	sms configuration	platform app
✔️	translations	platform app
✔️	usage analytics	platform app
✔️	user	platform app
✔️	user profile	platform app
➖	charts	non-platform app
➖	event capture	non-platform app
➖	event charts	non-platform app
➖	event reports	non-platform app
➖	gis	non-platform app
➖	interpretation	non-platform app
➖	immunization analysis who	non-platform app
➖	maintenance	non-platform app
➖	maps	non-platform app
➖	metadata browser who	non-platform app
➖	pivot tables	non-platform app
➖	tracker capture	non-platform app