add fly.io PR review apps

[chris48s]

closes #7924
closes #8337

We lost access to Heroku review apps, but they were very useful 😭

I had a look into https://github.com/superfly/fly-pr-review-apps but this only works on PRs from branches on the source repo (not PRs from forks) because a fly token must be exposed to the workflow. Even if we could expose a fly.io token to PRs from forks, we really don't want to do that.

This PR is based on the code in https://github.com/superfly/fly-pr-review-apps but adapted for our needs. Merging this PR will give us a workflow which we can trigger manually. We give it a PR number when we trigger it and that will create or update a review app for us using fly.io. Some notes:

• It is up to us to manually create/update the review apps. Review apps will not be auto-created or auto-update when the PR is updated after creation. That said, I think this should be an improvement over having to check PRs out locally to review them.
• It is up to us as maintainers to check to make sure PRs we deploy are not going to exflitrate all our secrets or whatever before we click deploy. Any code changes are going to get pulled into a context with a fly.io token in the environment.
• I've set a token in the repo secrets.
• One thing we do need to consider is that review apps will have some cost attached to running them, but we have the budget available in opencollective and I think the convenience is worth paying for.

[chris48s]

Just to clarify: This will fail to deploy a review app if there is a merge conflict.

[calebcartwright]

I'm having a bit of deja vu so apologies if this has been asked and answered before, but is there any reason we wouldn't want to check out the PR ref directly? I.e.

git fetch pull/$PR_NUMBER/head:pr-$PR_NUMBER && git checkout pr-$PR_NUMBER 

[chris48s]

I don't think we've discussed it before.

At this stage, there is a very pragmatic reason why I'm doing it this way: There is no open PR which has these files in the source branch yet. So if I check out the source branch for any existing PR, fly.toml won't be on it. Merging the PR source branch into the working branch allowed me to test this workflow on some real PRs.

Once we get to the stage where all the open PRs are based off a master branch that has these files on it, it might make more sense to check out the branch directly. That would also allow us to deploy review apps for PRs where there is a merge conflict.

I think in the first case, being able to use this on existing open PRs is useful.

[chris48s]

One thing I realised when I tried to make a preview app for #9014 was that applying the diff doesn't work if there are binary files in it.

I've switched to merging the source branch in 9d93b98 (which solves that, but will still fail if there is a conflict), but still sticking with merging for now for the reason above (no open PRs have this code on the source branch yet)

[github-actions]

	Messages
📖	✨ Thanks for your contribution to Shields, @chris48s!

Generated by 🚫 dangerJS against 9d93b98

[calebcartwright]

Does the creation of this file have the potential to impact any of other workflows/processes? (e.g. the staging/prod apps)?

[chris48s]

Our existing fly.tomls live elsewhere and just says "grab an image from DockerHub and deploy it". Its shieldsio/shields but it could be any old image on DockerHub so what's in this repo only really impacts anything to the extent that the code in it is in the docker image we deploy.

Adding this here does mean if you were to flyctl launch from the root of this repo, that would create an app called shields-io-review-apps (which doesn't currently exist in our account). If we don't want it in the root, I think we could also move it somewhere else (maybe under .github) and specify the path to the config file in deploy-review-app.sh with the --config arg.

You have made me think this doesn't need to go in the docker image though, so I've added it to .dockerignore in f8baf18

[calebcartwright]

Couple questions but otherwise lgtm