ci: heavy linux workload on ubuntu-latest-4-cores

[bruno-garcia]

Also fixes #473 (comment)

Looking at the large runner here, looks like we need to use the right name

#skip-changelog

[bruno-garcia]

Status: Downloaded newer image for ghcr.io/epicgames/unreal-engine:dev-slim-5.4.1
29d835b3ddc370bc69d060b9993670e0e890fc1a2313a49416fe5c09f66844c8
+ docker logout ghcr.io
Removing login credentials for ghcr.io
+ docker exec --user root unreal useradd -u 1000 -g 1000 --create-home gh
useradd: UID 1000 is not unique
Error: Process completed with exit code 4.

@tustanivsky any idea what's up?

[bruno-garcia]

A problem I see already is that it'll run the build in series. Since we have 1 agent, and 3 parallel builds for it.

[tustanivsky]

Status: Downloaded newer image for ghcr.io/epicgames/unreal-engine:dev-slim-5.4.1
29d835b3ddc370bc69d060b9993670e0e890fc1a2313a49416fe5c09f66844c8
+ docker logout ghcr.io
Removing login credentials for ghcr.io
+ docker exec --user root unreal useradd -u 1000 -g 1000 --create-home gh
useradd: UID 1000 is not unique
Error: Process completed with exit code 4.

@tustanivsky any idea what's up?

After investigating this for a while I've figured out that there are some differences between the standard and large GitHub runners in terms of user configuration. On both of them we run agent on behalf of runner user however it has different UIDs:

• 1001 on standard runner:

$ cat /etc/passwd
...
runneradmin:x:1000:1000:Ubuntu:/home/runneradmin:/bin/bash
runner:x:1001:122:,,,:/home/runner:/bin/bash

• 1000 on large runner:

$ cat /etc/passwd
...
runner:x:1000:1000:Ubuntu:/home/runner:/bin/bash

At the same time UE docker container provided by Epic Games already has a ue4 user with UID 1000 which causes the above error when we're running the following command:

sentry-unreal/.github/workflows/ci.yml

Line 183 in 311b005

docker exec --user root unreal useradd -u $uid -g $gid --create-home $user

This step is required to install UE4 CLI that we're using later on to build the plugin and run tests. Since it's just a convenience utility for us I'll see if we can get rid of it and use the Unreal's build tools directly to avoid dealing with that matching UIDs issue.

[bruno-garcia]

@bitsandfoxes should we make this a matrix instead? So it runs on both 4-cores and 4-cores-arm64?

[bruno-garcia]

Oh actually that won't work because there are not arm64 docker images for UE?

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

[tustanivsky]

There's no official arm64 docker images for UE so it won't work (for now at least). However we may utilize our new arm64-runner for the future CI pipeline improvements (#575) and proper support of this arch (#473)

[tustanivsky]

Removing ue4cli usages from the existing workflow spared us from creating additional user in docker container and allowed to avoid UIDs collision problem entirely.

Also, with this large GitHub runner we now can package the sample app in CI for all three UE version within a reasonable amount of time and without a risk to run out of space.

[tustanivsky]

@bruno-garcia Are there any additional changes required within the scope of this PR beyond the introduction of the ubuntu-latest-4-cores runner to our CI pipeline?

[bitsandfoxes]

Could you explain what's going on here? If I read that right:

1. We're first building the project via the RunUAT.sh but with all the flags.
2. We're copying some logs to somewhere else
3. We're running some tests but from within the Editor

Do I get that right?

[tustanivsky]

Yes, that's correct.

The ue4cli tool handled all the flags internally so the corresponding command looked cleaner but since we had to get rid of it, we now must specify them explicitly.

Caching build logs can be useful if some issues during the sample packaging arise. This step was moved here as it has finally become possible for all supported UE versions (and not only for 4.27 as previously) to complete the packaging without the risk of running out of disk space.

To run tests we need the Editor executable, it's a common syntax for the corresponding command.

[bitsandfoxes]

Merging this to move forward on it.