Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

e2e docker environments are hard to tweak/adjust setup, because docker config files are "hard coded" in various scripts #9337

Open
haszari opened this issue Aug 27, 2024 · 2 comments · May be fixed by #9364
Labels
focus: devops Release processes, monitoring, automations, dev tools, CI/CD pipeline type: developer experience type: enhancement The issue is a request for an enhancement.

Comments

@haszari
Copy link
Contributor

haszari commented Aug 27, 2024

Describe the bug

docker-compose automatically searches for a compose.yml and an optional, local developer override file compose.override.yml (aka docker-compose.yml and docker-compose.override.yml in earlier versions. From the docs:

Using -f is optional. If not provided, Compose searches the working directory and its parent directories for a compose.yml and a compose.override.yml file. You must supply at least the compose.yml file. If both files exist on the same directory level, Compose combines them into a single configuration.

This flexibility is really useful, especially when debugging problems, or trying out different options as part of iterating on and improving our system.

Our e2e setup hard-codes the main compose config in various helper scripts. This means that it's not practical to override any docker settings locally – to do so, you'd need to hack the helper scripts. Examples:

step "Starting client containers"
docker compose -f "$E2E_ROOT/env/docker-compose.yml" start
if [[ "$E2E_USE_LOCAL_SERVER" != false ]]; then
step "Starting server containers"
docker compose -f "$E2E_ROOT/deps/wcp-server/docker-compose.yml" start
fi

step "Starting CLIENT containers"
redirect_output docker compose -f "$E2E_ROOT"/env/docker-compose.yml up --build --force-recreate -d wordpress
if [[ -z $CI ]]; then
docker compose -f "$E2E_ROOT"/env/docker-compose.yml up --build --force-recreate -d phpMyAdmin
fi

step "Starting SERVER containers"
redirect_output docker compose -f docker-compose.yml -f docker-compose.e2e.yml up --build --force-recreate -d

Proposed improvement

  1. Restructure our e2e config so each container uses the default compose filenames.
  2. Update the scripts to not explicitly specify the compose file -f – so we get the default behaviour.

This would allow me to use override files to fix the config of the db containers.

Alternatives:

Why I care

I haven't been able to run the e2e environment. The db container dies very soon after launch.

I suspect this is because our default docker setup exposes mysql data (for server and client) as a shared folder. My main development environment uses docker-compose.override.yml to use a named volume instead, which works perfectly for me (and I think it's a more robust approach – I don't think we need to expose the data folder). See related issue for main dev env:

If I could use an override file for all the various WordPress containers needed for e2e, I could specify a named volume instead of a mount, and 🤞 hopefully write and run more e2e tests.

@haszari haszari added type: bug The issue is a confirmed bug. type: enhancement The issue is a request for an enhancement. type: developer experience focus: devops Release processes, monitoring, automations, dev tools, CI/CD pipeline labels Aug 27, 2024
@haszari
Copy link
Contributor Author

haszari commented Aug 30, 2024

Noting another blocker to working around this problem…

git clone --depth=1 --branch "${WCP_SERVER_BRANCH-trunk}" "$WCP_SERVER_REPO" "$SERVER_PATH"

  • The setup.sh script clones the server repo (if using "test against local server" option E2E_USE_LOCAL_SERVER).
  • So the configuration, docker env, scripts etc need to be 100% bulletproof for this local server option to work.

If the docker env, or other config, needs tweaking for whatever reason, this needs to be pushed up to default branch of server repo.

A way around this would be params or environment variables to allow devs to override stuff (e.g. specify a different branch, or override docker config).

That would allow more workarounds (and also make it easier to iterate / experiment with improvements). However would be preferable to make the default docker config more robust (work every time for everyone).

@haszari
Copy link
Contributor Author

haszari commented Aug 30, 2024

In the meantime, I'll try the remote server option, as a workaround - reduce the complexity of my local setup (~3 fewer containers).

@haszari haszari linked a pull request Aug 30, 2024 that will close this issue
6 tasks
@vbelolapotkov vbelolapotkov removed the type: bug The issue is a confirmed bug. label Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
focus: devops Release processes, monitoring, automations, dev tools, CI/CD pipeline type: developer experience type: enhancement The issue is a request for an enhancement.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants