Skip to content
This repository has been archived by the owner on Jun 27, 2022. It is now read-only.

[BUG] ELB throws 500 bad GW #12

Open
emctl opened this issue Feb 18, 2022 · 6 comments · Fixed by #13
Open

[BUG] ELB throws 500 bad GW #12

emctl opened this issue Feb 18, 2022 · 6 comments · Fixed by #13
Assignees
Labels
bug Something isn't working

Comments

@emctl
Copy link

emctl commented Feb 18, 2022

Tested this with defaults apart from image (I used 23.0.1) and while everything provisions correct, NC never passed the health check and is in a 500 loop.
I unfortunately don't have time to investigate any further today but can follow up as needed over the weekend. I just wanted to flag this to see if anyone else is in the same boat?

I have not tried this with the default image from the template as I want to use the latest available

@f7o
Copy link
Contributor

f7o commented Feb 20, 2022

It takes loads amount of time to spin up the container. This is an issue with the official docker image and how they initialize nextcloud. At the first run and on every upgrade they use rsync to copy over the application files to a persistent file storage (EFS in this case). Initially EFS has no IOPS provisioned and is therefore quite slow.

I will increase the Health check grace period on the ECS service configuration. Tested with 2500, that should be enough for now. See https://github.com/aws-samples/aws-serverless-nextcloud/blob/main/ecs-nextcloud.yml#L623

We are trying to get the nextcloud docker image in line with best practices, such as keeping it stateless. There is no reason to require a persistent volume for it. See nextcloud/docker#1548

@f7o
Copy link
Contributor

f7o commented Feb 21, 2022

I will further push for improvement of the official docker image or provide an alternative docker image which follows container best practices.

@f7o f7o closed this as completed in #13 Feb 21, 2022
@emctl
Copy link
Author

emctl commented Feb 24, 2022

@f7o - FWIW, I tried this again with multiple different NC images waiting 30+ mins between redeploys with the same problem. Happy to spin up again for the needs of debugging if needed. This was the exact same issue, the image was running, all metrics looked fine (including EFS).

@f7o
Copy link
Contributor

f7o commented Feb 24, 2022

Can you share logs from ECS tasks? Especially the initial task running when EFS and RDS is deployed the first time?

Which AWS Region did you deploy into?
I assume the cloudformation stack was created successfully, it is just the application never coming up, including health checks?

If, for what ever reason, the first container being started does not finish the initial installation routine and is being stopped and replaced with a new one, the whole installation is broken. One has to prune the EFS and remove all tables from the RDS database to get a clean installation.

@f7o f7o self-assigned this Feb 24, 2022
@emctl
Copy link
Author

emctl commented Feb 25, 2022

Can you share logs from ECS tasks? Especially the initial task running when EFS and RDS is deployed the first time?

Which AWS Region did you deploy into? I assume the cloudformation stack was created successfully, it is just the application never coming up, including health checks?

If, for what ever reason, the first container being started does not finish the initial installation routine and is being stopped and replaced with a new one, the whole installation is broken. One has to prune the EFS and remove all tables from the RDS database to get a clean installation.
@f7o
Sorry for the delay -

  • Default region from quickstart (i.e. eu-west-1)
  • Stack (and nested stacks) all provision successfully

Defaults apart from usernames/pw's for the db and nc itself.

looks like it doesnt like a non default db name -
image

I deleted the stack and retried to default db names -

This seems to work so I geuss this is due to changing db names - should that be removed as an input?

@f7o
Copy link
Contributor

f7o commented Mar 1, 2022

okay, seems like you are right and non-default db name does not quite work. i will investigate further

@f7o f7o reopened this Mar 1, 2022
@f7o f7o added the bug Something isn't working label Mar 1, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants