Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

There is no need to explicitly list variables in envsubst #473

Open
sadiqkhoja opened this issue Aug 31, 2023 · 1 comment · May be fixed by #830
Open

There is no need to explicitly list variables in envsubst #473

sadiqkhoja opened this issue Aug 31, 2023 · 1 comment · May be fixed by #830
Assignees
Labels
ops Docker, nginx, ops to deploy Central refactor Improves code without altering behavior

Comments

@sadiqkhoja
Copy link
Contributor

envsubst '$DOMAIN $BASE_URL $SYSADMIN_EMAIL $ENKETO_API_KEY $DB_HOST $DB_USER $DB_PASSWORD $DB_NAME $DB_SSL $EMAIL_FROM $EMAIL_HOST $EMAIL_PORT $EMAIL_SECURE $EMAIL_IGNORE_TLS $EMAIL_USER $EMAIL_PASSWORD $SENTRY_ORG_SUBDOMAIN $SENTRY_KEY $SENTRY_PROJECT' \

@sadiqkhoja sadiqkhoja self-assigned this Aug 31, 2023
@matthew-white matthew-white added ops Docker, nginx, ops to deploy Central refactor Improves code without altering behavior labels Aug 31, 2023
@alxndrsn
Copy link
Contributor

alxndrsn commented Dec 6, 2024

This won't work for files which contain other strings of the form $name, e.g. nginx config:

$ envsubst <<< 'proxy_set_header Host $host;'
proxy_set_header Host ;

Given this limitation, here are some options:

double-escape dollars which should be preserved:

$ envsubst <<< 'proxy_set_header Host $${W}host;'
proxy_set_header Host $host;

add second conversion layer

$ envsubst <<< 'host={{host}} shell=$SHELL' | sed 's/{{\([^}]*\)}}/$\1/'
host=$host shell=/bin/bash

automatically generate SHELL-FORMAT

printenv

$ envsubst "$(printenv | awk -F= '{ printf "$%s", $1 }')" <<< 'host=$host shell=$SHELL'
host=$host shell=/bin/bash

compgen

$ envsubst "$(compgen -v | awk '{ printf "$%s", $1 }')" <<< 'host=$host shell=$SHELL'
host=$host shell=/bin/bash

use perl in place of envsubst

...and require substituted vars to take the form ${CAPS_AND_UNDERSCORES_ONLY}

$ perl -pe 's/\$\{([A-Z_]*)\}/$ENV{$1}/g' <<< 'host=$host shell=${SHELL}'
host=$host shell=/bin/bash

alxndrsn pushed a commit to alxndrsn/odk-central that referenced this issue Dec 6, 2024
For nginx config, a new approach is implemented with perl.  This is because unrestricted use of `envsubst` on nginx config files will replace nginx variables like $host, $request_uri with an empty string.

Closes getodk#473
@alxndrsn alxndrsn self-assigned this Dec 6, 2024
alxndrsn pushed a commit to alxndrsn/odk-central that referenced this issue Dec 9, 2024
For nginx config, a new approach is implemented with mawk.

This is allows an equivalent to `envsubst`, but:

* no need to explicitly list all variables
* throw error on missing variables
* do not replace nginx vars like $host, $request_uri with an empty string

Closes getodk#473
alxndrsn pushed a commit to alxndrsn/odk-central that referenced this issue Dec 9, 2024
For nginx config, a new approach is implemented with mawk.

This is similar to envusbst, but more ergonomic:

* no need to explicitly list all variables
* throw error on missing variables
* do not replace nginx vars like $host, $request_uri with empty strings (in contrast to envsubst when executed without an explicit variable list)

Risks:

There are a couple of changes which may break existing deployments:

* changing client-config.json.template
* requiring all substituted variable to be defined

Closes getodk#473
alxndrsn pushed a commit to alxndrsn/odk-central that referenced this issue Dec 9, 2024
For nginx config, a new approach is implemented with mawk.

This is similar to envusbst, but more ergonomic:

* no need to explicitly list all variables
* throw error on missing variables
* do not replace nginx vars like $host, $request_uri with empty strings (in contrast to envsubst when executed without an explicit variable list)

Risks:

There are a couple of changes which may break existing deployments:

* changing client-config.json.template
* requiring all substituted variable to be defined

Closes getodk#473
@alxndrsn alxndrsn linked a pull request Dec 9, 2024 that will close this issue
2 tasks
@github-project-automation github-project-automation bot moved this to 🕒 backlog in ODK Central Dec 19, 2024
@matthew-white matthew-white moved this from 🕒 backlog to ✏️ in progress in ODK Central Dec 19, 2024
@matthew-white matthew-white linked a pull request Dec 19, 2024 that will close this issue
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ops Docker, nginx, ops to deploy Central refactor Improves code without altering behavior
Projects
Status: ✏️ in progress
Development

Successfully merging a pull request may close this issue.

3 participants